In other, 2.7 news: I'll upgrade when I have some time, not before friday. If anybody wants me to Not Update for a reason, please tell.
Notices by halcy:icosahedron: (halcy@icosahedron.website), page 19
-
i.w admininstrative notices (polyhedral@icosahedron.website)'s status on Monday, 21-Jan-2019 18:43:21 JST i.w admininstrative notices -
i.w admininstrative notices (polyhedral@icosahedron.website)'s status on Monday, 21-Jan-2019 18:42:20 JST i.w admininstrative notices logs will stay on the main HDD, but since the main DB will be someplace else, it'll allow for more time before things break hard.
* I will make postgres monitoring yell into a place that is not The Void when things go wrong so that next time I will actually notice.
* I will make a buffer file that I can easily delete to free up some space when needed in case the disk runs full again in the future -
i.w admininstrative notices (polyhedral@icosahedron.website)'s status on Monday, 21-Jan-2019 18:42:16 JST i.w admininstrative notices database, freed up some space by cleaning out some temporary files and restarted the database. As it now had some space to work with, and the archival command worked again, it started copying over all the old transaction logs and eventually cleaned them out and resumed normal operations.
3) What will be done to prevent this in the future?
Various things:
* I will move the main database files to its own drive. This is going to be neccesary in the near future anyways. The transaction -
i.w admininstrative notices (polyhedral@icosahedron.website)'s status on Monday, 21-Jan-2019 18:41:40 JST i.w admininstrative notices extended amount of time. This, in turn, was because the command copies the logs to a computer in my home via rsync over ssh, and my ssh port forward through my NAT broke.
2) How was this resolved?
When I was made aware that i.w. wasn't working (thanks dot and colon_three), I logged into the server via ssh and began figuring out what had happened. I first fixed my NAT by rebooting the NAT device. I then took down the site entirely to make sure that the problem does not get worse, stopped the -
i.w admininstrative notices (polyhedral@icosahedron.website)'s status on Monday, 21-Jan-2019 18:41:07 JST i.w admininstrative notices Okay, as promised, here is the slightly longer postmortem. Sorry, no timestamps like all the Cool People, too lazy for that.
1) What happened and why?
The primary HDD that contains the postgres database and transaction log ran full because the transaction logs did not get deleted. This was because I have transaction log archiving set up, which allows for point in time recovery of the database, and the archival command (that copies the log files to an off-site location) failed for an -
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 18:20:29 JST halcy:icosahedron: well, here's hoping that 1) the state government has 250€ in a budget pot for this left somewhere 2) I can find somebody who can sign off on spending those 250€ real quick because I am not an authorized to spend money type person
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 18:17:14 JST halcy:icosahedron: like, god, I'm sure there's legal bull shit reasons for this that exist, and everybody I actually called on the phone was really nice but also
u g h
We also asked Other State Public Broadcaster about a similar clip and they just went "yeah sure np go ahead" and why can't everybody
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 18:03:58 JST halcy:icosahedron: me: "hey dear State Public Broadcaster, who is a public institution with an educational mandate. member of a university, also a public institution with an educational mandate here, acting on behalf of the state ministry of education. can we use this 4 minute clip about literally our research that you shot at our institute in a public, non-commercial exhibit for free? p.s. as you well know our budget that we have for this is literally zero euros."
them: "sure you can, for 250€"
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 07:51:05 JST halcy:icosahedron: @tom bother :( if it helps to stay a day in bremen before/after and fly into/out of bremen / hh, I‘m driving anyways
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 07:30:38 JST halcy:icosahedron: @tom come to Saarbrücken after (before?)
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 06:50:42 JST halcy:icosahedron: Man it messe me up how you watch the credits of animated movies and they have like 10 people credited with rigging
like, jesus
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 06:48:12 JST halcy:icosahedron: @urien i‘m pretty sure they went off cat videos for animation ref where they could and I love it
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 06:11:40 JST halcy:icosahedron: they‘re like little cats! i want to hug them!
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 06:07:50 JST halcy:icosahedron: We‘re watching the How to train your Dragon movies and I love the dragons
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 05:40:44 JST halcy:icosahedron: @codl if you do go for mods, I can absolutely recommend installing a iain banks culture ship names list because those are a riot
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 05:36:53 JST halcy:icosahedron: @codl on large maps in the late game, when you have most of the galaxy explored and get to 40k+ fleets
it‘s the one thing about stellaris that annoys me a bit
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 05:07:29 JST halcy:icosahedron: @codl are you in the Oops All Lag phase yet
Also is that vanilla or did you add a name mod
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Monday, 21-Jan-2019 02:41:44 JST halcy:icosahedron: @dotUser it KEEPS HAPPENING
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Sunday, 20-Jan-2019 11:56:17 JST halcy:icosahedron: Please allow for hiccups while everything recovers
-
halcy:icosahedron: (halcy@icosahedron.website)'s status on Sunday, 20-Jan-2019 11:51:15 JST halcy:icosahedron: Excuse the brief service interruption. Will do a proper postmortem when I habe time, short version is disk was full due to backup not working properly.