Tech talk:Incidents

I don't think I'm supposed to make incidents so instead of making Incidents/2015-03-21, I'll just comment here. The site decided to go down tonight, been this way for hours. Takes an hour of fighting with it to get anything other than: 504 Gateway Time-out The server didn't respond in time. Chemtrail Morgellons (talk) 03:27, 22 March 2015 (GMT)

Downtime this week (May, week 22)
There has been a lot of downtime this week (504 Gateway Time-out - "The server didn't respond in time"). Today from 10:00 - 12:00. Yesterday in the afternoon. Often.

We are using Orain for the Permanent Future Lab Wiki these days. Of course it's important that the platform is online.


 * 1) Does someone know what is the cause for the downtime?
 * 2) And: Is there anything I can do to help? (Programmer. Located: Rotterdam / The Hague, The Netherlands)

BartWR (talk) 11:39, 28 May 2015 (BST)
 * Hi! Basically Orain is undergoing a distributed UDP DDOS. The attack itself isn't taking down anything, but digital oceans default response when seeing a large network spike such as this is to turn off the networking for the machine for a few hours (hence the downtime). We are currently working on a fix, which might involve moving hosts or other big changes..  ·addshore·  talk to me! 11:52, 28 May 2015 (BST)
 * Ok :) Thank you for your response. I hope the downtime will absent from now. Good luck with the server migration - if there'll be any. BartWR (talk) 17:24, 28 May 2015 (BST)
 * Right now we have just migrated our DNS servers to cloudflare. The next step will be rotating all of our server IPs which will result in some more downtime, although this should be less than an hour for the changes to ALL servers. After this the DDoS should no longer be affecting us.  ·addshore·  talk to me! 17:25, 28 May 2015 (BST)
 * Great! Thanks :]
 * All migrated, all behind cloudflare, there should be no more downtime due to this DDoS  ·addshore·  talk to me! 12:52, 29 May 2015 (BST)
 * Take a look at Tech:Incidents/2015-05-ddos or my post here for details on what happened.  ·addshore·  talk to me! 13:49, 30 May 2015 (BST)