Tech:Incidents/2014-06-12

A missed payment resulted in Prod4 being temporarily suspended.

Timeline

 * 05:26 BST - Pingdom reports orain down
 * 06:06 BST - RamNode has suspended prod4 again reports JL via email
 * 07:46 BST - Addshore facepalms via email
 * 20:15 BST - Paid and all back up.

In Hindsight

 * "As Labster's pointed out several times, we need redundancy not Single Point of Failures though I'll leave that for you to deal with :)"

Action taken

 * John has been granted access to the main RamNode account and methods to gain access to the PayPal account when necessary.