Reply to post: A very big bank

Today's budget for application improvements is brought to you by the letters "Y", "K" and the number "2"

Simon Millard

A very big bank

Whilst working for a very big bank in 1999, Y2K testing was done in a Ring Fenced Environment. The RFE wasn't ring fenced enough. The testers forgot to ring fence UDP connections and, unfortunately, when testing a shutdown process, the system sends a call to the servers in the cluster via a UDP port - can you guess what happened?

The live system shutdown with a WTF happened moment. Later we had a grovelling apology once the service was restored and the RFE had it's barn door closed, now that the horse bolted.

Additionally at 11pm, we suddenly lost one of our French servers. Some bright spark decided not to risk have the system running at midnight (GMT-1) and had forgotten to tell the support team who were doing 24/7 monitoring.

Hey ho. Fun times.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER

Biting the hand that feeds IT © 1998–2020