Yesterday the internets nearly imploded when Facebook went down from about 8:30 A.M. EST until around noon EST. Here’s the technical explanation for the downtime:
Today we made a change to the persistent copy of a configuration value that was interpreted as invalid. This meant that every single client saw the invalid value and attempted to fix it. Because the fix involves making a query to a cluster of databases, that cluster was quickly overwhelmed by hundreds of thousands of queries a second. To make matters worse, every time a client got an error attempting to query one of the databases it interpreted it as an invalid value, and deleted the corresponding cache key. This meant that even after the original problem had been fixed, the stream of queries continued.
Lol wut? Instead of doing an in-depth analysis of Facebook’s technical difficulties (because, frankly, I don’t understand that or care as long as the porn still gets to my laptop somehow), I am instead going to do a break down on some of the funnier tweets made about The Great Three Or Four Hours Or So We Couldn’t Get On Facebook…gate of 2010.
First, some people noticed that Farmville was unplayable:
(click pictures to enlarge)
Sidenote: if it actually bothered you that your imaginary crops might wither, please go to Kiva and look at all the people with real problems.
Later, several of the Twitter top trending topics were predictably Facebook related:
And some people offered theories on the downtime:
And some rushed to take credit:
Luckily, some were able to keep calm and put this whole thing into perspective: