Opinion At the time of writing, it has been exactly 100 hours since Facebook, Instagram and WhatsApp crept back out of the void onto the internet. They’d been gone for six hours – or seven billion dollars, if you measure out your life by Zuck’s net worth, which we don’t recommend.
The time for hot takes has passed. We are now in the brief window for sober analysis before everyone forgets it ever happened. Next Thursday, by current standards.
There are three angles that matter. One is the pure technical side – how a giant corporation, built out of the most resilient networking technologies ever created, just vanished. One is what it teaches us about Facebook’s importance to our daily lives. The last is what it tells us about Facebook itself, where it goes next, and whether its strip-mining of societal values for profit will continue.
The technical side is simply put. Someone, most likely a hapless sysadmin, tried to check backbone availability, but a malformed command shut down access to the company’s DNS servers instead.
It did this by cancelling the routes into Facebook that the internet knew about. The safety checks to stop that happening were broken, there was no Plan B and, as a bonus, most if not all of Facebook’s internal tools, comms, virtual and physical access systems shut down too. Hilarity ensued.
Did it matter?
Yes and no. There are lots and lots of small businesses that rely on Facebook pages to talk to their customers. They had six hours of uncertainty.
WhatsApp has been widely adopted as a team channel by all sorts of people who had to wait to plot, plan or parcel out work.
Families dealing with sickness or crises remotely suffered.
You don’t turn a multibillion-user service off without pain, and most of that was felt by people with the least heard of voices. But think what would happen if Google, Microsoft or Amazon had a total service outage like that. Even the partial ones we’ve seen have created universal concern that was absent here.
Instead of concern, there was a sigh of relief all around, and a surge of schadenfreude among that hapless admin’s peers. That’s unusual and noteworthy.
Pity the lonely, unloved Facebook network admin
Most sysadmins and other ops types have huge sympathy when an internal fumble by one of the clan causes public conniptions. There but for the grace of God is the rule – and the #HugOps tag is deployed in fraternal and sororal support. This time, not so much. Normally, everyone in the business recognises that work is work, but the social network seems like social death for network admins.
This failure of respect amplifies the technical aspects of the outage. The raw components from which Facebook is built, the technologies and the people in the engine room, can do much better when allowed. You can build secure backdoors that bypass your main networks. You can build watchdogs that fall back to a known safe state when things stop happening.
You can’t foresee everything, but you can assume systemic failures whatever the cause. Yet a single point of failure took everything down. Efficiency had won over redundancy. The worst case that never happens, happened. Too much security is never enough – until it’s just too much.
And these are managerial decisions.
When we look at how Facebook’s business model is constructed, we see the same thinking. Concerns of malfeasance from the outside world must be locked out, no matter how badly the internal systems are behaving. Maximal efficiency, a single-minded approach to gathering data and selling it – allegedly without nuance or safeguard – is the one true way.
Whistleblowers are the enemy. The media is to be ignored, rebuffed or treated like idiots, and what the hell do regulators or governments know anyway? Best practice is what Facebook says it is, not what everyone else decides.
This is hubris, characteristic of a company that is too powerful to care. State telcos had it. IBM in the 1960s had it and Microsoft in the 1990s – neither are by any means cured. And Apple, Google, Amazon all have it too.
But where Facebook is uniquely vulnerable is that if it goes away, it doesn’t much matter, even in the medium term. It is a social network, and nothing but, and users can rebuild that social interaction on a different platform in weeks. Others stand ready to take the advertising and analytics spend.
Unlike Microsoft, Amazon and Google, unlike the telcos and IBM, Facebook provides no essential services to business or state. Quite the opposite – it threatens, and it has money, but it has no leverage. A loss of confidence at the top, or a sufficiently puissant legal or regulatory challenge, and it could vanish as quickly as its BGP routes, an irrelevant Yahoo! for the 2020s.
Facebook embodies the single point of failure. It has no friends it cannot buy, just enemies it cannot pay off. Six hours on a Facebook-free planet felt like six thousand hours too few, and now we know it. Does Facebook? ®