RIP Twitter
Started: 2022-11-17 22:10:31
Submitted: 2022-11-18 00:31:43
Visibility: World-readable
When the birdsite implodes I will be at @jaeger@sfba.social
I signed up for Twitter on the 13th of February 2009, when Kiesa was pregnant with Calvin and had just spent two weeks in the hospital in Denver to ward off preterm labor. My first tweet was:
Brought Kiesa home from the hospital. Calvin's stable at 34 weeks GA. Looking for an amusing distraction for the evening.
(And now I'm wondering about the datestamp I see on the tweet (7:05 PM · Feb 13, 2009): Given I'm currently in Pacific time, am I seeing it converted into my current local time? Twitter probably could guess my local time zone but might not have bothered. The time as represented in the json in my data archive is "Sat Feb 14 03:05:00 +0000 2009", so it does look like it's converted in Pacific time, even though I originally tweeted it in Mountain time. This is why I go to a ridiculous amount of trouble to get correct timezone metadata on my photo archive. But I digress.)
I got my first smartphone a couple of days later, and quickly installed a Twitter client on my new phone. I was using Windows Phone at the time (it seemed like a good idea at the time), and I think I needed to use a third-party app. I announced Calvin's birth on Twitter, along with a very-fuzzy early smartphone photo hosted on twitpic.com, that's somehow still up, despite pervasive Internet bit-rot.
As of the last time I downloaded my Twitter archive (on the 27th of October this year, just as the new owner of the company was carrying a sink into Twitter headquarters on Market Street) I've posted 15,650 tweets. Most days Twitter is the first thing I see in the morning and the last thing I see at night. I get much of my news first from Twitter, though often it's of the form "what are the people I follow referencing". When local news events happen — the floods in Boulder in 2013, the North Bay fires in 2017, the CZU Lightning Complex fire in 2020) they tend to overtake my feed — not specifically because I only follow local people, but because I've cultivated enough local news-adjacent sources and the people I follow are talking about it.
In March 2020, as a certain global pandemic overtook our lives, I had to take a break from Twitter, because the doom-scrolling was too much for me. I also unfollowed a handful of accounts, because while I respect that they wanted to deal with their pandemic experience by tweeting about it, their doom-and-gloom was not good for me. Within a couple of weeks I returned to Twitter, carefully, once the worst of the doom had subsided and I had a better idea how to manage the rest.
Along the way I created a couple of side accounts. First was "Version 2.1", to present Julian's view of the world, which I started a year before he was born, just in case. At first I used the account to project a sarcastic view of the world; now many of the things are actual quotes from him. During the pandemic I created a new account for our cat Willow, because the Internet needed more cat photos to get it through 2020. Now that we have a new cat Rio I've started posting photos of her.
I downloaded an archive of my main account as ownership changed hands; but I forgot to download an archive of my side accounts and now I think it's too late.
My Twitter handle, @calvinsdad, is obviously a reference to my son Calvin; but I think only two people that I interacted with on Twitter actually knew me first as Calvin's dad in real life: his preschool teacher (who I think has fallen off Twitter; last I checked they no longer had a visible Twitter account), and his third-grade teacher (who was still on Twitter, last I checked).
And now Twitter has a new owner, and he seems to be doing the best he can to destroy the platform I've lived on for the better part of fourteen years.
I understand how we got here: the world's richest man made a stupid joke about a hostile for buying Twitter at $44 billion, and then followed through with his stupid joke to the point where the company board realized they were never going to get that much for the company any other way. The board interpreted their mandate as "maximize shareholder value", ignoring the impact to other stake-holders: the 7500 Twitter employees, and the millions of people who use Twitter every day: the long tail of people who have found a community on the platform unlike any community anywhere else. Not to mention the people in countries with governments hostile to opposition organizing using Twitter to communicate and organize, protected by Twitter's robust legal-affairs team. (I'd like to believe that, as the company was pursuing that guy through the Delaware Court of Chancery, they hoped that he would just give up and pay the $1 billion separation fee, rather than somehow come up with the money to buy the company.)
It's been a couple of tumultuous weeks since the world's richest man decided to squander his fortune on destroying a social media platform. He brought in group of goons from Tesla and asked everyone to print off all the code they'd written recently. He laid off half the company, in a manner that was probably illegal but the only remedy is paying enough severance so it probably comes out in the wash. (While watching the looming Twitter layoff I flashed back to my own experience years ago as Qualcomm was figuring out who to lay off. Twitter's process was a complete clusterfuck, but at least their employees didn't have to spend two months waiting to find out what was going to happen.)
The shoes continued to drop over the next two weeks. There were a bunch of new parody accounts that popped up, and even though "comedy [was] now legal" the accounts got banned because the new owner got his feelings hurt. The midterm election briefly overtook Twitter as the top news item, but even as the expected red wave turned into a purple rain Twitter had more surprises for us. There were a bunch of departures from the legal and compliance teams, with some consternation about what this meant for FTC consent decrees. Late last week was the disastrous roll-out and roll-back of the verified blue check-marks by paying $8, and the totally-predictable results as new "verified" accounts sprung up to say things like "insulin is now free" and (apparently) tank stock prices.
Over the weekend everyone was talking about "1000 poorly-batched RPCs". (I blocked the new owner (whose name I'd prefer not to mention out loud) on Twitter earlier this year as he was bloviating on about buying Twitter, so if anyone I follow directly retweets him I remain blissfully ignorant, but I still see his tweets as ghostly "This tweet is not available" messages in quote-tweets and screenshots from the people I follow.) Then on Monday this week the new owner decided to turn off a bunch of microservices just to see what would happen, because "micro" probably means "unimportant" or something (with a hilarious subtweet by Google's SRE account about microservices).
Today was the deadline for the remaining Twitter employees to click a form to sign up to build "Twitter 2.0" which would be more "hardcore" than the existing version of Twitter, whatever that meant. Most of the survivors of the last round of layoffs decided this was there best chance to get out while they still could; they declined to click the form, leaving the company to scramble to try to figure out how to convince people to stay. Whole teams quit together, a massive exodus of talent and institutional knowledge, who will probably go on to do great things at other companies, but Twitter as we know it is now dead.
On Twitter this afternoon, as I watched most of the remaining Twitter employees announce they were leaving, and everyone else realized this meant the site was probably going down forever, the site had the energy of a bittersweet graduation farewell, if school were also on fire and the new school principal had set the school on fire and were running around with a flamethrower begging us to stay.
With most of the company gone, Twitter now gets to test the apocalyptic scenario of what happens to infrastructure when the humans who tend to it are no longer around. Things will seem normal at first: jobs will auto-scale up to their programmed limits, and servers will restart and load-balancers will route around them. But things will start to go wrong: cron jobs will fail because of some weird bug and logs won't get rotated and disks will fill up and services will stop responding and pieces of the site will stop working.
Important warning signs will go overlooked because some senior SRE who thought something looked hinky in a monitoring graph won't be around to investigate why the load spiked in one cluster at 02:00 in the morning (still below capacity, so no alarms fired) to discover that there was a rare combination of eight different unlikely things happening at once; when left unidentified and unfixed, next time nine different unlikely things will happen at once and the cluster will crash and cascade out to the rest of the system, and Twitter will go down and will never come back up.
I don't know the details of Twitter's systems and infrastructure, but I spent four years as an SRE at Google and I have my share of war stories from the trenches: the time the customer routed all of their traffic to a single instance; the time a single bad system clock on a single machine in a downstream service that my service depended on caused an outage across a region; the time a previously-undiscovered bug in a storage layer happened to break and caused a global outage; the time a "safe" change caused a global outage. There were the availability caches (intended to make the system go faster) that turned out to be capacity caches (makes the system work at all). There were the times (more than once!) that we had to use systems designed to handle DoS attacks to selectively block traffic to a single service to allow the backend service to restart. Every one of these incidents were new and unique (which is why I remember them, years later), and they took teams of people with deep institutional knowledge to resolve.
Every time I took the pager for App Engine I wondered, in the back of my mind, what fresh new hell I was going to get paged for this week. Every time I knew I could handle whatever came my way, because I was rested and trained and ready, and if all hell broke loose I could commandeer the war room and get everyone I needed on the SRE team and the dev team and all of the SRE teams and dev teams that my service depended on to help. Above all I felt safe: no one was going to second-guess my decisions in the moment or fire me because I did the wrong thing.
None of these things hold true at Twitter any more, which is why I think the service is going to go down, hard, soon. (I honestly give it 50/50 that it'll last until Thanksgiving, next week.)
I will be on the Bird Site until it crashes; and when it finally cascades into failure I will be on Mastodon at @jaeger@sfba.social.
RIP Twitter. We'll miss you.
