Lessons learned from making the front page of Hacker News and /r/technology
(AKA the legally required ‘my WordPress blog went down after making Hacker News’ post-mortem post)
A few days ago one of my posts made the front page of both Hacker News and /r/technology. It was a bit of a surprise to me. But it was an even bigger surprise to my $5 Linode VPS, which quickly collapsed under the strain.
Thankfully I was home and blogging when it went down, so I noticed right away and quickly tried to diagnose why on earth PHP was using so much CPU. But the idea that it was due to a traffic influx never even crossed my mind. So I spent the first 30 minutes poking around and troubleshooting.
Eventually though, I thought to myself what if it was due to traffic? So off I went to my Cloudflare dashboard. And sure enough, traffic!
But of course once again my silly self struck again and I didn’t think for a single moment that it was genuine, organic, human visits on my read-by-a-dozen-people blog. Nope, in my head this of course had to be a DDoS attack – it was the only logically answer! So I turned on Cloudflare’s “I’m Under Attack” mode and left it at that.
As I sat there though, I pondered what if I was perhaps Fireballed or something? (And this all had to be just pondered because at this point I had no analytics on my blog. It was good for the privacy of my twelve readers. But not so good when I was trying to work out where a load of traffic was coming from). So, I went to daringfireball.net. But nope, I hadn’t been linked to on there. Mmm.
I then rather pathetically typed my blog into Google News, seeing if that might shine any light. But nope.
Hacker News maybe? And… cook a cat! My latest post at the tippy top. Lots of actual humans on my blog! Panic mode engaged.
And also, how do I fix my downed blog?!
Well, it’s complicated by this blog being used to being powered by WordPress. Self-hosted WordPress blogs have a long and storied tradition of going down after making Hacker News – a tradition my blog shamefully continued. In a world of increasingly static sites which survive any amount traffic, the LAMP based WordPress blog – which goes down as easily as Twitter in 2008 – does look a tad dinosauric and inefficient in comparison.
Thankfully in the end though, fixing the problem (at least temporarily) was actually fairly simple thanks to the fluidity of the cloud. I just threw horsepower at the problem and simply resized my server (Linode did this in just over two minutes, which I found rather impressive).
Well, it’s now a week or so on. And now that the dust has settled, here are some lessons and curious things I discovered after making the front page of /r/technology and Hacker News.
How much traffic does /r/technology and Hacker News send?
This is a little tough to know exactly as during the first hour of being on HN my blog was mostly down. But by the looks of it – and rather surprisingly – HN sent many more visitors than /r/technology, a subreddit with over 11 million readers (though to be fair, my post made the top of HN and only sixth on reddit). Here was the number of unique visitors on day one:
- Hacker News: circa 38,000+
- /r/technology: 19,771
The majority of traffic is phone traffic
I’m sure this comes as no surprise to people who often look at the analytics of their website. But as someone who has only done so for the first time it certainly came as a surprise to me that 69% of the visitors I got were on their phone.
Whenever I get the yearly itch to redesign the look of this blog I do of course always ensure it looks okay on a phone, but it’s an afterthought for the most part. My priority instead is how it looks on a desktop. But apparently I’ve got this backwards. Nowadays the focus should evidently be very much be on the smart phone, as it’s how the majority of visitors will experience the site.
Is Cloudflare worth it?
As the owner of a rarely visited blog, having my site run through Cloudflare felt like storing a pencil sharpener in a shipping container. So it was fun to see it actually be called into action and have proper traffic to deliver.
Cloudflare did let me down in some ways. Its “Always Online” feature didn’t save the day. It’s supposed to show a cached copy of my blog if the server goes down. But it apparently relies on the Internet Archive’s slow-to-crawl Wayback Machine for this (no hate, Internet Archive! You’re an amazing free service and one of the best corners of the web and I love you). And as the blog post that made Hacker News was only a day or so old there wasn’t a cached copy to serve. Which was a shame.
But when it came to caching and serving files Cloudflare did very well, with a 95% hit rate. Of the 107 GB of bandwidth sent out, my origin server handled just 4 GB of it. CloudFlare did the rest. And I’m sure they delivered it all far faster to the non-European visitors than my London-based Linode would have.
And the thing is, I titled this section ‘is Cloudflare worth it?’ But like most individuals, I’m not on the paid plan – just the free tier. So yes, Cloudflare is very much worth it. Aside from a few lock-in concerns and its tendency to present too many CAPTCHAs to genuine visitors, in my mind it continues to be almost a requirement for any website to be proxied through Cloudflare. It’s a remarkable service and tool.
What sort of server do you need for your WordPress blog to survive a Hacker News onslaught?
Do you run a self-hosted Wordpress blog and want it to stay up and running if you make Hacker News and /r/technology? Well, it looks like a $5/mo VPS isn’t going to be enough.
My blog was hosted on a VPS with 1 vCPU and 1 GB RAM when it went down. When things went south I upgraded to a $40/mo one with 4 vCPU and 8 GB RAM, which proved to be overkill – though I did want to guarantee no more downtime.
So how much compute do you need? Well I spent a lot of my time anxiously monitoring htop throughout all this. And from the looks of it the minimum requirement if you want to even stand a chance of your WordPress blog surviving a Hacker News beating is 2 vCPU with 2GB of RAM.
And this is presuming you have a WordPress caching plugin installed and CloudFlare handling the vast majority of static files. It also presumes you’re using Linode, who have pretty high-end CPUs (AMD EPYC 7601’s in my servers case). If you’re using an alternative with a less beefy CPU - like Digital Ocean or Vultr - you might need more than 2 vCPU to be safe.
You should also harden your WordPress site
It’s important to prepare your site for a potential influx of visitors. I know, I know, no one reads your blog. No one reads mine either. But for one day they did. And it was rather embarrassing when it immediately melted.
So be over prepared. If you self-host WordPress, you need to take it a little seriously, and some steps are likely required:
- Install a caching plugin. There are many. I use WP Super Cache. You don’t stand a chance without one.
- Use Cloudflare. It’s a very useful tool. It caches static content and speeds up your blog, as well as protecting you from bots.
- If your blog does go down, you’ll want to know. UptimeRobot can check every five minutes and email you if there are any problems - and all for free.
- Go overkill when it comes to hosting. I just presumed that because I wasn’t using some over-sold shared hosting service that my blog would be able to handle a sudden influx of traffic. But I was wrong. If I was just linked to by a fairly popular blog I might have been okay. But Hacker News and reddit is a different kettle of fish, and your site will likely go down if you end up being featured there. So sadly you’re just going to have to stump up the cash and pay the roughly $20/mo for a really good server if you want to self-host WordPress and survive a Hacker News-sized influx. That’s just the WordPress penalty I’m afraid.
With all that being said… bye-bye WordPress
After all this bother I’ve actually decided to do the cliche thing and say goodbye to WordPress and instead go the simple, static route. This blog is now powered by Hugo and hosted on Amazon S3. And thank goodness I no longer have to worry about MySQL databases, spammy plugins or wp-login.php attacks from Ukrainian hackers.
Finally
Finishing up. Overall it was really rather fun making Hacker News and /r/technology. And despite there being over 1,200 comments submitted across the two sites (apparently people like talking about their hatred for ads almost as much as they like talking about their hatred for Netflix subscription price increases), basically zero were mean to me – which was a nice surprise.
Also, hello to the new people who now follow the blog! Glad to have you here. Expect two posts a year :/
And one final note for older readers who subscribed via RSS: the RSS feed URL is no longer https://imlefthanded.com/feed/. It is now https://imlefthanded.com/index.xml. Apologies for the annoyance, but it’s probably best to update the URL for this blog in your feed reader of choice. Or you can get new updates via Twitter if you prefer. Thanks!