Self Hosting #1 - My server died
2024-09-07
I am hosting a bunch of different self-hosted services that I more or less use. Nothing really critical, sometimes it's just for the fun of installing and trying a new tool. I also have a few services I use a lot, like FreshRSS or Shaarli. But self-hosting comes with its lot of hurdles.
I have been doing that since 2012. Back then I was even hosting my email server - a word of advice: don't do that. For that I had a small dedicated server with a French provider, for about €20/mo. In 12 years the price hasn't moved that much, but neither did the hardware.
In 2017, after 5 years, my first dedicated server died. And chaos ensued. My emails were out of service, and a bunch of other things I was hosting back then. I immediately contacted the support with high hopes, but they basically told me that the hard drive was probably fried and... that's all. They could commission me a new one. I accepted, slightly defeated but at least I had daily saves. Except when they decommissioned the fried one, they removed the FTP backup server associated with the plan. And again there was nothing they could do. So I had a not very recent backup locally. Not great. Obviously it happened in the summer, when I had much funnier plans than doing sysadmin grunt work.
So I made a couple of important decisions back then to avoid getting to that level of stress again.
- First, I need my emails to work at all time, it's still a critical way of communication, and I can't comfortably be out for a couple of weeks again. So I moved my emails to a paid service provider - Fastmail - and it's been one of the best decisions of my self-hosting journey. There is a lot to say, I may write a log about that subject in the future.
- Second, all these services installed manually on a single server were impossible to maintain, and even more to replicate on a new server. I dockerized everything and learned to use Docker in the process.
- Third, be careful about backups and do not rely too much on the hosting provider. Also, backup configuration files! I had all the data, but none of the configuration for nginx, databases, etc.
And that new server running Docker had been running for seven years with an uptime of four; not sure what happened in 2020. Then, I don't even remember why, I decided to run the update on that server, which probably had a bunch of vulnerability. Then I restarted. And it never got back up. The support was as useful as expected: "I can't remotely reach it, it has to be considered dead.". And that was it, after seven years of good services. RIP.
I had already decided that when this dedicated server would pass, I would stop paying the hosting provider and start to host it at home. First I moved to the US, and it doesn't make sense to keep a dedicated server in France. And I no longer have any service that requires a lot of traffic or a good availability. If my website or my RSS reader are down for a few hours during a power outage or something, it's not a big deal.
So I picked up a cheap mini PC ($170) on Amazon after reading endless threads on Reddit and folks, that's how my *real* self-hosting journey started.
I didn't expect that it would be so complicated to get everything running properly again because ISP in the US are so shitty, due (I think) to lack of competition. But that's a story for another time.
Note: this capsule is not actually running on my home server. Because it's part a separate project with pseudo-anonymity, it runs on cheap VPS.