Today marks the beginning of the end of me having an Ubuntu machine at home, and I have mixed feelings about that. By the weekend the last machine that I do have, my network file server and general dogsbody machine, will have been replaced and its replacement will not be running Ubuntu.
The primary purpose of the machine is to be a point of backup for my laptop and other devices, as well as a host for the large and valuable content collections such as my photos, music, purchased TV shows, movies, etc.
Since this collection is multiple terabytes in size there just isn’t a viable cloud storage solution. Firstly getting content to the cloud would be a long and difficult process, secondly the actual costs of that much storage are still reasonably prohibitive compared to home solutions and thirdly since a lot of this content is in the form of high quality media, while my home Internet connection can stream it, the download bandwidth costs of cloud storage providers are equally cost prohibitive.
So I still need some form of fast and reliable file storage at home, at least for the foreseeable future. And this is where Ubuntu comes up short.
For the last few years I’ve done what anyone would have done, I purchased a small form-factor machine, loaded it with SATA drives and installed Ubuntu 10.04 LTS using Software RAID to deal with the reliability factor.
This has all worked fine, the box even survived a transatlantic voyage; what it hasn’t survived is the upgrade to Ubuntu 12.04 LTS. At some point after the upgrade the box did not come back up after a reboot; after searching for a monitor to plug into it to find out what was going on, I was dismayed to see a message about the RAID being in degraded mode and the boot not continuing.
My first reaction, naturally, was that one of the disks had finally given out; so, knowing that the Ubuntu initramfs is too limited to debug, I booted a USB image and grabbed the various SMART utilities to figure out which disk had been thrown and needed replacing.
Mysteriously they all checked out. I rebooted back into Ubuntu, and it came up just fine. Weird. And a subsequent reboot works fine too.
At this point my disk utilization is well over 90% and I’m already starting to consider my options for expanding it, I’m still thinking dodgy disk and so begin accelerating that process. The most obvious option is just to buy larger disks; the next option would be to buy more smaller disks, but this would require additional SATA capacity in the machine; the final option would be to buy a proper RAID array or even a NAS of some description.
I’m wary of NAS, the last one I bought, while admittedly a relatively budget option, ie. under $1,000, just didn’t perform. It didn’t have the power to actually get data from its disks and out of the network port in anything like a timely manner, certainly not enough for 1080p 7.1 streaming, for example.
And then the server throws a disk again, but at least this time a monitor is plugged into it so I can see the messages I missed last time. And this time I stay in the initramfs and do a little bit of poking around.
I realize there’s nothing wrong with the disks at all.
The problem is Ubuntu 12.04 LTS.
I do what anyone else would do with a problem, and hit Google, Stack Overflow and Launchpad to find a workaround. And what I find saddens me; huge numbers of people reporting that their RAIDs frequently boot in degraded mode. Bugs are marked “Invalid”, “Won’t Fix” or “Unassigned”.
Now I know this used to work, because I wrote unreasonable amounts of the code that did it. So I quickly dived in to see if there was an obvious bug fix to find that all the code I’d written had been ripped out, not replaced with anything better, just gone. All that remained was the “upstream” code that had existed before I started, or at least an updated version of it.
I dug through the history to figure out if I was missing something, expecting that things were no longer required and that new ways of doing things had been put in place, but that wasn’t the case either. The history clearly showed a different story: faced with the pressure of updating to a newer upstream release of various utilities, for no reason other than to keep roughly in step with Debian, all of the bug fixes, patches and changes to make things work had been dropped because they were “hard to merge”.
Now I don’t want to come across as bitter at this point that my work had been dropped, because that’s not my feeling at all. I entirely understand and appreciate the decision that must have happened here.
Canonical has limited resources of its own, and a small hobbyist developer community around it. Those resources have to be spent wisely and not squandered. The Ubuntu focus right now is on the desktop, and on Unity; the Server focus is a lesser one, and entirely aimed at cloud hosting and guests — though given that the Canonical VP of Cloud couldn’t even be bothered to turn up for his scheduled panel at the most recent CloudOpen conference, it’s hard to fathom how much of a focus even that is for them anymore.
So if they have a low server focus, and what they do have is for cloud, then it’s no surprising that support for things like Software RAID aren’t a priority worth spending resource on. Cloud guests and hosts access storage over a network using protocols like NFS or (ugh) iSCSI.
Simply put, the home server is a uninteresting and dying product, and I’m a weird outlier for still having one at all.
This wasn’t quite the end though, I still had disks to replace and storage to sort out. If Ubuntu couldn’t do Software RAID reliably anymore, it could still at least do Hardware RAID. I looked around for Hardware RAID boxes, especially single enclosure ones that could just plug into the box and go.
This seemed like a good plan except that high performance Hardware RAID devices come in two fundamental flavors: Thunderbolt, which Ubuntu does not support; and Ethernet, which means the Ubuntu machine is superfluous to requirements.