Dec 13, 2007

Obsessive Compulsive or How to Roll Your Own NAS

So here's a story of why I usually tell people to buy a Dell instead of building your own computer. If you're only interested in setting up a NAS, just skip to the last section.

The History
Up till last year I had archived almost all my data on optical media, first burned on CD-ROMs, then on DVD-ROMs as they became more affordable. However, as the stacks of CDs grew, I found myself getting impatient swapping through CDs or DVDs to look for some old file. Especially when it came to the photo library, this started becoming rather time consuming. There's also the risk that the dyes on the discs could break down over time and I'd risk losing the data.

With the rapidly falling price of hard disks, I figured I could use a RAID array of some sort to have all the data available "online", and maintain some level of safety for the stuff I'm emotionally attached to.

At the time, some consumer level NAS products were starting to become available. NAS stands for Network Attached Storage, which is basically a fancy term for a computer that sits around purely for the purpose of making files available on the network. The features I wanted were:

1. Some level of security against losing files (RAID 1 or RAID 5)
2. Fast file copies (Gigabit ethernet)
3. Convenience (constantly on -> low power)
4. Quiet (since it sits in my room)
5. Future proof (flexibility to upgrade or add disks)
6. Reasonably affordable

At the time, none of the products on the market really met all these needs; items notably 2,5 and 6 were usually missing. I should note that the current generation of products out there feature most of these, except maybe 5 still might be hard to find.

In any case, I figured I might be able to build my own for less than the retail models (around $800 for a 600GB Buffalo Terastation at the time). I ended up getting a $300 fanless VIA EPIA EN12000, which consists of a fanless CPU on a motherboard that had basic onboard graphics, video and audio (which I don't need), gigabit ethernet, 2 ATA controllers, and 2 SATA ports, allowing me to connect up to 6 hard disks. On top of this I needed to get a stick of RAM, a power supply, and some hard disks (I got 2 250GB SATA drives). I always wanted to find a small case that I could stick it all in, but I never actually found the perfect case.

The key to getting this to work was FreeNAS, a small FreeBSD package that fit in under 32MB (at the time). You install it on a USB stick, and boot off of it, and configure the settings via a web browser. I was initially impressed by how easy it was to install. Download the image, burn onto CD, boot from CD, and it works. From there you can install it onto a USB stick in a minute, and reboot and run from USB instead of CD. This left all the hard drive connections available for storage, instead of requiring one for the operating system. It also gave me a sense of security that if anything ever happened (like I lost the USB stick), I could replace it very quickly.

Anyways, this actually cost about the same as the 600GB TeraStation, except that I had 100GB less, and no case. And then it occurred to me that the hard drives were actually pretty hot, and noisy, and I wondered if it defeated the purpose of getting the relatively expensive fanless CPU and power supply.

The "Upgrade"
Well roll ahead a year and I've pretty much filled up the 250GB (they two drives are in RAID1, so I have 2 copies of everything). Now I can get 2 650GB ATA drives for $250. I imagined a simple upgrade. FreeNAS development has progressed, and I figured I could try a new version of that as well.

Downloading the latest build of FreeNAS, burning it to CD, and booting worked fine and only took a few minutes. I could see the updated web interface allowed for more features, and my data was still accessible. However, installing the OS to USB just wouldn't work. I did a quick search on the FreeNAS forums, and didn't find a solution. If I had spent more time searching, I could have found a solution and saved myself an entire day of trying to figure out what was going on.

I'd say FreeNAS has pretty good functionality, but it's not a commercial product, and it has a number of rough edges, and documentation is one of them. If you're technically adept and familiar with UNIX, it's surprisingly easy to use. If not, there's probably a number of places where there's a potential to get stuck. There is a bit of actual "documentation", but most of the knowledge is actually in forums where people discuss problems. It takes a bit more effort to delve in there and find the solution to your problem (although in retrospect, it's likely that someone has seen your problem before). In the end I figured out that I the installation was failing because the USB drive was plugged in when the computer was starting; plugging it in after FreeNAS was finished solved that issue. Oh well, at least I familiarized myself with FreeBSD in the process.

Since I didn't take the "upgrade" path (so as to not ruin my original settings, in case I messed them up), I had done a fresh install to a new USB disk, and it took me a little while to look through all the configuration settings and set them all to my liking.

After everything was working, I decided to add in the new drives, and... nothing, my computer wouldn't even POST with the new drives plugged in. After an evening of trying to figure out what was going on, I figured it was either a BIOS problem, or something wrong with the ATA controller on the EPIA motherboard, since it was fine as long as the disks weren't plugged in. I tried flashing the BIOS to a newer version, which seemed to hang for about a half hour, upon which I got impatient, reboot the machine, and found that it no longer POSTed at all.

So this leads to the first trip to Fry's, for a new $39 motherboard (let's call it #2) and CPU and finally, a case. Come home and set it up, and it doesn't turn on, at all. The motherboard has been returned before, so that's probably the problem. Go back to Fry's for motherboard #3 at $50, and manage to get that booting. I can boot into FreeNAS, but it hits a similar problem once I plug in my new disks. I finally search through the forums and realize I need to tweak the BIOS to disable the hard disks completely. If I had figured it out earlier, I probably wouldn't have bricked my EPIA. So fine, I get the machine set up almost completely, but two things bug the hell out of me. First, the fan on this CPU is blazingly loud. Secondly, this motherboard will not boot with a keyboard plugged in.

The CPU is a $55 Celeron 430, which is a fairly new design, and relatively power efficient. I measured the whole board to run at around 45W, which is much worse than the EPIA, which ran between 13W-17W. I figure if I lower the clocks, I should be able to get the CPU to a state where it can run without the fan. What I need is a better motherboard that allows some overclocking settings, so I get an ABIT branded motherboard #4 for $80. It's a slight improvement, since it allows booting without the keyboard, and I can overclock, but I can't lower the clocks any further. So it's back to Fry's for a Gigabyte motherboard #5 at $85, which has a pretty new G31 chipset. This one's got all the nice BIOS settings, and I can lower the clocks till the motherboard runs fine without a fan, and it's "only" using about 38W. Plus the motherboard has a nice smart fan feature that will actually turn on the fan when the CPU heats up (unlike the cheap #3 that had the fans on full power constantly). Great, except that FreeNAS crashes once it starts trying to boot from USB. It actually boots fine from CD. I try to get this working for a day, but I suspect it has something to do with FreeBSD 6.2 not supporting the G31 chipset.

So it's back to Fry's again for another Gigabyte motherboard #6, which is based on the older 945G chipset, has the same clocking features as the G31 version, and is only $55.

Conclusion: Roll Your Own NAS, the quick way
The current market for home NAS hardware is much broader than what was available a year and a half ago. There's a number of brands that will sell you a small box that can fit 2 disks for about $150 (without the disks). Higher end products like the most recent Terastation gives you about 1TB of storage for about $650. They're definitely much more convenient than my setup, they're more power efficient and they're much smaller too. I never ended up finding a nice small case that fit perfectly, so I ended up with a standard PC case.

The power consumption problem is not too bad. My system draws between 38W-70W. The Buffalo Terastation runs at about 60W. The Infrant ReadyNAS NV+ draws around 26-50W.

On the plus side, I have 4 disks, but still have space for 2 more disks, plus I have PCI and PCIe slots to add SATA controllers, so I could potentially have a lot more storage. The main benefit though, is that I know even if my CPU or motherboard fails in any way, I can replace it with pretty much off the shelf parts to get my data back (I may have some trouble in terms of getting the perfect quiet system though). My coworker got a $150 Linksys NAS200 that just died on him after a couple of months. I'm hoping he's lucky and that the NAS200 runs Linux, he might be able to set up a Linux machine and get the data back off of his drives.

Here's my build your own NAS cookboook:
1) Get a Celeron 430 processor, around $50
2) Get a Gigbyte GA-945GCMX-S2 motherboard, around $55
3) Get 512MB of DDR2 memory, about $15
4) Get a 64MB or more USB memory stick. You might have these lying around, you can probably get one for a few bucks or free somewhere.
5) Get a case and power supply. I got a CoolerMaster Elite 330 for $40 and it has a lot of drive bays.
6) Get as many hard disks as you want.
7) Put the system together, hook up a CD-ROM drive (just borrow one).
8) Download and burn FreeNAS onto a CD
9) Boot the FreeNAS CD
10) Insert USB stick
11) Follow the FreeNAS menu to install the OS on the USB stick
12) Turn off the machine and remove the CD-ROM, put in
13) Plug in the network, turn on the machine, and use the menu to set the IP address
14) From another computer on the network, you can access the FreeNAS machine with your web browser, and you're on your way.

One other interesting product that came out fairly recently is the Intel D201GLY2 motherboard. It's got a low end Celeron 220 soldered on, runs at about 19W and sells for $70. It's got 1 ATA and 2 SATA ports, 10/100 ethernet (no gigabit), and 1 PCI slot.

At this point, I might try to recover the BIOS on the EPIA board. If I can manage to do that, I might set up a second system to leave at my parent's house. I'm theorizing I can get my box to sync with that one, so that I'd have an extra level of security, in case something physically happens to my box here.