Dec 13, 2007

Obsessive Compulsive or How to Roll Your Own NAS

So here's a story of why I usually tell people to buy a Dell instead of building your own computer. If you're only interested in setting up a NAS, just skip to the last section.

The History
Up till last year I had archived almost all my data on optical media, first burned on CD-ROMs, then on DVD-ROMs as they became more affordable. However, as the stacks of CDs grew, I found myself getting impatient swapping through CDs or DVDs to look for some old file. Especially when it came to the photo library, this started becoming rather time consuming. There's also the risk that the dyes on the discs could break down over time and I'd risk losing the data.

With the rapidly falling price of hard disks, I figured I could use a RAID array of some sort to have all the data available "online", and maintain some level of safety for the stuff I'm emotionally attached to.

At the time, some consumer level NAS products were starting to become available. NAS stands for Network Attached Storage, which is basically a fancy term for a computer that sits around purely for the purpose of making files available on the network. The features I wanted were:

1. Some level of security against losing files (RAID 1 or RAID 5)
2. Fast file copies (Gigabit ethernet)
3. Convenience (constantly on -> low power)
4. Quiet (since it sits in my room)
5. Future proof (flexibility to upgrade or add disks)
6. Reasonably affordable

At the time, none of the products on the market really met all these needs; items notably 2,5 and 6 were usually missing. I should note that the current generation of products out there feature most of these, except maybe 5 still might be hard to find.

In any case, I figured I might be able to build my own for less than the retail models (around $800 for a 600GB Buffalo Terastation at the time). I ended up getting a $300 fanless VIA EPIA EN12000, which consists of a fanless CPU on a motherboard that had basic onboard graphics, video and audio (which I don't need), gigabit ethernet, 2 ATA controllers, and 2 SATA ports, allowing me to connect up to 6 hard disks. On top of this I needed to get a stick of RAM, a power supply, and some hard disks (I got 2 250GB SATA drives). I always wanted to find a small case that I could stick it all in, but I never actually found the perfect case.

The key to getting this to work was FreeNAS, a small FreeBSD package that fit in under 32MB (at the time). You install it on a USB stick, and boot off of it, and configure the settings via a web browser. I was initially impressed by how easy it was to install. Download the image, burn onto CD, boot from CD, and it works. From there you can install it onto a USB stick in a minute, and reboot and run from USB instead of CD. This left all the hard drive connections available for storage, instead of requiring one for the operating system. It also gave me a sense of security that if anything ever happened (like I lost the USB stick), I could replace it very quickly.

Anyways, this actually cost about the same as the 600GB TeraStation, except that I had 100GB less, and no case. And then it occurred to me that the hard drives were actually pretty hot, and noisy, and I wondered if it defeated the purpose of getting the relatively expensive fanless CPU and power supply.

The "Upgrade"
Well roll ahead a year and I've pretty much filled up the 250GB (they two drives are in RAID1, so I have 2 copies of everything). Now I can get 2 650GB ATA drives for $250. I imagined a simple upgrade. FreeNAS development has progressed, and I figured I could try a new version of that as well.

Downloading the latest build of FreeNAS, burning it to CD, and booting worked fine and only took a few minutes. I could see the updated web interface allowed for more features, and my data was still accessible. However, installing the OS to USB just wouldn't work. I did a quick search on the FreeNAS forums, and didn't find a solution. If I had spent more time searching, I could have found a solution and saved myself an entire day of trying to figure out what was going on.

I'd say FreeNAS has pretty good functionality, but it's not a commercial product, and it has a number of rough edges, and documentation is one of them. If you're technically adept and familiar with UNIX, it's surprisingly easy to use. If not, there's probably a number of places where there's a potential to get stuck. There is a bit of actual "documentation", but most of the knowledge is actually in forums where people discuss problems. It takes a bit more effort to delve in there and find the solution to your problem (although in retrospect, it's likely that someone has seen your problem before). In the end I figured out that I the installation was failing because the USB drive was plugged in when the computer was starting; plugging it in after FreeNAS was finished solved that issue. Oh well, at least I familiarized myself with FreeBSD in the process.

Since I didn't take the "upgrade" path (so as to not ruin my original settings, in case I messed them up), I had done a fresh install to a new USB disk, and it took me a little while to look through all the configuration settings and set them all to my liking.

After everything was working, I decided to add in the new drives, and... nothing, my computer wouldn't even POST with the new drives plugged in. After an evening of trying to figure out what was going on, I figured it was either a BIOS problem, or something wrong with the ATA controller on the EPIA motherboard, since it was fine as long as the disks weren't plugged in. I tried flashing the BIOS to a newer version, which seemed to hang for about a half hour, upon which I got impatient, reboot the machine, and found that it no longer POSTed at all.

So this leads to the first trip to Fry's, for a new $39 motherboard (let's call it #2) and CPU and finally, a case. Come home and set it up, and it doesn't turn on, at all. The motherboard has been returned before, so that's probably the problem. Go back to Fry's for motherboard #3 at $50, and manage to get that booting. I can boot into FreeNAS, but it hits a similar problem once I plug in my new disks. I finally search through the forums and realize I need to tweak the BIOS to disable the hard disks completely. If I had figured it out earlier, I probably wouldn't have bricked my EPIA. So fine, I get the machine set up almost completely, but two things bug the hell out of me. First, the fan on this CPU is blazingly loud. Secondly, this motherboard will not boot with a keyboard plugged in.

The CPU is a $55 Celeron 430, which is a fairly new design, and relatively power efficient. I measured the whole board to run at around 45W, which is much worse than the EPIA, which ran between 13W-17W. I figure if I lower the clocks, I should be able to get the CPU to a state where it can run without the fan. What I need is a better motherboard that allows some overclocking settings, so I get an ABIT branded motherboard #4 for $80. It's a slight improvement, since it allows booting without the keyboard, and I can overclock, but I can't lower the clocks any further. So it's back to Fry's for a Gigabyte motherboard #5 at $85, which has a pretty new G31 chipset. This one's got all the nice BIOS settings, and I can lower the clocks till the motherboard runs fine without a fan, and it's "only" using about 38W. Plus the motherboard has a nice smart fan feature that will actually turn on the fan when the CPU heats up (unlike the cheap #3 that had the fans on full power constantly). Great, except that FreeNAS crashes once it starts trying to boot from USB. It actually boots fine from CD. I try to get this working for a day, but I suspect it has something to do with FreeBSD 6.2 not supporting the G31 chipset.

So it's back to Fry's again for another Gigabyte motherboard #6, which is based on the older 945G chipset, has the same clocking features as the G31 version, and is only $55.

Conclusion: Roll Your Own NAS, the quick way
The current market for home NAS hardware is much broader than what was available a year and a half ago. There's a number of brands that will sell you a small box that can fit 2 disks for about $150 (without the disks). Higher end products like the most recent Terastation gives you about 1TB of storage for about $650. They're definitely much more convenient than my setup, they're more power efficient and they're much smaller too. I never ended up finding a nice small case that fit perfectly, so I ended up with a standard PC case.

The power consumption problem is not too bad. My system draws between 38W-70W. The Buffalo Terastation runs at about 60W. The Infrant ReadyNAS NV+ draws around 26-50W.

On the plus side, I have 4 disks, but still have space for 2 more disks, plus I have PCI and PCIe slots to add SATA controllers, so I could potentially have a lot more storage. The main benefit though, is that I know even if my CPU or motherboard fails in any way, I can replace it with pretty much off the shelf parts to get my data back (I may have some trouble in terms of getting the perfect quiet system though). My coworker got a $150 Linksys NAS200 that just died on him after a couple of months. I'm hoping he's lucky and that the NAS200 runs Linux, he might be able to set up a Linux machine and get the data back off of his drives.

Here's my build your own NAS cookboook:
1) Get a Celeron 430 processor, around $50
2) Get a Gigbyte GA-945GCMX-S2 motherboard, around $55
3) Get 512MB of DDR2 memory, about $15
4) Get a 64MB or more USB memory stick. You might have these lying around, you can probably get one for a few bucks or free somewhere.
5) Get a case and power supply. I got a CoolerMaster Elite 330 for $40 and it has a lot of drive bays.
6) Get as many hard disks as you want.
7) Put the system together, hook up a CD-ROM drive (just borrow one).
8) Download and burn FreeNAS onto a CD
9) Boot the FreeNAS CD
10) Insert USB stick
11) Follow the FreeNAS menu to install the OS on the USB stick
12) Turn off the machine and remove the CD-ROM, put in
13) Plug in the network, turn on the machine, and use the menu to set the IP address
14) From another computer on the network, you can access the FreeNAS machine with your web browser, and you're on your way.

One other interesting product that came out fairly recently is the Intel D201GLY2 motherboard. It's got a low end Celeron 220 soldered on, runs at about 19W and sells for $70. It's got 1 ATA and 2 SATA ports, 10/100 ethernet (no gigabit), and 1 PCI slot.

At this point, I might try to recover the BIOS on the EPIA board. If I can manage to do that, I might set up a second system to leave at my parent's house. I'm theorizing I can get my box to sync with that one, so that I'd have an extra level of security, in case something physically happens to my box here.

Nov 24, 2007

Inspiration

Tonight I walked outside and looked up. High altitude clouds, like waves in the sand, stretched in all directions, veiling the brilliant full moon and dispersing its halos across the sky.

After a day of big, thick, low hanging clouds, the sky felt so open and so far away.

I was out in the suburbs of Cupertino, but for a moment I recalled the open skies of Kashmir. There the mountains were a little taller, the sky a little darker, and the people a little more hospitable.

In August I piggybacked a trip to our company's Pune office with a photography workshop by VII photographer Gary Knight.

I really had very little idea of what the workshop was like. Other than that it was in Kashmir, and had something to do with photojournalism and storytelling, and it was not for beginners. When I saw the website in May, I just thought "Damn, this is f!@#ing cool", and filled in the registration without too much thought.

A few days later, I got a reply regarding the registration, was out $1000 in deposit, and I emailed the coordinator (whom I later discovered was Gary's wife) with a few more questions. I got this as a reply, and although initially I felt like this description still left a lot of holes about workshop, in retrospect I'd say it's perfectly accurate:

"With regard to an average day I'm not sure there's any such thing with Gary's workshops. They're not structured in a classroom style and a lot will depend on who the participants are and where they are with their photography.
The way it normally works is that everyone has a meeting with him when they arrive (it's to your advantage to arrive the evening of the 22nd as you could then see him either pm on the 22nd or am on the 23rd and be out shooting the same day. Before you arrive you should research some story ideas. Often what you end up photographing is not what you set out to do but it's a really useful exercise in getting you focused and off the starting block.
Gary talks with everyone individually and gets an idea of their photography and what they hope to achieve in the week. You'll then work with a fixer on your story (if necessary) and go out and start work. In the afternoon/early evening there's an editing session followed by dinner with the group. It's all fairly convivial and open to change."

Unfortunately, I slacked off with the preparation for most of June and July, and by August I found myself scrambling preparting for the work part of the trip, that I really didn't have much preparation at all for the photo part. I did use the trip as an excuse to get some new gear (Canon Rebel XTi, and a the 10-22 EF-S lens). I had thought, maybe when I'm in India for work, I can catch up on my research on Kashmir, but really, that didn't happen.

The one interesting tidbit I picked up was an Indian Express (delivered daily to my hotel) about independence in Kashmir. Another editorial expressed desires of reunification between India and Pakistan (it was approaching the 60th anniversary of India's independence from the British Empire.

So as the plane headed through the mountains towards Srinagar, the summer capital of Jammu & Kashmir, I had a great sense of trepidation. The greenery below and the blue skies above were a far cry from the vast stretches of slums in Mumbai and the brown-grey haze over Delhi. The Kingfisher airline flight landed in the small airport, alongside military jets and helicopters. It's one of those airports where attendents roll a staircase on wheels up to the door of your plane, where you disembark and entire an empty hall, guarded by men in army fatigues and submachine guns. While you wait for someone to cart in your luggage, you fill in registration forms for foreign nationals, as locals pester you with offers of beautiful houseboats that are cheaper than the one you already have booked. At this point I was still a little hesitant to mention why I was there. I didn't want to be accidentally mistaken as a journalist, who tend to be less welcome than tourists.

I had email confirmation that Gary had arrived. I had the address of the houseboats I was supposed to head to. I had a fair bit of Indian currency. I had the expectation that there would be a driver to pick me up. But as I walked out of the single building arrivals terminal, along a long driveway to a gate thronged with rickshaw drivers, I really felt like I had absolutely no clue what would happen next.

Aug 27, 2007

Let me take you there


IMG_1743
Originally uploaded by dragonx

I don't know if one flies to the other side of the planet to find oneself. It was never really my goal.

What I did find though, was the end of routine.

Hopefully I'll remember enough to share it later.

Aug 18, 2007

Thoughts on flying

I think it's the relative efficiency of travel in North America - great highways and extensive air coverage, that you would never think of travelling for more than 24 hours at a time. Even flying from Toronto to Hong Kong, practically the other side of the planet, only takes around 18 hours.

So I didn't really expect that it'd take more than 36 hours to get to our Indian office. I think it didn't seem quite as long last time, since I stayed in a hotel overnight in between flights in Delhi. This time it was an 8 hour layover in Hong Kong, followed by a 3 hour drive from Mumbai to the office.

The highlight really was Hong Kong. I landed, got through customs quickly (it was 5am), got some HK dollars at an ATM in the airport and got on the Airport Express train to Central. The ticket is about US$20 for a single day round trip ticket (unfortunately, if you're not coming back on the same day, then it's the same price for a one way ticket). I was intending to go grab some wonton noodles, but ended up cabbing over to my uncles' place for a quick shower, hung out with my cousins for a while, and had a Shanghainese "brunch" with my grandmother before getting back on the Airport Express and getting back to my gate with plenty of time to spare.

Oh, and there's free wifi in the departures area in the Hong Kong airport.

On the topic of free wifi at airports, it seems like Northwest lounges don't secure their wifi, so hanging out around their lounge will get you free internet access. You can also find free wifi by the Fairmont Hotel located inside the Vancouver airport. I haven't found anything in the Toronto airport yet though.

Aug 11, 2007

A Good Day

Fly into YYZ, get picked up by my sister, have Shanghainese food for lunch, and a 3 hour pho dinner. It was a good day.

Wake up and chat with a reporter for 2 hours about journalism and photography, get some work done, pick up my shoes, dumplings, opera in the park, and dessert at Blowfish. It was a good day.

Climbing in the morning (although, I feel like I should have done better), where they played the entire Shins album, smoked salmon sandwiches for lunch, watching M&D pledge their undying love for each other, Congee Wong, wedding banquet, watching Garden State before going to the airport and (hopefully) getting my visa approved (again). It was a good day.

Thank God for good days.

May 24, 2007

He says this is us

I want to run
I want to hide
I want to tear down the walls
That hold me inside
I want to reach out
And touch the flame
Where the streets have no name

I want to feel sunlight on my face
I see the dust cloud disappear
Without a trace
I want to take shelter from the poison rain
Where the streets have no name

Where the streets have no name
Where the streets have no name
We're still building
Then burning down love
Burning down love
And when I go there
I go there with you
It's all I can do

The city's aflood
And our love turns to rust
We're beaten and blown by the wind
Trampled in dust
I'll show you a place
High on a desert plain
Where the streets have no name

Where the streets have no name
Where the streets have no name
We're still building
Then burning down love
Burning down love
And when I go there
I go there with you
It's all I can do
Our love turns to rust
We're beaten and blown by the wind
Blown by the wind
Oh, and I see love
See our love turn to rust
We're beaten and blown by the wind
Blown by the wind
Oh, when I go there
I go there with you
It's all I can do

-U2

May 9, 2007

Self-discipline

This M&M addiction is getting out of hand.
It must stop.
After this pack.

Mar 20, 2007

I Bet Your Baby Drives Better Than You Do

Dear Lady,

Just because you had one accident, that "Baby On Board" sign won't save you from another. Let's think for a second what that sign really means. It suggests that you're carrying something valuable in that death-mobile of yours that careens wildly down the expressway. Obviously that nine months of your life (give or take a few is worth) more than the decades invested into that scientist, engineer, doctor, lawyer, sales clerk, or general miscreant you plan to wipe out of existence, without a mere glance to your mirrors, let alone blind spot.

Really, what do you expect that sign to accomplish? It's far too small to warn the world of either your 4-lane wide lane ego or lane change. You'd think flashing lights and a siren on a tank would be more appropriate to protect both your little bundle of joy and warn your fellow drivers to get the hell out of your way.

I would even go as far as to recommend putting some eyes and a brain on board that grinding metal box of destruction that you send flailing through the streets of California. On the other hand, maybe not. You know, just in case Jr. there miraculously survives to adulthood and wonders "what the heck was Mom thinking with that sign?"

Sincerely,
A Concerned Driver.

Feb 18, 2007

Roast Suckling Pig

Unlike woe, I don't recall the fortune of experiencing the Chinese New Year festivities in Hong Kong. Miss CoolBeans asserts that her childhood memories of the celebrations are highly memorable.

I must have missed out in Toronto, but I can't complain here. Sizzling 74F/23C weather seems to be just the ticket to spur on a whimsical Saturday. Rock climbing, dim sum, and hanging out with the old crew. There was a celebratory air in the crowds at the taiwanese Ranch 99 supermarket, and the visible traffic and street parking around dinner time.

Really, I can't think of anything more to ask for... hm, maybe some red pockets? Must remember to ask for those tomorrow!

Jan 18, 2007

Clouds

As I left an all-afternoon meeting and walked between buildings on our campus, I was shocked by the deep blue sky and the glowing pink and purple clouds.

It's frickin beautiful.

Sad thing is I'm sure they're there almost every evening. I just rarely go out and see them.

Jan 17, 2007

Japanese Game Shows

I really don't know what to make of Japanese games shows.



Jan 14, 2007

Reflections

I haven't really penned (or typed) down any real reflections in the last while. Occasionally something worth writing about will come to me; a fit of inspiration, a sight, a sound, a sermon, or a revelation. But invariably, by the time I find some time to blog, it's fled from my mind. You need a certain environment to think and write, and that just hasn't come up much lately. Either that, or the brief moments where I do find myself reflective, it's just too far away from a computer.

In some ways I miss the BlackBerry.
I have a very short memory span these days.

Either that or something like this happens, I start blogging and then get distracted by IMs for an hour and lose my train of thought.

So I guess I'm just teasing.

Jan 10, 2007

Happy 200,000

Image017.jpg

The Camry was fairly underutilized when I got it from my parents, but I've put it to good use since!

Jan 6, 2007

COOKIES! COOKIES! COOKIES! part deux

I thought they would be good
I mean they looked good
but I didn't expect them to be this good
like heavenly
mmmmmm butter

COOKIES! COOKIES! COOKIES!

OMG I GOT COOKIES!

thanks R. you're the best!