Mark Koch's Weblog Mark Koch's Weblog

A couple years ago, I built a 1TB file server using standard computer components. I wanted a place to put my photos, music and movies and be able to access it via Samba or NFS from other computers in my house... oh, and I'm a cheapskate so spending too much money was not an option. I used an old chassis and motherboard that I had along with some other components that I already had. For the drives, I hunted around and found that CostCo had a pretty good deal on 250MB drives. I bought two to start with. I also bought a cheap IDE-133 card since the on-board IDE controller was only IDE-66.

I put my file server together and downloaded the latest Fedora Core 4. Installation of the OS went smoothly enough for the root drive but things soon took a turn for the worst. After alot of research and very experimental drivers and software RAID utilities, I had my system working. Since I only had two drives, RAID-5 was out of the question. I decided to concatenate the drives. I wasn't worried about backups since all my data was backed up on CDs and my music and movies were safely on their original disks and boxed up in storage somewhere. Worst case, I would have to spend a few days feeding CDs back into the system if things failed.

Things worked happily for about a week. I was still in the process of adding my movies and songs to the system when it started acting weird. The system went into read only mode. Looking at the log files, I saw lots of errors accessing the disks. So I did what any sysadmin would have done. I rebooted it. Upon reboot, I saw that my raid was gone. From my notes, I retyped the commands to create the RAID again and the RAID was working OK again.... for a few days. I went through this cycle of use, reboot and re-create my raid to the point of having a script that would do that all for me.

Another few months passed and finally one weekend, the Linux file server which had been unnaturally resurecting itself started to have performance problems. I had added two more disks with the intent to add two more. Since Linux RAID wouldn't let me add new disks into an existing RAID, I came up with a strategy to move the files around to different raid sets to balance out which files went where. I started to notice that 1 or 2GB files were taking over night to copy from one drive to the other. Something was wrong here and no amount of Googling would turn up a good answer. I decided to just deal with it and let the computer do it's thing. At least my files were still there.

A few months later, I attended a presentation at Sun when Solaris Nevada was only a few builds old. The presenter used her AMD64 bit laptop to walk through many Solaris Nevada demos including DTrace and ZFS. She made both DTrace and ZFS look too easy, it seemed to good to be true.

I decided to bite the bullet and build a new file server with Solaris and ZFS. This time I had a few extra dollars to spend so I beat my inner cheapskate into submission and headed to NewEgg.com online. I started with a CoolerMaster Stacker 810 series chassis. It's one of the few chassis that has 12 5.25" drive bays out the front and a relocatable power button panel. Using their included 3.5" drive adapter ( and two more purchesed ones) I was able to put twelve hard drives into the space of nine 5.25" bays, and each 4 disk adapter has a silent fan on it. I was able to loan 12 500GB drives for this project. With nine 5.25" bays filled up, I filled the other three with the power button panel at the bottom, a DVD-ROM drive at the top and a 80GB root drive below that. I pulled the ASUS motherboard out of my Myth-box-slash-failed-experiment-since-it-also-ran-linux box and repurposed it for this project. That motherboard had 1GB RAM and a AMD64 3400+. I researched the Sun hardware compatability list and found a cheap quad SATA controller that should do the job. It turns out that the chip on that controller was used in a Sun product somewhere, so I had a reasonable expectation that it would work pretty well. The SATA controller came as RAID-configured. I needed JBOD mode for this thing to work correctly so I re-flashed the controllers from a download from the chipmakers site.

I installed Solaris without much trouble. OK, I admit that I've installed Solaris many many times in the past 17 years, so things were pretty easy for me. However, I was new to ZFS and was still skeptical about how simple it might be to set up and whether my data would still be there a few days later. I decided that I wanted to have a mirrored "data" pool for my most precious stuff like Quicken backups, family photos and laptop backups. I also wanted a Raid-Z "media" pool for movies and music that I knew I had a hard copy of and wouldn't cry if all was lost.

Using examples I got from a stack of ZFS presentation slides, I created my pools using only a few commands.... and that was it. But wait! I said to myself. What about formatting, partitioning and newfs? Hmmm... not needed. I was kind of bummed at this point. I had expected to spend the evening and possibly the whole weekend seting this thing up and futzing with it. No dice. My ZFS set up literally took about 5 minutes and has been running flawlesly for four months now. I've even upgraded from Solaris build 34 to build 42 to build 47 without any problems. I've had to come up with a few new hobbies because Solaris has become too easy to use.

With 750GB and 1TB drives coming out soon, I look forward to being able to swap out the 500GB drives one by one. ZFS doesn't care how big the drives are! It will determine the best use of the drives you put into the pool. And unlike the Linux RAID system, I can even grow my pools by simply adding new drives.

In summary, although I have used SunOS/Solaris for about 17 years, ZFS and RAID are pretty new to me. The Linux path was akin to throwing me to the wolves. It was difficult and frustrating for this experienced user who only wanted to put his files somewhere on the network. In contrast, setting up ZFS was easy to understand and easy to use. And so far, it just works.

Jonathan and Scott talk alot about cost of ownership. I'm just a hobbyist when it comes to using Solaris and ZFS at home but my free time is still worth something to me. I could be doing something else like watching my movies and listenting to my music instead of Googling for "linux+raid+failed+wtf+how+to+fix+my+computer".