James C. Liu's Weblog

Wednesday December 06, 2006

Swapping drives between Solaris machines

AMD Geode NX and motherboard combo deals

In the constant search for more low-power, home system deals, another processor I've had good success with is the AMD Geode NX. The Geode is a line of x86-compatible 32-bit processors AMD offers for various embedded applications, usually in extremely low-power, space-constrained applications. I'm not sure what the roots are of the Geode line, but at the higher end of the spectrum is the Geode NX and from what folks are saying on the web, it looks like these have roots from the Athlon XP line and might actually be the cream-of-the-crop CPUs that are down-clocked and can run stably at 1.2V (rather than the usual 1.6V) so as to reduce power. I've only seen Geode NX 1750 1.4 GHz cpus sold on combos with motherboards retail but AMD advertises multiple speed versions of the Geode NX, probably for volume OEMs.

About two months ago, there was a sale at Newegg.com on a particular Geode NX/Motherboard combo. The price on the combo was $69.99. This deal was for a PCChips M863G(v7.0) micro-ATX socket-A board with SiS 741GX/964L chipset and integrated graphics, AC'97 audio and SiS900 fast ethernet. The combo included a factory mounted, Geode NX 1750 idling which supposedly idles at 14 Watts going up to 20 Watts. The combo included a generous heat sink and super loud 70x15mm fan that ate 7.3Watts by itself! (Fan and H/S replaced with much quieter CoolerMaster unit running at 7V and 2200 rpm). Rev 7.0 of this board does support a 1.2V core cpu voltage setting that lets the Geode NX achieve its low power status. Newegg seems to be sold out of this combo, but they get more once in a while. (Picture courtesy of Newegg.com).

Fig. 1. PCChips M863G board with bundled AMD Geode NX 1750 CPU from Newegg.COM
 

Directron.com is also sold-out on a similar combo with a Biostar M7VIG 400 board with all-in-one VIA KM266 Pro chipset. The Biostar board should have been the better board and I actually bought this board first, paying a few bucks more because I knew the chipset better and thought it was well supported. But after buying it, I wasn't fully happy because the board revision and BIOS didn't support the Geode NX's low-voltage mode. So while the cpu could run cooler and with the same clock rate, 1.4V was the lowest voltage it could drop to, so basically, I was sort of running the equivalent of an Athlon XP at 30 - 35 Watts, when the cpu should be able to operate at 14 - 20 Watts. I also had some issues with Xorg on the Unichrome graphics which I was able to work around, but it wasn't a clean install. The Xorg workaround will be explained a little further down.

So with the Biostar-VIA board not being as low-power as I thought, and Newegg having the similar combo with SiS chipset, I didn't hesistate; I bought one. As luck would have it, not 10 days after I received the first board, Newegg had the $10 rebate offer that would save more on the same item, so I had to take advantage of that deal too and get a second combo to score on the rebate. I did this pretty much not having checked if the SiS graphics, network and audio would work - the deal was just too good to pass up.

Sad days; Retirement of two old friends coming soon.

The goal of buying those Geode NX combos was to begin the process of retiring a pair of 8 year old boxes that have served me well over the years. These were proprietary BookPCs I bought back in 1998 with a tiny Super Socket 7 mobo, special 95 Watt power supply, and were short depth. These had the VIA Apollo MVP3 chipset with ECC SDRAM support and I maxed each one out with 512 MB of branded ECC memory and AMD K6-2 450 MHz. These were tough boxes and the weight alone on these small but heavy units was definitely old school manufacturing. They had Davicom 9102 NICs and I only put a disk drive into each, removing optical drive and FDD after the installation for reduced cabling and improved airflow. I've upgraded these boxes over the years. First 6.4GB IDE drives in both, then 40 GB Ultra quiet drives. Today, one runs Linux kernel 2.4 still and the other runs Solaris Nevada. Both have run so long that the power supply and CPU fans have clogged up from dust and seized up and the power supplies (which are very hard to find) have blown. I spent a couple of days back in 2001 relearning basic circuit analysis again and researching power supplies. After a couple of trips to Halted Supply Co. (HSC) near Lawrence Expressway and Central in Santa Clara, I finally got a bag of high frequency switching capacitors in various sizes at about $0.35/each. So for about $1.50 per power supply plus $7 for a new fan, I had some quiet and good-as-new power supplies and was back up. Since 2001, I take the servers down at least every 100 days or more often during Spring and Summer to dust off the systems, replace fans that are noisy, etc. I still have a spare set of capacitors out there for one more repair.

It's amazing how much dust can accumulate inside a server box when you run it at home. With humans shedding skin flakes, pollen and dust from outside in the garden, those servers were actually like air filters for the house. After cleaning, I was shocked at how warm the side of the case was; and then it hit me that the dust build up had been preventing heat from radiating out the sides of the case too, and that sent more heat out the back through the power supply. So almost every 2 to 3 months, the servers get taken down on a Saturday evening, at 1 am or so, and the cases are cracked open and the units go outside to the patio where I try to stay barefoot and grounded and then blow on the boards with some type of compressed air. Some stubborn soot gets caught between pins or in nooks and crannies like on the cpu heat sink and I use a soft tooth brush with Swiffer dust cloths to clean and wipe. The units go back inside and get tested to see if cpu, case and P/S fans are noisy or wobbling after service. I stock 50mm, 60mm and 80mm fans in 10mm and 15mm thickness (and other size fans too) for this purpose. They get swapped if noisy and I usually switch pin order to bias the fans at 7V versus 12V so as to reduce noise. Not all fans and motherboards support this, so you need to make sure the ones you get aren't smoking or failing to start doing this. Then the systems get closed up and put back into service.

It'll be sad retiring these old friends, since they handle email, web, Java servlets and JSPs, firewalling and database for close to 10 domains and do it without any real performance issues so far. But each consumes about 44 Watts in power when I have other servers that are 1.5 times faster at 19 Watts, which could tremendously improve battery life on the two 1500VA UPS units I own and allow me to consolidate all the switches and routers onto a shared UPS rather than adding separate smaller UPS units for those. With newer, even faster chips using about the same or less power, I may even be able to use just one box for all services and consolidate completely and really saving money, power, the environment, and all that good stuff.

Installation Dilemma - Slim Drive or No Drive?

Ever since Sun produced the Netra X1 line of products, I've loved the low 1U, shallow 13 inch depth case. Those boxes sort of epitomize the whole idea what a small, cool looking, but industrial server should look like. I wished someone made an affordable case in a similar form factor that was a bit quieter, and equally attractive. About the closest thing I can find today is a SuperMicro SC513 or SC512 1U chassis. But at close to $180 for the case, it's not cheap. And to add cost, it requires expensive slim optical and floppy drives. It's certainly too rich for my tastes and I've never investigated the acoustics. Those old Netra's though were pretty quiet, if I recall.

A compromise of sorts is to go with a BookPC form factor chassis. The first gen of these cases were like the ones I described above; they came with small proprietary boards and power supplies and actually were the size of a large telephone book. The whole barebones kits could be gotten for under $100 + shipping. A good feature was they did take standard-sized optical and floppy drives, usually over the motherboard, but it got cramped inside pretty fast. Back in the late 1990's, not all DIMMs were low-profile. Some were 1.4 inches tall and too tall to allow sufficient clearance between drive bracket above and cabling that went over the DIMMs situated underneath or partially below the drive bays. Subsequent revisions of BookPC cases have gotten longer and deeper so the board is entirely clear of the drive bays. Prices have also gotten longer and deeper and they start around $60 and go up to $100 or $200 for some sleek all-aluminum cases. (If I'm gonna pay $200 for an all-aluminum case, it better protect the board from EMP from the next nuclear detonation in my neighbourhood! LOL!)

And the peeve I've had with cheaper BookPC cases are they're really noisy due to all the cheap small fans because the makers know that there will be folks out there who think they can save a few bucks and get a small case for that Quad Core/Quad GPU gaming system. So, yes, that's why they have a bunch of fans and they're noisy. And that's why I focus on finding low-power processors and motherboards to reduce volumetric heat generation on these small boxes. (Note: Before video games, kid brothers would watch their 2nd grade sisters use a Hasbro Easy Bake Oven with the 100 Watt light bulb heat source - it bakes -real- cookies. Understanding the heat generated by a little box might be something to bring back into 2nd grade education so big boys don't grow up and try to shove a 200Watt heat source into a small case. Note2: We could educate the manufacturers too... only, most aren't socialized in America with Easy Bake Ovens - they're just putting in noisy fans to cover themselves and their distributors against too many RMAs on melted chasses).

For all my complaining, the compact BookPC size and shape does appeal to my sense of aesthetics, and with some re-wiring and soldering skills, I can usually lower the voltage on case and power supply fans to make them slower and quieter, yet still sufficent to cool a lower-power system. And that's what I did to a couple Enlight 7396AM1 low-pro cases. These have a high quality, sound insulated chassis, front USB, comes with a fairly quiet power supply and Directron.com has them on clearance for $19.99 with $13.99 shipping, which gets cheaper if you buy more than one case. (See figure below - courtesy of Directron.COM):

Fig. 2. Enlight 7396AM1 micro-ATX case for $19.99 + S/H at Directron.COM
 

I bought two of these last month, and then realized that the drive bays take a normal floppy, but -slim- CD/DVD drive. I was sort of in a dilemma. I didn't want to shell out $75 each for 2 slim DVD burners when I had perfectly good regular NEC 3550As in stock. It would sort of defeat the whole idea of buying these cases for $19.99 + shipping. Then I asked myself the question if I really needed an optical drive or even floppy on the box, since the reason for these systems were to replace those two 8-year old AMD K6-2 450 MHz bookpc servers. Neither have any optical or floppy drives in them; they were removed after OS installation, leaving just the hard drive.

Chicken or Egg Solaris Install? How about transplants?

So I don't have slim optical drives to do the Solaris install using the small Enlight cases. In addition, the PCChips M863G with SiS chipset boards support Novell IPX Netware Boot but not PXE, so a DHCP boot of the system isn't going to be easy to implement. And even if the board supported PXE boot, the default Solaris install doesn't have the SiS900 network driver. I guess if we had PXE, I could disassemble the x86miniroot in Jumpstart to add Murayama's sfe driver (Howto provided in a previous Blog) and that would give us an active network interface to complete the Jumpstart install. But the boards don't have PXE, only IPX Netware boot so thinking about PXE boot without a PXE boot capable NIC would be moot. I checked my inventory of Intel and 3COM ethernet cards and none had the optional PXE boot ROM. And the only one with the PXE boot option ROM I have in stock is a PCI-e Intel e1000g. Great. More shopping, but that would take a bit of time and I wanted to install the systems, there and then.

So the best solution I initially thought of was to temporarily attach a standard DVD-ROM drive and have it perched outside the propped-open case and do the install once on each disk. That'd expose the system for just an hour or two and we could then close them up. But that's still an hour or 2 or more. Plus the whole hokey setup with cable/ribbon hanging out and a bare drive setting there spinning loudly for 2 hours while I'm trying to watch a remake of Van Helsing with Hugh Jackman, isn't what I had in mind. And looking over my desk, I had a bunch of IDE Seagate Barracuda IV ST380021A drives, in clear plastic clam shells, just sitting there, recently swapped out of a couple of test boxes with Solaris Nevada b52, which got newer SATA drives.

A conversation with a colleague earlier in the Spring of this year came to mind. We were all sitting outside the Sun Santa Clara Auditorium after a Silicon Valley Open Solaris Users Group (SVOSUG) meeting and a bunch of folks were going over to Denny's or IHOP for a late bite. Dan Price had just given an S10 Next Gen Overview and he covered a lot of things. And our SATA team had given a talk on the new SATA framework. And one of my colleagues who works on x86 boot and ACPI was there. I think we were waiting for a couple of guys still inside and probably cleaning up, and I asked my colleague about why if I switch disks from one box to another, I can't get Solaris to boot, except for Safeboot.

My colleague gave me that funny look, like, "Why would anyone wanna do that?" I explained that from a customer support perspective, it'd be cool to flash a Solaris image to disk, then ship it and let it boot up and do self-config. He countered that, no, this isn't a big feature demand for Solaris and asked rhetorically how many folks would ever use that feature anyway? Well, I wasn't sure. But on Linux, I do this all the time - preflash a disk and when I need to do a quick build, I slap the drive in there, Kudzu kicks in, and voila... system is configured in a couple of minutes. Solaris gets stuck in reboot-hell if we try this. So after some debate, which went nowhere, I brought up the Jerry Seinfeld episode that got me to start watching that sitcom. And this was the first episode I ever chanced upon where Seinfeld has his little comedy clips at the open and close of each episode. He's standing there talking about the black box.

"Ya know, when a plane crashes... the only thing that survives is the BLACK BOX... Ever wonder why they don't just make the WHOLE plane.... out of THE black box???" (laughter).

In all seriousness, we know that Solaris safeboot has the hooks to rebuild the boot-archive and device trees, and the installer figures out the devices and puts a permanent map of that onto the filesystem somewhere. So why can't we put those same hooks into the regular Solaris boot or multi-boot? That's what I asked my colleague.

He gave me that, 'James, you're naive and uninformed about Solaris x86 boot'-look and just told me that it's not a widely used feature, and that's not what multi-boot does, and started down the path of how the GRUB works with multi-boot and the whole secondary boot blah, blah, blah, blah. Which I took as obfuscating the issue with detailed specs that don't matter to end users. Anyways, to make a long story short, I was miffed and decided against Denny's, and went home to eat cold supper made by my wife, who's always been a lot warmer and at least seems more understanding.

Fast forward 9 or 10 months and here, I have two perfectly good disks, almost new, and pre-installed. I searched the web and managed to find a few Sun FAQ/Developer discussion board topics. It was clear that at least a couple other folks wanted to do the exact thing and actually had the gumption to ask us how. One guy got pretty far in the boot but still didn't get it going. Our standard answer was to boot the kernel in debug mode and look at the output. That's fine if you're a Sun Engineer and know how to look at the messages and hex scrolling off the console. But to for most folks, it's nonsense and just shows a lack of empathy on our part.

But within the various tips, I put together the various tips and gave it a try; magically, 5 minutes later, it was all working on the new SiS chipset motherboard. Here are my steps:

1. Boot into Solaris Safeboot mode. You can get access at the Grub menu, usually the 2nd option.
2. Mount the found Solaris partition on /a Safeboot will usually find the slice on the disk with Solaris and ask if you want it to mount on /a. Select Yes.
3. Move /a/dev, /a/devices, and /a/etc/path_to_inst to another name (I just append .orig) and then create new directories, (mkdir) /a/dev and /a/devices, and touch /a/etc/path_to_inst.
4. Run "devfsadm -r /a" to rebuild the device tree
5. Edit /a/boot/solaris/bootenv.rc and modify the line with "setprop bootpath '/pci@0,0....' to match the path you'll find mounted for /a (i.e. run a 'df -k' command, and you should see /a mounted from /dev/dsk/c1d0s0 or something, then run 'ls -l /dev/dsk/c1d0s0' or whatever your device listed was, and you should see the actual link point to ../../devices/pci@0,0/...) The path to bootpath you want should be the hard disk which is mounted as /a and you just need to find the expanded /devices/pci@0,0/... path and put that in the bootenv.rc file on the Solaris root filesystem on the hard disk (sans the /devices/ prefix of course).
6. Now run "bootadm update-archive -v -R /a" to rebuild the boot-archive on /a
7. run a 'touch /a/reconfigure'
8. Run "cd /; sync; sync; sync; umount /a"
9. and finally reboot.

There may still be issues if you have changed the IDE priority (master/slave) of the hard drive or moved it to a different SATA socket. In these cases, you probably will successfully boot but run into problems with mounting filesystems not found, in which case, boot into safeboot again, and edit /a/etc/vfstab to correct.

Once the system boots, it does retain its legacy settings for network and naming service which may be totally bogus, in which case, I check for any missing drivers (like an SiS900 Fast Ethernet module) and either transfer the source/binaries for the modules via CD/DVD media or USB which usually will work. I'll then delete any /etc/hostname.[NIC#] files and run a sys-unconfig on the system, and reboot again.

Yes, not the most elegant of solutions, but it could be scripted and one of my colleagues down the hallway thought we could do our customers right but putting that script into the safeboot and documenting that we put it there. For now, I hope the instructions help others who might be in the same boat. It takes about 5 - 10 minutes to swap the drive and reboot and reconfigure the system if you know what you're doing. That's a lot better than an hour or two re-installing or upgrading and it also means I can keep pre-flashed drives just sitting around to save time on installs and testing.

BTW, Solaris b52 runs well on the PCChips M863G motherboard with AMD Geode NX 1750. The SiS 741GX/964L chipset functions normally, and Xorg even finds and configures the SiS onboard graphics with no work required. The graphics support on the SiS chipset, while supported in Solaris, is really grainy. And the graininess is particularly worse on one of the systems than on the other. I guess I could stick an AGP card into in the slot (and I did try an older ATI Radeon unit and it looked beautiful), but it eats more wattage with the optional graphics card. And since my plan is to mainly run these headless, I really shouldn't care. But this isn't the first time I've had graininess issues with Integrated graphics, especially on these small form factor boards. I suspect it has something to do with the number of layers in the board (usually cheap boards use fewer layers) and the integrity of the analog/vga signal that goes through the motherboard where there may be a lot of RF interference to get to the back I/O plate. I've used the same graphics chip and software driver on different boards and sometimes the image is crystal clear. And of course, with an optional AGP card in the slot, the board is raised and separate from the board and less likely to have interference due to wires too close in proximity. I get similar graininess with the VIA mini-ITX systems, but it's not as bad as on the SIS chipset on this particular board, plus an older celeron system I used to have with an SiS chipset and SiS Mirage graphics looked great, so it isn't the chipset.

The goal is to run them as network servers, so I plan to disable the graphical login and run only in text mode anyway. Next, I compiled and installed the sfe-2.2.0 gldv3-nemo driver from Masayuki Murayama's Free Solaris NIC collection and it just works. Honto ni, arigatou gozaimasu, Murayama-sama! (someone send this guy a case of Sapporo Nama Biru and Pizza Hut vouchers or a free Shinkansen Ticket to Sapporo where he can pig out at the beer factory at the "Ghengis Khan" Mongolian BBQ Tabe/Nomihoudai [all-you-can-eat and drink] !)

Note about Xorg on older Unichrome (not Pro) Graphics

I mentioned above I had an issue with graphics installing Solaris on the VIA Unichrome graphics. This was on the Biostar M7VIG 400 board. Back in Nevada b30-something timeframe, more than a year ago, I was having some graphics issues on my EPIA mini-ITX boxes which only impacted the graphical installer, and the VGA text modes. But Xorg, itself, worked fine. Only, if you selected Text Console as the login option, you'd get a dark and illegible screen or sometimes a pastel screen. But the version of Xorg did work and very stably if you always used the graphics. We had a few talks with some of the VIA folks who tried to reproduce this on newer mini-ITX boards and they didn't see the same problem. The funny thing was we knew this was partly a problem with the Solaris driver and partly in hardware rev, because in going from an older rev of a PCChips M789CG v2 to a newer rev, v3, both with VIA Unichrome graphics, suddenly the problems went away in the same build of Solaris and all the modes: the VGA text, console login and X graphics worked fine - no driver changes. A friend mentioned also that a 1.3 MHz Nehemiah on CN400 chipset with Unichrome worked fine, but I saw weird pastels or garbelled text consoles on my older EPIA ME6000, M10000, and EPIA 800 systems.

Some time around the build 48 time frame, I was going through and upgrading a bunch of my systems when suddenly, all of my VIA Unichrome systems would power-up in blanked out graphics mode (i.e. there was no VGA signal coming from the graphics port and the monitor would blank with a yellow blinking standby-mode light). The behaviour was very peculiar. And nothing was showing up in the logs. I tried to log in, and yes, it was as if X was thinking that it was running because I could see the disk spinning as if I succeeded in logging in, and ssh'ing from another machine showed that I had active Shell processes on the console, even though the console was blanked (like someone turned off graphics or blanked the screen in low-power mode or something like that).

Back in the old XFree86-to-Xorg transition days of Linux, I used to run a bunch of Biostar M6VLR boxes with the old Trident Cyberblade graphics embedded as part of the VIA PLE133T chipset. Fedora Core 1 and 2 had fits with the Trident Cyberblade. And even FC3 still had some issues. A more stable driver binary was available and the trick was to use it to replace the default driver module used by the Xserver. Using the same trick, I loaded a Solaris DVD from a previous build and found the ./Solaris_11/Product/SUNWxorg-graphics-ddx package and copied the ./archive/none.bz2archive to /tmp and unpacked it using:

        # bzcat none.bz2 | cpio -C 512 -idukm

This created /tmp/X11 and inside /tmp/X11/lib/modules/drivers/ was the via_drv.so file.

I copied the old VIA graphics driver in /usr/X11/lib/modules/drivers to via_drv.so.orig and then copied the one unpacked in /tmp/X11/lib over to /usr/X11/lib, clobbering the old version. Because the sizes differ slightly with each build of Solaris, I couldn't tell which was the same or different. So I methodically went back checking each build of Solaris until about build 42 or 43 before I found a version of via_drv.so that worked with the graphics and didn't blank the screen. But it did cause the Biostar board with Geode NX cpu to revert back to bad console text. So at least it seems that the Biostar boards were old versions of the VIA Unichrome hardware (reaffirming my disappointment with that first Geode NX/Biostar combo). But I had a work around to get X graphics working.

I know build 53 put back some big changes to graphics, and improves AGPgart. I don't know if it fixes other graphics issues; but it's worth a try. And b54 just got posted internally. Unfortunately, that Geode NX box is up near Vancouver, Canada right now and in a powered-off state with no LOM (lights-out management). I'll be up there for Christmas in a couple of weeks and then I'll give it a try, unless the weather warms up a bit to let me get out there and go fishing for some winter steelhead. I'm usually more worry free in winter while fishing. The black bears are hibernating and not likely to be up stalking me while fishing. I'll save that for another blog.

December 06, 2006 05:12 PM PST Permalink

Tuesday December 05, 2006

Solaris x86 Desktop Hardware Shopping

I'm not sure if it's the Christmas holiday coming up soon, or if it's just coincidence, but the 3rd person this week emailed me this week asking for recommendations on a Solaris x86 box. Two were asking about low cost laptops for kids that will dual boot, and the other was for a home server. There's also been this interesting internal discussion on small form factor Solaris x86 boxes that's garnered a lot of enthusiasm and suggestions for small appliance systems.

I'm by no means the expert on hardware, especially the high end. I know you can buy that premium stuff online at the Sun shop which has a new look and feel. Surprisingly, I needed to do a double take when I just looked at this site, just now, because while I sometimes have a disdain for big-iron workstations, a dual core Opteron workstation starting at $995 from Sun has to be a printing error, right? It's missing an extra digit somewhere. No, it actually looks like a real price, and it looks like it comes with 1 yr hardware support too. That's pretty hard to believe coming from Sun. We're actually selling them cheap. I'll need to look further into the shop to see what other stuff we sell. But back to the really low-end do-it-urself boxes and shopping for Hardware.

I've been known to have a weakness for big sales (except ones requiring rebates), and I have a hoarding instinct for this computer stuff. Some folks refer to this as a disease. And yes, admittedly, I've got some variation of the illness. But I reassure myself that I'm not the worst. I do have lots of spare parts, but it's all well organized into bins and boxed and bagged properly in anti-static bags. And I tend not to carry anything expensive or power-hungry. For example, the fastest CPU I keep in stock is maybe an Athlon-64 3000+, and I only have one of these. And the biggest graphics card is maybe a 128MB AGP8x unit. The rest is slower, really cheap, but extremely usable and great for fixing computers and great for building Solaris boxes. I'm referring here to my home inventory; not my work, where we get to play sometimes with some bigger, enterprise type systems.

And my stockpile of stuff has come in handy on occasion, like when my nephew's budget "educational software" (i.e. gaming) system died recently due to a bad power supply that then led to a fried motherboard. I had both a compatible micro-ATX motherboard in stock, plus a newer, quiet power supply too and he was back up and running; Or when my Dad's box suffered back-to-back power outages during a recent late summer rain storm that shorted out the power supply, I had a surge protector and new special TFX12V power supply in stock, and still have another in stock just in case. Or if my psychic powers are right, I predict my Bro-in-law will have paging issues with a particular PC game that is memory hog and that extra stick of 512MB DDR333 will be the salvation he will be looking for... just mark my words. I've got it in stock and people will need it!

If you stockpile more than 5 boxed motherboards, or 5 new cases, or more than 5 separate power-supplies, then probably you fall into the class of electronics junky known as a "Hardware 'Ho". The etymology, I'm told, is not from anything related to the "Ho, Ho, Ho..." from a cheerful Santa at Christmas, but from the slang term for "Whore" implying, you'll pay anything to get close to computer hardware. I suffer similar issues with being a Fishing Tackle 'Ho, and I'm also a recovering Bicycle Gear 'Ho. But that said, there are ways to live with being a HW Ho and not break the bank. For example, one way is to focus your energies on low-cost, low-power stuff that works well with Solaris x86. That's how I channel the urge spend all available money on hardware on the biggest, baddest and most power-hungry systems into a contest of who can achieve the best price/power/performance. So, for example, instead of say, blowing $650 on a graphics card to get better frame rates on Quake 4, I simply evaluate 3 integrated graphics motherboards under $50 and experiment with which one has the best graphics performance for the dollar. Another part of my therapy is to go to Fry's for lunch (their small cafe inside the store has surprisingly decent sandwiches) and buy no hardware. The goal is to resist the urge to spend money frivolously while being tempted by a store full of stuff ready to buy. Becoming a discriminating shopper for Solaris-compatible stuff is a good way to curb that urge to waste money, and we'll go more in-depth below.

Step 1: Solaris End-User Requirements

As far as I know, Solaris isn't really an OS for gaming, at least not yet. It's possible to dual boot a Solaris box with Windows so that you can have an awesome PC gaming system, but get real work and do real Solaris development using the same box. But as a home-user, I use it for word processing, spread sheets, some digital photo stuff, email client, web browser client, playing and ripping audio, as a file/web/email server and firewall. I also VPN into work from home using IPSec and I burn a lot of CD and DVD iso images. I find these were most of the things I used to do with Linux, but less and less because on Solaris, the software now performs as well if not better than Linux, and has pretty solid stability even across kernel versions. I can tell that it won't be long before, at least in our household, Linux won't be active, although, I plan to always keep a current copy of a popular free distro around to play with.

More than just software and performance requirements, I also have environmental requirements. My living room and bedroom are places where these systems will stay. People coexist here and must endure the noise. People living in a place also stirs up air and dust too, and the PCs need to behave stably with lots of dust fouling during hot summers and cold winters.

If your requirements are like mine, nothing there really needs a lot of big-honkin' iron processing. Some of the audio ripping and conversion, or editing photo images does take some CPU and memory, but for the most part, any Pentium-3 class machine with more than 600MHz and 512MB of PC133 SDRAM is almost adequate for Solaris. Anything much faster, will be spent idling, or burning more Watts which costs money if folks actually stopped to think of the costs to power and cool a system. In addtion, it could shorten the life of the system because that extra power requirement goes into more fans to suck more air in to cool the system, depositing more dust that can foul and burn out power supplies and overheat capacitors on the motherboard causing failure. Some of my colleagues refer to their home racks as "air filters" for the home, since they suck up so much air and deposit soot inside the case.

This doesn't mean we should just stop buying newer, high speed hardware. In fact, quite the opposite. While the CPU and memory might have achieved performance adequacy 5 years ago for home users, some newer systems may provide even better power economy while running faster, as well as support newer, faster chipsets, graphics and monitors, faster networking and USB peripherals, and are likely to be more available in stores at a lower price, than having to try to find it on some auction site because the original part is no longer made. Sellers rarely make overt offers for low-power, high-value hardware; you need to look for these deals.

Step 2: What works with Solaris x86

To find out the official list of hardware items that work with Solaris, there's an official site called the BigAdmin HCL. On that page, you'll get a list of servers, desktops, motherboards and laptops that are certified or reported to work with official Solaris 10 and Solaris Express. The HCL allows companies and individuals to buy a Support Contract from Sun for an annual fee. Using the OS is still free. Folks can also download the Hardware Compatibility Test Suite (HCTS) from Sun and run this on their hardware and submit their entries. Version 3.0 of HCTS recently went public and it runs for about 12 hours and tests the filesystems, cpu, memory and network pretty intensely during that time. It requires two machines (a system-under-test SUT, and a Test Manager/Server) hooked up back-to-back, or through a seperate switch on a private network to run the fully networked test. The Solaris Marketing folks used to have a Tshirt Give-away for anyone who registered, downloaded the HCTS and tried to run it, then made a Report-To-Work submission or certification. I made a bunch of submissions, but found out later than Sun employees were forbidden to participate in that contest, and anyway, they didn't have 3XLT tshirts to fit me. Bummer.

Unofficially, Solaris x86, especially, the latest build of Open Solaris, supports a lot more hardware. In general, Solaris will run on most major CPUs (AMD, Intel, VIA) and motherboard bridges and north/southbridge chipsets that support Intel's x86 instruction set or the AMD64 instructions (or variant, such as Intel's version - EM64T). This includes standard I/O like IDE/ATAPI interfaces, any SATA interfaces that support IDE legacy mode, systems with PCI, PCI-X, and PCIe bus, older ISA bus support for PS/2 keyboards, mouse, peripherals and Serial and Parallel I/O, most on-board USB, and systems with an AGP slot. Typically, Solaris can be install in console mode on any x86 system with about 128MB of memory, some type of ATAPI optical drive and some type of supported hard disk with at least 5GB of free space. But if you want something usable, 512MB or more RAM is good, and 80 GB disk might be a start. By default Solaris x86 auto-senses 32-bit or 64-bit architecture and boots the OS as appropriate. And yes, the 64-bit version of OS is fully compatible with 32-bit user applications.

Where Solaris doesn't work out-of-the-box is for certain on-board and common peripheral devices like new native SATA and RAID controllers, Wireless 802.11, some ethernet chips, some new graphics cards, or and with some types of high-definition (HD) audio devices. Solaris also lacks software and drivers with proprietary peripherals, like Point-of-Sale scanners, printers, card-swipes, kiosks, etc. Many network, audio, and storage controller drivers are available from third parties, and in the open source space and there are companies that sell proprietary drivers solutions that run on Solaris as well. There are a few drivers in the GNU GPL space that Solaris might be better off having, but those are getting fewer as hardware vendors are coming over to Solaris and we apply more engineering resources for porting. For the time being, it still pays to stick with what we know.

Step 2a: Motherboard Selection

With some experience, we find that most chipsets work with Solaris and it isn't really a particular maker or brand that causes incompatibilities, but it's an onboard peripheral bundled with that particular chipset that may not have a supported driver. When that is a native-only storage controller or NIC or Graphics port, then immediately the Solaris install is more difficult. So the trick isn't so much choosing the maker or the chipset, but knowing what peripherals come with a particular board or chipset-bundle. In some cases, the behaviour of a peripheral, like a native SATA controller, can be adjusted to set to use legacy-mode only, in which case, the board works fine with Solaris.

I like the low-end and all-in-one chipsets because this category provides excellent price/performance and low-cost to buy and own. Counter to what others tell me, I actually think the cheapest and most mass market motherboards are some of the most abused and tested board on the market, and therefore, manufacturers tend to make them more robust to human abuse than on server boards that cost 10 times as much, but have a much smaller marketshare. Certainly, the margins are much greater on server hardware (yeah, we at Sun should talk... we're in that business). But the whole ideal of achieving volume Solaris for the Proletariat means (a) making it cheap to get the OS, and (b) having it install and run well on a damn cheap box!

Rather than go through each chipset and what works, it's probably easier to say what is not likely to work. Since most CPUs, bridges, buses, and standard I/O controllers to memory and disk have to be somewhat compatible with Windows/x86 instructions and memory flow, for the most part, Solaris just works and recognizes and uses that hardware like any other x86 operating system. Standard drivers are in place for the usual ISA legacy devices like serial, parallel, ps/2 mouse and keyboard. Also supported for the most part and standard USB devices and USB storage, and ATAPI drives like IDE disks and CD/DVD ROM and Burner drives and any SATA drives that are running in Legacy IDE mode.

Step 2b: Graphics Selection

What might NOT be fully supported in the install media are the graphics, NICs, audio devices, and WiFi networking chips. These don't necessarily mean that all is hopeless. Some drivers can be downloaded from the web if another system is available and networked and you have something, like a USB jump drive to copy drivers after the initial install for configuration. In other cases, there may be no support for the onboard device, but you might be able to install an add-on card that works with the board you have. Graphics is a good example. Up until build 53 of Solaris Nevada, the AGPgart driver didn't support the onboard VIA Unichrome Pro graphics on any of my newer VIA systems. But those systems all had optional AGP4x/8x slot or PCIe and it was possible to buy a very inexpensive ($15 - $20 online) ATI- or nVidia-based AGP card from a previous chipset line (e.g. ATI Radeon 7000 or nVidia MX-4000). Laptops are different story when it comes to graphics - pretty much, you're stuck with what you get. In these cases, Solaris supports the slightly higher end embedded graphics chips like the ATI Radeon Xpress 200M or the nVidia GeForce 6100/6150. Solaris has lots of issues with Intel embedded graphics, even among chipset variants and vendor implementations that should be supported by the Xorg Intel embedded graphics drivers. I often have the same graphics non-compliance issues with Linux, and luckily, it's been getting less frequent with both Linux and Solaris, as it should since both leverage the Xorg source. But I try to avoid embedded Intel graphics on a system for running Solaris graphics mode. But ATI and nVidia seem like well support brands these days. We even support the newer nVidia Quadro line in Solaris.

Step 2c: NIC selection

With NICs, the problem is less difficult. There are many free drivers out there for Solaris. Murayama-san puts out a bunch of open source, free Solaris NIC drivers. So the installation may not find any drivers for the ethernet devices onboard, but if you have a VIA, SiS, Tulip, Davicom, or other mass market embedded NIC, then most likely, there's a driver for it. What folks may want to avoid are the newer embedded PCI-Express GigE NICs. The drivers may actually use the same core logic, but the PCIe bus signals and handles interrupts differently over PCI. We do support most high-end PCIe server NICs, but each has a specific initialization and so this is device specific and our driver requires some amount of code to correctly initialize each variant of that MAC. So it isn't as easy as trying to stick that vendor/device ID into the /etc/driver_aliases file and run devfsadm to initialize one of these PCIe embedded NICs. It may work, but usually not. Luckily, if you need a NIC and this isn't some laptop where you have no choice, then you can get a Realtek 8139-based Fast 10/100BT PCI card for about $0.99 on sale (sometimes $0.49), and I've purchased 3 Realtek 8169 10/100/1000 GigE PCI cards which I paid $4.99 for each. This is brand new. Not used.

Step 2d: Audio Selection

Solaris 10 and later supports most Intel Spec AC'97 audio, VIA 823x Audio, and some legacy Sound Blaster stuff. That covers about half of all the audio on most boards these days. There are still a few boards with AC'97 audio which the Solaris included install media won't recognize, and there is the new class of High Definition or HD Audio chips. For folks with an unknown AC'97 type of audio controller/codec combination (read your spec sheet or prtconf -pv and find the PCI vendor and device ID and web search that device to find out what it supports), most likely, you have a good chance if you download Jurgen Keil's audio drivers and install them. Jurgen supplies his own versions of VIA and AC97 Intel audio drivers. These have different module names and will not clobber the existing drivers, and his installation package does a good job of pruning and transferring PCI vendor and device IDs to his driver control where both the Sun drivers collide with his drivers. Jurgen's drivers are especially useful with some AC97 codecs that have a non-standard sample rate conversion hardware. Where the Sun audio810 driver fails, or takes a huge performance hit going with a software sample rate converter (am_src module), Jurgen's driver allows one to compute the proper sample rate and set it as a configurable in the /platform/i86pc/kernel/drv/audioi810.conf which usually gets the driver to play audio at the proper speed. The information on how to do this is actually in the .conf file itself and has come in handy on a Compaq motherboard with Intel 815 chipset and AC97 audio. Jurgen's VIA drivers eat a bit more memory for buffer space, but lower the interrupt rate dramatically, eliminating almost any hiccups on some very slow first-gen EPIA 500 fanless C3 mini-ITX systems and allow those to play flawless audio.

As of Nevada build 44, Minskey Guo and the Beijing team did a put back of a quick High Definition compatible audiohd driver implementation. Minskey says he was "helping" me, but actually, he did most of the heavy lifting on the all-nighter coding and testing. I ought to send this guy a case of quality beer and certificates for Pizza Hut in Beijing. I wonder if they deliver like in Tokyo?

HD audio comes from a 2004 Intel spec and is pretty much different and not meant to be compatible with AC97. First of all, the spec which is available from Intel's site separates the combined AC97 controller/codec into separate HD audio controller interface and compliant codec. Capabilities are for much higher bit rate (upto 192kbps for 8 channel - 7.1 audio) versus the old AC97 (48kbps for 6 channel - 5.1 audio). This allows motherboard vendors more performance and choice in selecting the codec and implementing rich audio support for multiple devices, playing different streams simultaneously. At the same time, if the driver is implemented with good support for codec discovery, it's possible to implement a universal audio driver that can ideally parse and initialize any codec. All communications between audio applications to the device are through the controller and through defined kernel DMA structures. This eliminates applications ever needing to hold onto pesky base address register values once the memory segments, driver and codec initialization are complete. The only problem is that support for codec discovery isn't trivial and can take time to test and properly debug. Also, we learn in the hardware business that every codec isn't quite the same and there's always some errata on either the controller or the analog pinouts the codec is hooked up to, such that the discovery may not provide correct information. It's quite a bit easier and faster if the register specs are available to simply check for codec device and do the initialization in the driver code or to use an external configuration file that enumerates name=value pairs of properties that specify where the standard pins for that codec are. It makes it easy for vendors to ship audio codec initialization files too as properties. But that all depends on if vendors are willing to give out the specs on their codecs. Which is really the crux of the problem, since that's been somewhat hard to get except for the Realtek, which have just been excellent Solaris partners. So far, the Nevada audiohd driver can support most Realtek HD audio codes like the 260, 880, 883, and 885 and others. And this supports stereo audio playback and recording. But the number of tested HD Audio controllers isn't great. But if you have a Realtek codec or think you may have one, you may still be able to get the driver to work by editing the /etc/driver_aliases file, and searching for "audiohd". Then add an entry in the same format for a new device. For example, on my MSI K8NGM2 motherboard system with MCP51 chipset and HD Audio controller, the PCI device node blocks me from seeing the codec directly, but a "prtconf -pv" shows the controller as "pci10de,26c" which I added to the list, then ran devfsadm -i audiohd, reboot and suddenly, the audiohd driver works great. The driver may work for quite a few other audiohd controllers that are back-ended using Realtek HD Audio codecs. But we didn't have too many systems at this point to play around with, although a number of laptops are now coming out with HD Audio standard. Many don't use a Realtek HD audio codec, but some other brand, like a SigmaTel.

Support is unlikely until we get specs, or if we have time to work out a really good parsing algorithm for codec discovery. But for the time being, you're not all shot on Solaris if you have HD Audio. In fact, there may be a version of OSS (Open Sound System) from 4Front Technologies that works on Solaris x86. 4Front has been doing high quality commercial audio drivers for UNIX for years. And their latest stuff is free for personal use and has a 6 month license that is renewable for free every 6 months, or can be purchased perpetually for just $49 to end the hassle of re-submitting for that license key. I've tried the OSS stuff and it does work with the ATI SB450 HD Audio controller and codec underneath on my new bargain Toshiba M115-S1064 laptop. The audio worked fine out of the box, but had some issues with standard Solaris audio support devices (like the Gnome volume control) not responding on the laptop (since there is a manual dial for audio volume), but otherwise it sounded good. I have some poorly ripped Enrique Iglesias and Led Zepplin tracks that had lots of Cymbals and high pitch audio that got a bit over saturated and tinny in this driver. But I peeked at the hdaudio.conf driver (no collisions in the name space again) and I downgraded the quality of the input audio stream the driver was expecting to medium (instead of the default - high) and suddenly, the tinniness went away and I was hearing great audio again. It pays to try different settings in the driver if it has a .conf file. Hopefully, between the existing drivers, Jurgen's drivers, and OSS drivers, most audio chips will just work with Solaris.

Step 2e: WiFi Selection

I used to think selecting a WiFi chip was specific to laptops. But recently, in Sunnyvale, the Mtn.View based company: MetroFi Wireless, has installed a bunch of WiFi base stations. These are advertising supported services, but it's changed my outlook on how to help the families of kids at my local elementary get online using broadband. Instead of deploying a donated and refurbished Linux Box with some semi-expensive and rare controller-based modem for a cheap dialup service, the PTA might be able to collect donations and get some cheap PCI WiFi cards, install them into these systems, and then use wifi to get on the network with no monthly or annual dialup fees. We wouldn't be limited to Linux and its LinModem support, which is mercurial at best with the plethora of Host-signal processing (i.e. controller-less) modems. We could go stock 802.11g and with the right booster antennas, users should could get good WiFi signal.

That's motivated me to look more closely at WiFi support and not just on laptops, but at built-in, cardbus, and PCI card WiFi NICs. A big list of WiFi drivers recently got into build 53 of Nevada. These support PCI, mini-PCI and cardbus versions of popular 802.11b/g chipsets including the Intel Pro Wireless 2100 and 2200bg/2915abg, the Prism/Orinoco chipsets, the Atheros 52xx series, the Ralink Technologies RT2500 series chipsets, the Realtek 802.11b wireless chipset and the Cisco Airnet chipsets. The Atheros driver works with most of the previous mini-PCI laptop ABG cards. It doesn't quite work yet with the newer AR5006 PCI express line which is starting to become more prominent with laptops. The same story goes for the Intel ipw3945 which is also a PCIe device showing up in lots of laptops, and latestly, the Broadcom BCM4306 line of WiFi chips fills in the remainder. So none of the newer Wifi chips are supported... yet. But my bet is on the AR5006 since it's already available in the MADwifi driver on Linux so it shouldn't be too long. I don't have too much public news about the ipw3945 or the bcm4306 except that we don't have the specs to write the driver, but some reports claim that the NDIS wrapper can provide a usable and stable driver for either device on most laptops.

But besides laptops, what's interesting is the availability of WPA on top of a lot of PCI-bus WiFi-BG cards. To my surprise, a lot of the cheapest (e.g. Airlink 101) PCI WiFi cards are based either on the Atheros (Super g) 5212 chip, or the RaLink RT2500 chipset (regular G). These are cards that show up frequently on sale for under $20 at stores like Fry's. And the RT2500 version have been down as low as $7 for the card (limit 2 per customer) which is why I have two at home. For folks that don't want to rewire their homes with Cat 5e/6 cabling, and have noise already on the PowerLine and can't get more than 6Mbps/sec throughput on PowerLine accessories, then 54 Mbps is a refreshing increase and it only takes switching to wireless. So Solaris users can simply go out and get a cheap card for their desktops too and enable WiFi without shelling out a lot of money to go wireless. Not bad at all, I'd say.

Step 3: Where to buy this stuff?

I shop for most of my stuff at these sites:

1. www.newegg.com - probably where I spend the most money. Awesome website, great customer product reviews, and awesome prices. Once you New Egg...
2. www.ewiz.com - less selection than newegg, 5% cheaper on boards than newegg in a lot of cases and a local Bay Area company with insanely fast service - order it and get it in 24 hours!
3. San Jose Mercury News - click on the [Newspaper Ads Online] link and see daily Fry's ads. If I see something cheap, a quick trip at lunch with 3 other colleagues in a carpool is the usual menu/venue. And you can order a decent lunch at good prices at the Fry's coffee shop inside the store.
4. www.directron.com Another low cost online retailer. Houston, Texas-based. No sales tax to CA and good service. I get a lot of quiet-PC gear, adapter, fans, and cases from them.
5. Starmicro.Net A Bay Area company with fast service and some really cheap prices on older memory and cpu.
6. www.compuvest.com A Washington-state based company with a long history of low prices and good mail order service. They carry a large variety of older technology and new stuff at competitive prices and out-of-state means no sales tax either.

This is by no means an exhaustive list. Sometimes TigerDirect.COM has a good deal if you're into waiting for Rebates (I'm not), or SurplusComputers.COM will have some good deals too which are online-only and not available if you go to the store for pick-up, which is a bummer, since I live pretty close to those guys. There's also www.axiontech.com, another Texas-based company that caries some good motherboard prices, just to name a few.

I tend not to shop eBay that much. There used to be deals in the past, but most sellers have jacked up the price on shipping and handling to the point where the prices are ridiculous. Often, with computer gear, the shipping and handling are more expensive than the item. Who in modern history would pay $25 per disk for a -USED- IDE 40GB drive? But that's exactly what it comes to with these online auctions and the bidding wars in the last few minutes of every auction close. The going rate for disk storage is about $0.40 per Gig and that's out-the-door prices after tax and shipping on a -NEW- disk drive retail. I usually wait for a sale on some Seagate 5 year warranty SATA or IDE drives and get storage at close to $0.25/Gig. Folks are paying upwards of $0.60/Gig for used stuff and that's just stupid and buying from some kid with a 97.9% rating. For that amount of shipping and handling, I'd expect a 110% rating if it existed. But again, eBay does serve its purpose for those desparate and in search of something the other vendors don't carry any longer. I picked up a couple of Intel D815EEA boards for $16/each with shipping recently to build a couple more Solaris boxes to go with a couple of Intel PIII 733 MHz cpus I bought for $8 each at Compuvest.

My Top Pick for Solaris System Right Now

Clearly, there are a lot of choices that anyone could make right now about what parts to throw into a Solaris box for home use. And it's a lot of fun to build and install systems, especially if in the end, they go to help families get low-cost broadband. But I do have a favourite box for at least I think this is the best deal and the one, right now, I wish I had in my home. It appeals to all my personal criteria for the right blend of a) price, b) performance, c) quietness, d) power consumption, e) Solaris support and f) aesthetics. It starts with the PCChips V21G flex-ATX motherboard with soldered-on VIA c7 processor at 1.5GHz. It has a new CN700 chipset that supports IDE and SATA and has integrated VIA Unichrome Pro graphics, VIA rhine Fast Ethernet, and DDR2 support. The motherboard and cpu all-in-one is just $68 at ewiz.com and installs Build 53 of Nevada without any real issues. I'd stick a 120GB Maxtor SATA drive with 8MB buffer which was on sale at Fry's for $49 recently, and then a 1GB stick of DDR2-533 Kingston memory for $95, and put that inside an Antec Minuet 300 Case w/ 300 Watt P/S for $79 with a silver bezel NEC 3550A or 7170A-0S DVD burner for $30 and a Multi-format, Silver bezel USB 2.0 flash reader for floppy drive bay install for $14. Total cost would run upwards of $380 when all shipping and handling was done. Not the cheapest build possible, but a very usable system that's small and relatively quiet. If I were going for a true budget server system and forego a DVD burner and get a cheap case, I'd probably use an Enlight 7396AM1 BookPC case with 180W P/S and save $50 on the case, $44 on optical drive and flash reader. I'd also get a stick of 512MB DDR2 for half the price and save $50 since most of the time, I would run this without X. That'd save a considerable amount of memory more than ample for a simple httpd/j2ee server and mail server. That'd make a box for about $250 that could serve up a lot of services.

How would you install a box without a working network interface (yet) or an Optical drive?

I'll talk about options for that in the next blog.

December 05, 2006 03:23 AM PST Permalink