James C. Liu's Weblog

Wednesday November 29, 2006

Solaris Install Experts - the New Chic

Quite a few years ago, I met up with this big guy at Tokyo University. His name was Ohno-san. He had a similar build like me. Big, round, husky. He road a big Motorcycle that really ate gas - sort of a status symbol as big Honcho for being one of the early contributor's to Japan's WIDE network. I remember his greeting to me - the once over stare and then grin and a modest handshake. Not a limp-fish, and somewhat out of character for a Japanese person. He said immediately that he could tell I was a techie/geek and I must know my UNIX systems, since most of us are pretty big guys with similar builds that look like we exercised our forearms lifting slices of cold pizza for most of our lives. But I lacked the facial hair and bad hair. Oh well for stereotyping folks like us. But I don't mind the status and any implied chic other folks -think- I may possess.

There seems to be a revival of that UNIX chic these days. It's not just inside the company. I see it with some vendor/partners and some of the academic/EDU folks. Solaris is cool again, and folks who can install it and fix other folks computers have a certain chic. Granted, we're a couple levels below the God-hood of a kernel developer, but we interface at a higher level with desparate folks in management and marketing who want to try out Solaris but haven't got a clue how to install it properly on their system.

Pepboys (Geek version): Computers Like Us - Colleagues Love Us

Solaris installation is a lot like car maintenance. Almost anyone could probably pick it up if they had the inclination to research a little bit and try to exercise some Emersonian Self-Reliance. The argument against everybody doing this (within a company or outside even) is the notion of Comparative Advantage. This idea says that everyone has their personal strengths and contributes in their own way. Folks have proven mathematically that Comparative Advantage allows multiple parties to optimize their productivity so all sides can benefit. And hence, we all specialize in our particular fields. And it's just my long-winded explanation of why there's a Chic associated with being able to install Solaris. Simply because folks like us are in demand. And so the ones who are less self-reliant will want to schmooze with us to defrag their laptops and partition a slice to install Solaris on it.

But it's still a lot like car maintenance. And a cornerstone in the mechanics trade has been that customers go back to the mechanics they trust, and with an evolving relationship, customers grow to respect the journeymen with lots of experience. Solaris installation ain't much different than doing tune ups. The more systems we get to work on, the more tips and tricks we learn. One of the things we pick up is what to buy, what works well, and what has good price-performance. Yes, it applies to names like Toyota and Honda too, but we're talking systems, and it isn't always the high-end chipsets that are well supported by Solaris.

In-Flight Across the Chasm

And whether that compatibility is a result of more community users hammering on the platform, or that the platform is more compatible and therefore more people are using it, it's clear that Solaris's recent popularity is coming from the x86 side. Sure, it runs on SPARC and we try to insure that out-of-the-box, Solaris just runs well and tuned on SPARC. But our SPARC customers just expect that and depend on it. The Solaris x86 side has been more mercurial. Instead of specifying specific supported hardware (which we Sell), we've had to provide an OS with broad support for many 3rd party devices. Linux has done a great job crossing this chasm of device driver support. Many vendors are providing drivers up front now for Linux. But the Linux kernel and headers are GNU GPL'd and the license can be somewhat severe for enterprises that have trade secrets. Some vendors have tried to play a risky game using shim layers in their device driver to insulate themselves from the GPL. But if they do it right, they end up with massive build environments for Linux because to support compatibility with the ABI, they need to maintain copies of kernel source, headers, and compilers. If they do it wrong, like a couple of embedded switch companies in Europe recently, then they may be forced to open source all their proprietary software on the device which also leverages GPL code or face a massive recall of all sold network appliances in the last several years.

The problem isn't so bad with GPL applications living outside the kernel. Applications are fairly safe if they only link to high-level libraries. But device drivers are kernel modules and live in the same process space that the kernel lives in and rely on GPL headers to compile. To get around GPL, shim layers of GPL code that then link to standalone proprietary object binary code seems to be the standard these days. But compiling a kernel module such that it has no errors in loading (e.g. the kernel taint statement), doesn't work well. The solution, if done right, is to compile a target object binary driver module for that explicit kernel with that explicit distro of Linux with that specific version of compiler, just as an added safe measure. But this grows into a support nightmare for vendors pretty quick. I support at least a couple of partners who support several popular distros each keep about 29 GB and 40 GB respectively in build environment (yes, Gig as in Billion). All this to prevent the taint message from showing up during modload. So it's not surprising that from where my group sits, more and more vendors are actually trying to contact us about driver porting to Solaris x86. There Solaris driver build environments are just a couple to several megs. Mostly documentation and make stuff. Not actual code.

And so, Solaris is trying to cross that chasm today. Fortunately, I can see the other side, and as Joerg Schilling predicted about 1.5 years ago, he said if we kept working on x86 at this rate, we'd cross that chasm pretty soon. Thanks Joerg for believing in us. We're not there yet and so we're still working really hard. But it won't be long now. Along the way, some of us have picked up some useful ways to get Solaris up and running on our systems.

Installation Tips and Tricks - a Summary

It's not all about the drivers. There some other pre-requisites to installing Solaris that should be covered. I'll be blogging more about the tips and tricks of trying to install Solaris in later blogs. But it starts with hardware choice and what not to spend money on. I'm into frugality, and it's often the cheapest, all-in-one motherboards and hardware that have support these days. It's more about selecting the right chipsets on a motherboard. I'm not a big-time gamer, so I don't go after the super high-end market. Plus, if I'm running Solaris, most likely, I'm doing simple home stuff - like audio, some digital camera stuff, some word processing, running some web sites and mail servers, firewalling the rest of my house network, etc. I make it a point not to spend more than $50 on a motherboard. If it's on sale, even better. And if it's a combo with all-in-one graphics, audio, LAN for under $75, even better.

It seems like the Optical media install is the de facto standard by which we judge usability. But more and more, I use network installs. It's amazing how many motherboards support PXE boot these days, and from a previous BigAdmin article, myself and a colleague tried to put a quick cheat sheet on how to set up a network install server and add more drivers to the netinstall image. But did folks know that they can pre-flash disk drives with Solaris on them and then re-configure them? I have a couple of servers at home and in the office that don't have an optical drive and don't have PXE boot bios extensions. I installed Solaris on a disk stuck in an install machine that flashed a netboot image onto the disk. And then I stuck it into the server box with no optical drive. There are issues of course with the boot-archive, the old device tree, etc. Linux does really well in this area with their Kudzu and admittedly, I wish Solaris were better. But installation isn't something most folks do that often. And so if instead of a single utility doing it for us, someone just had a complete set of instructions on how the darn boot-archive, path-to-inst and device tree worked together, then we might be able to reconfig the drive to the new hardware in say, less than a couple of minutes, it might not be so bad. Better yet, would be to script the process and have it as a command in the safeboot image. That's not there now, but something some of us are suggesting go in there in the future.

Lastly, once the system is installed, there's all the standard software that folks should stick on their system that isn't on there by default. There's the standard pathing for user shells that we should setup so commands are easily found too. There are also some nagging problems with devices that don't work well or at all. They don't impact the core Solaris kernel, but they may make the system unusuable (e.g. graphics is incorrectly sized, sound doesn't play, or some devices not functioning. Sometimes, an existing driver might actually work, only the vendor and device ID were not recognized in the OS database. Other times, the driver may be available as a free or commercial 3rd party just not on the install media. But their are quick ways to find basic drivers for network, audio, wifi and other components. These and other subjects will be topics in the next couple of blogs as I have time.

Looking back on the past couple of years and my experience with Solaris x86, I've gone through about 4 or 5 cycles where I've attempted to install latest current OS onto all my home and office boxes. At first, it was with mixed success, implying that a good fraction of the drivers were missing or so poor in performance and reliability that it made it unusable on the system. But lately, that's changed. Many do work for the low-end, budget systems in fact. Moreover, lately, the installations have been relatively easy. So it may not actually be so hard to achieve that high-level of fashion and popularity that being a Solaris install wiz, at least for a short while; most folks will still think installing Solaris is hard. But it's not something you need to share with everyone. And maybe you don't need to sit around all day eating cold pizza slices doing it to become a master at it. So you can have the sysadmin chic and still maintain the "girlish figure." The key is to enjoy this upcoming new year with Solaris, and sandbag a little when friends and colleague beg and grovel to have you install their laptop systems. Tell them you have a backlog and need more time. That might be true sometimes, but most of the time, I'd just take the fishing pole out and go fish a few hours while the install completes in about 30 minutes. I brought in a new custom fishing rod into the office recently. It's up on top of my locking bookshelf unit. Colleagues think it's just ornamental to go along with my fishing pic on the door. Hah! November 29, 2006 06:09 PM PST Permalink

Thursday November 16, 2006

What? Solaris Life Style?

Fishing Vacation Recap

Okay, so I haven't had time to blog in a while. Or fish for that matter. The exception was a short vacation back not too long ago to Vancouver to install a couple of new workstations in the vacation home up there, put in a second security camera and to figure out what was eating and extra 2kW-hrs/day in electricity besides the frige. I did manage to get some fishing in there, and a little scientific method of observing when the fish arrive relative to the tide, yielded big results. Here's a 10+kg salmon caught from shore just outside the house. Yeah, that's me in the picture. Took me 20 minutes to land the fish and two boats anchored just off shore to watch me bring it in. I wish I had a net with me, but I rarely bring one for fear of "Murphy's" Corollary - which states: If you bring a big net, you won't need it. But the fish looks like a Chinook on the outside. Black mouth, massive size, fiesty. And I got a kick cleaning it in the driveway. More than several of my neighbours (who own boats) drove by, hit the brakes, then backed up and lowered their passenger side power windows. "Ooooooh. N-I-C-E fish!" they all said, licking their chops. Amazingly, this salmon had no pink flesh. It's known as a "White Chinook" or "Ivory King." And up in B.C., there appear to be anywhere from 1% to several percent of the Chinook population that have the same genetic mutation where they lack the enzyme to breakdown and absorb beta-carotene. Hence, their flesh isn't pink. The flavour also is very sweet and mild, like a trout or char, and hardly like regular salmon. A delicacy in my book, for sure.

The two workstations I installed up there were systems for each bedroom. I found this killer store in Richmond, B.C., just off Bridgeport Rd., not to far west of the Home Depot/IKEA. It's called CanComputer and they have some good prices that compare well with the US. I may have mentioned some series of shops along the Bridgeport ICBC row (the section of Road where lots of auto-body repair shops are). Most are hole-in-the-wall shops with small store fronts, little retail display space, but large back rooms with inventory. CanComputer is like one of those shops, just a little fancier and cleaner. But they're all more mail order fulfillment businesses as opposed to Best Buy (which moved in last year, next door to the Future Shop at Landesdown Mall), which is over-priced anyway. Plus, the folks at CanComputer seem to know there specs a bit better on parts than the Future Shop or BestBuy folks. At least I was impressed that a sales person at CanComputer actually knew the difference in form factor for a x16 lane PCI-express slot versus AGP slot on an AMD socket 939 board and quiet power supplies with 20+4 versus 24+4 pins power rails. But neither BestBuy nor Future shop sell many computer parts - rather, they sell whole finished systems, so they aren't the kind of shops you'd expect true geeks/nerds to frequent.

I also had the privilege to bump into Nick Harrison at the Richmond, B.C. Costco. I was doing a run for DVD videos and on the lookout for Season 9 of Stargate SG-1, and met a semi-tall, medium build guy in the same video section looking at Sci-Fi serials. For folks who don't know, Nick Harrison is the guy who plays the Dark Knight in some of the Arthurian episode in last year's Stargate SG-1. He played roles as Jaffa in previous seasons as well. But it's just a kick to meet up with folks from popular Sci-Fi shows who are nice people and approachable. And with all the movies and TV shows they make up in Vancouver, that's another bonus for going up there. BTW, Season 9 of SG1 is out on DVD now. In the US, it's $32 at Costco.

Solaris Life Style?

So what's to do during those lazy late summer/early fall evenings in the great North? Well, for one thing, you can sit around and rip CD audio into mp3s to stuff them on a new, cheap, no-name MP3 player. I picked one up for fishing and roadtrips in California for around $50. It's a Truly MP310 in lime green with FM radio, colour display and 512MB of flash memory. I saw a recent sale at Fry's down in California on the same unit for about the same base price but $35 after rebate (curse short profanity to self...should've waited!) My sister always recommends waiting for these cheap electronic gizmos. "They'll get so cheap, you'll get 'em free with a Happy Meal in a year," she says.

Usually, I carry a trusty laptop with Linux on it, but as you all know, upgrading is a crap shoot, and sometimes, in the middle of an upgrade to a new kernel version, the Linux ABI story ain't the best and something go wrong. So my triple boot laptop has been booting Solaris x86-only in recent weeks/months. And it'll stay that way until I find some time to fix the install without hosing all my data on the existing slice. Well, I thought it was a bummer that I didn't have Linux with me, since, for things like CD-ripping and burning, it's been very handy and relatively easy to use. In fact, I believe the ability to rip and burn audio and mp3 CDs (as well as hook up digital cameras and other device support), has been critical to Linux's success. Without those utilities, there wouldn't have been the widespread adoption of Linux in the enterprise. In essence, Linux offered the Geeks something other Unices couldn't - LIFESTYLE.

Getting the powers-that-be who direct Solaris to view Multi-media support on Solaris, has been a long struggle for a lot of folks inside, and for many years. Senior/Upper management got the need for an Office Suite pretty quick. And Open Office/Star Office is just an awesome product. But office productivity doesn't equate to LIFESTYLE. A word processor isn't fun. You use it when a paper is due and you gotta get the darn thing out. You use a presentation package when customers are gonna show up the next day and you need to make something look pretty to sell them on some product or idea. And a spreadsheet... well, it's useful when creating PTA invoices to submit for reimbursement.

I thought Solaris was a long way from offering any Lifestyle, but I was wrong.

Stuck without operational Linux, I gave Solaris a try. And reading the man page on the cdrw(1) command, I quickly found out that it's possible to rip audio tracks and quite easily. For example, if you want to rip the 7th audio track off of any audio CD to a .wav file, then just do this:

    % /bin/cdrw -x -T wav 7 myaudiotrack.wav

So imagine what any shell scripter or Java programmer could do with a command line this simple... well, yes, it was easy enough to script this and rip and re-title every single song off of a CD and it's pretty fast too - probably less than 4 minutes to rip an hour of music into .wav format.

Once in .wav format, to convert it to .mp3, we can use any program. My preference is for Lame. And getting any version 3.9.x or later compiles cleanly and easily on Solaris x86 and runs fairly quickly, ripping at close to 6x playing speed for 128kbps bitrate. Usage is also straightforward. My ears aren't finicky so I don't need a higher bit rate. And if your cheap mp3 player is only 512 MB, the files are smaller than ripping using a higher bitrate.

  % /usr/local/bin/lame -h -b 128 --ta "Artist" --tt "Title" myaudiotrack.wav myaudiotrack.mp3

You may want to keep the .wav files around if you ever plan to burn a backup of your CD. But I usually delete them and just keep the .mp3 files. Some folks don't like the fact that .mp3s are lossy and noticeably lossy at 128kbps, so you can rip at a higher rate and store them which is still much smaller than the .wav files. I convert the .mp3 back to .wav files using mpg123 which is available freeware, and included on the Solaris 10 update 1 companion software distribution. If you've installed the companion software, then you'll find it in /opt/sfw/bin/mpg123. The command to convert songs back to .wav is:

    % /opt/sfw/bin/mpg123 --wav myaudiotrack.wav myaudiotrack.mp3

When you have all the tracks you want in .wav format, burning the audio CD is straightforward. Just use:

   % cdrw -a [track1.wav] track2.wav ....] trackN.wav]

So all this playing around with audio clips made me reconsider if Solaris can yield some semblance of LIFESTYLE. Yes. I think it can. The real question though is whether it could load songs onto the el cheapo MP3 player which is supposedly a usb-storage device. Funny thing is that the manufacturer claims that one can use Linux 2.2 and later with this player. But some users have had issues with kernel 2.6 and the usb port with this device. I had some issues with Fedora Core 3 at home, which has been upgraded a few times. There appeared to be an "Error USB Port 1" on Linux in the log files. I didn't expect much on Open Solaris Nevada build 48 if Linux couldn't even see this.

So on to Solaris. And as expected, nothing happened. Well there was some output in the /var/adm/message file on Solaris, but the usb-storage didn't mount. Yes, I was a little disappointed, until I got an idea to contact some of our Beijing driver folks working on USB. I got my hands on an early version of USB driver and after installing, I tried plugging the device in and, WOW! The device mounted and was found in /media. I found out that the MP3 player only supports USB 1.1, so it's a slow dog to upload songs. But after a couple of hours of chugging along, I managed to load up some oldies and goodies for my exceptional entertainment-on-the-go. I've checked recently with build 51 of Solaris Nevada, and yes, the new USB is in there, so soak up the Lifestyle folks. And if you have some cool scripts or Java GUI front end for the command line utils to rip and process audio clips, let me know. I would upload mine to Java.NET, except they're pretty embarrassing right now. Maybe later after code clean-up.

[disclaimer: Please use information here only to create -legal- archives of music you own. Don't be a spoil-sport and steal music. Support your favourite musicians and their labels]. November 16, 2006 05:08 PM PST Permalink