Managing on the Bleeding Edge Eric J. Ray's Weblog

Wow! It's great to finally get to talk about the OpenSolaris 2008.11 release and the tremendous accomplishment that it represents. I've been watching the OpenSolaris Install Development team (my team!) working on this for months--starting before the dust had even settled from the 2008.05 release--and they've just done a wonderful job. 

What's cool about this release? Well, there's a whole bunch, from Time Slider to the great accessibility features to vastly improved hardware support, but I'd like to focus on the Install developments, simply because that's closest to my experience (and my team's incredible effort). What did they do for this release?

Well, first, there's the AI (Automated Install) project, which delivered a great prototype of the hands-free install features that we plan to be supporting in the next release. It's a huge step and a vast amount of work, complicated by some staffing changes mid-stream. The team pulled together and really made huge progress. I do expect that there will be some rather substantial changes for the next release, because I keep talking to engineers who have an ever-growing list of "well, if I can, I want to fix" tasks for the next few months. That said, even in the current form, AI is a super achievement and a great foundation for our planned SPARC support and other enhancements. This project took the lion's share of the engineering effort from the Install team over the last several months, and it shows the commitment.

Next, there's DC (Distribution Constructor), which underwent a complete rewrite from the prototype code we delivered in May to final, supportable, and polished code for this release. Watching the engineers on this project was fun, because they just kept  pushing harder and harder to get the code done, and done early enough to allow some stabilization time before the rest of the code from all of the other parts of OpenSolaris hit. Obviously, as we're creating our own release engineering tools--and your tools, as a member of the community--it's a big task, and one that's very intolerant of mistakes. It's clear from how quickly the post-2008.11 improvements are being completed that this code is a significant milestone, and a great place for anyone wanting their own OpenSolaris-based distro to start. 

 ICT (Install Completion Tasks) is the un-sung hero here--the engineers who drove this simply saw some loose ends from the last release that needed attention, and they dove in and got it done. It wasn't sexy or exciting, but it really needed to be done, and the effort here shows in the overall fit-and-finish of the 2008.11 Install technologies. Thanks, guys!

 Snap Upgrade, finally, is my biggest "wow!" from this release, I think. (That changes, based on what I've looked at last, but for now, it's the one.) I just find it incredible that I can install a system with the 2008.05 release, and do a full upgrade, while the system is running, in only a few minutes, and easily--safely--choose to move forward, or stick with the old bits. After ~15 years with various Unix and Linux flavors, and the trepidation that system upgrades always bring with them, it's mind-boggling that upgrades now take virtually no time, accrue virtually no risk, and just work. From the engineering management standpoint, the most impressive part is that the engineers working on this project made it look utterly simple. No sweat, no drama, no hassle--just hg commit after commit after commit, and results in spades.

My comments here cannot be complete without acknowledging the rest of the folks who helped make these accomplishments possible. OpenSolaris Install benefited from having really good support from the QE, Tech Pubs, and HCI teams, and many other groups across the organization. Some of the team members really pitched in behind the scenes, in non-obvious ways, to make it possible for these teams to succeed. And the teamwork within the organization to ensure success just delighted me.

 Honestly, I cannot imagine being any prouder of a team than I am of this one--working with these very talented, hard-working, and diligent engineers is utterly delightful. 

 Thanks so much, to all of you--you've done a great job, and you're already delivering for the next release. Wow. 

Ask me how I spent my Saturday morning...go on, ask...

Well, if you insist, here's why I spent my morning reconfiguring my whole home email setup.

Yesterday morning, I noticed that my usual morning email supply (in my personal account) was rather sparse. All of the election email, kids grade reports, and daily reminders of stuff, all of it--it was conspicuously absent. After doing some investigation, it became clear that email just wasn't passing from the spam filtering service I use to my home network. After ruling out all other options, I ended up on the phone with Comcast, only to learn that "they're having problems with port 25". Sigh. I decided to let things sit for the day, and hope that they'd get it fixed. I'm tough. I can handle a day without email.

Well, they didn't fix it. After googling for "Comcast port 25", without quotes, it became pretty clear that I wasn't likely to get the problem fixed, and I'd have to work around it. (In brief, they seem to have started blocking all traffic on port 25, regardless of the direction or the purpose. I don't even send mail on port 25, but SMTP does use port 25 to get mail to me.)

I ended up having to configure a new mail server (Postfix) in a Solaris Zone on a system that we have at a colo facility. That system now accepts incoming mail from the spam filtering service and spools it. Then I had to set up a MTA (UWimap), with SSL, of course, to be able to get to that mail. I then set up fetchmail to retrieve that mail and feed it back into the home server, so we can all get our email again, from the home server under the stairs. (Of course, in the process, I sorta set up a mail loop that resulted in 40,000 extra emails for my wife--whoops. Had to fix that too.)

As I was tinkering, my wife asked "what do normal families do in these cases?" Well, I guess they just continue using Gmail or Hotmail or whatever, and life goes on. I dunno. Maybe I've been a geek for too long, but the idea of not running my email service myself just seems odd.

Although today, I'm thinking I might want to give that a try.

 As I look back at getting the home network server moved over to Solaris, one of the things that's  most remarkable is the fact that I didn't have to build anything from source. I'm pretty sure that's never happened to me before, even on some of the more popular Linux distributions.

Thanks, Dennis Clarke and Blastwave (http://www.blastwave.org/). Virtually everything I needed or wanted was either included in the Solaris distribution I choose, or was at the end of pkg-get. As a result, all of the time and effort in the process was in tweaking things like my Samba configuration and the like, to account for the fact that the kids are now users of the system, so it needs to be a bit more bulletproof than I've bothered with in the past. That's really cool, particularly given the time that I've spent in the past crawling through various Catch-22 permutations (on lots of different systems) to get a server in the state that I want it.

The one exception in terms of software availability was Unison, which is a file synchronization utility that I like for cross-platform file syncing. I considered setting up one of the Rsync clones on the kid's computers, but didn't feel like messing with the ssh agent complexity needed for that to work appropriately. Even getting Unison running under SMF wasn't a big deal, though--that's just cool.

 

After literally years of good intentions, I've finally managed to get my home network server flipped from an aging Linux distro to Solaris (Solaris Express, 12/06, to be specific)...whew. It's funny how well-entrenched technology gets, and the "if it ain't broke, don't fix it" rule of thumb takes over.

 

On thinking about it, I believe that we've had a home network server for about 12 years now, ever since we had a 10base2 network, Windows for Workgroups, and a honking Windows NT 3.51 server. That server was actually the impetus to getting into Unix in the first place-- we hit one of those nasty BSOD events that totally trashed the file system, and I had to rebuild the server and restore everything from scratch. Again.

I headed out to the computer store and found a copy of Yggdrasil Linux, along with a book to help out, and went home to get to work. It was a heck of an interesting learning process. I had done some sysadmin work on Ultrix and AIX systems in a previous life, and spent a little time with Sun products, including the spiffy SparcStation5, which just smoked everything else I'd seen. I hadn't, however, ever used a *nix system that was all mine. It was fun!

As I recall, it probably took a week or so to get things up and running sufficiently to have a network connection, X running, and Samba sharing files with the Windows boxes. Cool. (Funny how the time to configure doesn't change, although the scope of what you can configure in that time does...it's Moore's Law in reverse, I guess.)

Over the years, we migrated from one Linux distro to another, from Yggdrasil to Slackware to Redhat to SuSE, usually based on either cool new claims or hardware death that provided the opportunity to upgrade. Recently, though, life's just been too busy to upgrade, and the existing server wasn't broken, so just kept on going. (There's clearly a challenge here for Sun as a whole--when the existing systems are good enough and not actively failing, it's hard to get the needed catalyst to actually upgrade.) Fortunately, a fading Dell PIII server provided the impetus to put a used Xeon to work and reduce the server sound level by 30 dB.

With Solaris working beautifully on most x86-class hardware, not to mention that I've been working in the Solaris organization for about 7 years, it was clearly well past time to get the home network on Solaris. Over the next few postings, I'll try to share some of the things I did to get the server up and running, and happily serving the family needs. Stay tuned...