Matt Ingenthron's Stream of Consciousness

Wednesday June 14, 2006

Happy Birthday to OpenSolaris!

As someone whose primary job these days is several layers above the OS (regardless of kernel), but has a long history with Solaris, I figured in celebration of the birthday of the OpenSolaris project, I could give my perspectives on how the Solaris community has changed.

Call me self-centered, but to me part of OpenSolaris is all about acknowledging and growing the community on Solaris for x86 (and now x64). Why do I say that? Many of the advocates who told Sun walking away from Solaris on x86 is stupid are now some of the major contributors to the Solaris community. Folks like Jürgen Keil and Phil Brown have long been advocates of Solaris on the systems they use. They go out and attack tough problems, document solutions on websites, and generally work hard to grow the community for themselves.

Also, this is where I've seen the most personal use. The momentum that some of these guys kickstarted by telling Sun what the right direction to take with Solaris is has dramatically helped my personal use of Solaris.

Here's a story I've not shared before, because it's akin to airing dirty laundry. When I arrived at Sun, responsible for a general technical role in the sales organization, I was issued a Fujitsu laptop. Sun, at the time, had an arrangement with AOL for ISP services and had an ITOps build of software for the Windows 98 SE on the laptop.

As someone who prior to Sun had been a user and an advocate of Solaris on x86, it was amazing to me to see that Sun wasn't flying its own planes. I promptly obtained a copy of Solaris for x86, and set to work on it. I found, at the time, a small community (which has grown dramatically!) of folks inside and outside Sun who were helping each other with resolving issues on just such hardware. I eventually found that with the particular system I'd been issued, it just couldn't be loaded. This was not because the OS didn't support the hardware after install-- there were just too many goofy little tricks I had to go through with the ancient install process, which had been originally built around 386 class machines and things like ISA and MCA being bresent in the system.

The good news is about 6-8 months later, when I tried again with the same system and an updated release of Solaris, it loaded. However, that wasn't exactly fruitful either-- no modem support meant no networking from home, which was afterall the secondary place I'd be using just such a system (the primary being a customer's site).

Fast forward to today. As I mentioned above, the community has dramatically grown. Now, not only do I have Solaris (Express Community Release) loaded on my system, but I can confidently load regular builds of the latest/greatest and use them! I have wifi. I have cd burning support. I have good (though not perfect) graphics drivers. We even have VPN support at Sun through a project called punchin. I have software from a great many other companies on the system, including Adobe/Macromedia which now allows me to be targeted by a great number of advertisers on C|Net with video adverts(okay, so it's not all good). I use ZFS regularly with Zones to build up demo environments of Java Enterprise System software and take them back to a previous pre-demo state. Plus, compared to early 2000, the community is now so large that chances are, if I run into something (which happens infrequently now), chances are the solution is out there in the community. Chances are also good now that Sun is even involved in helping that community, instead of living behind the firewall (circa 2000).

So, happy birthday to OpenSolaris. It's come a long way in the last year, and it isn't showing any lack of momentum.

What do I hope to see in the next year?

• An easier path for contributions into the source code base-- I know folks are working hard on that
• Implementations of projects like NWAM
• More collaboration with other Open Source groups (there's been some good stuff with DTrace and ZFS); mostly this means people (sometimes including those at Sun) need to give up religion and focus on compatiblity
• A few more drivers that would make life easier, like SpeedStep; Intel contributes quite a bit to Linux-- I'd rather see this stuff more modular and available with a license that has less restrictions (that may not be the problem, but it is my perception)
• A bit better support for all of the chipsets from graphics manufacturers-- this is a call to those companies, as the state of that industry makes it hard for the OpenSolaris community to do it alone

I guess the most telling thing is that this list isn't that long. For that reason, I'd have to consider the first year of OpenSolaris a success. I'm sure others will have their yardsticks, but I fervently believe this list is the right track and the other stuff (i.e. marketshare, more IHV support) will come in time.

Technorati Tag: OpenSolaris

Tuesday April 25, 2006

How to shrink a VMWare OpenSolaris disk image

Okay, so maybe you got here via a search. I'll tell you right now, I've duped you. I don't actually shrink it. It's really creating a flash archive and building a new VM, but it's still worth throwing out there.

The basic steps are

1. Create a flash archive and get it to the VMWare host OS (I'm assuming VMWare Workstation, server is definitely a different approach)
2. Make the flash archive available from the host OS (or some other system if you're not using workstation
3. Create a new VMWare Image
4. Boot it off of an ISO or the network with the Solaris installer
5. Select the flash archive when using the installer

Now, being more specific. I created a flash archive like so:
flarcreate -x /var/tmp/flash -n myFlar -e "a flash archive so I can shrink the OS" /var/tmp/flash/myFlar.flar

Next I set up Cygwin. In my case, I'm working on something for a customer, and VMWare Workstation is the required host OS. I also set up the Cygwin NFS server, and enabled it as described in this howto. Note you don't have to click around the GUI to start services. Typing net start "Cygwin nfsd" to start the service is functionally equivalent. I exported the directory the directory I was going to use. Then I copied the file over to the directory.Update:Be careful, as of this writing, cygwin's NFS server doesn't support large files, so your flar will need to be under 2GB.

After that, it's simply a matter of an install from that flar. Boot the Solaris installer (either PXE boot or off an ISO image) and select install from a flash archive. Specify the flar at the right screen.

Then let it cruise along until it's done.

Yeah, there are probably other ways to accomplish this (ufsdump, for instance), but this was the one I chose. Regardless, the set of steps is nearly the same. It's definitely a lot of work, but it was worthwhile as I was able to more than halve the size of the VMWare image. In this case, it's effort worthwhile, as there may be many, many people using this VMWare image.

Technorati tag: OpenSolaris ( Apr 25 2006, 01:32:45 PM PDT ) Permalink

Wednesday August 31, 2005

SoCal OpenSolaris presentation

The good folks at the San Gabriel Valley Linux Users Group have invited me to reprise the OpenSolaris talk I did for another local LUG.

You can find the details here, but here's a summary of the agenda:

"Matt will discuss OpenSolaris's goals, how licensing works (and why you shouldn't be afraid of it) and how to get involved. This meeting will also cover what OpenSolaris is (and isn't) and have a brief demo. Take a look at Sun's OpenSolaris site before attending to develop your own questions about the project."

Technorati Tag:OpenSolaris

Tuesday August 09, 2005

Announcement: Adam Leventhal scheduled in LA

We've tightened down the schedule and we are in fact bringing Adam Leventhal for one evening meeting in LA on August 15th. In case you didn't see the previous posting, Adam is part of Solaris Engineering, and has spent a fair amount of time enhancing DTrace for the Solaris 10 and OpenSolaris release. He's recently been given an award by InfoWorld for innovation in Solaris, so he's definitely one of the thought leaders in the world of Unix. He looks the part too! (sorry Adam, couldn't resist...)

Adam's ready to come in and talk about DTrace, show some examples, and then talk for a few minutes about OpenSolaris. If there is something in particular you want to see, post a comment and we'll see what we can do. Since Adam did the DTrace and Java session (TS-5211) at Java One and has spent a fair amount of his time making DTrace useful for userspace code, I may see if the local Java guys have interest in seeing how DTrace and Java can be used together. Let me know (post a comment) if you'd be interested in seeing that.

Here are the details:

Meeting date/time: Monday, August 15th from 7pm - 9pm

Meeting Location:
Sun Office in El Segundo
222 N. Sepulveda Blvd., 10th Floor
El Segundo, CA 90245

Parking is worked out, Sun will validate parking in the parking structure attached to the building. You can enter from Grand Avenue or Sepulveda Blvd. It's the building at the corner that says "Oracle" on top of it.

Tuesday August 02, 2005

Adam Leventhal visiting LA

To: UUASC Members

Adam Leventhal will be in town for some customer meetings right after he speaks at Linux World. We might be able to make him skip dinner and talk with us instead. :)

I may set up Adam for a special evening talk on August 15th in LA or August 16th in OC. Is there any interest in attending this?

Adam is a Staff Engineer (I hope I got that title right) in the Solaris Kernel group. He's contributed significantly to OpenSolaris's DTrace (also in Solaris 10). He's an excellent speaker too!

Post a comment saying if you'd like to see Adam in LA on the 15th or OC on the 16th.

Update: Sorry, I should have been more clear, please be specific about whether you would attend LA (El Segundo) or OC (Irvine) or both. I don't want to take Adam away from dinner if we'd only have one person attending. :)

Monday August 01, 2005

Interactive Unix mention in "SCO OpenServer Is a Winner"

Hi Steven,

Regarding your article, I don't know for certain, as I'd never really worked with Interactive Unix, but I believe it is in fact Sun, not SCO who acquired Interactive.

I'm in the LA field office of Sun, and until recently, the old Interactive Unix office was still in tact (albiet with a Sun logo) along the side of I-405. We've consolidated it into this office (they were only a few miles apart), but a large number of the Interactive guys are still here on the 9th floor of my building toiling away at Solaris (both x86 and SPARC).

Perhaps there's something more complex to the M&A lineage that I'm not aware of.

- Matt

p.s.: Why do you discriminate between Linux and Unix in the article? "Unix on Intel had long been a niche product, but Linux was bigger and better."

p.p.s.: it does occur to me though that Sun is acquiring Tarantella, which was the old SCO before it sold off the SCO kernel and IP to Caldera, subsequently to be renamed SCO. It's always funny how this stuff works out.

p.p.p.s: I would have posted this as a comment to the article, but the 15 or so required fields to register for eWeek and to be spammed (if I had so chosen, of course) weren't worth the effort. :) instead I'll post it to my blog. Feel free to post any comments there-- no registration required.

Tuesday June 14, 2005

Congrats Guys!

I just surfed the OpenSolaris site. Good stuff!

One quick note to tell you how real this is. I looked at the Nevada page, and while there isn't much content, the content that is there is nearly exactly what you'd find on the internal build page. On the internal page you would always find what the current build is, when the flags days are, etc. It looks like Dan or someone prettied it up for public consumption though.

Awesome stuff, I can't wait to see what others contribute.

Friday April 15, 2005

OpenSolaris User Group in LA?

This is just a quick blog entry to point to John's entry on the same. Perhaps I have some readers out there. Is there interest in such a group? If so, let us know one way or another!

By the way, the name of the group is still being looked into, so if you have any suggestions, by all means, post a comment on those too.

Thursday April 14, 2005

Usenix '05 BOF on Day 2

Thanks to Liane for leading last night's BOF.

Last night was one more late night of Usenix talking with Solaris users/developers about developing on Solaris and OpenSolaris.

First the things I thought of as interesting from the development parts of the conversation:

• One method of getting information out of a process that's failed that can be quite useful is to put a log into the process's memory space. This is a pretty neat idea in that if your users send you a core file, you can quickly fire up mdb(1) and peek into that debug log. This gives you all of the advantages of getting your debug log info, with none of the negative side effects of throwing it to disk.
• In Solaris, even device drivers (that use the proper DDI public interfaces of course) should be stable across releases [i.e. Solaris 8 -> 9 -> 10 -> 11(?)]. If you test your device driver and it doesn't work, then that's Sun's bug, not yours and we'll fix it right away.
• Unlike any other OS I'm familiar with, Solaris really allows you to write one app in a structured way and have it run on a variety of platforms without needing to throw a boatload of #IFDEFs. One of these I'd known about is that Solaris will, on x86 platforms, use the hwcap facilities to mount the libc that best matches your platform. So if you have a screaming fast AMD chip, you'll get the sort that uses all the SSE and 3Dnow extensions to be even more screaming fast. The one I didn't yet know about was the madvise capability for an app to tell the system how you want your memory to be allocated. Then the OS can, based on what's appropriate for that platform, allocate memory in whatever way makes the most sense. So if your app will have lots of threads on a large multithreaded system accessing the same hunk of memory, as Bart said, it'll peanutbutter the memory across the whole thing. At the same time, if you put it on a HyperTransport 2-way system, it'll optimize for the in-core memory allocators. Cool stuff!
• By the way, that hwcap stuff isn't just for Sun with libc-- it's intended for app developers to be able to distribute different optimized libraries with their applications, meaning that if you were, for instance, developing RealPlayer and wanted to ship some optimized codecs for different hardware capabilities, one could do that. Again without a boatload of #IFDEFs.
• One thing that I've described to many, many customers, but never knew what the document was called is the application packaging guide. Haven't you ever wondered why Sun does /opt /var /etc /etc/opt /var/opt? It's not to make you have honkin' long environment variables--there is a method to the madness-- and if you keep track of it carefully your stuff will fit into the zones of Solaris Containers, Live Upgrade and the other cool stuff.
• SMF ships with a "generic limited net" profile that one can use to, out of the box, limit what's running to a reasonably secure system.

Then there was a fair amount of discussion on OpenSolaris. We were lucky enough to have Roy Fielding from the OpenSolaris CAB there at the BOF-- and we didn't know he was planning to be there. Thanks for coming out Roy!

I had the opportunity to shake Roy's hand and ask him if the stuff that came up in the BOF was within a reasonable standard deviation of the bell curve. He said that the questions were all the same stuff as the ones the analyists asked last week and I think we had good answers for all of them at least in terms of strategy, if not concrete implementation answers.

One thing that Roy mentioned that I think should be pushed out is that day one, you'll likely see Sun doing most of the work in the OpenSolaris codebase. Don't take that to mean that Sun will not be elevating members of the community to higher levels of status, even committers on various sub-projects. But, like in the Apache community, developers will need to earn their position in the projects. That all makes sense to me.

All in all, I learned quite a bit, and I'm glad I could make it. Thanks to all the community members who came out.

Wednesday April 13, 2005

Solaris 10 BOF at Usenix

Last night was one of two BOFs Sun is holding/sponsoring [i.e. free alcohol :) always guaranteed to make the majority of a long-hair unix crowd happy].

There were 8 or so of us from Sun there. I think we're all bloggers actually. Dan Price (the main speaker), Bart Smalders, Alan Duboff, David Bustos (non-blogging heathen as John would say), Richard McDougall, Liane Praza, Spencer Shepler and some less-than-real engineering types myself and John Clingan. [John's more of an engineering type than I am. I just try to keep up with all the brains].

Dan did an awesome job, especially in light of the litany of questions thrown at him by 80 some people, with no particular order. It must be Dan's familiarity with the Kernel that allows him to handle interrupts so gracefully. They were great questions. I just emailed Dan the same, but he brings sooo much credibility to these kinds of discussions because he's spent the last year or two talking to customers and building things into Solaris based on customer feedback. For those who don't know, Dan was a big part of the Solaris Containers team (i.e. zones and resource management, amongst other things).

I'll try to write more later (as I have to run off and see a customer here in a few), but in no particular order here is my stream of interesting tidbits:

• Popularity of alcohol - Dan had to interrupt himself when the cart came in
• Oracle and DTrace - we'd love to see Oracle more instrumented with DTrace so customers can correlate different Oracle events all the way down into things happening in the kernel (i.e. I/O and memory)
• VMware support wanted - mention of VMware host support by one member of the audience brought APPLAUSE from the rest of the audience-- [please!!! contact VMware about this!!! Sun would love to see this, but it's paying customers that will get it going, not Sun asking VMware]

Update to come later, but join us at tonight's BOF if you can! It's free!

Update: There were some comments to the effect that the two hour talk (that finally we started dimming the lights on after 3.5 hours) was great because Dan was "super honest and way more geeks get more out of this than being in the EBC with Jonathan Schwartz" [sorry Jonathan-- their words, not mine :)]. Dan did what sometimes I will do when talking with customers one on one. Admit when your (or your companies) actions are, looking backwards, not the greatest or can't do everything you want to do. Our customers are people too and if you're honest with them about how things really work and show them why you're spending your limited resources where you're spending them, they may not like it, but at least they know you're showing up and taking the feedback.

As Dan/Bart proved last night, they do in fact take feedback.

That's one reason that Tim Bray and the other folks who pushed for blogging we owe a debt of gratitude to.

Also updated to added Spencer Shepler of NFSv4 fame-- sorry Spencer :)

Tuesday February 01, 2005

Great deal on Techbargins.com

One of my colleagues, Daryl Doami, pointed to this deal on Techbargins.com.

Where else can you get something that has had billions of R&D spent on it for free?

Thursday September 16, 2004

ZFS Usage the /. community thought of

This made me laugh out loud, so I had to share it.