It Must Be Time for Tea

Thursday August 11, 2005

Send In the Builds

Send in the builds
There ought to be builds
Why aren't they here?[1]

It's been a hectic couple months since the OpenSolaris launch. Some folks have been asking why isn't Sun producing the regular biweekly builds like we said we would, and I'd like to let everyone know what's happening. First, some background:

The Solaris group puts out a numbered build every two weeks. The ON gatekeepers take a snapshot of the ON gate every other Monday evening. This gives them a week to deal with any problems found in Pre-Integration Testing (PIT). They then deliver packages built from the snapshot into the WOS. The Release Engineering staff produces ISO images from the WOS packages. These images then show up in places like Solaris Express.

What I did for the OpenSolaris Pilot, and what I planned to do after the launch (until we get a source code management system installed), was to merge the biweekly ON snapshot into the workspace that I use to generate the OpenSolaris deliveries. During the Pilot this typically took a couple days per delivery, and most of that was waiting for builds to complete. So I figured I could get a bunch of other work done after the launch while still producing the OpenSolaris deliveries at regular intervals.

Silly me.

The short explanation is that I've been extremely busy, and producing the deliveries hasn't been at the very top of my priority list. Instead, I've been working with some of the other core OpenSolaris team to improve the automation of the delivery process and to make it so that others will be able to produce the deliveries. To make things worse, I just moved and bought a house, so I'm having to take vacation time a fair amount right now.

Since it's been a couple weeks since the last delivery, I'm going to back-burner that other work and start integrating the crypto code that Darren Moffat and Dan McDonald have worked to make available. In parallel, Steve Lau is going to start on the build 20 delivery.

So my best guess is that the next delivery will be somewhere around 17-20 August. I'm not sure how much of the crypto code the next delivery will contain. I plan to work that out with Steve next week.

Steve has also been working on automation for the delivery process. So I'm hoping that soon after the next delivery we'll be able to get on a more regular schedule.

During the Pilot I tried to send out regular status announcements (weekly, if I remember correctly). This kept everyone informed as to when the next delivery was likely to happen. And for the folks external to Sun, it provided a useful heartbeat, showing that the Sun folks were still there and still working on things. Cyril Plisko has suggested that I resume regular status updates. Given the erratic delivery schedule since the launch, I think he has a point. But I want to also consider venue and frequency. I'd like for opensolaris-code to stay as technical as possible. opensolaris-discuss is an alternative, but it has a lot of traffic, and I don't want to add to that unless the postings are really useful. So right now I'm thinking of setting up a biweekly schedule and then only sending out mail if it looks like we'll miss the advertised delivery date.

[1] With apologies to Stephen Sondheim.

Technorati tags: OpenSolaris

Thursday June 30, 2005

Useful URLs for OpenSolaris Newbies

At the user group meeting last Tuesday, Keith Wesolowski and I gave a presentation on using BFU and how to recover if you brickify your system. Actually, Keith did most of the work. But I did make up a short handout with some useful URLs, and someone suggested I put them up online.

So if you weren't at the meeting, the links were for

• the 20050612 release notes. New deliveries will have the same URL except the date will change. Please read the release notes before playing with any new download--this is where we warn you about changes to procedures (which we hope to minimize) and gotchas (which we really hope to minimize).
• the Developer's Reference
• the opensolaris-help archives
• Rich Teer's article on Building and Installing OpenSolaris (part 1)
• the Studio 10 (compiler) downloads
• compiling OpenSolaris with gcc

Technorati tags: OpenSolaris

Thursday June 16, 2005

The Hitchhiker's Guide to OpenSolaris

OpenSolaris: Sun Microsystems is open sourcing its Solaris operating system. The initial delivery is based on an intermediate version of the Nevada release of the Solaris system. It contains the OS/Net (aka ON) consolidation. Don't Panic! Additional pieces of Solaris will be open sourced in the future. The OpenSolaris team welcomes your feedback on what parts of Solaris should be open sourced next.

The Hitchhiker's Guide to OpenSolaris, despite being banged out in a couple hours and not being nearly as funny as Douglas Adams's Hitchhiker's Guide to the Galaxy, scores over the older and more pedestrian OpenSolaris Developer's Reference in two important respects. First, it is much smaller. Second, it contains the words ``Don't Panic'' in large letters, scattered through a variety of entries [1].

Consolidation: The Solaris operating system (also known as the WOS, or Wad Of Stuff [2]) consists of several consolidations, such as OS/Net (ON), X, GNOME, Admin, Install, SFW, and Devpro. Each consolidation is built as a unit and typically delivers multiple packages.

There are a number of implications of this structure. One is that different consolidations have different policies for how they build and test their code. In fact, the different consolidations don't even have uniform policy for where they put their source.

You also need to understand that some stuff that you might expect to be in ON is, in fact, in a different consolidation.

The upshot is that most Solaris kernel engineers have no idea where to find the source for the package tools or libm.

ON: This is the only consolidation in OpenSolaris at this time. The open code in ON consists of 27,916 files and 9,185,408 lines of text (as of Opening Day). The code is broken down into the kernel (uts[3]), basic libraries (lib), basic command-line utilities (cmd), some developer tools that aren't part of any Solaris product (tools), and boot code (grub, psm, and stand). User-level header files mostly live in head, though they may also be found in lib. Kernel header files live in uts and are used by both kernel and user-space code. Package definitions are in pkgdefs. Common code that is shared between the different trees is in common. See the Developers Reference for more details.

The ucbcmd, ucbhead, and ucblib directories contain compatibility libraries and utilities for SunOS 4.x. Despite the name, these trees are not general-purpose BSD compatibility source, though there are rumors of developers who would like to change that.

Some fairly large chunks of ON are missing in the OpenSolaris tree. Don't Panic! A lot of what's missing is cryptography source or things that won't build unless the crypto source is in the source tree. The OpenSolaris team was hoping to have the crypto code available at launch. But for various logistical and legal reasons, they were not able to make it happen. With a couple small exceptions, this entire wad of code should be available in the next delivery.

Another chunk of code is missing for legal reasons. The writer of this guide is not a lawyer, does not deal with lawyers, and wishes he didn't have to deal with all the secrecy that the legal issues seem to require.

A third chunk of code is missing because it will not build, because it depends on something in one of the first two chunks. Sun's ON development team is somewhat fanatical about clean code, and that includes not having random bits of stuff floating around in workspaces. The OpenSolaris team is considering making this third chunk available separately. Though after the crypto code is available, this third chunk should get a lot less interesting.

There is one final chunk of code that is missing, sort of. This is code that belongs to consolidations other than ON. Some commonly requested items from this chunk include libm, the packaging tools (pkgadd, etc.), the Solaris installer, and various open source utilities that are delivered via the SFW consolidation.

opensolaris.org has a list of the things that fall into the first three categories.

I own the workspace that OpenSolaris is currently built from. I've been at Sun for almost 13 years. For most of that time I worked on NFS, which is just a small piece of ON. One of the reasons I signed up for OpenSolaris was to learn more about the rest of ON. It's been educational and fun.

Source Code Management: The initial source code delivery system for OpenSolaris is biweekly tarballs. Don't Panic! Sun realizes that this isn't tenable for serious development work. The original plan was to (initially) provide a one-way bridge from Teamware (Sun's SCM tool) to CVS. But CVS doesn't handle renames, which are rather common in ON. So the current plan is to provide a one-way bridge from Teamware to Subversion. As described in the Development Process Plan, external developers will submit diffs to Sun engineers, who will apply the changes to the internal workspace, which will then be reflected externally via the bridge.

Unfortunately, the Teamware/Subversion bridge is currently not owned by anyone. If you're interested in working on this, please speak up on the Tools discussion list.

The long-term goal is for the master workspace to be external, and for external developers as well as Sun engineers to have commit privileges.

Nevada: Nevada is the code name for the Solaris release that will follow Solaris 10 (S10). Why the name Nevada was chosen, instead of, say, Fred or--to pick something totally absurd--S11, is a story that will have to wait for a future edition of this guide.

I was hoping to get this written for Opening Day, but like the rest of the OpenSolaris team, I was pretty busy with the launch itself. I'm hoping to add more entries as time permits.

[1] This will have to do, at least until the W3C adds <friendly> to HTML.

[2] I'm not making this up.

[3] UNIX Timesharing System

Technorati Tag: OpenSolaris

Technorati Tag: Solaris