Philosophy

I believe in open communication and open development.

• Communication
  Our (Sun's) communication with the OpenSolaris community has been poor. We need to be clearer about both making unilateral decisions (explaining why we decided whatever) and sharing information with the community in general. People who have been to any number of Solaris all-hands meetings in the last 11+ years can attest that I am one of the chief questioners, always trying to keep the lines of communication with management as open as possible.
• Development
  Solaris has been open source since 2005, but for it truly to flourish, it needs to have open development as well. The sponsor program has been a band-aid while we worked on opening up our tools and processes and moving our source repositories outside the firewall, all prerequisites for open development, but the community is thirsting for more, and we need to provide it. This in my view should be our #1 priority to both keep current community members interested and attract new community members. As ON tech lead, I have actively sought (and found) people to work on migrating our tools from being Teamware-based to being Mercurial-based, and I plan to continue driving both this migration, then the move of the ON source repository outside the firewall, then whatever else will be needed so we can have true open development.

Employment history

• Sun, December 1996 - present
  • ON C-team tech lead, August 2007 - present: lead engineer for the ON consolidation, responsible for overseeing all changes made to the source base
  • Network Auto-Magic (NWAM) project lead, February 2005 - August 2007: led story-boarding, requirements, architecture, design and implementation of Phase 0 and initial implementation of Phase 1.
  • Zones development, April 2002 - February 2005: wrote zonecfg, libzonecfg, most of zoneadm and large parts of zoneadmd, plus worked on the booting sub-system; brought libtecla to Solaris for use in zonecfg.
  • ON gatekeeper, October 1999 - April 2002: lead administrator of gate machines, also core C-team member, responsible for keeping up the quality of the ON consolidation
  • Sendmail, December 1996 - October 1999 (still have responsibility for it, but since late 1999 it has been ¼ time or less): upgraded ancient Solaris version, kept Solaris up-to-date, eventually deprecated all old Sun hacks or got them integrated into the sendmail.org version. Contributed many enhancements, including anti-spam features and got Solaris to a unified config file. FAQ maintainer since 1997.
• InReference, August 1995 - November 1996
  Wrote a Usenet search engine (code: Python and C).
• HP, June 1989 - August 1995
  Supported various network applications and protocols on MPE and HP-UX. Added 8BITMIME support to sendmail, offered patch to Eric Allman.

Education

• MS, UC Santa Barbara (Electrical & Computer Engineering), 1989
• BS, Caltech (Engineering & Applied Science), 1987

Personal

• Married since 1992, 10-year-old daughter, (almost) 7-year-old son
• Physical fitness enthusiast: running, ultimate frisbee, cycling, hiking
• I serve as an usher at our church most Sundays
• We live in San José, CA (USA)

My old desktop (with SCSI disks) died, and I'd been limping along on a loaner until the new one (with SATA disks) arrived a few days ago. The steps needed to migrate the data from one to the other turned out to be remarkably simple, so I thought I would share them in case anyone else ever runs into this. The old box had a single pool:

NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
space                    68G   16.6G   51.4G    24%  ONLINE     -

with two file-systems therein:

NAME                   USED  AVAIL  REFER  MOUNTPOINT
space                 16.6G  50.3G    99K  /export/home
space/jbeck           12.2G  50.3G  12.2G  /export/home/jbeck
space/local           4.25G  50.3G  4.25G  /usr/local

First, on the new box, I ran:

% zpool create space mirror c2d0 c3d0

Then:

• I temporarily tweaked /etc/ssh/sshd_config and /.ssh/authorized_keys to allow root from the old box to ssh to the new without being prompted for a password (not sure if this step was needed or not)
• Since my desktop is also my mail server, I disabled sendmail to make sure that no mail would come in during the migration
• On the old box I ran:

% zfs snapshot -r space@today
% zfs send space/jbeck@today | ssh newbox zfs recv -d space
% zfs send space/local@today | ssh newbox zfs recv -d space

The two send/recv runs took roughly 30 and 10 minutes respectively, much less than the almost two hours that the back-to-back tar method used to take to do this. Finally, on the new box, I ran:

% zfs set mountpoint=/export/home space
% zfs set mountpoint=/usr/local space/local
% zfs set sharenfs=on space/jbeck space/local

And that was it! I'm a happy ZFS user, but note that Matt told me that he is working on an RFE to make the above even simpler:

I'm working on some enhancements to zfs send/recv that will simplify this even further, especially in cases where you have many filesystems, snapshots, or changed properties. In particular, you'll be able to simply do:

# zfs snapshot -r space@today
# zfs send -r -b space@today | ssh newbox zfs recv -p -d newpool

The "send -b" flag means to send from the beginning. This will send a full stream of the oldest snapshot, and incrementals up to the named snapshot (eg, from @a to @b, from @b to @c, ... @j to @today). This way your new pool will have all of the snapshots from your old pool.

The "send -r" flag means to do this for all the filesystem's descendants as well (in this case, space/jbeck and space/local).

The "recv -p" flag means to preserve locally set properties (in this case, the mountpoint and sharenfs settings).

For more information, see RFEs 6421959 and 6421958, and watch for a forthcoming formal interface proposal.

Nice work, ZFS team.

People a little older than me (I was 3 at the time) will remember the "black power salute", one of the most famous photographs in sports history. In the 1968 Olympic Games in Mexico City, while the national anthem played during the awards ceremony of the men's 200-meter run, gold medalist Tommie Smith and bronze medalist John Carlos stood with their heads bowed, each with a black-gloved fist held straight up. Smith raised his right fist to represent black power, Carlos his left to represent black unity, and both had taken off their shoes and just wore black socks to represent black poverty. The silver medalist, Peter Norman from Australia, wore a pin on his sweat jacket as a show of unity. Smith and Carlos were sent home from the games as a result of their stand, and were harassed and threatened when they got back to America.

Well, that was 1968. As years went by, more and more people saw their stand in a more positive light, understanding that they were saying, in a dignified way, that it was not right for Black Americans to be given equal treatment only when convenient for the country, such as in war or in sports, but treated as second class citizens otherwise. So by 2005, Smith and Carlos tend to be viewed as heroes more than anything else, and heroes overdue for recognition at that.

As it turns out, both Smith and Carlos (and Lee Evans, who won gold in the 400-meter run that year, and participated with them in their planning of "the stand") went to San Jose State University, and a few years ago a student there learned about their story and thought the university should do something to commemorate their stand. It took a few years, but last night (37 years and 1 day after their courageous stand), there was a ceremony on campus at San Jose State, with Smith, Carlos, Norman and Evans all in attendance, where a 20-foot statue was unveiled. I was on hand for the event, and took some pictures.

In the cropped picture above, Carlos is on the left in the bluish suit with the white goatee, Smith is on the right in the greenish suit, Norman is standing behind Carlos' left shoulder, and Evans is standing behind Smith's left shoulder; San Jose Vice Mayor Cindy Chavez is presenting them with an award. In the uncropped version of the picture, you can see how big the (still veiled) statue is.

The statue, seen unveiled, is magnificent. The skin and hair and socks are bronze, while the sweat suits are ceramic tiles. Norman's spot on the dais was intentionally left empty, to make the statue interactive, so people could pose "taking a stand".

I think I was about 7 when I first learned about "the stand", probably during the 1972 Olympic Games in Munich. I remember thinking that they must have been very brave. As I got older, and eventually ran track in high school and college myself, I came to think of them more as heroes, the ultimate in track: winning a gold medal and doing something noble with one's moment of glory. It is hard to put into words how uplifting it was to see these heroes in the flesh, and to hear them speak. Smith and Carlos both credited God for helping them get thru their trying times and both urged the young people present at the ceremony to take a stand and do what's right. Amen, brothers.

As soon as Zones integrated into Solaris 10, we started getting some RFEs for ways to make it better. One of them was to add command history to zonecfg; now that OpenSolaris has been launched, I thought I would share how I did it.

Some existing libraries were suggested for enabling this; after studying them, I recommended libtecla, written by Martin Shepherd, a staff scientist / programmer in the Astronomy department at Caltech, because of its feature set (command-line editing and history, highly customizable, etc.), quality, and license.

Getting libtecla ported to Solaris was trivial, as it already compiled and linked fine. I just had to massage it into the Solaris format, which did not involve any significant code changes, but mainly Solaris Makefiles, lint library stubs, etc. These can be found under usr/src/lib/libtecla for the curious.

The more interesting work was tweaking zonecfg to use libtecla, which included the following steps, which are listed here in a way intended to make it easy to follow for other people wishing to do the same with their application(s):

• header file:
  #include <libtecla.h>
• some constant definitions:

  #define	MAX_LINE_LEN	1024
  #define	MAX_CMD_HIST	1024
  

• global variable:
  static GetLine *gl; /* The gl_get_line() resource object */
• initialization: near the beginning of main():

  	if ((gl = new_GetLine(MAX_LINE_LEN, MAX_CMD_HIST)) == NULL)
  		exit(Z_ERR);
  

  Note that Z_ERR is a constant with the value of 1, but the important thing for the general case is that it is some non-zero value.
• (optional) customization of tab-completion:

  	if (gl_customize_completion(gl, NULL, cmd_cpl_fn) != 0)
  		exit(Z_ERR);
  

  Note that in the case of zonecfg, cmd_cpl_fn() is not particularly interesting, mainly consisting of lots of calls to cpl_add_completion() to cover various cases in the grammar; see the cpl_complete_word(3TECLA) man page for details.
• clean-up: near the end of main():

  	(void) del_GetLine(gl);
  

• the heart of it:
  If your program has some sort of construct like:

  	for (;;) {
  		...
  		prompt = prompt_value();
  		print_prompt(prompt);
  		line = read_input();
  		if (line == NULL)
  			break;
  		handle_input(line);
  		...
  	}
  

  then it would be rewritten thus:

  	for (;;) {
  		...
  		prompt = prompt_value();
  		line = gl_get_line(gl, prompt, NULL, -1);
  		if (gl_return_status(gl) == GLR_SIGNAL) {
  			gl_abandon_line(gl);
  			continue;
  		}
  		if (line == NULL)
  			break;
  		handle_input(line);
  		...
  	}
  

  In the case of zonecfg, however, it was slightly more complicated because of its use of lex(1)/yacc(1), so I needed to convert the char *line into the FILE *yyin, then call yyparse(). So the code became:

  	for (;;) {
  		...
  		prompt = prompt_value();
  		line = gl_get_line(gl, prompt, NULL, -1);
  		if (gl_return_status(gl) == GLR_SIGNAL) {
  			gl_abandon_line(gl);
  			continue;
  		}
  		if (line == NULL)
  			break;
  		(void) string_to_yyin(line);
  		yyparse();
  		...
  	}
  

  where the string_to_yyin() function is thus:

  static int
  string_to_yyin(char *string)
  {
  	if ((yyin = tmpfile()) == NULL) {
  		zone_perror(execname, Z_TEMP_FILE, TRUE);
  		return (Z_ERR);
  	}
  	if (fwrite(string, strlen(string), 1, yyin) != 1) {
  		zone_perror(execname, Z_TEMP_FILE, TRUE);
  		return (Z_ERR);
  	}
  	if (fseek(yyin, 0, SEEK_SET) != 0) {
  		zone_perror(execname, Z_TEMP_FILE, TRUE);
  		return (Z_ERR);
  	}
  	return (Z_OK);
  }
  

  where execname is a global variable, zone_perror() is a perror(3C)-like function, and Z_TEMP_FILE is a constant which maps to an error message indicating "Problem creating temporary file", localized appropriately.

For those who would like to see all of zonecfg.c in context; it can be found at usr/src/cmd/zonecfg/zonecfg.c.

Technorati Tag: OpenSolaris
Technorati Tag: Solaris

Most people who have spent any time on any version of Unix know that "rm -rf /" is about the worst mistake you can make on any given machine. (For novices, "/" is the root directory, and -r means recursive, so rm keeps deleting files until the entire file system is gone, or at least until something like libc is gone after which the system becomes, as we often joke, a warm brick.) Well a couple of years ago one Friday afternoon a bunch of us were exchanging horror stories on this subject, when Bryan asked "why don't we fix rm?" So I did.

The code changes were, no surprise, trivial. The hardest part of the whole thing was that one reviewer wanted /usr/xpg4/bin/rm to be changed as well, and that required a visit to our standards guru. He thought the change made sense, but might technically violate the spec, which only allowed rm to treat "." and ".." as special cases for which it could immediately exit with an error. So I submitted a defect report to the appropriate standards committee, thinking it would be a slam dunk.

Well, some of these standards committee members either like making convoluted arguments or just don't see the world the same way I do, as more than one person suggested that the spec was just fine and that "/" was not worthy of special consideration. We tried all sorts of common sense arguments, to no avail. In the end, we had to beat them at their own game, by pointing out that if one attempts to remove "/" recursively, one will ultimately attempt to remove ".." and ".", and that all we are doing is allowing rm to pre-determine this heuristically. Amazingly, they bought that!

Anyway, in the end, we got the spec modified, and Solaris 10 has (since build 36) a version of /usr/bin/rm (/bin is a sym-link to /usr/bin on Solaris) and /usr/xpg4/bin/rm which behaves thus:

[28] /bin/rm -rf /
rm of / is not allowed
[29] 

I post to Usenet occasionally about Zones, more often about sendmail. More to follow...