mrj's Weblog

Congratulations to Intel on their new Intel Xeon 5500 series. It is a truly remarkable CPU, and in my opinion, will be looked back on as a game changer in the industry.

There are features which are easy to see, such as the integrated memory controllers and the Quick Path Interconnects (QPI) (which connect the CPUs and IO bridges to each other).

But, as you would expect, they also continue to improve the guts of the CPUs. For example, with the 5500 series and associated IO chipsets, comes virtualization improvements including better virtualized CPU performance, and the ability to safely passthrough an IO device to a virtualized guest.

With Intel's help, we backported a lot of the new functionality from xen-unstable.hg into OpenSolaris xVM in April of 2008, including the big ones, Extended Page Table (EPT) support and Virtual Processor IDs (VPID). So we've been ready for a while :-)

With the EPT support, you can bypass the shadow page code in the hypervisor for fully virtualized guests. This gives you a nice performance improvement, and has the added benefit of a not having to run a *lot* of complex, and occasionally, buggy code.

I've just scratched the surface of the virtualization improvements in the Intel Xeon 5500 series. For folks who enjoy CPU technology, we live in exciting times.

I've been playing around lately to see how small of a Solaris image I can get that will boot to a shell prompt... It turns out, it can get pretty small.. And it boots to the prompt in ~ 1 second... Although there's not a lot you can do with it :-)

SunOS Release 5.11 Version onnv-3.3-mrj 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
WARNING: Last shutdown is later than time on time-of-day chip; check date.
strplumb: failed to initialize drv/dld
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/ramdisk:a             38255   15146   19284    44%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                 1768156       0 1768156     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
# du -sk *
31      boot
952     dev
1       devices
67      etc
6087    kernel
4849    lib
8       lost+found
1931    platform
11277   proc
122     sbin
1441    system
1       tmp
59      usr
3       var
# 

Following up on Bart's comment, you can certainly use -R to perform the same operation... Not sure why I didn't think that could be used for pkg set-authority, but it can..

Funny since I use -R for my custom opensolaris builds..

root@unknown:~# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0d0s0       491522  367690   74680    84%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  767856     336  767520     1%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap3.so.1
                      491522  367690   74680    84%    /lib/libc.so.1
fd                         0       0       0     0%    /dev/fd
swap                  767520       0  767520     0%    /tmp
swap                  767536      16  767520     1%    /var/run
root@unknown:~# pkg list | wc -l
      65
root@unknown:~# ps -ef
     UID   PID  PPID   C    STIME TTY         TIME CMD
    root     0     0   0 06:52:45 ?           0:01 sched
    root     1     0   0 06:52:46 ?           0:00 /sbin/init
    root     2     0   0 06:52:46 ?           0:00 pageout
    root     3     0   0 06:52:46 ?           0:00 fsflush
    root     7     1   0 06:52:47 ?           0:02 /lib/svc/bin/svc.startd
    root     9     1   0 06:52:47 ?           0:27 /lib/svc/bin/svc.configd
    root   549     1   0 10:58:06 ?           0:00 /usr/lib/inet/inetd start
    root   201     1   0 06:53:19 ?           0:00 devfsadmd
  daemon   293     1   0 06:53:40 ?           0:00 /lib/crypto/kcfd
   dladm    15     1   0 06:52:48 ?           0:00 /sbin/dlmgmtd
    root   298     1   0 06:53:44 ?           0:00 /usr/lib/picl/picld
    root   198     1   0 06:53:19 ?           0:00 /usr/lib/sysevent/syseventd
    root   554   552   0 10:58:06 ?           0:00 /usr/lib/saf/ttymon
    root   552     7   0 10:58:06 ?           0:00 /usr/lib/saf/sac -t 300
    root   580     1   0 10:58:08 ?           0:00 /usr/lib/ssh/sshd
  daemon   525     1   0 06:57:48 ?           0:00 /usr/sbin/rpcbind
    root   544     1   0 10:58:03 ?           0:00 /usr/sbin/nscd
    root   512     1   0 06:56:07 ?           0:00 /sbin/dhcpagent
    root   553     1   0 10:58:06 ?           0:00 /usr/lib/utmpd
    root   571     7   0 10:58:08 console     0:00 -bash
    root   376     1   0 06:53:54 ?           0:00 /usr/sbin/cron
    root   667   571   0 11:03:32 console     0:00 ps -ef
    root   567     1   0 10:58:08 ?           0:00 /usr/sbin/syslogd
root@unknown:~#

Anyway, here is the sequence using a -R.. This works for 99.x% of the cases.. But I would expect to fail for the same cases lu will.. i.e. say you need a new version update_drv, etc. For those cases the chroot will get you through it with some skilled sequencing.. Of course, the chroot approach can have it's own set of problems :-)

beadm create snv109
beadm mount snv109 /mnt
pkg -R /mnt set-authority -O http://pkg.opensolaris.org/dev opensolaris.org
pkg -R /mnt refresh
pkg -R /mnt install SUNWipkg
pkg -R /mnt install entire@0.5.11-0.109
bootadm update-archive -R /mnt
beadm umount snv109
beadm activate snv109

One of the things I have found over the years to be helpful as a Solaris kernel developer is liveupgrade (lucreate(1M)).

For example, I will usually have 3 boot environments (BE) on my systems at one time. The first BE usually has the base build that my gate is a child of. The second BE is my test BE which I BFU, and generally seem to kill quite often. And the third BE is an old build so I can go move my test BE to the old build all the way up to the most recent build (luupgrade(1M)).

OpenSolaris uses beadm which is a nice replacement to the lu commands. I was a little disappointed when I first started using it because I didn't think it was as flexible as I wanted.. But after playing with it for a while, I have a setup which I'm pretty happy with at the moment.

What I want to be able to do is to have the stock OpenSolaris BE, which gets upgraded at major releases. But I also want to have BEs which are based off of various development builds, and test BEs which I can BFU, etc. This is really nice since these will be COW based clones due to zfs.

So the question was how to update an alternate BE and at the same time, not modify the current BE. i.e. not having to update the authority, upgrade SUNWipkg, etc. The solution is pretty simple, chroot...

Here's how I create a b108 BE on a stock 2008.11 system, while ensuring I keep the 2008.11 bits unmodified. I can switch back to the stock OpenSolaris bits at any time with a beadm activate opensolaris.

beadm create snv108
beadm mount snv108 /mnt
mount -F proc /proc /mnt/proc
chroot /mnt
pkg set-authority -O http://pkg.opensolaris.org/dev opensolaris.org
pkg refresh
pkg install SUNWipkg
pkg install entire@0.5.11-0.108
bootadm update-archive
exit
umount /mnt/proc
beadm umount snv108
beadm activate snv108