I just pushed the changes that add LZMA to (Open)Solaris
and also allow lofi(7D) to use LZMA as one of the
supported compression algorithms.

On an snv_111 machine, here's what you will see -

Usage: lofiadm -a file [ device ]
       [-c aes-128-cbc|aes-192-cbc|aes-256-cbc|des3-cbc|blowfish-cbc]
       [-e] [-k keyfile] [-T[token]:[manuf]:[serial]:key]
       lofiadm -d file | device
       lofiadm -C [gzip|gzip-6|gzip-9|lzma] [-s segment_size] file
       lofiadm -U file
       lofiadm [ file | device ]

So, if you take large'ish file and compress it with gzip
and lzma, the size difference is quite noticeable.

contraption# du -h solaris.orig
 2.2G   solaris.orig
contraption# lofiadm -C lzma solaris.orig
contraption# du -h solaris.orig
 555M   solaris.orig
contraption# lofiadm -U solaris.orig
contraption# lofiadm -C gzip solaris.orig
contraption# du -h solaris.orig
 702M   solaris.orig

With LZMA support now available for both userland and
kernel consumers, it should be very easy for other Solaris
utilities (zfs?) to provide support for it.

The OpenSolaris LiveCD contains hsfs filesystems that
are compressed with lofi compression, primary among
these are solaris.zlib which maps to /usr and solarismisc.zlib
which maps to /mnt/misc. 

The /usr filesystem contains essential components to
allow for the LiveCD to boot into a desktop. As a result
the layout of this filesystem is carefully ordered such
that accesses are sequential as opposed to being completely
random. This careful ordering of contents allows for the
LiveCD to boot into a desktop in a reasonable amount of
time (~3 minutes on most systems).

Since hsfs is the only OpenSolaris filesystem that allows
files to be ordered a certain way via the specification of
'-sort' flag to mkisofs(8), it was the obvious choice for
the /usr filesystem. And, the primary reason why compressed
lofi is used for the LiveCD as opposed to, say, ZFS or dcfs(7FS).

More details can be found in Moinak's slides here.

I recently wrote that LZMA has been used to pack more languages
onto the LiveCD. Here are some charts that show how LZMA
stacks up against someof the other popular compression algorithms.
(apologies for the poor image quality, open in another window 
for a clearer image)



These tests were run on a LiveCD archive using 7za(1). 
As you'll note, the compression ratio provided by LZMA is
about 35% better than gzip-9. However, LZMA is more
CPU intensive and as a result the compression and
decompression speed is slower than the alternatives.

So, for some use cases the cpu versus compression
tradeoff might make LZMA unsuitable but for the LiveCD
use case, it is reasonable provided we architect our
solution such that the decompression speed isn't a bottleneck
(Compression speed isn't a problem for the LiveCD architecture)

The OpenSolaris 2008.05 release that is going to come
out sometime in May is going to have two versions of
the same LiveCD, one with a limited set of languages and
locales and another one with a more fuller set of languages.

One of the big challenges with creating a LiveCD with a
full set of languages was that there was limited amount
of available free space on the CD to allow for including
all the languages. How do you cram more stuff on the CD?
Compress it harder, I say! Even better, compress it with
LZMA!

The OpenSolaris kernel did not have an in-kernel implementation
of LZMA that could be taken advantage of (why do we need an
in-kernel implementation, I'll answer that in a separate blog entry). 
So, in our quest to provide one, we started looking at the LZMA SDK. 
Some of the challenges with porting the source from this SDK to the  
OpenSolaris kernel were that our lawyers were not amenable to the licensing 
terms and the compression code was all written in C++ (which, 
for the uninitiated, is strongly desisted in the kernel).

If you've ever dealt with lawyers you'll be quick to spot
that the licensing can be particularly troublesome. It was. 
But only until we contacted with author of LZMA, Igor Pavlov.
Igor was not only willing to relicense the source code under
CDDL (which would ofcourse be agreeable to the lawyers) but
also willing to re-write the compression code in C. And, he 
did that in just a matter of couple of weeks -  truly outstanding. 
That, to me, is the power behind open source and the sharing 
opportunities it provides for the broader good.

So, thank you Igor for an excellent compression algorithm
in LZMA and thanks for all your assistance in making the
OpenSolaris 2008.05 release what it is. We look forward to
working with you in the future too.

At the last ATLOSUG  meeting couple of days ago, Ryan 
Matteson  talked about iSCSI in OpenSolaris. He even did a demo. 
I never realized it was just *that* easy to setup the target 
and the initiator except ofcourse for setting ACLs which 
seemed like a royal PITA.

Ryan's presentation can be found here .

Couple days back I happened to notice an article on the Sun internal webpage the summary
of which said something like,

"Sun Provides Sun Ray Solution to Chitkara Institute

At Chitkara, Sun has installed four high end SUN servers with the best in class Solaris 
operating system. 300 thin clients are spread all over the campus connected on Nortel 
switches through a Fibre backbone.

The Sun Ray architecture is comprised of Sun Ray thin clients and Sun servers. By 
moving resources to a central location on the network and removing complexity, a much 
more flexible and easy-to-maintain environment is achieved."

(Apparently, the actual article appeared in Chandigarh, India, Tribune News Service)

And, the first thought that came to my mind is,"Chitkara? Is it the same as the Chitkara I 
knew when I was in high school in India? If so, there's an Institute by that name?" Once I 
read the entire article, it was clear to me that it was the Chitkara I used to know.
The same guy who taught Math classes out of his house (couple miles from my parents' 
house) when I was about to get into engineering school (around 1994). And, yes, it had 
been turned into a full scale Institute or rather a group of Institutions. Wow!

This was an amazing news from my perspective on a couple of counts. 

First, one of the well known educational institutions in the city of Chandigarh, where most 
of the population has typically been Microsoft savvy (that is true for majority of India as well), 
had adopted a Sun solution running Solaris! This is a huge acknowledgement of the fact that 
Solaris has come a long ways in terms of being more approachable and user friendly. Mind 
you, there still is a lot of a catchup still left to do but it is approachable enough for students in 
this case.

Secondly, there were going to be Sun Rays in classrooms. Just like a bunch of other educational 
institutions around the world have adopted. I've always thought of the Sun Ray technology as 
one of the best pieces of technology created at Sun. And, where better to put them than in the 
classrooms -- no need to have 300 different workstations that need to be maintained on a regular 
basis and draw a significant amount of power, the students can't do crazy stuff on 'em and they 
can't even physically break the damn thing coz it's just a dumb thin client (well, for the most part). 
Just ideal! Plus, you get all the benefits of a carrier grade OS in Solaris.

And, thinking about it for a moment - it doesn't even have to be Solaris. You could very well be 
serving up Windows or even Linux sessions (how long will it be before a Mac OSX virtual client 
is available?). So, you could very well bring up a screen on the Sun Ray that gives the users the 
ability to choose from the available operating environments -- they could choose whichever they 
like potentially based upon the kind of activity they wish to undertake.

Having been a long time user of the Sun Ray at Home solution, I wonder how long will it be 
before service providers like Comcast, in addition to providing you internet access, also serve up 
virtual desktops for an additional charge. The users benefit because they don't have to be sysadmins 
anymore and the service provider benefits because it gets to deliver value added services.

So, really how long will it be before we hit a tipping point for the Sun Ray technology?

md_ops_t stripe_md_ops = {
 stripe_open, /* open */
 stripe_close, /* close */
 md_stripe_strategy, /* strategy */
 NULL, /* print */
 stripe_dump, /* dump */
 NULL, /* read */
 NULL, /* write */
 md_stripe_ioctl, /* stripe_ioctl, */
 stripe_snarf, /* stripe_snarf */
 stripe_halt, /* stripe_halt */
 NULL, /* aread */
 NULL, /* awrite */
 stripe_imp_set, /* import set */
 stripe_named_services
};

 /*
 * Fixup
 * (1) locator block
 * (2) locator name block if necessary
 * (3) master block
 * (4) directory block
 * calls appropriate writes to push changes out
 */
 if ((err = md_imp_db(setno)) != 0)
 goto cleanup;

 /*
 * Create set in MD_LOCAL_SET
 */
 if ((err = md_imp_create_set(setno)) != 0)
 goto cleanup;

struct frame {
        long    fr_local[8];            /* saved locals */
        long    fr_arg[6];              /* saved arguments [0 - 5] */
        struct frame    *fr_savfp;      /* saved frame pointer */
        long    fr_savpc;               /* saved program counter */
#if !defined(__sparcv9)
        char    *fr_stret;              /* struct return addr */
#endif  /* __sparcv9 */
        long    fr_argd[6];             /* arg dump area */
        long    fr_argx[1];             /* array of args past the sixth */
};

000002a1012203d1 vpanic(1295800, 7aabd868, 7aabd880, 851, 2400, 2a1012210fc)
000002a101220481 assfail+0x74(7aabd868, 7aabd880, 851, 18c6000, 1295800, 0)
000002a101220531 nfs4_make_dotdot+0x4f4(2a101220df8, 2388873b24c20,
    fffffffffffffff8, 301412eb920, 2a101221238, 1)
000002a101220941 nfs4lookupnew_otw+0x7d8(301ef0d4dc0, 2a101221530,
    2a101221528, 301412eb920, df8475800, 38285c955c0)
000002a101220a71 nfs4_lookup+0x114(301ef0d4dc0, 2a101221530, 2a101221528,
    301412eb920, 0, 391f52d44a8)
000002a101220b41 fop_lookup+0x28(301ef0d4dc0, 2a101221530, 2a101221528,
    7aa69c2c, 0, 600045703c0)
000002a101220c01 lookuppnvp+0x344(2a1012217f0, 0, 600045703c0, 2a101221528,
    2a101221530, 6000008dbc0)
000002a101220e41 lookuppnat+0x120(301ef0d4dc0, 0, 1, 0, 2a101221930, 0)
000002a101220f01 lookupnameat+0x5c(0, 0, 1, 0, 2a101221930, 0)
000002a101221011 vn_openat+0x164(1, 400, 1, 1, 0, 1)
000002a1012211d1 copen+0x260(ffffffffffd19553, 87aa3, 0, 50400, 0, 1)
000002a1012212e1 syscall_trap32+0x1e8(87aa3, 0, 50400, 0, 0, 0)

/*
 * nfs4_make_dotdot() - find or create a parent vnode of a non-root node.
 *
 * Our caller has a filehandle for ".." relative to a particular
 * directory object.  We want to find or create a parent vnode
 * with that filehandle and return it.
.. snip

int
nfs4_make_dotdot(nfs4_sharedfh_t *fhp, hrtime_t t, vnode_t *dvp,
      cred_t *cr, vnode_t **vpp, int need_start_op)

> 000002a101220531+0x7ff::print struct frame
{
    fr_local = [ 0, 0, 0x381a4c49000, 0x2a101221108, 0x2a101220ee0,
0x2a101221118, 0x7aabd800, 0x7aabd800 ]
    fr_arg = [ 0x2a101220df8, 0x2388873b24c20, 0xfffffffffffffff8,
0x301412eb920, 0x2a101221238, 0x1 ]
    fr_savfp = 0x2a101220941
    fr_savpc = 0x7aa6b474
    fr_argd = [ 0x1, 0x5bc679f3060, 0, 0x2a101221250, 0x200000000,
0x5bc679f3178 ]
    fr_argx = [ 0 ]
}

static int
nfs4lookupnew_otw(vnode_t *dvp, char *nm, vnode_t **vpp, cred_t *cr)

> 000002a101220941+0x7ff::print struct frame
{
    fr_local = [ 0x391f52d4450, 0x381a4c49000, 0x2388873c342a0, 0x600074432a8,
0x2388873b24c20, 0, 0x1, 0x391f52d44e8 ]
    fr_arg = [ 0x301ef0d4dc0, 0x2a101221530, 0x2a101221528, 0x301412eb920,
0xdf8475800, 0x38285c955c0 ]
    fr_savfp = 0x2a101220a71
    fr_savpc = 0x7aa69d40
    fr_argd = [ 0x38f6901b110, 0x311fe247dc0, 0x2a101221704, 0x2a1012216fc,
0x2a101220c71, 0x7aa7ad20 ]
    fr_argx = [ 0x2a101221264 ]
}

> nfs4lookupnew_otw::dis!grep i0
[ .. elided ]

> 0x301ef0d4dc0::whatis
301ef0d4dc0 is 301ef0d4dc0+0, bufctl 301ecba50c8 allocated from vn_cache

> 301ef0d4dc0::print vnode_t v_data |::print rnode4_t r_svnode.sv_name
r_svnode.sv_name = 0x391d8a24090
> 0x391d8a24090::print nfs4_fname_t fn_parent fn_name
fn_parent = 0x3b6565ba920
fn_name = 0x3292727cbe0 "uts"
> 0x3b6565ba920::print nfs4_fname_t fn_parent fn_name
fn_parent = 0x353987b1550
fn_name = 0x4f42a102fe0 "src"
> 0x353987b1550::print nfs4_fname_t fn_parent fn_name
fn_parent = 0

nfs4_make_dotdot+0x27c:         mov       %l2, %o0
nfs4_make_dotdot+0x280:         mov       0xc, %o3
nfs4_make_dotdot+0x284:         call      +0x15c58      nfs4_end_fop
nfs4_make_dotdot+0x288:         mov       %l7, %o5
nfs4_make_dotdot+0x28c:         ba        -0xd0         nfs4_make_dotdot+0x1bc
nfs4_make_dotdot+0x290:         cmp       %i5, 0
nfs4_make_dotdot+0x294:         add       %fp, 0x797, %o4
nfs4_make_dotdot+0x298:         mov       %l2, %o0
nfs4_make_dotdot+0x29c:         mov       0xc, %o3
nfs4_make_dotdot+0x2a0:         mov       %l1, %i1
nfs4_make_dotdot+0x2a4:         call      +0x15c38      nfs4_end_fop
nfs4_make_dotdot+0x2a8:         mov       %l1, %o5
nfs4_make_dotdot+0x2ac:         ld        [%fp + 0x7bb], %i3 <------
nfs4_make_dotdot+0x2b0:         cmp       %i3, 0

> 0x2a101220941+0x7bb::print nfs4_error_t
{
    error = 0
    stat = 0t10006 (NFS4ERR_SERVERFAULT)
    rpc_status = 0 (RPC_SUCCESS)
}