The Random Bit Bucket: Moinak Ghosh's Weblog

I have been away from blogging for quite a while as things have been going extremely tight for me both on family and work front. I have been managing through a difficult family situation and many concurrent tasks at work, esp. since I got busy with the Slim Install project.

However the work has been quite involved and interesting. One of the goals of the Slim Install work is to provide a single CD install experience where you boot from a single LiveCD and get a base OS image installed quickly. Subsequently you can pull in additional packages from a repository as and when you need stuff. The second part will be provided by the new IPS packaging being developed.

At present we have an OpenSolaris LiveCD that can boot into a Gnome Desktop. We also have a modified Caiman Installer that uses the new Target Instantiation module and the Transfer Module that I have been working on. So we have a LiveCD from which we are actually able to install bits onto a ZFS root on harddisk. There are still some issues while booting off the harddisk after installation that is being wctorked on at present. Sundar Yamunachari and Jan Damborsky has been working hard on Orchestrator and Target Instantiation while Sarah Jelinek as usual has been working her charms on adapting Caiman to Slim. The new Solaris Installer is charming isn't it

The current Slim Install prototype in development contains a variety of stuff. It pulls in some technologies that I had initially developed for BeleniX like lofi compression and CDROM filesystem enhancements. We have been using the framework from the Live Media Kit to build the Slim LiveCD prototypes, but eventually will be using the Distro Constructor, once it stabilizes. This prototype pulls in a build of the full open-source Xorg tree. I have been building a complete Xorg tree for OpenSolaris based on the FOX project. These changes and changes from Martin Bochnig's work on Xorg for SPARC will soon make their way into the FOX mercurial repository. Slim Install also includes a slightly different JDS build from Nevada. This JDS build removes some SUN branding and a couple of non-redistributable components. Laszlo (Laca) Peter from the JDS team has been providing JDS built against the FOX packages for Slim.

The lofi compression and my HSFS Filesystem enhancements are making their way into OpenSolaris now. The  HSFS codereview is complete. It was done openly on the caiman-discuss@opensolaris.org mailing list and the review process provided a huge amount of valuable feedback that helped me to refine my changes a lot beyond the original work done for BeleniX. Very soon you will see much better data CD/DVD performance on OpenSolaris. I have more changes planned but those will come in a second installment.

Alok Aggarwal has been working on getting the lofi compression changes into OpenSolaris and he has finished some stuff that I left unfinished and made some excellent enhancements to the original code. He is currently running through the PSARC case you can see a discussion thread here http://opensolaris.org/jive/thread.jspa?messageID=161019&tstart=0.

Apart from these there have been other activity going on with Sanjay Nadkarni and Dave Miner leading the efforts and sitting on everyone's heads, keeping all in line at the same time getting their hands dirty with code and technical nitty-gritties.

As you can see a lot of activity is going on and I have basically had my head down plowing through work. I recently spent a 2 month stint in the US wandering from East to the West coast and finally onto Colorado (God's own Country in the US) interacting with the team members. If you want to get into the thick of action here are some things you can do:

Subscribe to and participate in caiman-discuss@opensolaris.org discussions.

Subscribe to and participate in  pkg-discuss@opensolaris.org discussions.

Browse the numerous materials at  http://www.opensolaris.org/os/project/caiman/ and suggest new ideas, comments etc. Browse the IPS repository at ssh://anon@hg.opensolaris.org/hg/pkg/gate pkg-gate .

Play with the Live Media Kit and contribute enhancements/ideas.

Participate in the NOSI P-Team meetings, details here http://www.opensolaris.org/os/project/caiman/NSI_Meetings/

Participate in the Slim Install meetings, details here http://www.opensolaris.org/os/project/caiman/Slim_Install/Meeting_page/

Lets get a world class OpenSolaris LiveCD and Install enperience. Of course with a LiveCD framework and a distro constructor there are lots of other things to look at like Damn Small Solaris aka Damn Small Linux and an OpenSolaris equivalent of FreeNAS.

I had a totally unexpected sighting of a comet. Date: Sept 10, 2007, Time: 7:20 PM, Place: SUN Microsystems office, Broomfield, Colorado. It had been a completely overcast and rainy day with clouds clearing somewhat towards the evening. I was working late mucking around with my FOX build and hsfs testing. After a while I went to the window and was watching a great sunset behind the Flatirons, Rocky Mountains. It was then that I noticed a long red streak in the sky moving towards the sunset.

Fortunately I had my digicam (Canon S3 IS) with me and took a video:

I could see this guy moving towards the sunset for about 9 minutes and it has 2 tails.

Correction: I was initially thinking this as a Comet, but how could it be missed by others if it's visible to the naked eye. The comment on my blog is right. This is a transcontinental airliner, since if you notice carefully the head of the thing is not glowing. The 2 streaks were the jet trails reddened by the setting sunlight.

As I had blogged earlier, I was working on a fix for the Windows Vista and Solaris dual boot issue. It is finally integrated in OpenSolaris and is available in Solaris Express Build 70, BUG#6510526. So now you need'nt have to do any gymnastics if you already have Vista on your machine. Just install Solaris and Vista will still boot. The fix was ultimately done in fdisk.h and affects both installgrub and fdisk.

Of course if you want to resize your Vista partition then you can use Vista's own safe volume resize option. Go to Computer Management->Disk Management and right-click on a disk partition.

I was lucky. For the first time in my life I was able to watch a meteor shower. The Perseids meteor shower peaked on Aug 12th which was last Sunday and there were good views from the Northern hemisphere. I am currently in the US and was in Mountain View, California last weekend which had excellent clear skies.

I got to see at least 5 large meteor streaks within 20 minutes, near to the constellation of the Big Dipper in spite of the hotel lights. It was very exciting for me. I had a few rare glimpses of meteors in the past, but nothing like what I saw on Sunday. The other time I had similar excitement was a long time back when I was able to watch Halley's Comet using a home-brew refractor telescope that I had constructed out of cardboard tubes for holding blueprint paper, a 0.5 diopter (2m focal length) objective that my dad got made after haggling with an optician store and a 6x eyepiece.

I am not too much of an Astronomy hobbyist, but do a bit of reading of Astronomy and Cosmology topics.

I put in a few changes to the HSFS module in the recent BeleniX 0.6.1 release. Firstly sync my HSFS enhancements to the current HSFS source in OpenSolaris. Secondly fix a panic in the HSFS module and thirdly modify the I/O scheduling logic a wee bit to get better CLOOK behavior and improved performance.

The fix for the panic had been the hardest part and the panic used to manifest occasionally in earlier releases with an incomplete stacktrace. In 0.6 however it started happening every time on many machines if 0.6 was being booted off a CDROM media. It turned out to be mistake in the readahead code which was unmapping a page before it was to be used. Well it was quite a bit of a corner case: the panic would happen servicing a sector being readahead when there are two adjacent non-interleaved sectors and when it's previous sector is on a page boundary and it was waiting too long in the I/O queue and was serviced when it's deadline timer expired. It's something that happened with a previous read in the past causing a failure in the future.
Some corner case! Quite a bit of struggle to figure that out. Thanks to Pramod and Pavan a couple of kernel folks for their help. Because of the way the VM pages were being handled for the readahead case a sector read on a page start boundary will cause the page to be mapped out (via ppmapout) so the subsequent sector read will have nowhere to write the data to. Normally this is not a problem because the I/O scheduler coalesces individual reads, but a deadline expiry changes that.

Looking at one's own code after a period of time can help to uncover logic flaws. My HSFS changes implementing the CLOOK scheduling logic lacked a little bit of effectiveness. In CLOOK that disk head moves in one direction servicing all the requests in the way till there are no more in that direction. It then revs back to the beginning and starts again. The way I had implemented this, the I/O queue structure had a next pointer to the next higher logic block address request or NULL if there aren't any. This pointer would be set in the scheduling logic when it is looking for requests to service. It will set the next pointer to the pending I/O request in the queue after the last request being serviced in the current iteration of the scheduling function. The next iteration would pick up this pointer (if non-NULL) and start from there.

However there is a possibility that more I/O requests might get enqueued in between the last serviced and the next to-be-serviced one. These would get skipped to be serviced later. Not quite the behavior we want. So I changed this to record the Logical Block Address of the last serviced request instead of a pointer. Then the scheduling function will probe the queue for an I/O request with a higher Logical Block Address. By this approach we would also catch those I/O requests we re missing earlier. This change appears to have helped and initial tests have indicated an additional performance gain of 10%-12%.

I am traveling again and this time while on the flight I started reading "Coalescent" by Stephen Baxter. In Chapter one page 3 he mentions the following about Italian traffic:

"The narrow cobbled streets are always crowded with traffic, with cars and buses, lorries and darting scooters. Italians don't drive as norther Europeans do. They just go for it: they swarm, as Peter McLachlan would have said, a mass of individuals relying on the unwritten rules of the mob to get them through."

This brought a smile to my lips. Haha wait till you see Bangalore or Chennai traffic: cars, buses, lorries, scooters, motorcycles, bicycles, push carts, cycle carts, occasional tractors, bullocks and people walking and last but not the least the indefatigable Autorickshaw. Essentially moving objects of every shape, size,  description and velocity massing on the street in typical Brownian Motion!

And this is what I navigate every day as I go from home to office and back. It is like an obstacle course video game, except that you play it in real life. Scary ? Well you kinda get used to it.

It is quite gratifying to see the BeebEm project: http://www.mikebuk.dsl.pipex.com/beebem/ moving along nicely. I can enjoy all those fantastic BBC Micro (Also called Beeb) programs and get to dust off my old memories.

This machine was one of the most versatile microcomputers of it's time with the BBC Basic language being the most advanced Basic dialects available in any microcomputer at that time.  BBC Basic included constructs like Procedures, Functions, support for recursion, typed pointers, typed variables etc. when various other dialects of BASIC were still stuck with GOSUB. The micro itself was amazingly versatile having features like Paged ROMs, VGA out, RGB out, basic networking, Parallel port, Serial port, external floppy disks, Advanced 4-channel sound, co-processor etc.

My first introduction to Computers began with this box at the encouragement of my Mother. I wrote my first BASIC program to accept two numbers, add them and display the result. From then on it's been a journey of discovery and fun. Every now and then I will discover some new nifty info, trick, tweak etc. about this machine. Then I will be the happiest person in the whole family. I had even attended a couple of coding competitions on this machine and came away happily with prizes. One of the prizes was the excellent book Variations in 'C' by Steve Schustack that got me introduced to the 'C' language, but that's another story.

This machine was first introduced in 1981 but was available in India only about 8 years later as the SCL Unicorn brand! I was exposed to this wonderful gadget in the April of 1990. I would go to my Mom's office: Birla Industrial and Technological Museum regularly to marvel at  the Scientific miracles and also get access to a bunch of computers including the BBC Micro.

I thoroughly devoured the ins and outs of this machine for close to 3 years before moving onto the comparatively cranky 4.77 MHz IBM PC-AT that seemed like a slow dinosaur compared to this sleek and fast 2 MHz beast! Most of my understanding of how Computers work were built on this machine, things like: Basic system architecture, Data Structures and Algorithms, OS stuff, 2D and 3D graphics, Animation, Assembly, and all the typical blah blah.

It was fun to do 3D graphics stuff with BBC Basic and eke out the last Hz of performance from the 1 MHz 6502 processor when doing hidden line removal and animating the results. I learned about memory paging and wrote a simple paging system in BASIC. This was a big paintbrush-style graphics editor that wouldn't all fit into the available 8K RAM after the memory-mapped graphics (20K) and OS workarea have taken their toll. The BASIC program was divided into multiple procedure modules which would be loaded on demand, from floppy, into the same region of memory overwriting the earlier contents.

BBC Basic supported inline Assembly code and it was a lot of fun. One weird thing to try was to update the memory mapped display faster than the Vertical retrace resulting in strange flowing curtain-like effects on the screen. BBC Basic allowed user defined characters for creating sprites. I still remember the hours I spent designing custom character sets and sprites for games on 8x8 grids. I'd even spend weekends poring over pictures drawn on graph paper and vectorizing the images by hand and transferring to the BBC Micro as DATA statements for rendering on screen.

I'd grab every BBC Micro book I could find on bookshelves and streets. I still have a good collection of those. I was always sad that I never could get my hands on the newer BBC models like B+ and Master 128. However BeebEm supports emulating those and is real fun to play with. I am enjoying those classic games like Elite and Chuckie Egg as well.

We use Google Analytics on the BeleniX site and here are some statistics for the past one month, May 13 2007 - June 12 2007.

Overall 24308 visits. You can see the spike on May 23rd after the 0.6 release announcement which clocked 4979 visits.

Map Overlay view shows the visits by Continent. Highest number of visitors are from Europe which clocked 12077 visits with Americas coming in second at 8250 visits:

In Europe, Western Europe has higher visit count at 4037 with Eastern Europe at 2951. This image shows details for Western Europe. Germany has the highest number of visitors at 1882 followed by France at 1091:

3168 Visitors came from Asia with highest number of visitors from Japan at 849 followed by India at 708, China at 366, Taiwan at 170. In India 109 visitors were from Bangalore followed by 85 from Delhi and 59 from Mumbai. The following image shows overall Asian region:

Finally in the Americas region 6487 visits were from North America followed by 1294 from the South. Bulk of the North American visitors were from the US numbering 5637. In the US 885 folks were from California followed by 731 who have no homes as per Google or probably hiding behind firewalls,proxies or some such . The Image below shows the US region:

When working on the second release of BeleniX 2.5 years back which included Xorg 6.8 and Xfce for the first time, I needed a way grab the Monitor's EDID data. EDID stands for Extended Display Identification Data and is explained in this Wikipedia article. I modified the ddcxinfo utility used in Linux (KnoppiX) and used it for BeleniX. The ddcxinfo utility would process the EDID data and generate modeline and monitor refresh information while providing some useful options. Xorg auto-configuration would try to do that same but had vagaries. In general ddcxinfo seemed to produce better results using it's own logic.

That utility still persists today and had been giving good results till BeleniX 0.5.1. But in BeleniX 0.6 it has stopped working. Why ?  Because of the integration of Direct Boot support in OpenSolaris! This needs some explanation.

In order to get the Monitor's EDID information one has to perform a DDC probe a technique defined by Video Electronics Standards Association or VESA for short. Typically on PCs there are BIOS extensions that support some of the VESA functionality including the ability to perform a DDC probe (Vesa Bios Extensions or VBE). So this is also called VBE probe. You typicall call int10h function 1 to fetch EDID data from the monitor. This is all fine but the problem is BIOS works in real mode while all modern Operating Systems including OpenSolaris run in protected mode. The BIOS is not active in protected mode. So to call a BIOS function you'd need to either use virtual 8086 mode or use an x86 real mode emulator. In Linux the kernel provides a vm86 system call that allows one to execute code in real mode that includes int10h BIOS calls. Vm86 uses the Virtual 8086 mode. There is a library called Linux Real Mode Interface that allows easily making real mode BIOS calls utilizing vm86.

Such a thing is not available on OpenSolaris however. But before Direct Boot was integrated OpenSolaris used the multiboot program to boot the kernel. Grub will load and pass information to the multiboot standalone program which will do some basic setup, load and jump into the the kernel. Multiboot ran in real mode and among other things used VBE to grab the monitor EDID information block. This can be seen in the vgaprobe.c file. Multiboot used to attach this raw EDID data as a property of the vgatext device node. Vgatext is the module on OpenSolaris that provides the text mode VGA console (80x25) on x86 platforms. This property was accessible via the Platform Information Control Library (See: man -s 3PICL libpicl). So my port of ddcxinfo utility simply queried PICL to extract the raw EDID information from the "display-edif-block" property. This would not work if Nvidia's binary driver was active since Nvidia Xorg binary driver uses the nvidia kernel module that replaces vgatext on machines with a supported Nvidia display card. This is a not a problem since the Nvidia driver does some custom probing and does a good job of selecting a usable display resolution and refresh.

Now the Direct Boot project got rid of multiboot for various reasons and along with it went the vga probing stuff. So vgatext no longer had the EDID information attached. I did not have time to look at this before BeleniX 0.6 release, so ddcxinfo fails to get EDID data and generates a set of standard predefined VESA modelines and some additional ones calculated using the VESA Generalized Timing Formula.

I started looking at this beginning of last week and noticed that Xorg itself would do a VBE Probe to get the EDID info. Sample /var/log/Xorg.<displaynum>.log extract:

(II) RADEON(0): initializing int10
(II) RADEON(0): Primary V_BIOS segment is: 0xc000
(II) RADEON(0): VESA BIOS detected
(II) RADEON(0): VbeVersion is 512, OemStringPtr is 0x00002100,
        OemVendorNamePtr is 0x0000211c, OemProductNamePtr is 0x00002132,
        OemProductRevPtr is 0x00002137
(II) RADEON(0): VESA VBE Version 2.0
(II) RADEON(0): VESA VBE Total Mem: 131072 kB
(II) RADEON(0): VESA VBE OEM: ATI MOBILITY RADEON 9600
(II) RADEON(0): VESA VBE OEM Software Rev: 1.0
(II) RADEON(0): VESA VBE OEM Vendor: ATI Technologies Inc.
(II) RADEON(0): VESA VBE OEM Product: P11
(II) RADEON(0): VESA VBE OEM Product Rev: 01.00
(II) Loading sub module "ddc"
(II) LoadModule: "ddc"
(II) Loading module ddc from path "/usr/X11/lib/modules/"
(II) Reloading /usr/X11/lib/modules//libddc.so
(II) RADEON(0): VESA VBE DDC supported
(II) RADEON(0): VESA VBE DDC Level 2
(II) RADEON(0): VESA VBE DDC transfer in appr. 2 sec.
(II) RADEON(0): VESA VBE DDC read successfully
...
...
(II) RADEON(0): DDC Type: 3, Detected Type: 0
(II) RADEON(0): EDID data from the display on port 1 ----------------------
(II) RADEON(0): Manufacturer: IDT  Model: 2  Serial#: 0
(II) RADEON(0): Year: 1990  Week: 0
(II) RADEON(0): EDID Version: 1.3

Which means that it is able to make the int10h call even in OpenSolaris. A lot of digging later I found that Xorg used a real mode x86 emulator aptly called x86emu that was originally authored by Scitechsoft and their original now obsolete version is available here. The x86emu code in Xorg is at hw/xfree86/x86emu. Xorg has another module called vbe that calls into the int10 module for VBE BIOS calls. The int10 module uses x86emu which has the ability to execute the Vesa Bios Extensions. The DDC (Display Data Channel) protocol support is in hw/xfree86/ddc.

As far as I looked almost all of the Xorg Video drivers do have code to probe the Monitor and extract EDID data. Even the Nvidia binary driver does this, but probably in it's own way. In fact the big advantage is that the drivers can and do print this EDID data by calling some generic functions. See: print_edid.c.

The drivers will dump the raw EDID data block to Xorg.<displaynum>.log if verbose logging is enabled, in other word we execute: Xorg -logverbose 10. However that command starts the Xserver and it dosen't exit by itself. We need to run the Xserver, have it probe and log everything and come out, so that our little ddcxinfo utility can process that data. That is fortunately possible using: Xorg -logverbose 10 -probeonly. Sample output showing an EDID dump for the LCD monitor on my Ferrari 3400:

(II) RADEON(0): EDID (in hex):
(II) RADEON(0):         00ffffffffffff002494020000000000
(II) RADEON(0):         00000103801e17780acd7591554f8b26
(II) RADEON(0):         21505400000001010101010101010101
(II) RADEON(0):         010101010101302a78f0501a0f403070
(II) RADEON(0):         130031e41000001e0000000f0008fa22
(II) RADEON(0):         ff023e07bbffffffff00000000fe0049
(II) RADEON(0):         44540a202020202020202020000000fe
(II) RADEON(0):         004e31353050320a202020202020003c
(II) RADEON(0):

Thus I modified the ddcxinfo utility to parse the verbose Xorg logfile and extract out the raw EDID data and interpret it. This is working as of now and will make it's way into BeleniX 0.6.1. The Nvidia binary driver also logs raw EDID data but in a slightly different format. ddcxinfo-belenix can handle that as well.

Having done all this I feel a desire to see the day when this extra processing will become obsolete. Simply running Xorg will give an optimum display out of the box. It is good to note that development efforts in Xorg is moving in that direction. For eg. a lot of effort has been put into the Xorg radeon driver to improve auto-configuration. In fact the ddcxinfo utility in BeleniX 0.6.1 will not apply it's own logic if it detects a supported Nvidia or ATI card, leaving the auto-configuration upto the driver.

Last weekend Eben Moglen was in Bangalore and we able to arrange for him to talk at our Bangalore office on GPLv3, Patents and sundry. It happened due to Balaji Kutty who has friends in FSF India and we could grab a free slot on Eben's schedule. Eben is an interesting and entertaining speaker who would make you laugh at regular intervals. To me personally he also came out as a practical and down to earth person. I was also quite impressed at his technical depth that you don't expect from a lawyer

However he speaks long sentences and you will have to hang on to every word if you want to get what he is talking about. He explained in general about sharing stuff, and the ideas beghind GPLv3. He also illuminated on the Novell-Microsoft thing and the Tivo issues.

He had good things to say about SUN and mentioned that a significant portion of Debian repository (probably 30%) is FOSS code from SUN. Other significant comments he made are below. These are not exact speech transcripts but essentially the words he used in his long sentences:

SUN with it's open innovations in the past and FOSS efforts today is converging on the same direction as the FSF. In fact SUN could, moving forward, in principle be seen as the commercial avatar of the FSF.

SUN's opening of Java and the statement that they are using GPLv2 now and will move to GPLv3 when it is finalized came at a crucial time that provided the support for GPLv3 to move forward. At that time the GPLv3 process was in trouble with most of the participants threating to walk out if their demands were not met. SUN was the only organization who did not threaten, rather held productive discussions on the issues. SUN's statements with OpenJDK helped to bring others in line.

Obviously he lightly pushed for OpenSolaris to be released under GPLv3. He also mentioned that it was SUN who first brought the idea of GPL-ed hardware to the FSF and got them interested via the OpenSPARC project.

Here's a pic of Eben with some of the geeks. Sorry for the low quality, but that's all that the mobile cam could muster. You can see Balaji on the extreme left:

BeleniX 0.6 got released last Tuesday after quite a gap of about 6 months. Several months of this time have been way too busy for most of the folks including myself. As I had blogged earlier, travels and engagements took a significant toll in terms time and energy.

Even then work had been progressing slowly and various folks have been doing their bits. It was the middle of April that I finally decided to put all the pieces together  and get the delayed release done. But it was easier thought than done - of course you can think at the speed of Tachyons swimming past Einstein's limits. You can however work only at the pace of dead slow human I/O mechanisms and frustratingly slow home computing power. Restricted budgets matter sigh. Even more importantly luck matters, and it can run out at the worst possible moment!

Here's a sequence of happenings leading up to the release. A little long read:

1. First on my agenda was to get complete Solaris packages built for the OpenSolaris base. Of course doing a make install in the usr/src/pkgdefs failed with missing objects. Closed source non-redistributable stuff. So I updated the prototype files for the package in question and repeated the build. It failed for another package. I kept updating and rebuilding till I fixed them all; 30 packages in total took about 3 hours of drudgery.
   
2. Next move to Xorg 7.2. After pondering a little I decided it was best to base my work on the X-Consolidation source drop. Since this source drop is partial I had to implement a full modular Xorg build adding more Makefiles and packages. This was a significant undertaking, a lot of which was grunt work. Build, fix, build fix, build, build packages, fix packages, add packages, build packages, fix packages, install packages, fix packages, fix Makefiles, build, build packages, install packages, fix ... All these took 10 days of my outside-work waking hours and some sleeping hours as well!
3. Next update various libraries and software in BeleniX. Several days of effort and you have to rebuild some other dependent software as well.
4. In all these I had to generate ISO images, burn them on a DVD-R/W and test now and then. One complete ISO generation took at least 3.5 hours, during which I will have to wait doing nothing but monitor the build for errors! You can't do any other work as the system will be heavily loaded and sluggish to even keypresses. I'd need to free up as much RAM in my 1GB as possible and use either CDE or better still TWM! After all this you find a *single character syntax error* in a startup script and possibly repeat the process #%@$%*
   
5. And while doing heavy duty development, even my Athlon64 dual-core 2.6 GHz with 7200RPM Sata drive just feels way too sluggish slowing down my pace of work. Just the time one could put one of the M-Series screamers to good use: http://blogs.sun.com/jonathan/entry/i_love_mainframes SIGH.
6. Next build Compiz 0.5.0. This went fine without troubles.
7. Integrate the Solaris Parted port. Lots of compiling of libsigc++, Glibmm, Gtkmm, cairomm, GParted. Stuff mostly worked. Fixed a few issues.
8. Update startup scripts to integrate the Compiz desktop and various other script cleanups. Rebuild and test ISO multiple times.
9. Add/fix various other things including KSH93, Compact shell, Usbdump etc.
   
10. Rebuild and test ISO. Give ISO to a couple of other folks to test who come up with a bunch of issues.
11. Fix issues and look at updating to Kdevelop latest release. This is when disaster strikes just 5 days before the projected release date of 22nd May ...
12. My 1KVA APC UPS powering my workhorse, for some inexplicable reason decides that the Battery is screwed. It refuses to pass the self-test and power the equipment :-O  I am able to run the UPS without batteries but that is not of much use.
13. On the Friday before 22nd I decided to get the battery replaced. In the worst case shell out cash and buy a new one. Morning I rushed to a nearby APC authorised center and they asked me to go the shop from where I purchased the UPS. They did not have spares either.
14. I had purchased the UPS from SJP Road in Bangalore, a market for all kinds of Electronics and Electrical stuff you can find. Heck you can even get old Sparcstation 20s or Ultra 5s if you search hard enough. That road is however far from my house. So I went to office to catch up on some work and finish lunch. Since I was late I had to park my car on a road outside the office - sorry parking full
    
15. After lunch I grabbed the 5.1KG battery and hopped on an Autorickshaw in the blazing Sun. There is simply no way you can find car parking in the congested SJP road.
16. The shop guys gave me APC 24x7 service numbers which I have to call and they will provide on-site service. Exasperation! Why didn't you give me the damned numbers when I purchased the Unit. The user manual did not list the numbers unfortunately.
17. So now back to square zero. How do I work at home ?  I remembered a spare 0.5KVA UPS lying at office that I had returned along with the old home PC that SUN had given me a long time back. I rushed to office grabbed the 7.5 KG unit rushed 6 floors down the stairs out into the blazing Sun and then into my Car.
18. Reaching home I called UPS and spoke to a very helpful customer service person who asked me the serial number and to do some tests which appeared to be successful. Finally he asked me to do a 15hour zero load battery charging test.
19. I hooked up the 0.5KVA UPS and was able to work on my system.
20. The Kdevelop build failed with some inexplicable C++ template error Ugh! I decided to give up on Kdevelop for now and look at getting Koffice 1.6.2
21. But the CD was getting close to capacity and I had to figure out innovative ways of squeezing out more space and trim down Koffice components to get the most important Kword, Kpresenter etc. in and leave space for future when I will have full packaging.
22. Unfortunately it is not possible to predict exactly how much extra space my efforts will lead to till I build the complete ISO, 3.5 hours ouch. On top of that the 0.5 KVA UPS was quite old and was not able to switch fast enough on a power failure that happened during the fag end of an ISO generation process. Comp shuts off and I had to repeat the process.
    
23. Finally I decided to look at the CD build procedure and see whether it is possible to improve things. 22nd was getting near, I was hardly sleeping and still time was not on my side. The situation was: while generating the compressed files the program will compress a 128K chunk and write it out. Essentially thousands of disk I/O requests to compress 1.8GB. Since I have large swap and sufficient RAM I decided to create the file entirely in swap and finally copy it out to the appropriate location. Believe it or not this shaved off almost an hour from the whole build, hardly much disk I/O, yay The machine was running much cooler than the room heater it used to be earlier, I was being eco-friendly and reducing hdd wear and tear .
    
24. Finally after much testing during the next 2 days I managed to push 0.6 out. During this I found GParted corrupting libc heap and coring in a malloc at the last momment! I decided to try and debug this using Libumem at 2:30 AM in the night and the problem disappeared! I fired off a CD build set an alarm for 5:30 AM morning and crashed. I made a change to always run GParted with Libumem.
    
25. BTW a 15 hour battery charge did not fix the UPS problem and APC replaced the entire unit no questions asked after 2 working days. I was quite happy with their service.
26. Finally on 23rd I gorged on Rum, Fries, Fish and Chicken with a friend and slept for 8 hours at a stretch.

Linux For You one of India's premier Linux magazines has shipped Solaris Express Build 60 DVD on their current May issue. Joe George and myself were surprised on receiving a call from them at the last momment when they had already decided to ship Solaris Express and were trying out SXCR Build 59 for the last time before sending the image for mass replication.

They were having an Xorg issue which turned out to be an EDID probe failure due to a KVM switch. But there was another problem. We then went through a rush hour to burn and courier them a couple of DVDs of Build 60 which is a good stable build. Their office is in Delhi.

Ananth had to write an illustrated article on how to install SXCR. I spent a late night create an OpenSolaris goodies bundle that included all the opensource drivers, Nvidia driver, Compiz, Inetmenu, Gnome Themes and a variety of sundry other stuff. All one has to do is to extract the tarball and run ./install. Everything gets setup and the system reboots automatically. That is also included in the additional CD accompanying the issue. We wouldn't want new users having to hunt the net for OpenSolaris goodies after installing it esp. if their Ethernet card does not work!

The distribution of this mag is quite vast numbering tens of thousands and that many copies of OpenSolaris/SXCR went out. The only sad part: Build 60 has a bug where the GUI installer hangs after System Identification for any non-US locale. US locales work perfectly. This apparently is a Java bug. The workaround is to use the Text Mode Installer or select a US locale and change it later. We are trying to tackle the fallout using every means of communication: Blogs, Forum posts, mailing list notifications, subscribing to the LFY forum, Wiki page: http://www.genunix.org/wiki/index.php/Tweaking_Solaris etc.

Just a while back I was casually browsing The Register's newsboards when I came across this article: Oracle Shuts Windows on Data integrity drive

This comment in the article quoting an LSI senior had me wondering:
"End-to-end data protection cannot be achieved by a single vendor, and therefore requires a multi-vendor initiative such as the DII"

If this is true then what about this: http://blogs.sun.com/bonwick/entry/zfs_end_to_end_data,
ZFS's claimed End to End Data Integrity.

Am I missing something or ZFS is simply being ignored here ...! 

There was another thread of discussion on OpenSolaris Discuss on the Vista Dual boot issue and that prompted me to re-look at the problem. The Bcdedit dance is much too cumbersome and none of the Shareware GUIs around Bcdedit make much sense. The complete discussion thread is here:

 http://www.opensolaris.org/jive/thread.jspa?messageID=105480&#105480

After some digging it boiled down to the Windows DiskID being the critical factor here. You can read all about the Windows MBR DiskID or Signature here:

 http://www.multibooters.co.uk/mbr.html

In fact folks are still able to dual boot Vista with Linux using Grub. Linux flavors use grub-install to prepare the MBR and grub-install (or setup) preserves the Windows DiskID. Solaris however was using the home-grown installgrub(1M) and as is explained in the thread , it failed to preserve the Windows DiskID and thus Vista won't boot.

I have got around to making a couple of changes to installgrub to preserve the DiskID and luckily for me, we had a bunch of Sony Vaio laptops coming in for Managers preinstalled with Vista. So I got to test it on those machines using the following steps:

1. Resize Vista using a Gparted CD.
2. Boot into Vista and check everything is OK
3. Take a backup of the updated MBR
4. Install Solaris Express B60
5. Copy my modified installgrub binary to the system
6. Restore the original Vista MBR using dd
7. Now run the modified installgrub

Well I have finally got around to building binary packages and an install script to easily get the Compiz floss installed on OpenSolaris. I spent a day installing Compiz on various machines  and testing my scripts and making a few people happy in the process. One person is itching to show off his all singin' all dancin' SX installation on a Compaq Presario to his friends bragging about Beryl on OpenSuSE.

I have been looking at the Beryl crashes and several core dumps are showing similar stacks so I am getting closer to figuring the problem. A stable Beryl would be really nice, though Compiz is a little lighter and faster than Beryl.

Anyways here are the instructions on how to get Compiz on your Nvidia display card equipped box:

• First off you *need* an Nvidia 3D accelerated card, either a GeForce or Quadro. If you do not have an Nvidia display card you are out of luck :-|
• Secondly you need at least Solaris Express Build 60. It should also work on Builds 61 and 62.
• Thirdly you must be using JDS. This Compiz build has only been tested with JDS.
  
• Grab the latest Nvidia Solaris Driver and install it from here: http://www.nvidia.com/object/unix.html
• Grab the Compiz + Xorg additional stuff package tarball from here: http://www.genunix.org/distributions/belenix_site/binfiles/compiz-pkgs.tar.gz
• Extract the tarball and cd into the compiz-pkgs directory.
• Run the installer script as root user: ./install
• Logout and login again as the normal user. Nothing new happens.
• Now add some basic Compiz configurations by executing: /usr/bin/add-compiz
  
• Now start Compiz by executing the following in a terminal window:
• gtk-window-decorator --replace & compiz --replace gconf &
• Now if you like the Jazz you can make Compiz the default by adding the commands to the auto-start list:
• Go to  Gnome Menu -> Preferences -> Sessions
• Click on "Startup Programs"
• Click on "Add" and add the command: gtk-window-decorator --replace
• Click on "Add" again and add the command: compiz --replace gconf

Now you are all set to enjoy Compiz. Here are some basic usage tips:

• Gnome Menu -> Preferences -> Compiz Settings Manager allows you to tweak the settings and enable/disable plugins. Keep in mind that the "Annotate" plugin can cause some heartburn.
• Ctrl + Alt + Mouse Drag on Desktop rotates the Cube
• Mouse wheel on Titlebar shades/unshades windows
• Ctrl + Right Click zooms the display. Mouse wheel or Enter will un-zoom.
• Alt + Mouse Wheel on a Window adjusts the Window Transparency
• Jerking Mouse Pointer to Top Right desktop corner gets you the Expose effect
• Ctrl + Alt + Up Arror also gets you the Expose effect
• Ctrl + Alt + Down Arrow gives a panoramic view of all the desktops
• You can drag a Window from one cube surface to another
• Every Gnome desktop has it's own independent cube
• Shift + F9 toggles a Raindrop effect on the screen
• Shift+F10 toggles slow motion