Thursday November 30, 2006

twm(1) - The Directors Cut

As a postscript to my blogs on squeezing Solaris 10 onto an antique PC (here and here), Dave Levy commented that I should provide a screenshot of Tom's Window Manager, as a means to point out the irony that doing a screen grab of TWM is a recursive problem - the tools ain't there to do it. Oh well. I shan't restate the case for it because the Wikipedia entry sums it up - "Although it is now generally regarded as the window manager of last resort, a small but dedicated minority of users favor twm for its simplicity, customizability, and light weight". You can leap from the Wiki entry to an interesting interview with Tom himself.

	As you can see, twm(1) has the rich functionality that you need but without all that troublesome clutter.
	Solaris Performance and Tools and Solaris Internals (2nd Ed) make a firm foundation for any tuning exercise such as this. Rich Teer's Solaris Systems Programming also provides useful supporting documentation.

The quid pro quo I negotiated with Dave was that he in turn would blog an analysis of the case for commercial enterprises such as Sun open-sourcing their software set in the context of classical economic theory. I know I don't understand this properly because listening to Dave explain it is like holding up the TV aerial - it seems clear while he holds it up and explains but the moment he finishes and puts the aerial down, my screen goes fuzzy again.

Monday November 20, 2006

Filebench: Request For Enhancements<

The time has come to get Filebench cleaned up (I can't use the word "Productised", I just can't) and this task has fallen to me. Filebench, for those of you not aware is an I/O load-generation framework for simulating the effect of applications on file systems. - its an open source project hosted on Sourceforge and is used a lot within Sun for research, benchmarking and product development. I've been asked to tidy it up, document it and generally give it some love.

There are two real reasons why Filebench is a good thing. Firstly its cheap. Setting up and running some of the industry standard benchmarks (and all application stacks) consumes an enormous amount of engineering time that could be better spent thinking great thoughts, or snowboarding. Filebench gives you back this time. Secondly it provides a language to express the conversation between an application and its persistence layer (what we used to call "storage" before those Java fiends got hold of it). The semantics of this language are very rich and have taken a lot of thought - they might not be right yet - but we can express workloads that mimic real applications without the expense of real applications. How we transfer that converation from the "Record" button (i.e a trace) to "Play" (i.e a workload) is a topic which seems to have kept a lot of academics and others from their snowboards but I'll come back to their endeavours in a future blog.

Whats on The ToDo List?

There are quite a few things on my "To Do" list for Filebench which have been put on it people using it in Sun, our customers and the wider community. Here's what I have:

• A User guide. There is currently no documentation for Filebench. This was cited as the greatest barrier to wider adoption by people I talked to when I presented at the UKCMG conference earlier in the year.
• Multi-client functionality. It needs to work in environments such as clusters, grids, and with proxy filesystems such as CIFS, NFSv3 (and 4) and others such as QFS and SANergy
• Work with block devices. We need this to play with iSCSI, apart from anything else.
• Creation of community site to support further development: It would be good to have more fully formed website and wiki to support Filebench. Sourceforge offers support for this but its all a bit bare-bones at present.
• Tidying up of stable ports to Linux and Microsoft. There are currently ports to these operating systems but they are not entirely stable or properly tested and documented. It is not yet clear whether a Microsoft port should utilise the native C environment or a Microsoft supplied or third party Posix emulation layer.
• General fixing of minor bugs, integrating a backlog of existing bug-fixes, attending to RFEs and so forth.

Anything Else?

If you want to add to this list (or indeed, take things from it; the joy of Open development) your first stop is either mailing me (dominic dot kay at gmail dot com) or the Open Solaris Performance Alias. ( Nov 20 2006, 01:33:01 PM GMT ) Permalink

Fat Software Part 2

I found a little more fat to trim from the midget system I described in my last post. Well actually I found quite a bit but it took me down a cul-de-sac of unbootability so here, after more experimentation are the modifications that are "safe" by which I mean, of course, totally unsupported and unsupportable by Sun but useful if you have to run Solaris on an old, memory-constrained, probably beige, system.

Such a system will never run such new-fangled protocols as Infiniband, FibreChannel and iSCSI, I don't need NFS or any of the enterprise management software. I really don't actually need SCSI or IPv6 - but you can't unbolt those.

So, for the record and so you can cherrypick your own modifications, here are the scores - memory is in 4 Kb pages: I have 63374 of these to my name:

I'm calling it a day at this point. A more diligent man would put all this in a Jumpstart script but time is money and that would be a sign of obsession. The out of the box installation had 7% of its memory free and I've managed to take that to 71%. The kernel (which of course is not pageable) is down to 28 Mb. This old HP Brio has a new lease of life. But brio is the quality of being active or spirited or alive and vigorous - so I'm off to get a life.

Wednesday November 15, 2006

French Customs and Fat Software

Never ever ever tell French Customs that you have just driven from Amsterdam - you will lose an hour of your life. This happened to me at Calais on my way home to UK. I always get stopped by Customs when I'm on my own. Its because I drive a big shiny German car but wear scruffy clothes and don't shave often enough for their liking. If I cleaned up and put a suit on for these journeys, I'd get left alone but I always forget. If I was a customs officer, I'd stop me.

Anyway, due to my arrival from the intoxicant capital of Europe I got the full treatment this time. They even X-rayed my spare tyre (no, the one in the car, not 'round my waist) and it is in fine health. They were very suspicious when I told them I don't smoke - not just dope, but anything. They all smoked - continuously. Then when they found that the 6 bottles of wine I had (only 6, Monsieur?) were from Portugal and not their beloved France, that was it. Out came the back seat of the car, pockets were emptied. carpets were lifted. I just kept smiling - They have guns after all, and latex surgical gloves - far more scary. Anyway, if I'm ever stopped again; I've just come from Lourdes.

I was coming home because remote working isn't working any more: my laptop is broken. Since my daughter peeled all the keys from the keyboard, it has never been quite the same and now one of them is permanently detached - queue renditions of "U picked a fine time to leave me, Lucille", "I'll never find another U", etc. Try working without this vowel - its impossible. So off to the laptop garage it goes.

This leaves me with a 8 year old PC in my office which was given to me as an alternative to putting it in landfill. Bye bye $3500 Ferarri laptop, hello ageing HP Brio. It hasn't much memory (256 Mb), or a DVD drive, or sound and the screen has got pen marks all over it because it came out of a goods-despatch office in a warehouse. But hey, it was free.

I load up Solaris 10 6/06 (Developer install profile), log in to Gnome and it...eh...stops. That is to say the visual element of the great Solaris experience stops - the meat grinder noise continues under the desk. Since then, its taken me a fair amount of performance tuning to get useability out of it - a sort of "back to basics" exercise.

My first suspicion was that the delicious new GUI that Solaris sports was the culprit for all the thrashing below. I used to say that using Unix was fine so long as one remembered that the first ten years would be the worst part. Now? You don't have to know anything about anything to be a performance analyst. You just read Solaris Performance & Tools and you're away. Observability? Look at this:

# echo "::memstat" | mdb -k 
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                      19522                76   31%
Anon                        20766                81   33%
Exec and libs                7685                30   12%
Page cache                   2095                 8    3%
Free (cachelist)            10547                41   17%
Free (freelist)              2759                10    4%

Total                       63374               247

...then turn to page 646 of the book, distill the content onto a Powerpoint slide and present your invoice Voila! (as the Customs men were dying to shout, but couldn't).

So is the windowing software fat? Well actually no. It is fatter than it was as the graph below shows (the Y axis indicates where my precious 247 megabytes have gone - for all the data points we have one terminal window open on the desktop) but not really podgy like some operating systems window mangers. The real problem though was that unknown to me (then), the web management software starts a honking great JVM - thats where all the memory goes.

So my choice was to go out and buy some hardware or execute a slash and burn strategy on memory consumption. Well the PC was free, the OS was free and the authors gave me the book for free (all as in "free beer"). Enough said. I did the numbers at each stage as you can see in the graph below:

Stage 1. This is my baseline - With Gnome and 1 terminal window open.

Stage 2. With CDE, 1 terminal window open.

Stage 3. Disable the windowing system altogether (dtconfig -d).

Stage 4. In /etc/rc2.d get rid of (i.e rename) S90wbem and S90webconsole: As I mentioned they start a very large (for the RAM I have to play with) JVM. This was the biggest single win.

Stage 5. In /etc/rc3.d, rid yourself of S76snmpdx, S77dmi and S82initsma

Stage 6. Disable "uneeded" (opinions differ here) service deamons using svcadm disable: sendmail, nfs/cbd, nfs/mapid, nfs/server, nfs/status, nfs/nlockmgr, nfs/rquota, nfs/client, network/ssh, system/power, system/picl, filesystem/autofs

Stage 7. Fire up twm(1) using xinit(1). For those who cannot remember Toms Window Manager, you can enjoy its rich functionality by choosing a Failsafe session from the options on your Gnome/CDE login screen and then logging in. In the (only) shell window you are presented type/usr/openwin/bin/twm and there you are. Now run away. Fast.

There is probably more fat I could trim (please email me your favourite culprits: dominic dot kay at gmail dot com) but the effort/mempages ratio has peaked and the day-job beckons.

And so here I am. I have an antique computer running a windowing system no-one can remember how to use but my ps(1) listing doesn't scroll off the screen anymore and it boots up in bounded time and goes like the wind. It runs Openoffice, Firefox, Sun Studio 11 (with special "compile during coffee break" option enabled). My X Windows Users and Administrators Guides (from O'Reilly circa 1992, from my attic yesterday) have new value and I'm high on life - thanks to losing "U" in Amsterdam.

I should make it clear by way of postscript that I was not in Amsterdam because it is the preferred haunt of those who like their tobacco to have a fuller flavour but because my brother has a fabulously comfortable house there with a spare room, a DSL connection and a fridge full of food. As far as intoxication went - I took everything from my cellar dated 1995 to enjoy with him. Most of the claret was starting to fade. I take this as a sign that I should giddy-up and attack the 1996's at Christmas. Mais ne pas les drogues et stupifients. No, I haven't got any. Really.