Monday November 20, 2006

Filebench: Request For Enhancements<

The time has come to get Filebench cleaned up (I can't use the word "Productised", I just can't) and this task has fallen to me. Filebench, for those of you not aware is an I/O load-generation framework for simulating the effect of applications on file systems. - its an open source project hosted on Sourceforge and is used a lot within Sun for research, benchmarking and product development. I've been asked to tidy it up, document it and generally give it some love.

There are two real reasons why Filebench is a good thing. Firstly its cheap. Setting up and running some of the industry standard benchmarks (and all application stacks) consumes an enormous amount of engineering time that could be better spent thinking great thoughts, or snowboarding. Filebench gives you back this time. Secondly it provides a language to express the conversation between an application and its persistence layer (what we used to call "storage" before those Java fiends got hold of it). The semantics of this language are very rich and have taken a lot of thought - they might not be right yet - but we can express workloads that mimic real applications without the expense of real applications. How we transfer that converation from the "Record" button (i.e a trace) to "Play" (i.e a workload) is a topic which seems to have kept a lot of academics and others from their snowboards but I'll come back to their endeavours in a future blog.

Whats on The ToDo List?

There are quite a few things on my "To Do" list for Filebench which have been put on it people using it in Sun, our customers and the wider community. Here's what I have:

• A User guide. There is currently no documentation for Filebench. This was cited as the greatest barrier to wider adoption by people I talked to when I presented at the UKCMG conference earlier in the year.
• Multi-client functionality. It needs to work in environments such as clusters, grids, and with proxy filesystems such as CIFS, NFSv3 (and 4) and others such as QFS and SANergy
• Work with block devices. We need this to play with iSCSI, apart from anything else.
• Creation of community site to support further development: It would be good to have more fully formed website and wiki to support Filebench. Sourceforge offers support for this but its all a bit bare-bones at present.
• Tidying up of stable ports to Linux and Microsoft. There are currently ports to these operating systems but they are not entirely stable or properly tested and documented. It is not yet clear whether a Microsoft port should utilise the native C environment or a Microsoft supplied or third party Posix emulation layer.
• General fixing of minor bugs, integrating a backlog of existing bug-fixes, attending to RFEs and so forth.

Anything Else?

If you want to add to this list (or indeed, take things from it; the joy of Open development) your first stop is either mailing me (dominic dot kay at gmail dot com) or the Open Solaris Performance Alias. ( Nov 20 2006, 01:33:01 PM GMT ) Permalink

Fat Software Part 2

I found a little more fat to trim from the midget system I described in my last post. Well actually I found quite a bit but it took me down a cul-de-sac of unbootability so here, after more experimentation are the modifications that are "safe" by which I mean, of course, totally unsupported and unsupportable by Sun but useful if you have to run Solaris on an old, memory-constrained, probably beige, system.

Such a system will never run such new-fangled protocols as Infiniband, FibreChannel and iSCSI, I don't need NFS or any of the enterprise management software. I really don't actually need SCSI or IPv6 - but you can't unbolt those.

So, for the record and so you can cherrypick your own modifications, here are the scores - memory is in 4 Kb pages: I have 63374 of these to my name:

I'm calling it a day at this point. A more diligent man would put all this in a Jumpstart script but time is money and that would be a sign of obsession. The out of the box installation had 7% of its memory free and I've managed to take that to 71%. The kernel (which of course is not pageable) is down to 28 Mb. This old HP Brio has a new lease of life. But brio is the quality of being active or spirited or alive and vigorous - so I'm off to get a life.