Tuesday December 20, 2005

Getting To Grips With NTFS

Getting to grips with NTFS

This blog entry is for the amusement of Solaris folks. Seasoned enterprise Windows administrators, look away.

Figure 1: The Windows Server 2003 Disk Management GUI - Looking Glass, it ain't!

For reasons best not gone into, I have to do I/O benchmarking on Windows Server 2003. To be fair, the command shell language has grown up a lot since I used to teach it to help-desk unfortunates 10 years ago. Scarily, it has adopted Unix style I/O redirection (e.g "2>&1" and so forth) and flow control making it the bastard son of Kornshell and DOS Batch. Hmm. Variable substitution is still reassuringly hellish though.

Configuring up filesystems is not a million miles from format(1M) and metainit(1M). In order to make it more interesting I am working with a server connected to a KVM switch which exports the Windows screen over HTTP. This has some interesting effects;

• My mouse pointer and the mouse pointer on the server get further and further from each other over time (presumably due to lost packets on the net). This results in quite a performance - a sort of mouse driven tai-chi in very slow motion.
• I am working on a laptop running Windows XP. When I went for a coffee break, the server screensaver came on - to log back in I have to give the three fingered salute (ctrl-alt-del). In order to do this without rebooting my laptop I had to walk over to a neighbouring Sunray! It was only later I discovered that the makers of the KVM software have actually thought of that and included menu options for sending that key combination.
• The backslash character is completely unmapped. I have tried every combination of characters on my keyboard and all possible Alt-nnn combinations. I am reduced to running scripts where I have cut and pasted the hateful character in by hand. Ditto the pipe symbol.

Anyway enough whinging. How is it going? Well. To start with, its worth saying that S2003 comes with a graphical volume manager.(Control Panel -> Administrative Tools -> Computer Management -> Disk Management). This, as the Help menu item will re-assure you, is provided to Microsoft by the VERITAS Software Corporation, about which I have written before. Now you have found the GUI, forget you ever saw it for two reasons:

• Its hobbled. Put more delicately it "prohibits you from inadvertently performing actions that may result in data loss". I have to confess I have not even bothered to explore what actions these might be. Bring 'em on.
• As GUIs go, its not very good. This is strange because Veritas are adept at GUIs that have to deal with multiple layers of abstraction. This one is firmly two dimensional. The idea of configuring up filesystems from a couple of hundred spindles using this tool was not a pleasant one. Throw it away and...

Real men in this part of the operating system forest use Diskpart - a command shell which you can use interactively or you can pass it scripts on the command line.This is combined with the Format command, so named to confuse Solaris folk. Diskpart plays the role of format(1M) and metainit(1M). Format plays the role of newfs(1M) or mkfs(1M). Do keep up!

Interaction

I was going to list a quick sample of the filesystem creation process but the KVM web server makes it all too painful to gather the data.. Instead, I feel I probably haven't irritated Jonathon enough by 'fessing up to using XP (my two years at Dell with a Sun Ultra 1 Creator on my desk for similar purposes caused much management wailing and gnashing of teeth) so I'm going to whole-heartedly recommend Bill Stanecks' Windows Command Line Pocket Book from Microsoft Press. If you were brought up in a Unix shell, starting here will cut through a ton of larger books.

The Striping Conundrum

Anyone starting I/O tuning wants to match the modal read and write request of their application to the capabilities of the underlying hardware. The mechanism for achieving this is the volume manager (the functionality of which may or may not be part of the file system but never mind). The mechanics of QFS for example are set out here and its counterpart for Solaris Volume Manager is here. Similar information for Veritas Volume Manager is in this large PDF manual. The point I am making is that anyone with the vaguest interest in tuning their I/O subsystem ends up in this section of the manual for the product of choice. With Windows Server 2003, it was quite hard to find anything on this topic. My web search revealed many pages with bland reassurances similar to;

"With a striped volume, data is divided into blocks and spread in a fixed order among all the disks in the array, similar to spanned volumes. Striping writes files across all disks so that data is added to all disks at the same rate."

This really tells me nothing, or rather it begs more questions than it answers: How big are these blocks? What is the order (round-robin, parallel, ...). How do I change these things? Tantalising isn't it. More digging revealed this gnomic utterance;

"For Windows Server 2003, the size of each stripe is 64 kilobytes (KB)."

Thats it. End of story: I expected more, I really did - especially as I mentioned, its got Veritas written on the label. If I'm missing something (another manual? strange Registry witchcraft?) please let me know (I'm Dominic Kay, I work at Sun; take a wild guess at the email address). I know I could download Mark Russinovich and Bryce Cogswells debugging kit and plough the stack traces to reverse engineer the I/O subsystem for myself but you know that might be cheating. I've looked at the Enterprise, Datacentre and Storage versions of Windows Server 2000 and I can't see the vital difference from the vanilla flavour I'm looking for. By the way, the semantic splicing of those product names is a bit frown inducing, like "Large", "Big" and "Not Small".

So I am left with two possibilities as to why there are no tuning knobs for Windows Server 2003 volume manager;

1. What we have here is "Volume Manager Lite" and storage seriouzos must upgrade to er...Veritas Volume Manager, maybe.
2. This volume manager is so advanced, so weighed down by I/O pattern discovery heuristics and self tuning algorithms that it just does not need tuning. To meddle with this piece of software would be like taking a pasting brush to the Sistine Chapel.

We shall see. Talk to you later.