FileBench ported to Mac OS X
I
recently ported the benchmarking utility FileBench to Mac OSX. The
chief attraction of FileBench is, that one can define the I/O
operations put on the filesystem in something called the "Workload
Definition Language".
So - it's not a fixed benchmark, burden the system with the very same
load pattern over and over again but an amazingly flexible Software
which can simulate various workloads like file servers, databases,
web servers etc.
FileBench
was created by Richard McDougall who is a
Solaris and Systems programming guru and one of the authors of the
famous Solaris Internals book. More information about FileBench could
be found at the official touch base here. FileBench is now included
in OpenSolaris. Take a look here for the sources. There is also a
SourceForge-Version landing page for sources which build on Linux.
Here's an example, how
this definition language looks like:
set $dir=/tmp
set $nfiles=1000
set $meandirwidth=1000000
set $filesize=16k
set $nthreads=16
set $meaniosize=16k
define fileset name=bigfileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwidth,prealloc=80
define process name=filereader,instances=1
{
thread name=filereaderthread,memsize=10m,instances=$nthreads
{
flowop deletefile name=deletefile1,filesetname=bigfileset
flowop createfile name=createfile2,filesetname=bigfileset,fd=1
flowop appendfilerand name=appendfilerand2,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile2,fd=1
flowop closefile name=closefile2,fd=1
flowop openfile name=openfile3,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile3,fd=1
flowop appendfilerand name=appendfilerand3,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile3,fd=1
flowop closefile name=closefile3,fd=1
flowop openfile name=openfile4,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile4,fd=1
flowop closefile name=closefile4,fd=1
}
}
On
the FileBench homepage and in the distributions "workloads"
directory one can find many more examples how to write these
definitions.
Here
is an console-log of loading and running the varmail benchmark:
$
/opt/filebench/bin/go_filebench
FileBench
Version 1.4.4
filebench>
load varmail
1364: 10.634: Varmail
Version 2.1 personality successfully loaded
1364: 10.634: Usage: set
$dir=<dir>
1364: 10.634: set
$filesize=<size> defaults to 16384
1364: 10.634: set
$nfiles=<value> defaults to 1000
1364: 10.634: set
$nthreads=<value> defaults to 16
1364: 10.634: set
$meaniosize=<value> defaults to 16384
1364: 10.634: set
$readiosize=<size> defaults to 1048576
1364: 10.634: set
$meandirwidth=<size> defaults to 1000000
1364: 10.634: (sets mean
dir width and dir depth is calculated as log (width, nfiles)
1364: 10.634: dirdepth
therefore defaults to dir depth of 1 as in postmark
1364: 10.634: set
$meandir lower to increase depth beyond 1 if desired)
1364: 10.634:
1364: 10.634: run
runtime (e.g. run 60)
filebench>
set $dir=/tmp
filebench>
run 10
1364: 26.250:
Creating/pre-allocating files and filesets
1364: 26.252: Fileset
bigfileset: 1000 files, 0 leafdirs avg dir = 0, avg depth = 0.0,
mbytes=4602678819172646912
1364: 26.310: Removed
any existing fileset bigfileset in 1 seconds
1364: 26.314: making
tree for filset /tmp/bigfileset
1364: 26.314: Creating
fileset bigfileset...
1364: 27.009:
Preallocated 807 of 1000 of fileset bigfileset in 1 seconds
1364: 27.009: waiting
for fileset pre-allocation to finish
1364: 27.009: Starting 1
filereader instances
1364: 27.009: Starting
16 filereaderthread threads
1364: 28.010: Running...
1364: 38.011: Run took
10 seconds...
1364: 38.012:
Per-Operation Breakdown
closefile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4
95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3
95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3
96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3
96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3
96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3
96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2
96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2
97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2
97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2
97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1
97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu
1364: 38.012:
IO
Summary: 12515 ops, 1251.4 ops/s, (192/194 r/w) 4.2mb/s,
3587919346114us cpu/op, 40.9ms latency
1364: 38.012: Shutting
down processes
filebench>
stats dump "stats.varmail"
1364: 69.662: Stats dump
to file 'stats.varmail'
1364: 69.662: in
statsdump stats.varmail
filebench>
quit
$
more stats.varmail
Flowop
totals:
closefile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4
95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3
95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3
96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3
96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3
96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3
96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2
96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2
97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2
97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2
97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1
97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu
IO
Summary: 12515 ops 1251.4 ops/s, 192/194 r/w
4.2mb/s, 3587919346114uscpu/op
I've created a Diskimage
including an Apple Installer package which installs the software into
/opt/filebench. Just download the image
file and you could install the program in an Apple fashion:
As a next step the process
of creating the Mac-version from the OpenSolaris sources should be
automated like this:
Here's the first version of the script, the skeleton and a set of patches to automate this.
Posted at 01:18PM Mai 29, 2009
by schmidtm in Mac |