FileBench ported to Mac OS X
I
recently ported the benchmarking utility FileBench to Mac OSX. The
chief attraction of FileBench is, that one can define the I/O
operations put on the filesystem in something called the "Workload
Definition Language".
So - it's not a fixed benchmark, burden the system with the very same
load pattern over and over again but an amazingly flexible Software
which can simulate various workloads like file servers, databases,
web servers etc.
FileBench
was created by Richard McDougall who is a
Solaris and Systems programming guru and one of the authors of the
famous Solaris Internals book. More information about FileBench could
be found at the official touch base here. FileBench is now included
in OpenSolaris. Take a look here for the sources. There is also a
SourceForge-Version landing page for sources which build on Linux.
Here's an example, how
this definition language looks like:
set $dir=/tmp
set $nfiles=1000
set $meandirwidth=1000000
set $filesize=16k
set $nthreads=16
set $meaniosize=16k
define fileset name=bigfileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwidth,prealloc=80
define process name=filereader,instances=1
{
thread name=filereaderthread,memsize=10m,instances=$nthreads
{
flowop deletefile name=deletefile1,filesetname=bigfileset
flowop createfile name=createfile2,filesetname=bigfileset,fd=1
flowop appendfilerand name=appendfilerand2,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile2,fd=1
flowop closefile name=closefile2,fd=1
flowop openfile name=openfile3,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile3,fd=1
flowop appendfilerand name=appendfilerand3,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile3,fd=1
flowop closefile name=closefile3,fd=1
flowop openfile name=openfile4,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile4,fd=1
flowop closefile name=closefile4,fd=1
}
}
On
the FileBench homepage and in the distributions "workloads"
directory one can find many more examples how to write these
definitions.
Here
is an console-log of loading and running the varmail benchmark:
$
/opt/filebench/bin/go_filebench
FileBench
Version 1.4.4
filebench>
load varmail
1364: 10.634: Varmail
Version 2.1 personality successfully loaded
1364: 10.634: Usage: set
$dir=<dir>
1364: 10.634: set
$filesize=<size> defaults to 16384
1364: 10.634: set
$nfiles=<value> defaults to 1000
1364: 10.634: set
$nthreads=<value> defaults to 16
1364: 10.634: set
$meaniosize=<value> defaults to 16384
1364: 10.634: set
$readiosize=<size> defaults to 1048576
1364: 10.634: set
$meandirwidth=<size> defaults to 1000000
1364: 10.634: (sets mean
dir width and dir depth is calculated as log (width, nfiles)
1364: 10.634: dirdepth
therefore defaults to dir depth of 1 as in postmark
1364: 10.634: set
$meandir lower to increase depth beyond 1 if desired)
1364: 10.634:
1364: 10.634: run
runtime (e.g. run 60)
filebench>
set $dir=/tmp
filebench>
run 10
1364: 26.250:
Creating/pre-allocating files and filesets
1364: 26.252: Fileset
bigfileset: 1000 files, 0 leafdirs avg dir = 0, avg depth = 0.0,
mbytes=4602678819172646912
1364: 26.310: Removed
any existing fileset bigfileset in 1 seconds
1364: 26.314: making
tree for filset /tmp/bigfileset
1364: 26.314: Creating
fileset bigfileset...
1364: 27.009:
Preallocated 807 of 1000 of fileset bigfileset in 1 seconds
1364: 27.009: waiting
for fileset pre-allocation to finish
1364: 27.009: Starting 1
filereader instances
1364: 27.009: Starting
16 filereaderthread threads
1364: 28.010: Running...
1364: 38.011: Run took
10 seconds...
1364: 38.012:
Per-Operation Breakdown
closefile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4
95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3
95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3
96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3
96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3
96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3
96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2
96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2
97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2
97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2
97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1
97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu
1364: 38.012:
IO
Summary: 12515 ops, 1251.4 ops/s, (192/194 r/w) 4.2mb/s,
3587919346114us cpu/op, 40.9ms latency
1364: 38.012: Shutting
down processes
filebench>
stats dump "stats.varmail"
1364: 69.662: Stats dump
to file 'stats.varmail'
1364: 69.662: in
statsdump stats.varmail
filebench>
quit
$
more stats.varmail
Flowop
totals:
closefile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4
95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4
95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3
95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3
96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3
96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3
96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3
96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2
96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2
97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2
97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2
97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1
97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu
IO
Summary: 12515 ops 1251.4 ops/s, 192/194 r/w
4.2mb/s, 3587919346114uscpu/op
I've created a Diskimage
including an Apple Installer package which installs the software into
/opt/filebench. Just download the image
file and you could install the program in an Apple fashion:
As a next step the process
of creating the Mac-version from the OpenSolaris sources should be
automated like this:
Here's the first version of the script, the skeleton and a set of patches to automate this.
Posted at 01:18PM Mai 29, 2009
by schmidtm in Mac |
Great, pity for me it is Intel and not PPC MacOS.
I'm happy to build a PPC version if you tell me how.
Gesendet von Darren Moffat am Mai 29, 2009 at 03:11 PM CEST #
I just tried to build the last version from sourceforge on my mac yesterday. It failed with the error:
In file included from procflow.h:33,
from filebench.h:57,
from auto_comp.c:32:
vars.h:63: error: syntax error before ‘boolean_t’
vars.h:63: warning: no semicolon at end of struct or union
vars.h:63: warning: no semicolon at end of struct or union
...
I had to use the version 1.3.4 to be able to build it on my mac.
Have you built the last version from sourceforge? I'd be curious to know how you have proceeded!
Gesendet von Gaëtan am Mai 29, 2009 at 07:27 PM CEST #
Hi Darren, unfortunately I don't have an up to date development environment on my PPC Macs anymore.
These are the sources I've used for the Intel version:
http://blogs.sun.com/schmidtm/resource/filebench/filebench-1.4.4-MAC-NO-XANADU.tar.gz
It should compile on PPC fine. Give it a go and drop me a mail about the outcome. schmidtm at sun dot com
Gesendet von Matthias Schmidt am Juni 02, 2009 at 10:18 AM CEST #
Hi Gaëtan - the SF 1.4.4 version won't compile out of the box. I had to patch it 7-8 times. Fetch the sources from the comment above and give it a try.
Or to patch it yourself take a look at the separate patches here:
http://blogs.sun.com/schmidtm/resource/filebench/create-filebench-mac.tar.gz
Gesendet von Matthias Schmidt am Juni 02, 2009 at 10:22 AM CEST #