Benchmarking has been around for years, and I started playing around with writing them back in college.  I recall an assembly language class were the assignment was to implement the Sieve of Eratosthenes in as few bytes as possible. I "won" the assignment by doing it in 24 (or was it 25, too many years ago!) bytes, but that got me started on benchmarking as I instrumented the code to time how long it took and then proceeded to run it on all the different machines I could to see which one was the fastest.

Clearly though, having a benchmark written in 8086 assembly language had limited usefulness, so in later years I rewrote it in C and added several other standard algorithms to the stable that were executed including calculating PI, a selection sort, and computing Ackermann's function to test recursive performance. I proceeded to run it on every machine I could from PCs running MS-DOS to workstations running Solaris to PPC machines running LynxOS and AIX. In later years, I ran the same benchmark on UltraSPARC, SGI MIPS, and DEC Alpha machines as well as today's dual-core Intel chips. Somewhere along the line I starting writing more in Java than C and I ported the benchmark to Java to see how it compared, and several years ago I began using BlackBerry's and wrote a Java ME version.

As you can imagine given the wide range of machines I've run it on in the past, having an iPhone was just begging to run it there too. While I have creating a proper iPhone app version on my list of things to do, thankfully a jailbroken iPhone makes it pretty easy to run the C and Java versions I already had. A sampling of results appears below, but the summary for the iPhone is that the native version is about 5 times faster than the Java one and ever so slightly faster than a 75 MHz Pentium 5 of yesteryear.

Machine	Language	Math	Array	Sort	Sieve	Ackermann	Total
Motorola L6	Java ME	0.11	0.19	0.23	0.13	0.19	0.17
BlackBerry Curve (OS 4.5)	Java ME	0.95	2.21	2.96	1.72	2.96	1.81
iPhone 3G	Java SE	12.4	4.79	4.64	3.28	5.01	4.95
Intel 486DX2-66 (Linux)	C	10	10	10	10	10	10
SPARC-20 (Solaris)	C	66.4	20.3	17.6	20.1	4.1	12.1
Pentium 5 75 MHz (Linux)	C	21.3	19.6	28.6	23.2	33.4	24.2
iPhone 3G	C	65.9	21	23.4	19.2	46.7	27.9
DEC Alpha 275 MHz (OSF/1)	C	111.4	93.6	63.1	41.2	76.1	68.6
Pentium 600 MHz (Linux)	Java SE	191.9	106.1	99.1	68.9	78.2	96.1
Pentium 600 MHz (Linux)	C	216.4	319.3	219.8	171.5	131.2	194.1
MacBook Pro 2.33 GHz	Java SE	655	1096	794	549	334	587
MacBook Pro 2.33 GHz	C	1089	2413	664	800	1356	1037

You may also view the full list that I have maintained over the years.  You'll notice that I did my initial calibration on a 66 MHz 486 DX-2 so that it scores 10 in each of the tests.

Now, I'll be the first to admit that my benchmark is pretty simplistic and thus shouldn't be be the end all be all testing to compare machines or platforms. The results can also be significantly affected by the quality of the C compiler or JVM being used. However, the results can provide a rough comparison and are at least fun to look at.

For example a SPARC-20 has terrible performance with deep recursion. And as noted above the JVM on the iPhone is about 5 times slower than a native app whereas on an x86 box the JVM is about half as fast as native.

While I've run the benchmark on quite a few machines, I'd love to add more to the list.  If you'd like to run it, here is the binaries in a variety of formats:

• Linux on x86 - Download the binary, set the executable flag, and execute with "./sci <iterations>". On most of today's faster machines iterations should be 10,000 to have reasonable results.
• OS X on x86 - Download the binary, set the executable flag, and execute with "./sci <iterations>". On most of today's faster machines iterations should be 10,000 to have reasonable results.
• OS X on ARM/iPhone - Download the binary, use ssh/scp to get it on your jailbroken iPhone, and execute with "./sci <iterations>". The default iterations should be fine.
• Solaris on x86 - Download the binary, set the executable flag, and execute with "./sci <iterations>". On most of today's faster machines iterations should be 10,000 to have reasonable results.
• Solaris on SPARC - Download the binary, set the executable flag, and execute with "./sci <iterations>". On most of today's faster machines iterations should be 10,000 to have reasonable results.
• AIX on RS/6000 - Download the binary, set the executable flag, and execute with "./sci <iterations>". On most of today's faster machines iterations should be 10,000 to have reasonable results.
• Java SE - Download the jar and execute it on any machine you have a JVM with "java -jar SCIJ.jar <iterations>". On most of today's faster machines iterations should be at least 5,000.
• Java ME devices - Point your phone/device browser at the link and it should prompt to install the application. Run the app and if it executes in less than a few seconds change the iterations to a larger number to make it more accurate in calculating the benchmark.

• NetBeans 6.7 Released - New features including integration with Project Kenai, native support for Maven, support for GlassFish (v3) and more.
• Six ways to make SOA services more reusable - Nothing ground breaking or new, but always good to reiterate decoupling the transport from the logic, having standard interfaces, and testing for interoperability.