SPECjbb2005 can be run in a multiple-instance mode, where n JVMs run in parallel,
with the benchmark load distributed between the separate JVM processes.
Take a look at the first ever Multiple-Instance result here:
http://www.spec.org/jbb2005/results/res2005q2/jbb2005-20050613-00007.html
The first question you might ask is: What's bops and bops/jvm?
Here's the scores listed:
bops = 75,862
bops / JVM = 18,966
bops is the total benchmark score for the system. It is the sum of the work performed
by each of the individual JVMs.
bops / JVM is the average work performed by each individual JVM.
Now, how does this compare to a single instance result? Take a look at our Sun Fire V40z result:
http://www.spec.org/jbb2005/results/res2005q2/jbb2005-20050613-00003.html
Here's the scores listed:
bops = 37,034
bops / JVM = 37,034
bops and bops/JVM are identical. Makes sense since this is a single instance result and therefore there is only 1 JVM.
So how do you compare single and multiple instance results? First a bit of background.
The single instance results highlight hardware, OS, and JVM performance and scalability. The multiple instance results highlight hardware, OS, JVM performance and scalability, and JVM interoperability.
Java is deployed in the enterprise in many different ways and a common production environment is a large SMP server running many (10s to 100s) JVMs. The multiple instance configuration allow JVM and hardware vendors to demonstrate this common deployment models in a public competitive benchmark, especially useful on large SMP servers. This was a great idea.
So again, how do you compare single and multiple results? Use the bops score alone.
A comparison between these two configurations ends up being a limited comparison of system scalability and performance. Limited in that hardware and OS scalability is tested, but JVM scalability has been limited to include the effect of many JVMs working together.
So here's a comparsion on the Sun V40z 4-way result and the Sun V890 8-way results:
4-way V40z: 37034 bops
8-way V890: 75862 bops