BM Seer Facts & Questions from an Anonymous Sun Source

Sun Fire X4150 SPECjbb World Record x86 Single-JVM and x86 2 Chip Multi-JVM Performance

Monday Feb 18, 2008

The Sun Fire X4150 server equipped with 2 Quad-core Intel Xeon processors obtained World Record x86 single-JVM and x86 2 chip multi-JVM results on the SPECjbb2005 benchmark. Enhancements to the JVM had a major impact on performance.

The Sun Fire X4150 with 2 Intel X5460 quad-core processors and running Sun J2SE 1.6.0_05-p achieved x86 2-chip World Record performance of 303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark for multi-JVM results.

The Sun Fire X4150 with 2 Intel X5460 quad-core processors and running Sun J2SE 1.6.0_05-p achieved x86 World Record performance of 277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM for a single JVM on the SPECjbb2005 benchmark.

Using the same processor, the Sun Fire X4150 with Solaris 10 and Java HotSpot(TM) 32-Bit Server, beat the results of Dell and Lenovo which used Windows and BEA JRockit on the multi-JVM test.

The Sun Fire X4150 running the single-JVM SPECjbb2005 test easily beat all x86 results, topping the Dell R200 by 1.98X, the Fujitsu BX620 by 2.0X and the SGI XE240 by nearly 2.1X.

SPECjbb2005 Performance Chart (ordered by performance, bops: SPECjbb2005 Business Operations per Second (bigger is better), selected results... my best guess at the top ones for the engineers who act like lawyers you can go to www.spec.org to see all results as it clearly states at the bottom.

System Processors Performance
Chip Core Thr GHz Type SPEC-
jbb-
2005
bops 		JVMs SPEC-
jbb-
2005

bops/JVM
Multi-JVM, 2-Chip x86 Results (selected top see note above)
Sun Fire X4150 2 8 8 3.16 X5460 303297 4 75824
Dell PowerEdge 2950 2 8 8 3.16 X5460 303130 4 75783
Lenovo R515 2 8 8 3.16 X5460 294716 4 73678
Single-JVM x86 Results (selected top, see note above)
Sun Fire X4150 2 8 8 3.16 X5460 277585 1 277585
Dell PowerEdge R200 1 4 4 2.66 X3230 140220 1 140220
Fujitsu BX620 2 4 4 3.0 5160 138388 1 138388
SGI Altix XE240 2 4 4 3 5160 134561 1 134561
Dell PowerEdge 2950 2 4 4 3 5160 130589 1 130589

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 2/8/2008 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores) 303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM; Dell PowerEdge 2950 (2 chips, 8 cores) 303130 SPECjbb2005 bops, 75783 SPECjbb2005 bops/JVM; Lenovo R515 (2 chips, 8 cores) 294716 SPECjbb2005 bops, 73678 SPECjbb2005 bops/JVM.

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 2/8/2008 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores) 277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM; Dell PowerEdge R200 (1 chip, 4 cores) 140220 SPECjbb2005 bops, 140220 SPECjbb2005 bops/JVM; Fujitsu BX620 (2 chips, 4 cores) 138388 SPECjbb2005 bops, 138388 SPECjbb2005 bops/JVM; SGI Altix XE240 (2 chips, 4 cores) 134561 SPECjbb2005 bops, 134561 SPECjbb2005 bops/JVM; Dell PowerEdge 2950 (2 chips, 4 cores) 130589 SPECjbb2005 bops, 130589 SPECjbb2005 bops/JVM.

Results Summary

Reference Date: Feb 8, 2008
Multi-JVM 303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM
Single-JVM 277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM
System: Sun Fire X4150
Processor: 2 x Intel X5460 3.166 GHz
Operating System: Solaris 10 8/07
JVM: Java HotSpot(TM) 32-Bit Server, Version 1.6.0_05-p

[7] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 12:15PM Feb 18, 2008 by BM Seer in Benchmark Issues
Tags:
Comments:

I pity the engineer who had to sit there running SPECjbb2005 over again until it got 0.05% higher than the Dell number.

Posted by rick jones on February 21, 2008 at 07:58 AM PST #

I guess you also pity athletes who beat world records by 0.05%.

Posted by Dmitri Trembovetski on February 21, 2008 at 10:00 AM PST #

Clearly, this isn't about running jbb over and over it is about engineers who continually work on improving JVM for everyone and then an engineer publishing a results at various points.

Posted by BM Seer on February 21, 2008 at 12:15 PM PST #

Did you hack the BIOS?

Posted by Wes Felter on February 21, 2008 at 01:52 PM PST #

That is just an outright lie! The Sun result contains the same "hacked BIOS" that bmseer has shed great tears about for years.

Better get some insurance for that glass house of yours - and check your facts next time.

Waiting for a retraction...

Posted by PantsOnFire on February 21, 2008 at 08:16 PM PST #

I don't pity athletes who barely break records because we are (at least it seems) approaching the limits of the human body. We've not yet hit the limits of computers. And in this case "major impact" performance changes were touted, which sets an expectation of better than a 0.05% (aka a tie) difference.

If that four JVM figure wasn't run over and over again, one is left wondering just how bad it was before. Given the same CPUs at the same frequency, one would expect more from "major impact" performance changes.

It wasn't mentioned in your writeup, but I'm assuming the x4150 result is still under review as I couldn't find it on www.spec.org. So we cannot yet find complete benchmark results unless there is an FDR somewhere else?

Posted by rick jones on February 22, 2008 at 07:25 AM PST #

Ok, made a mistake (not a lie, get a life). But all of you think I'm am some official source, I am not. This is a blog. It seems that Sun used the same hacked BIOS as the rest of the Xeon crowd to keep up with the "Jones".

Leave BIOS default and see a 25% reduction in some performance:
http://blogs.sun.com/bmseer/entry/intel_defaults_and_judging_performance

I personally think changing a BIOS for a particular workload is not a good or normal thing to do.

Posted by BM Seer on February 22, 2008 at 08:28 AM PST #

Post a Comment:
Comments are closed for this entry.