The Sun Fire 25K and Sun J2SE 5.0_08 team up to demonstrate leadership on large servers running SPECjbb2005, increasing performance by 19.1% over our previous submission on the same hardware. Not bad for 6 months of performance work! The 72-way Sun Fire 25K score is 1,387,437 SPECjbb2005 bops, 19,270 SPECjbb2005 bops/JVM. That is 11% faster than the 128-way Fujitsu PRIMEPOWER 2500 and many times faster than IBM's fastest SPECjbb2005 result to date. The BMSeer once again beats me the punch talking about SPECjbb2005 results, he/she (who is BMSeer anyway?) has a great piece talking about this result. Required Disclosure Statement SPECjbb2005 Sun Fire E25K (72-way, 72 chips, 144 cores) 1,387,437 SPECjbb2005 bops, 19,270 SPECjbb2005 bops/JVM, Fujitsu PRIMEPOWER 2500 (128 chips, 128 cores) 1,251,024 SPECjbb2005 bops, 39,095 SPECjbb2005 bops/JVM, IBM eServer p5 570 (8 chips, 16 cores, 16-way) 244,361 SPECjbb2005 bops, 30,545 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 06/19/06 on www.spec.org.

Here's my latest round of platform performance comparisons using Scimark. This time I compare Java to C# and once again Java performance is looking quite good. Thanks to Tony Zhang, another colleague of mine on the performance team who ran the initial performance comparision a few months back and provided me the environment to re-run the tests with our latest JVMs. The system under test was a 4 CPU Intel Xeon MP server (4x2.78 GHz, 8 cores, 3.87 GB memory) running Microsoft Windows 2003 Server and .NET 2.0. The CLR version under test according to SciMark was 2.0.50727.42. We used the Scimark 2.0 C# port found here. The HotSpot server compiler (-server) was used for both J2SE 5.0_08 and Java SE 6 b87. SciMark was run with the large data set (-large). Also, I found the chart below in an interesting writeup showing similar performance comparisions with older versions of the JVM. I particularly like HotSpot's performance lead over JRockit.

I received a few comments on my previous blog entry saying the results were bogus since I used an old compiler. I quickly found another test system running Suse SLES 9 U2 with gcc 3.3.3 and repeated the test. If I get around to installing the latest Visual Studio I'll repeat the test there as well. The JVM versions are different as I wanted to quickly post the results. Guess what, the results are a lot better! I ran this several times and its quite repeatable. I appreciate comments so please let me know what your thoughts. Especially if there are issues with the choice of gcc 3.3.3. The system under test was a 2 x 3.0Ghz Intel Xeon MP System (4-core) running Suse SLES 9 U2 and gcc 3.3.3. The C code was compiled with full optimization as shown by the Makefile in the SciMark source package. This time no tuning parameters were used for either 5.0_08 or 6.0 b83. Here's some output from /proc/cpuinfo: vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) MP CPU 3.00GHz For background here's the skinny on SciMark2. Scimark2 is a set of simple numerical kernels and its performance is directly related to the performance and quality of the generated code. The tests are single threaded and have little to no garbage collection overhead. In short, a great set of applications to compare statically compiled C code and dynamically compiled Java. This time Java is 35% faster than C. Here's a breakdown of the subtests. C is only ahead on Sparse MatMult by a small margin. Any one interested to see how the other JVM vendors look? Can JRockit or IBM beat C?

This is quite cool. Andy Johnson, a colleague of mine on the Java performance team, did a few performance tests comparing Java to native C. SciMark2 was used for the performance comparision. The system under test was a 2Ghz Pentium white box running Windows 2000 and using the Microsoft Visual C/C++ 6. The C code was compiled with full optimization. The server compiler was used for both J2SE 5.0_07 and Java SE 6. Scimark2 is a set of simple numerical kernels and its performance is directly related to the performance and quality of the generated code. The tests are single threaded and have little to no garbage collection overhead. In short, a great set of applications to compare statically compiled C code and dynamically compiled Java. The chart below is quite revealing. Both the charts are normalized to J2SE 5.0_07. Native C is only 3% faster than 5.0_07 and Java SE 6 pulls ahead of native C by 2%. The following chart breaks the comparison down further. Remember SciMark2 is a composite benchmark and the overall score is a simple mean of each subtest mflops score. With that, Java is ahead in some cases, and behind in others. Actually Java is ahead in all cases except Sparse Matmult. Looks like we have something to look at for additional optimization.

The performance of Java on Sun CoolThreads servers continues to be impressive. Our latest round of improvements have increased performance on SPECjbb2005 by 17% on the Sun Fire T1000 and T2000. If you thought the competitive positioning of these systems was impressive before, take a look at them now. The charts below represent the competitive landscape for the Sun CoolThreads servers and by no means are they meant to be a complete comparison of all systems in the classes described below. If there are particular descrepencies that are annoying, please let me know. For more detailed information on the Sun Fire T1000 and T2000 and comparisions running competitive benchmarks check out BMSeer's blog. The first chart shows the competitive landscape for 1 RU servers. The Sun Fire T1000 shines compared to other systems in this space. The Sun Fire X4100 (powered by AMD Opteron CPUs) looks rather good as well. The second chart shows the competitive landscape for 2 RU and 4 RU servers. The Sun Fire T2000 shows impressive performance against the competition in this space as well. Now this is were the Sun Fire T1000 and Sun Fire T2000 truly excel. The first power performance graph shows a comparision based on performance per watt using the SPECjbb2005 bops metric. The data presented is limited to what I've gathered using The Sun Fire CoolThreads systems and what has been gathered on http://www.sun.com/coolthreads. Here's another look at power performance using the SWaP metric. The SWaP metric is similar to performance / Watt, but includes system footprint as a part of the equation. The Sun Fire T1000 number is impressive. The light bulb next to my workbench in my basement uses more power than this server. For those individuals who prefer a spreadsheet to charts, here the same information as show above. Finally, this chart shows the performance difference between J2SE 5.0_06 and J2SE 5.0_08 on the same hardware, demonstrating a 17% increase in performance on both the Sun Fire T1000 and Sun Fire T2000. If we can improve performance by 17% in 6 months, wait to you see what Java SE 6 ("Mustang") can do. Required Disclosure Statement: SPECjbb2005 Sun Fire T1000 (1 chip, 8 cores) 51,528 SPECjbb2005 bops, 12,882 SPECjbb2005 bops/JVM submitted for review; SPECjbb2005 Sun Fire T2000 (1 chip, 8 cores) 74,365 SPECjbb2005 bops, 18,591 SPECjbb2005 bops/JVM submitted for review; Sun Fire X4100 (2 chips, 2 cores) 38,090 SPECjbb2005 bops, 19,045 SPECjbb2005 bops/JVM submitted for review; IBM eServer p5 550 (2 chips, 4 cores) 61,789 SPECjbb2005 bops, 61,789 SPECjbb2005 bops/JVM; IBM x346 (2 chips, 4 cores) 39,585 SPECjbb2005 bops, 39,585 SPECjbb2005 bops/JVM; IBM eServer p5 520 (1 chip, 2 cores) 32,820 SPECjbb2005 bops, 32,820 SPECjbb2005 bops/JVM; IBM eServer p5 510 (1 chip, 2 cores) 36,039 SPECjbb2005 bops, 36,039 SPECjbb2005 bops/JVM; Fujitsu Siemens RX220 (2 chips, 2 cores) 61,155 SPECjbb2005 bops, 30,578 SPECjbb2005 bops/JVM, Dell PE SC1425 (2 chips, 2 cores) 24,208 SPECjbb2005 bops, 24208 SPECjbb2005 bops/JVM; Dell PE 850 (1 chips, 2 cores) 31,138 SPECjbb2005 bops, 31,138 SPECjbb2005 bops/JVM; Dell PE 2950 (2 chips, 4 cores) 64,288 SPECjbb2005 bops, 64,288 SPECjbb2005 bops/JVM; SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 6/02/06 on www.spec.org