David Dagastine's Weblog

%%TNuL836K%%

« September 2008
Sun	Mon	Tue	Wed	Thu	Fri	Sat
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30

Today

The first ever SPECjvm2008 result has been published on the SPEC website. The Sun result stands alone, with no other hardware or software vendor stepping forward. This is the example of the confidence we have in HotSpot JVM and OpenJDK, and their abilities to strive in a broad range computing environments.

Also note that this is a SPECjvm2008 Base submission, with No JVM Tuning Required. SPECjvm2008 consists of 11 sub-components, some of which have multiple sub-test, such as crypto, scimark, and xml. As I've said before, this is a bear of a JVM benchmark, so improving the composite score requires significant improvement of its parts. Sun has not only submitted the first SPECjvm2008 result, but did so by submitting a SPECjvm2008 base submission only, hopefully setting the tone for competitive comparison moving forward.

Confident Performance. It has a nice ring to it.

I'd like to thank Apache Harmony for their JDK library performance efforts. We were given a tip that the Harmony folks were doing some interesting work with the TreeMap collection class, and low and behold they were. The work is surrounding fat TreeMap nodes where each node contains several TreeMap entries. This greatly improves in order traversal of the TreeMap entries, and since SPECjbb2005 traverses over a TreeMap I can see why they did this. Controlled measurements showed the "fat" TreeMap significantly faster with in order traversals, and improved our SPECjbb2005 score by a solid 3-5% depending on the platform.

Considering the potential performance gain we started the effort of porting the Apache Harmony TreeMap code to JDK 6. We ran into a few performance snags in our performance regression testing, but with a bit more work we were able to develop a solution that was able to realize the performance gains without any negative impact on other workloads. We are now making the necessary steps to give back our code changes to Apache Harmony.

The new TreeMap is included in JDK 6 Update 6 Performance Release which is available for download at http://java.sun.com/performance.

Thanks again Harmony! Keep the JDK performance optimizations coming.

I'm please to announce the release of JDK 6 Update 6 Performance Release on SPARC platforms. This is the latest of our performance releases and is the culmination of our optimization efforts over the last year. With this JDK Sun achieved many world records on SPECjappserver2004, SPECweb2005, SPECjbb2005, and the first ever SPECjvm2008 submission. Please try it for yourself, download it at http://java.sun.com/performance.

Where are the x64 bits? We believe in reliable performance, not just top line performance on targeted workloads. To achieve this we run a lot of tests, and our testing back end is often much longer than the development cycle for any given release. Long story short we found an issue that needed another build, so the x64 binaries will be delivered at a slight delay. Since the performance of an unreliable JDK is zero, we thought you'd understand. I'll update everyone once the x64 binaries are posted.

Finally, I'd like to thank the Apache Harmony project for their JDK optimizations efforts. No, you're not seeing things, I'm thanking Apache Harmony. More details to come in my next blog posting.

SPEC has release SPECjvm2008 and ....

Its Free!!

The new benchmark is the replacement to SPECjvm98, the first SPEC Java benchmark and the beginning of a family of SPEC Java Benchmarks including SPECjbb2005, SPECjappserver2004, SPECjms2007, SPECpower_ssj2008, and a bit of SPECweb2005.

SPECjvm2008 leverages a wide range of workloads including Scimark, Compilation, SunFlow, XML, Derby, Startup, and many more, and its a fine Java benchmark for competition and research. Download it and give it a try.

Congratulations to the development team and a big thanks to our many contributors from the community.

Just in time for JavaOne, I'm pleased to announce two new SPECjbb2005 World Records on Sun Intel systems. The Sun Fire X4450, powered by Intel Xeon MP CPUs and Java SE 6 Update 6-P, now hold the 4 Chip Multi-JVM World Record and the Single JVM x86 world record.

World Record Performance on 4-Chip Systems running 8-JVMs: 464,355 SPECjbb2005 bops, 58,044 SPECjbb2005 bops/JVM.

World Record Performance on x86 Systems running a single JVM: 389,208 SPECjbb2005 bops, 389,208 SPECjbb2005 bops/JVM.

The single JVM result actually beats all single JVM results with 16 CPUs or less. Wow. Thanks again to the HotSpot JVM and Performance teams, great work guys. Thanks also to the fine engineers at Intel. Lets keep on trucking.

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4450
results submitted to SPEC. Other results as of 05/6/08 on www.spec.org. Sun Fire X4450 (4 chips, 16 cores, Sun JDK 6u6-p) SPECjbb2005 bops = 464355, SPECjbb2005 bops/JVM = 58044. SPECjbb2005 bops = 389208, SPECjbb2005 bops/JVM = 389208.

I'm pleased to announce the release of our second performance release, JDK 6 Update 5-P. Its available to download at http://java.sun.com/performance. This is our fastest JDK to date, released just in time for JavaOne.

Thanks to the entire VM Technologies team!

Java, Solaris and CMT Systems have achieved a Java performance triple crown with winning results in 3 key industry benchmarks.

SPECjbb2005 2 CPU Multi-JVM World Record: Sun SPARC Enterprise T5240

SPECweb2005 World Record: Sun SPARC T5220

SPECjAppServer2004 World Record Single Application Server: Sun SPARC Enterprise T5240

All running the upcoming JDK 6 Update 6 Performance release. Nice.

Java == Performance

Today Sun announced the next generation chip multi-threading (CMT) systems with the powerhouse UltraSPARC T2 Plus Processor and again Java is ready. We've spent the last several months optimizing the Sun JDK and HotSpot JVM for UltraSPARC T2 Plus and the results are impressive. Today, I again have the privilege to announce another Sun World Record, this time taking the lead with the new Sun SPARC Enterprise T5240 achieving a World Record 2 Socket SPECjbb2005 Result of 373,405 SPECjbb2005 bops, 23,338 SPECjbb2005 bops/JVM.

1.94x scaling with 2 x 1.4Ghz UltraSPARC T2 Plus compared to 1 x 1.4Ghz UltraSPARC T2
Faster than all 4-Socket SPECjbb2005 results from IBM, including the IBM System p 570 with 4 x 4.7Ghz Power 6 CPUs !
Faster than all 4-Socket SPECjbb2005 results from HP (Actually faster than all results from HP outside of their 64-way Itanium2 systems)

Wow. The UltraSPARC T2 Plus and Sun Java powered by the HotSpot JVM are a sweet combination, and this result was made possible with the highly optimized Java SE 6 Update 6-P which will be available in July 2008. Take a look at http://java.sun.com/performance for the latest performance release.

Many, many engineers (and dare I say a few managers) had a role delivering these results, and I'd like to personally thank the following folks whose passion and determination made this possible. Thanks for all the hard work!

Paul Hohensee
Tom Rodriguez
Vladimir Kozlov
Brian Doherty
Charlie Hunt
Alejandro Murillo
Xiaobin Liu
Coleen Phillimore
Igor Veresov
Peter Kessler
John Coomes
Tony Printezis
Rob Strout
Dave Cox
Jim Melvin
John Pampuch
Kirill Soshalsky

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org. T5240 results submitted to SPEC for review.
Other results as of 04/09/2008 on www.spec.org. Sun SPARC Enterprise T5240 (2 chip, 16 cores, 128 threads) 373,405 SPECjbb2005 bops, 23,338 SPECjbb2005 bops/JVM. IBM p570 (4 chips, 8 cores, 16 threads) SPECjbb2005 bops = 346,742, SPECjbb2005 bops/JVM = 86,686. IBM System x3650 (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 323,172, SPECjbb2005 bops/JVM = 80,793. HP ProLiant DL360 G5 (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 301,321, SPECjbb2005 bops/JVM = 75330. Dell PowerEdge 2950 III (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 305,411, SPECjbb2005 bops/JVM = 76,353
Technorati Tags: T5140, T5240, SPARC, CMT, Java, UltraSPARC, Performance, JVM, HotSpot

I have been elected to chair of the SPEC OSG Java Subcommittee, which is responsible for Java benchmarks such as SPECjbb2005, SPECjappserver2004, and SPECjms2007 (plus a few in development). The first month has been entertaining to say the least, but its been a lot of fun so far. Thanks to everyone in OSG Java for helping me take this on. I've got big shoes to fill and I can't help feeling intimidated from time to time. The folks on the subcommittee are great and their patience and support have been a big help. Thanks!

Powered by ScribeFire.

Intel has posted a "Chip Shot" about the Sun SPECjbb2005 World Record on Intel systems. The text is a bit familiar, but heck, its coverage.

http://www.intel.com/pressroom/chipshots/chipshots.htm

Powered by ScribeFire.

The SPECjbb2005 World Record results on Sun Intel systems are now live on the SPEC website.

World Record Performance on 2-chip x86 Systems running 4-JVMS: 303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM

World Record Performance on x86 Systems with a Single JVM: 277,585 SPECjbb2005 bops

The Sun JDK powered by the HotSpot JVM is the the most widely deployed, scalable, and reliable JVM on the planet, but we need your help to continue to improve. Please participate at OpenJDK and the performance forum at Java.net, let us know what your application needs for reliable performance. You can even dive in and help. Competitive benchmarks are fun, but at the end of the day its our customer's application that really matter, so let us know what we can do to help!

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4150 results submitted to SPEC. Other results as of 02/15/08 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM. Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVM. Dell 2950 III (2 chips, 8 cores, BEA JRockit 6.0 P27.4.0) 303,130 SPECjbb2005 bops , 75,783 SPECjbb2005 bops/JVM.

Powered by ScribeFire.

Some rather ridiculous comments from "Rick Jones" on the Bmseer's blog prompted me to write this. Rick stated:

"If that four JVM figure wasn't run over and over again,
one is left wondering just how bad it was before. Given
the same CPUs at the same frequency, one would expect
more from "major impact" performance changes."

We have made huge performance improvements over the last year. Yes, our previous capabilities on the Intel platform were less than what we can do now. I thought that would be obvious.

But you know what the biggest difference with our SPECjbb2005 World Record? Sun Hardware + Sun Solaris + Sun HotSpot JVM. Its not a result from any old box vendor who runs with the latest and greatest OS and JVM and claims the result as their own. This is a Sun World Record, through and through. Who else can deliver the entire stack, its a short list isn't it?

Hmm, Think of the possibilities for customers. Who do you call when you have a JVM issue? Sun. OS issue? Sun. Hardware issue? Sun. What if you need a specific feature which requires a tailored approach from hardware to OS to JVM? Sun. Thats the difference.

Powered by ScribeFire.

Our latest SPECjbb2005 World Record on the Sun Fire X4150 is getting some coverage on Sun web pages and blogs.

As always, the bmseer has something to say, check out his post on the SPECjbb2005 World Record here.

The Sun marketing folks posted a page as well, check it out here.

We're very excited about these results, and hope our customers are too. Let us know if you have any questions!

Powered by ScribeFire.

It was one year ago that Sun and Intel kicked off a collaboration to optimize Solaris and the HotSpot JVM for Intel Xeon systems. The last year has been a fun ride, working together we've made our performance steadily increase, and finally today is the day.

Without further ado,

Sun HotSpot JVM, Solaris, and the Sun Fire X4150 with 2 Intel X5460 quad-core processors now hold 2 SPECjbb2005 World Records!

World Record Performance on 2-chip x86 Systems
303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM running 4 JVMs!

World Record Performance on x86 Systems with a Single JVM
277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVM

Thats right, Sun is fastest on Intel DP, inching past BEA to take the lead. Game on JRockit.

We submitted two results, one with 4 JVMs, and another with 1 JVM. Why? When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations. With both single-JVM and multi-JVM results a more complete picture can be drawn of the performance capabilities a software/hardware stack. If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results. If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results. I encourage my colleagues from other JVM vendors to submit both single JVM results and multi-JVM results. We, and our competitors owe it to the Java community to show the whole truth about JVM performance. It's the Java community which will benefit from the active competition and drive JVM development forward. I've written about this before, take a look here for more background.

The JRE we used to achieve the world record results is Java 6 Update 5 Performance Release (Java 6 Update 5-p). This is our second performance release, and will be available mid-May 2008. Our first performance release Java 6 Update 4-p, is available on our performance page at java.sun.com/performance. Java 6 Update 4-p is a step along the way to our current high scores, and is only ~10% behind Java 6 Update 5-p in the general case. Go ahead, give it a try and let us know what you think.

Congratulations to the Sun and Intel Java performance collaboration team. For the last year we've worked together to deliver some grand performance improvements. Great work everyone, let's keep up the pace!

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4150 results submitted to SPEC. Other results as of 02/15/08 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM. Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVM. Dell 2950 III (2 chips, 8 cores, BEA JRockit 6.0 P27.4.0) 303,130 SPECjbb2005 bops , 75,783 SPECjbb2005 bops/JVM.

Single JVM scaling has been ignored in the realm of competitive benchmarking. Since the release of SPECjbb2005 and its support for multiple JVM configurations most JVM and hardware vendors have ignored the single JVM configuration. Which begs my first question, "Why not submit using a single JVM?" Simply put, SPECJbb2005 multi-JVM results tends to be significantly higher than single JVM results on the same hardware. Which leads to my second question, "Why are SPECjbb2005 multi-JVM configurations faster than SPECjbb2005 single-JVM configurations?". Now that's a good question.

Multi-JVM configurations tend to be faster for two general reasons. First, you can run many 32-bit JVMs instead of a single 64-bit JVM. 32-bit JVMs are anywhere from 15 - 20% faster than true 64-bit JVMs when running memory bound compute intensive applications that can't leverage the large heap available with a 64-bit JVM. Second, multi-JVM configurations allow the end user to bind or set affinity of a given process to a physical processor or cores. For example, lets say a given processor runs very well when the SPECjbb2005 JVM process is kept on a single socket or a few cores on a socket, but suffers when a single SPECjbb2005 JVM process runs across many sockets or cores. In this situation, it would be advantageous to run multiple JVMs each bound to a set of cores that are all or a subset of the cores available in a single CPU.

The example above is very prevalent in SPECjbb2005 results today. Many, many vendors submit multi-JVM results, even on 1U and 2U servers with 2 sockets or less. Sun has done the same. Many of the processors available in 1U and 2U form factors have processors that show significantly different performance characteristics when comparing single-JVM and multi-JVM SPECjbb2005 configurations, and since multi-JVM is faster, vendors chose to publicize that result. Until now...

When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations, and from this point forward I plan to advocate that Sun continue as it has done today and submit both single-JVM and multi-JVM configurations on new hardware platforms. With both single-JVM and multi-JVM results a more complete picture of the performance capabilities a software/hardware stack can drawn. If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results. If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results.

Unfortunately the competitive submissions on the SPECjbb2005 website are woefully incomplete with many vendors only submitting multi-JVM results. I implore all vendors to submit SPECjbb2005 single-JVM results along with the multi-JVM results. Sure they're not as fast, but at least you're not hiding behind the multi-JVM configuration, and then hardware and JVM vendors will work together to solve any problems. Its the best decision for JVM innovation, our customers, and the Java platform.

Below is our most recent SPECjbb2005 single JVM submission on a Sun SPARC Enterprise T5120 powered by the UltraSPARC-T2 processor. This result fully utilizes the 64 hardware threads available on the US-T2, up to 8X more threads than our 1U and 2U competition, with nearly picture perfect scaling to boot. This result surpasses all single-JVM submission from HP and Dell, and is faster than all 8-core or less single-JVM submissions from IBM. And just to sweeten it a bit more, this 64-bit result is only 13% behind the 32-bit 8-JVM SPECjbb2005 submission on the SE T5120 @ 192,055 SPECjbb2005 bops!

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org.
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.

Technorati Tags: CMT, T5120, T5220, US-T2, Java, Performance, Scalability, JVM

Powered by ScribeFire.

The UltraSPARC T2 has arrived and the Sun HotSpot JVM is ready. We've spent the last several months optimizing Sun's JVM for US-T2 and the day has finally come to share what we've been working on.

Two new SPECjbb2005 world records!

Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record SPECjbb2005 Single Socket result of 192,055 SPECjbb2005 bops, 24,007 bops/JVM.

9% faster than IBM p570 with 2 x 4.7Ghz Power 6 Processors!
4X less space, 2X less power!

Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record Single JVM result for all systems with 8-cores or less.

Beats all single JVM results from Dell and HP!

Record Breaking Perfomance

The Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 servers equipped with a single UltraSPARC T2 processor at 1.4GHz, delivered a World Record single-chip result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM. An average of 464 Watts of power was used to obtain this result. This result beats the previous SPECjbb2005 single-chip world record by 71%.

Record Breaking Scalability

The Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 servers delivered a World Record single-chip, single-JVM result of 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. This result beats all single-JVM SPECjbb2005 results from Dell and HP, and all 8-core or less single-JVM results from IBM. To top it off, the 64-bit single-JVM SPECjbb2005 result is within 13% of the performance of the 32-bit multi-JVM result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM, highlighting the flexibility of the Ultra SPARC T2 and Sun HotSpot JVM technology.

Reliable Performance: Sun's HotSpot JVM

Rock solid reliable performance is at the heart of HotSpot JVM development at Sun. Our dedicated effort has allowed us to deliver performance optimizations into JDK 6 update releases in a highly reliable fashion, with our first ever JDK 6 performance release coming your way in December 2007. Stay tuned for more information on the performance release and all of the cool optimizations we're working on for all of our supported platforms.

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org. Sun SPARC Enterprise T5220 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5220 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM. IBM p570 (2 chips, 4 cores) SPECjbb2005 bops = 175474, SPECjbb2005 bops/JVM = 87737. Dell 2950 (2 chips, 8 cores) SPECjbb2005 bops = 148,436, SPECjbb2005 bops/JVM = 74,218. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 158,174, SPECjbb2005 bops/JVM = 39,544. IBM p505Q (1 chip, 4 cores) SPECjbb2005 bops = 63544, SPECjbb2005 bops/JVM = 31772. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 80884, SPECjbb2005 bops/JVM = 80884

Technorati Tags: T5120, T5220, SPARC, CMT, Java, UltraSPARC, Performance, JVM, HotSpot

Powered by ScribeFire.

I'll be presenting tomorrow at Intel Developer Forum in San Francisco with an esteemed colleague from Intel, Kingsum Chow.

The session is SSGS003: "How to Get the Most Performance from Sun* JVM* on Intel® Multi-Core Servers".

Take a look at this link for more information, just search by last name.

The conference is rather interesting. I've run into a lot of Sun folks working on Solaris, and I've seen many folks our competitors in blue. I've looked for Henrik and friends, but haven't found them yet. Interesting, considering all the marketing babble I expected to see them here.

Here's what we hope you'll walk away with from my talk tomorrow (straight from the abstract):

An overview of how Sun* JVM* has evolved to take advantage of current and future Intel® Multi-Core processor-based servers since Sun and Intel teams started working on JVM performance improvements in early 2007
A good understanding how harnessing the best Intel® Xeon® processor features with Sun JVM will start producing immediate benefits
A quick overview of Intel Multi-Core architectures
An understanding of how Sun JVM takes advantage of Intel® architectures on three major operating systems (Linux*, Windows*, Solaris*)
A look at how to select JVM parameters for your application to get the most performance, and at employing VM parameters tuning as a last resort
A few common options for a list of application classes

Powered by ScribeFire.

The 72-way Sun Fire E25K with 1.80Ghz US-IV+ processors and powered by Java SE 6 Update 2 now holds the Single Instance SPECjbb2005 World Record with a result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM.

This result surpasses the previous high score held by Fujitsu PowerQuest 580 running BEA Jrockit and Itanium 2 processors by 4%. To top it off, this result was accomplished without using the fastest US-IV+ processors available (the fastest available is 1.95Ghz!) This makes the Sun Fire E25K running Solaris 10 and Java SE 6 Update 2 the fastest, most scalable computing platform for Java applications on the planet!

SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)

System	Date	Processors		Performance
System	Date	(Chips, Cores, Threads)	GHz Type	bops	JVMs	bops/JVM
Sun Fire E25K	5/07	72, 144, 144	1.80 US-IV+	1,149,100	1	1,149,100
Fujitsu PowerQuest 580	3/07	32, 64, 128	1.6 Itanium 2	1,105,465	1	1,105,465
Azul 7280B	1/07	16, 768, 768	?	872,972	1	872,972
Fujitsu PP2500	1/06	128, 128, 128	2.08 SPARC64 V	811,607	1	811,607

Sun results have been submitted to SPEC for review and are on track for publication.

Disclosure Statement:

SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 144 threads, 1.95 GHz) 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM submitted for review; Fujitsu PQ580 (32 chips, 64 cores, 128 threads, 1.6 GHz) 1,105,465 SPECjbb2005 bops, 1,105,465 SPECjbb2005 bops/JVM; Azul 7280B (16 chips, 768 cores,768 threads, ? Ghz) 872,972 SPECjbb2005 bops, 872,972 SPECjbb2005 bops/JVM; Fujitsu PP2500 (128 chips, 128 cores, 128 threads 2.08 GHz) 811,607 SPECjbb2005 bops, 811,607 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

Paul Hohensee and I are doing a session this week at Java One, TS-2885: High Performance Java Technology in a Multi-core World. We'll talk about current and upcoming multi-core computer architectures ,JVM technologies and optimizations , and application and JVM tuning . Finally I'll do a quick demo showing our recent performance optimizations coming soon in Java SE 6.0_02, identification of heap tuning opportunities and common application bottlenecks.

Come see our talk on Thursday at 4:10pm in Gateway 102/103. See you at Java One!

Based on feedback from my esteemed colleagues and readers I've updated this entry with a table of results. The tables are at the bottom after the charts.

Java 6 is finally here. Its our fastest, most reliable release and specifically targets out-of-box performance. What does this mean? Simply put it means no tuning options are needed for the JVM to achieve optimal performance. Looking at the bigger picture it means much more. No longer will you spend hours pouring over cryptic JVM tuning parameters to determine the optimal configuration for your application. No more expensive re-qualification of your application for special command line tuning. Java 6 makes performance tuning easy.

Java 6, powered by the recently open-sourced HotSpot JVM is impressive. Here's a summary:

On SPECjbb2005 the numbers are impressive. Java 6 out of the box is more than 40% ahead of the competition on Intel Core, and 30% ahead on AMD Opteron.
On Scimark Java 6 continues to show solid performance leading the performance of the competition by more than 40%.
On Volano, Java 6 improves performance by more than 20% over the most recent update of the JDK 5.

The out of box performance of a Java application is an intriguing and difficultengineering problem. The requirements of client and server applications couldn't be more different. On one hand client apps want fast startup and low footprint, on the other hand server applications want highly optimized code, throughput and low pause times; while both want reliability and compatibility.

Out of box performance is the right goal for JVM development, and future Java benchmarks should reflect that goal. Delivering optimizations quickly to allow high benchmark results is fun but it doesn't help customers unless they become part on the default runtime behavior of the JVM.

Just to be clear and to reiterate once again, the intention of the data charts below is to highlight the importance of customer experience and out-of-box performance to Sun Java Engineering. These are not meant to be high performance benchmark results. Hand tuning can change the results significantly.

The following is an out-of-box performance comparison on a Dell 2950 and a Sun Fire X4200. The Dell system is configured with 2 dual-core Intel 5160 processors (2 CPUs, 4 cores @ 3.0Ghz) and 16GB of RAM. The Sun system is configured with 2 dual-core Opteron 280 processors (2 CPUs, 4 cores, 2.4 Ghz) and 8GB of RAM. The Operating System installed on both systems is Red Hat EL 4.0 AS Update 4. The kernel version is unmodified from the base install, which is 2.6.9-42.ELsmp. The only variable in this configuration is the JVM.

The JVM distributions and versions tested were the latest versions publicly available at the time of testing. The BEA JRockit JVMs tested are downloaded from their main GA website and their 64-bit performance update website. The IBM JVM is the latest available on the IBM developer website.

Sun JDK 1.5.0_10

32-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) Server VM (build 1.5.0_10-b03, mixed mode)
64-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_10-b03, mixed mode)

Sun Java SE 6 build 98

32-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Server VM (build 1.6.0-rc-b99, mixed mode)
64-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-b105, mixed mode)

IBM JDK 5.0 SR3

32-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxi32dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20061001 (JIT enabled) J9VM - 20060915_08260_lHdSMR JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
64-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxa64dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20061001 (JIT enabled) J9VM - 20060915_08260_LHdSMr JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002

BEA JRockit 5.0_06 R26.4

32-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, )
64-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build P26.4.1-12-67782-1.5.0_06-20061003-1620-linux-x86_64, )

As stated above and in the title no JVM tuning options were used for these results. The results below are statistical comparisons. No less than 10 samples were performed, and a T-test (single-tailed) was used to ensure confidence in the result. The data is normalized to the 32-bit IBM JDK 5 SR3 result.

The first set of charts reflect performance on Intel's latest Core 2 micro-architecture. The results below, particularly the SPECjbb2005 results, strongly highlight a core difference in philosophy between Sun HotSpot and its competitors. If you look at highly tuned competitive submissions of our competitors, BEA JRockit in particular, have impressive numbers on the new chip. Our competitors have chosen to quickly deliver platform specific performance optimizations for the purpose of benchmark submissions but require the use of several tuning parameters to achieve that level of performance. Unfortunately this is quite misleading for customers. Yes, the benchmark numbers are good, but can a customer jump right in and use these features? If they were thoroughly tested and ready for prime time shouldn't they be enabled by default on the platforms that require them? We think so, and we have chosen differently, and thats the difference with HotSpot.

The first chart is SPECjbb2005. SPECjbb2005 is SPEC's benchmark for evaluating the performance of server side Java. It evaluates server side Java by emulating a three-tier client/server system (with emphasis on the middle tier). It extensively stresses Java collections, BigDecimal, and XML processing. The cool thing about SPECjbb2005 is that optimizations targeted for it also show performance gains in other competitive benchmarks, such as SPECjappserver2004, and a broad range of customer workloads. The benchmark results below are run in single instance mode.

SciMark 2.0 is a Java benchmark for scientific and numerical computing and is a benchmark where Sun's JVMs have continued to shine. Its a decent test of generated code, particularly for tight computational loops. However it is particularly sensitive to alignment issues and can show some level of variance from run to run, mostly in a bimodal fashion. The test has three modes of exectution; small, large, and default. This is the size of the data under test, more details can be found at the scimark website. All in all its a good set of microbenchmarks.

Note that the 32-bit JVMs in all cases are faster than the 64-bit JVMs when running on the Intel Core system. This is quite different than the AMD Opteron system further down the page where 64-bit is significantly faster. Since the Scimark 2.0 test is using the large dataset, its likely that the added pressure of 64-bit pointers on the memory subsystem increases bandwidth enough to impede performance, however this is just a hypothesis.

Volano is a popular Java chat server. The benchmark is quick and involves both a client and server instance. From a JVM perspective the workload is heavily dominated by classic Java socket I/O which is a bit long in the tooth, an NIO version would be quite interesting. That being said, some customers have found this benchmark quite useful so we continue to test it, however it is by no means our favorite benchmark as my friends at BEA have suggested. Running Volano the performance gaps are not as large, most likely because this benchmark has very little garbage collection overhead. BEA JRockit is showing good performance here with a result thats 19% over the baseline. Sun Java SE 6 shines as well with a result thats nearly 22% over baseline.

The second set of charts are run on a Sun Fire X4200 with AMD Opteron 280 CPUs. This is the identical system used in my previous blog articles on this subject, this time with updated JVM releases from Sun and IBM. I'm sure someone will be curious why I didn't compare the Intel and AMD based systems directly. The primary reason is simple, I'm writing about JVM performance, not CPU performance. That being said, I didn't have the latest AMD CPUs readily available. In short, Intel is faster when running some of these benchmarks, while AMD is faster on others. In general the memory subsystem differences between these platforms is prevalent when comparing the performance of Java benchmarks. Sun Java 6 is showing impressive results running SPECjbb2005 with a result 30% over baseline and ~15% faster than J2SE 5.0_10.

Scimark 2.0 is impressive on AMD Opteron as well. The large dataset is an interesting workload as its effect on cache can highlight memory subsystem limitations. If your application crunches on a large dataset, take a look at the large dataset of Scimark when comparing JVMs and system architectures.

Last but not least is Volano on AMD Opteron (and again, no this is not our favorite benchmark!). Java 6 shows a strong improvement of with results more than 20% greater than 5.0_10, pulling ahead of 64-bit BEA JRockit. Nice.

SPECjbb2005 Result Disclosure
Single Instance Run. SPECjbb2005 bops = SPECjbb2005 bops/JVM
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops

IBM 5.0 SR3 43,575 32,617

BEA JRockit 5.0_06 R26.4 26,071 26,092

Sun J2SE 5.0_10 49,308 46,080

Sun Java SE 6 62,246 56,488

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.

JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops

IBM 5.0 SR3 30,500 23,998

BEA JRockit 5.0_06 R26.4 19,309 19,185

Sun J2SE 5.0_10 35,297 31,096

Sun Java SE 6 39,973 34,975

JVM Version	32-bit SPECjbb2005 bops	64-bit SPECjbb2005 bops
IBM 5.0 SR3	43,575	32,617
BEA JRockit 5.0_06 R26.4	26,071	26,092
Sun J2SE 5.0_10	49,308	46,080
Sun Java SE 6	62,246	56,488

JVM Version	32-bit SPECjbb2005 bops	64-bit SPECjbb2005 bops
IBM 5.0 SR3	30,500	23,998
BEA JRockit 5.0_06 R26.4	19,309	19,185
Sun J2SE 5.0_10	35,297	31,096
Sun Java SE 6	39,973	34,975

SciMark 2.0 Result Disclosure
Large Dataset. Score is in SciMark MFlops
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score

IBM 5.0 SR3 171.49 207.21

BEA JRockit 5.0_06 R26.4 278.15 276.37

Sun J2SE 5.0_10 321.85 292.89

Sun Java SE 6 357.72 336.58

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score

IBM 5.0 SR3 175.02 180.46

BEA JRockit 5.0_06 R26.4 230.85 231.53

Sun J2SE 5.0_10 300.23 332.23

Sun Java SE 6 320.42 343.74

JVM Version	32-bit Score	64-bit Score
IBM 5.0 SR3	171.49	207.21
BEA JRockit 5.0_06 R26.4	278.15	276.37
Sun J2SE 5.0_10	321.85	292.89
Sun Java SE 6	357.72	336.58

JVM Version	32-bit Score	64-bit Score
IBM 5.0 SR3	175.02	180.46
BEA JRockit 5.0_06 R26.4	230.85	231.53
Sun J2SE 5.0_10	300.23	332.23
Sun Java SE 6	320.42	343.74

VolanoMark 2.5.0.9 Result Disclosure
Loopback performance test
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score

IBM 5.0 SR3 121,747 111,826

BEA JRockit 5.0_06 R26.4 128,185 146,012

Sun J2SE 5.0_10 120,048 116,959

Sun Java SE 6 149,198 142,602

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score

IBM 5.0 SR3 64,218 60,802

BEA JRockit 5.0_06 R26.4 73,627 76,675

Sun J2SE 5.0_10 66,955 64,316

Sun Java SE 6 80,592 75,156

JVM Version	32-bit Score	64-bit Score
IBM 5.0 SR3	121,747	111,826
BEA JRockit 5.0_06 R26.4	128,185	146,012
Sun J2SE 5.0_10	120,048	116,959
Sun Java SE 6	149,198	142,602

JVM Version	32-bit Score	64-bit Score
IBM 5.0 SR3	64,218	60,802
BEA JRockit 5.0_06 R26.4	73,627	76,675
Sun J2SE 5.0_10	66,955	64,316
Sun Java SE 6	80,592	75,156

SPEC(R) and the benchmark name SPECjbb(TM) are trademarks of the Standard Performance Evaluation Corporation. Competitive benchmark results stated above reflect experiments performed by Sun Microsystems, Inc. For the latest SPECjbb2005 benchmark results, visit http://www.spec.org/osg/jbb2005.

Java 6 is finally here. Its our fastest, most reliable release and specifically targets out-of-box performance. What does this mean? Simply put it means no tuning options are needed for the JVM to achieve optimal performance. Looking at the bigger picture it means much more. No longer will you spend hours pouring over cryptic JVM tuning parameters to determine the optimal configuration for your application. No more expensive re-qualification of your application for special command line tuning. Java 6 makes performance tuning easy.

Java 6, powered by the recently open-sourced HotSpot JVM is impressive. Here's a summary:

On SPECjbb2005 the numbers are impressive. Java 6 out of the box is more than 40% ahead of the competition on Intel Core, and 30% ahead on AMD Opteron.
On Scimark Java 6 continues to show solid performance leading the performance of the competition by more than 40%.
On Volano, Java 6 improves performance by more than 20% over the most recent update of the JDK 5.

The out of box performance of a Java application is an intriguing and difficult engineering problem. The requirements of client and server applications couldn't be more different. On one hand client apps want fast startup and low footprint, on the other hand server applications want highly optimized code, throughput and low pause times; while both want reliability and compatibility.

Out of box performance is the right goal for JVM development, and future Java benchmarks should reflect that goal. Delivering optimizations quickly to allow high benchmark results is fun but it doesn't help customers unless they become part on the default runtime behavior of the JVM.

Just to be clear and to reiterate once again, the intention of the data charts below is to highlight the importance of customer experience and out-of-box performance to Sun Java Engineering. These are not meant to be high performance benchmark results. Hand tuning can change the results significantly.

The following is an out-of-box performance comparison on a Dell 2950 and a Sun Fire X4200. The Dell system is configured with 2 dual-core Intel 5160 processors (2 CPUs, 4 cores @ 3.0Ghz) and 16GB of RAM. The Sun system is configured with 2 dual-core Opteron 280 processors (2 CPUs, 4 cores, 2.4 Ghz) and 8GB of RAM. The Operating System installed on both systems is Red Hat EL 4.0 AS Update 4. The kernel version is unmodified from the base install, which is 2.6.9-42.ELsmp. The only variable in this configuration is the JVM.

The JVM distributions and versions tested were the latest versions publicly available at the time of testing. The BEA JRockit JVMs tested are downloaded from their main GA website and their 64-bit performance update website. The IBM JVM is the latest available on the IBM developer website.

Sun JDK 1.5.0_10

32-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) Server VM (build 1.5.0_10-b03, mixed mode)
64-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_10-b03, mixed mode)

Sun Java SE 6 build 98

32-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Server VM (build 1.6.0-rc-b99, mixed mode)
64-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-b105, mixed mode)

IBM JDK 5.0 SR3

32-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxi32dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20061001 (JIT enabled) J9VM - 20060915_08260_lHdSMR JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
64-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxa64dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20061001 (JIT enabled) J9VM - 20060915_08260_LHdSMr JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002

BEA JRockit 5.0_06 R26.4

32-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, )
64-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build P26.4.1-12-67782-1.5.0_06-20061003-1620-linux-x86_64, )