Tuesday October 09, 2007
Tuesday October 09, 2007 Single JVM scaling has been ignored in the realm of competitive benchmarking. Since the release of SPECjbb2005 and its support for multiple JVM configurations most JVM and hardware vendors have ignored the single JVM configuration. Which begs my first question, "Why not submit using a single JVM?" Simply put, SPECJbb2005 multi-JVM results tends to be significantly higher than single JVM results on the same hardware. Which leads to my second question, "Why are SPECjbb2005 multi-JVM configurations faster than SPECjbb2005 single-JVM configurations?". Now that's a good question.
Multi-JVM configurations tend to be faster for two general reasons. First, you can run many 32-bit JVMs instead of a single 64-bit JVM. 32-bit JVMs are anywhere from 15 - 20% faster than true 64-bit JVMs when running memory bound compute intensive applications that can't leverage the large heap available with a 64-bit JVM. Second, multi-JVM configurations allow the end user to bind or set affinity of a given process to a physical processor or cores. For example, lets say a given processor runs very well when the SPECjbb2005 JVM process is kept on a single socket or a few cores on a socket, but suffers when a single SPECjbb2005 JVM process runs across many sockets or cores. In this situation, it would be advantageous to run multiple JVMs each bound to a set of cores that are all or a subset of the cores available in a single CPU.
The example above is very prevalent in SPECjbb2005 results today. Many, many vendors submit multi-JVM results, even on 1U and 2U servers with 2 sockets or less. Sun has done the same. Many of the processors available in 1U and 2U form factors have processors that show significantly different performance characteristics when comparing single-JVM and multi-JVM SPECjbb2005 configurations, and since multi-JVM is faster, vendors chose to publicize that result. Until now...
When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations, and from this point forward I plan to advocate that Sun continue as it has done today and submit both single-JVM and multi-JVM configurations on new hardware platforms. With both single-JVM and multi-JVM results a more complete picture of the performance capabilities a software/hardware stack can drawn. If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results. If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results.
Unfortunately the competitive submissions on the SPECjbb2005 website are woefully incomplete with many vendors only submitting multi-JVM results. I implore all vendors to submit SPECjbb2005 single-JVM results along with the multi-JVM results. Sure they're not as fast, but at least you're not hiding behind the multi-JVM configuration, and then hardware and JVM vendors will work together to solve any problems. Its the best decision for JVM innovation, our customers, and the Java platform.
Below is our most recent SPECjbb2005 single JVM submission on a Sun SPARC Enterprise T5120 powered by the UltraSPARC-T2 processor. This result fully utilizes the 64 hardware threads available on the US-T2, up to 8X more threads than our 1U and 2U competition, with nearly picture perfect scaling to boot. This result surpasses all single-JVM submission from HP and Dell, and is faster than all 8-core or less single-JVM submissions from IBM. And just to sweeten it a bit more, this 64-bit result is only 13% behind the 32-bit 8-JVM SPECjbb2005 submission on the SE T5120 @ 192,055 SPECjbb2005 bops!
SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org.
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.
Technorati Tags: CMT, T5120, T5220, US-T2, Java, Performance, Scalability, JVM
Multi-JVM configurations tend to be faster for two general reasons. First, you can run many 32-bit JVMs instead of a single 64-bit JVM. 32-bit JVMs are anywhere from 15 - 20% faster than true 64-bit JVMs when running memory bound compute intensive applications that can't leverage the large heap available with a 64-bit JVM. Second, multi-JVM configurations allow the end user to bind or set affinity of a given process to a physical processor or cores. For example, lets say a given processor runs very well when the SPECjbb2005 JVM process is kept on a single socket or a few cores on a socket, but suffers when a single SPECjbb2005 JVM process runs across many sockets or cores. In this situation, it would be advantageous to run multiple JVMs each bound to a set of cores that are all or a subset of the cores available in a single CPU.
The example above is very prevalent in SPECjbb2005 results today. Many, many vendors submit multi-JVM results, even on 1U and 2U servers with 2 sockets or less. Sun has done the same. Many of the processors available in 1U and 2U form factors have processors that show significantly different performance characteristics when comparing single-JVM and multi-JVM SPECjbb2005 configurations, and since multi-JVM is faster, vendors chose to publicize that result. Until now...
When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations, and from this point forward I plan to advocate that Sun continue as it has done today and submit both single-JVM and multi-JVM configurations on new hardware platforms. With both single-JVM and multi-JVM results a more complete picture of the performance capabilities a software/hardware stack can drawn. If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results. If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results.
Unfortunately the competitive submissions on the SPECjbb2005 website are woefully incomplete with many vendors only submitting multi-JVM results. I implore all vendors to submit SPECjbb2005 single-JVM results along with the multi-JVM results. Sure they're not as fast, but at least you're not hiding behind the multi-JVM configuration, and then hardware and JVM vendors will work together to solve any problems. Its the best decision for JVM innovation, our customers, and the Java platform.
Below is our most recent SPECjbb2005 single JVM submission on a Sun SPARC Enterprise T5120 powered by the UltraSPARC-T2 processor. This result fully utilizes the 64 hardware threads available on the US-T2, up to 8X more threads than our 1U and 2U competition, with nearly picture perfect scaling to boot. This result surpasses all single-JVM submission from HP and Dell, and is faster than all 8-core or less single-JVM submissions from IBM. And just to sweeten it a bit more, this 64-bit result is only 13% behind the 32-bit 8-JVM SPECjbb2005 submission on the SE T5120 @ 192,055 SPECjbb2005 bops!
SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org.
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.
Technorati Tags: CMT, T5120, T5220, US-T2, Java, Performance, Scalability, JVM
Powered by ScribeFire.
Java Screams on Sun UltraSPARC T2
The UltraSPARC T2 has arrived and the Sun HotSpot JVM is ready. We've spent the last several months optimizing Sun's JVM for US-T2 and the day has finally come to share what we've been working on.
Two new SPECjbb2005 world records!
- Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record SPECjbb2005 Single Socket result of 192,055 SPECjbb2005 bops, 24,007 bops/JVM.
- 9% faster than IBM p570 with 2 x 4.7Ghz Power 6 Processors!
- 4X less space, 2X less power!
- Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record Single JVM result for all systems with 8-cores or less.
- Beats all single JVM results from Dell and HP!
The Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 servers equipped with a single UltraSPARC T2 processor at 1.4GHz, delivered a World Record single-chip result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM. An average of 464 Watts of power was used to obtain this result. This result beats the previous SPECjbb2005 single-chip world record by 71%.
Record Breaking Scalability
The Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 servers delivered a World Record single-chip, single-JVM result of 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. This result beats all single-JVM SPECjbb2005 results from Dell and HP, and all 8-core or less single-JVM results from IBM. To top it off, the 64-bit single-JVM SPECjbb2005 result is within 13% of the performance of the 32-bit multi-JVM result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM, highlighting the flexibility of the Ultra SPARC T2 and Sun HotSpot JVM technology.
Reliable Performance: Sun's HotSpot JVM
Rock solid reliable performance is at the heart of HotSpot JVM development at Sun. Our dedicated effort has allowed us to deliver performance optimizations into JDK 6 update releases in a highly reliable fashion, with our first ever JDK 6 performance release coming your way in December 2007. Stay tuned for more information on the performance release and all of the cool optimizations we're working on for all of our supported platforms.
SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org. Sun SPARC Enterprise T5220 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5220 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM. IBM p570 (2 chips, 4 cores) SPECjbb2005 bops = 175474, SPECjbb2005 bops/JVM = 87737. Dell 2950 (2 chips, 8 cores) SPECjbb2005 bops = 148,436, SPECjbb2005 bops/JVM = 74,218. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 158,174, SPECjbb2005 bops/JVM = 39,544. IBM p505Q (1 chip, 4 cores) SPECjbb2005 bops = 63544, SPECjbb2005 bops/JVM = 31772. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 80884, SPECjbb2005 bops/JVM = 80884
Technorati Tags: T5120, T5220, SPARC, CMT, Java, UltraSPARC, Performance, JVM, HotSpot
Powered by ScribeFire.