David Dagastine's Weblog

« May 2008
Sun	Mon	Tue	Wed	Thu	Fri	Sat
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

Today

Sun has achieved a new world record on SPECjbb2005, beating IBM's latest result by 8%. Take a look at the press release here. IBM's score: 224,200 bops Sun's score: 241,560 bops SPECjbb2005 is the next generation Java server application benchmark. It replaces SPECjbb2000 on January 4th, 2006. IBM has stepped up and submitted several benchmark results. This is fantastic. Where's BEA, HP, and Fujitsu? Dig yourselves out of the quagmire of SPECjbb2000 and submit result on a viable benchmark for a change. SPECjbb2000 = Old benchmark, Invalid results for today's applications SPECjbb2005 = New benchmark, developed to model today's applications Where do you want your JVM and Hardware vendor concentrating their resources?

Comments:

I see that a "16-way" Sun system has 32 cores while a "16-way" IBM system has 16 cores. I wonder if the difference in price is greater or smaller than 8%...

Posted by Wes Felter on October 18, 2005 at 03:40 PM EDT #

2Wes:

From the  press-release:
"With today's announcement, Sun continues to assert its position as the world's leading UNIX server vendor, offering up to five times greater performance than previous generations at the same power, cooling and space costs - with no increase in overall system price"

Does this answer your question?

Dmitri
Java2D Team

Posted by Dmitri Trembovetski on October 18, 2005 at 05:40 PM EDT #

I don't see the IBM 224k or the Sun 241k results on the spec web sites -- maybe you can provide some more info for your comparison – How many JVMs are used by IBM and Sun in each of the benchmarks? Looking at the existing Sun publication on the 24-core machine it seems that Sun uses 4-cores per JVM. Is the 241K number configured the same way? Is the use of 6 JVM instances on a 24-core machine an indication that Sun JVM does not scale beyond 4-cores for this benchmark? Will Sun publish single JVM version of the benchmark to provide some assurance to customers that the combination of HW/OS/JVM scales beyond 4-cores? Jbb2005 maybe a better benchmark than jbb2000 but it sure obscures performance information – A bad JVM with scaling problems can hide behind multiple instances – This benchmark is no use in measuring scaling of an OS or HW since each JVM can run in its own isolated set of resources without any interaction or communication with the other JVMs. I am sure Sun, HP, IBM and the rest will not agree with these opinions since they seem to create benchmarks to serve their marketing machines instead of providing useful performance and sizing information to customers.

Posted by Robin on October 26, 2005 at 02:11 AM EDT #

The Sun 241K submission is with 8 JVMs, or 4 cores per JVM. This is by no means an indicator the Sun's systems don't scale beyond 4-cores and a bit of a stretch to come to that conclusion anyway. Large MP boxes are largely deployed with many JVMs (or possibly OS instances) and multi-instance SPECjbb2005 results demonstrate that. Multiple instances scores are about total system throughput, not individual JVM scalability. How is performance information obscured? How useful is a single-instance result on a 32-thread MP system when customers don't deploy it in that fashion? Sun will of course publish large MP single-instance SPECjbb2005 results. Stay tuned. The rest of your comments seem to be self serving. Multiple-instance results don't highlight JVM scalability, but they do highlight OS and HW scalability and performance. If you don't like SPEC benchmarks become a SPEC member and voice your opinion. Sun, HP, BEA, IBM, and Intel were closely involved in SPECjbb2005 development.

Posted by Dagastine on October 28, 2005 at 04:05 PM EDT #

There seems to be a new record at SPECjbb2005: IBM eServer p5 570: 244,361 bops (8 way, 16 cores)

Posted by anonymous on November 16, 2005 at 08:17 PM EST #

Name:
E-Mail:
URL:
	Notify me by email of new comments
	Remember Information?