Sun Fire X4150 SPECjbb World Record x86 Single-JVM and x86 2 Chip Multi-JVM Performance

Monday Feb 18, 2008

The Sun Fire X4150 server equipped with 2 Quad-core Intel Xeon processors obtained World Record x86 single-JVM and x86 2 chip multi-JVM results on the SPECjbb2005 benchmark. Enhancements to the JVM had a major impact on performance.

The Sun Fire X4150 with 2 Intel X5460 quad-core processors and running Sun J2SE 1.6.0_05-p achieved x86 2-chip World Record performance of 303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark for multi-JVM results.

The Sun Fire X4150 with 2 Intel X5460 quad-core processors and running Sun J2SE 1.6.0_05-p achieved x86 World Record performance of 277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM for a single JVM on the SPECjbb2005 benchmark.

Using the same processor, the Sun Fire X4150 with Solaris 10 and Java HotSpot(TM) 32-Bit Server, beat the results of Dell and Lenovo which used Windows and BEA JRockit on the multi-JVM test.

The Sun Fire X4150 running the single-JVM SPECjbb2005 test easily beat all x86 results, topping the Dell R200 by 1.98X, the Fujitsu BX620 by 2.0X and the SGI XE240 by nearly 2.1X.

SPECjbb2005 Performance Chart (ordered by performance, bops: SPECjbb2005 Business Operations per Second (bigger is better), selected results... my best guess at the top ones for the engineers who act like lawyers you can go to www.spec.org to see all results as it clearly states at the bottom.

System	Processors					Performance
	Chip	Core	Thr	GHz	Type	SPEC- jbb- 2005 bops	JVMs	SPEC- jbb- 2005 bops/JVM
Multi-JVM, 2-Chip x86 Results (selected top see note above)
Sun Fire X4150	2	8	8	3.16	X5460	303297	4	75824
Dell PowerEdge 2950	2	8	8	3.16	X5460	303130	4	75783
Lenovo R515	2	8	8	3.16	X5460	294716	4	73678
Single-JVM x86 Results (selected top, see note above)
Sun Fire X4150	2	8	8	3.16	X5460	277585	1	277585
Dell PowerEdge R200	1	4	4	2.66	X3230	140220	1	140220
Fujitsu BX620	2	4	4	3.0	5160	138388	1	138388
SGI Altix XE240	2	4	4	3	5160	134561	1	134561
Dell PowerEdge 2950	2	4	4	3	5160	130589	1	130589

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 2/8/2008 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores) 303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM; Dell PowerEdge 2950 (2 chips, 8 cores) 303130 SPECjbb2005 bops, 75783 SPECjbb2005 bops/JVM; Lenovo R515 (2 chips, 8 cores) 294716 SPECjbb2005 bops, 73678 SPECjbb2005 bops/JVM.

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 2/8/2008 on www.spec.org. Sun Fire X4150 (2 chips, 8 cores) 277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM; Dell PowerEdge R200 (1 chip, 4 cores) 140220 SPECjbb2005 bops, 140220 SPECjbb2005 bops/JVM; Fujitsu BX620 (2 chips, 4 cores) 138388 SPECjbb2005 bops, 138388 SPECjbb2005 bops/JVM; SGI Altix XE240 (2 chips, 4 cores) 134561 SPECjbb2005 bops, 134561 SPECjbb2005 bops/JVM; Dell PowerEdge 2950 (2 chips, 4 cores) 130589 SPECjbb2005 bops, 130589 SPECjbb2005 bops/JVM.

Results Summary

Reference Date:			Feb 8, 2008
Multi-JVM			303297 SPECjbb2005 bops, 75824 SPECjbb2005 bops/JVM
Single-JVM			277585 SPECjbb2005 bops, 277585 SPECjbb2005 bops/JVM
System:			Sun Fire X4150
Processor:			2 x Intel X5460 3.166 GHz
Operating System:			Solaris 10 8/07
JVM:			Java HotSpot(TM) 32-Bit Server, Version 1.6.0_05-p

Intel defaults and judging performance

Tuesday Sep 18, 2007

Intel non-default BIOS change results by 25%? Sure turning off prefetch is a technique but if you don't know if a priori if you should, then should you use it to judge performance?

Always interesting when you have more information. I guess our friends at AMD wanted everyone to see what our friends at Intel were doing so they submitted two SPEC results for them.

Case in point on Clovertown there are two AMD results on the same hardware that gives 25% difference.

Point: Normal mode = prefetch on
Gives 163,080 SPECjbb2005 bops
www.spec.org/jbb2005/results/res2007q2/jbb2005-20070326-00276.txt

Counter-point: Disable HW prefetcher in BIOS for benchmark imprv
Gives 203,754 SPECjbb2005 bops
www.spec.org/jbb2005/results/res2007q2/jbb2005-20070326-00275.txt

...both on the same hardware:
same: 2-socket SuperMicro X7DBE (Intel 2.66GHz Xeon quad-core X5355), 16 GB

Disclosure statement

SPECjbb2005 SuperMicro X7DBE (2 chips, 8 cores, 2.66 GHz) SPECjbb2005 bops=163080, SPECjbb2005 bops/JVM=81540 submitted by AMD; SuperMicro X7DBE (2 chips, 8 cores, 2.66 GHz) SPECjbb2005 bops=203754, SPECjbb2005 bops/JVM=101877 submitted by AMD; SPEC, SPECjbb are registered trademarks of Standard Performance Evaluation Corporation. Results 3/7/07 on www.spec.org.

JavaOne: reposting World Record Single JVM result

Thursday May 10, 2007

Sun's single-JVM World Record (table now shown ordered by bops/jvm):

• The 72-way Sun Fire E25K with 1.8 GHz US-IV+ set a World Record with a single-instance result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM on SPECjbb2005.
• The Sun Fire E25K with dual-core US-IV+ 1.8GHz and running Java SE 1.6.0_02 outperformed Fujitsu PrimeQuest 580 with dual-core Itanium 2 processors and running BEA JRockit by 4%.
• The 72-way Sun Fire E25K with 1.8 GHz US-IV+ processors set a World Record with a single-instance result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark, Sun has a much higher result SPECjbb2005 using multiple JVMs running on a 1.95GHz US-IV+ processor. The Sun Fire E25K with 1.95GHz US-IV+ processors set a World Record for systems with 72 or fewer chips, achieving 2,105,264 SPECjbb2005 bops and 29,240 SPECjbb2005 bops/JVM on SPECjbb2005.

SPECjbb2005 Performance (ordered by performance bops/JVM : SPECjbb2005 Business Operations per Second, bigger is better)

System	Date	Processors		Performance
		(Chips, Cores, Threads)	GHz Type	bops	JVMs	bops/JVM
Sun Fire E25K	5/07	72, 144, 144	1.8 US-IV+	1,149,100	1	1,149,100
Fujitsu PRIMEQUEST 580	2/07	32, 64, 64	1.6 Itanium2	1,105,465	1	1,105,465
HP Superdome	9/06	64, 128, 128	1.6 Itanium 2	2,054,864	32	64,215
IBM p5 570	1/06	8, 16, 32	2.2 POWER5+	326,651	8	40,831
Fujitsu PP2500	3/06	128, 128, 128	2.08 SPARC64 V	1,251,024	32	39,095
Sun Fire E25K	5/07	72, 144, 144	1.95 US-IV+	2,105,264	72	29,240

Sun results have been submitted to SPEC for review and are on track for publication.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.8 GHz) 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM submitted for review; Fujitsu PrimeQuest 580 (32 chips, 64 cores, 1.6 GHz) 1,105,465 SPECjbb2005 bops, 1,105,564 SPECjbb2005 bops/JVM. SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 2,105,264 SPECjbb2005 bops, 29,240 SPECjbb2005 bops/JVM submitted for review; Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 1,657,274 SPECjbb2005 bops, 23,018 SPECjbb2005 bops/JVM; HP Itanium Superdome (64 chips, 128 cores, 1.6 GHz) 2,054,864 SPECjbb2005 bops, 64,215 SPECjbb2005 bops/JVM; Fujitsu PRIMEPOWER 2500 (128 chips, 128 cores, 2.08 Ghz) 1,251,024 SPECjbb2005 bops, 39,095 SPECjbb2005 bops/JVM; IBM eServer p5 570 (8 chips, 16 cores, 2.2 GHz) 326,651 SPECjbb2005 bops, 40,831 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

Certified Results
	Performance:		1,105,564 SPECjbb2005 bops
			1,105,564 SPECjbb2005 bops/JVM
	Reference Date:		May 8, 2007
Systems:			Sun Fire E25K
Processor/GHz:			72 US-IV+ 1.8 GHz
Operating System:			Solaris 10
JVM:			Java HotSpot(TM) 32-Bit Server, Version 6.0_02

JavaOne: World Record Single JVM result

Wednesday May 09, 2007

Sun's single-JVM World Record:

• The 72-way Sun Fire E25K with 1.8 GHz US-IV+ set a World Record with a single-instance result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM on SPECjbb2005.
• The Sun Fire E25K with dual-core US-IV+ 1.8GHz and running Java SE 1.6.0_02 outperformed Fujitsu PrimeQuest 580 with dual-core Itanium 2 processors and running BEA JRockit by 4%.
• The 72-way Sun Fire E25K with 1.8 GHz US-IV+ processors set a World Record with a single-instance result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark, Sun has a much higher result SPECjbb2005 using multiple JVMs running on a 1.95GHz US-IV+ processor. The Sun Fire E25K with 1.95GHz US-IV+ processors set a World Record for systems with 72 or fewer chips, achieving 2,105,264 SPECjbb2005 bops and 29,240 SPECjbb2005 bops/JVM on SPECjbb2005.

SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)

System	Date	Processors		Performance
		(Chips, Cores, Threads)	GHz Type	bops	JVMs	bops/JVM
Sun Fire E25K	5/07	72, 144, 144	1.95 US-IV+	2,105,264	72	29,240
HP Superdome	9/06	64, 128, 128	1.6 Itanium 2	2,054,864	32	64,215
Fujitsu PP2500	3/06	128, 128, 128	2.08 SPARC64 V	1,251,024	32	39,095
Sun Fire E25K	5/07	72, 144, 144	1.8 US-IV+	1,149,100	1	1,149,100
Fujitsu PRIMEQUEST 580	2/07	32, 64, 64	1.6 Itanium2	1,105,465	1	1,105,465
IBM p5 570	1/06	8, 16, 32	2.2 POWER5+	326,651	8	40,831

Sun results have been submitted to SPEC for review and are on track for publication.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.8 GHz) 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM submitted for review; Fujitsu PrimeQuest 580 (32 chips, 64 cores, 1.6 GHz) 1,105,465 SPECjbb2005 bops, 1,105,564 SPECjbb2005 bops/JVM. SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 2,105,264 SPECjbb2005 bops, 29,240 SPECjbb2005 bops/JVM submitted for review; Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 1,657,274 SPECjbb2005 bops, 23,018 SPECjbb2005 bops/JVM; HP Itanium Superdome (64 chips, 128 cores, 1.6 GHz) 2,054,864 SPECjbb2005 bops, 64,215 SPECjbb2005 bops/JVM; Fujitsu PRIMEPOWER 2500 (128 chips, 128 cores, 2.08 Ghz) 1,251,024 SPECjbb2005 bops, 39,095 SPECjbb2005 bops/JVM; IBM eServer p5 570 (8 chips, 16 cores, 2.2 GHz) 326,651 SPECjbb2005 bops, 40,831 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

Certified Results
	Performance:		1,105,564 SPECjbb2005 bops
			1,105,564 SPECjbb2005 bops/JVM
	Reference Date:		May 8, 2007
Systems:			Sun Fire E25K
Processor/GHz:			72 US-IV+ 1.8 GHz
Operating System:			Solaris 10
JVM:			Java HotSpot(TM) 32-Bit Server, Version 6.0_02

JaveOne: More performance results

Tuesday May 08, 2007

Sun US-IV+ vs. POWER5+:

• Sun Fire E2900 with dual-core US-IV+ (24 threads) beats the the fastest IBM POWER5+ p5 570 result (2.2 GHz 32 threads) of 326,651 bops.
• How do they compare on pricing? Well IBM doesn't seem to post pricing online for expensive products over about $500,000 or 32 threads, so I can't get the public pricing for the IBM p570.
The only online pricing (a bit old) is from IBM p570 TPC-C disclosure report. IBM TPC-C result of 1,025,169 tpmC at $4.42/tpmC on a 16-core (8 processors, 32 threads) 2.2 GHz IBM System p5 570 (configuration planned to be available 05/31/06). look carefully at the line items, just turning on the processors is expensive: "MODEL 570 PERMANENT PROCESSOR ACTIVATION FEATURE *16 = $343,040" (note: that does NOT count any memory costs or actually getting the processors any of the 48 other p570 required line items). My comments on the TPC-C benchmark.
• Also see this for IBM hardware per core pricing on the high end - wow!

The Sun Fire E2900 with 1.95GHz US-IV+ achieved 332,917 SPECjbb2005 bops and 27.743 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark. The Sun Fire E2900 used Solaris 10.

SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)

System	Date	Processors		Performance
		(Chips, Cores, Threads)	GHz Type	bops	JVMs	bops/JVM
Sun Fire E2900	5/07	12, 24, 24	1.95 US-IV+	332,917	12	27,743
IBM p5 570	1/06	8, 16, 32	2.2 POWER5+	326,651	8	40,831

Sun results have been submitted to SPEC for review and are on track for publication.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPECjbb2005 Sun Fire E2900 (12 chips, 24 cores, 24 threads1.95 GHz) 332,917 SPECjbb2005 bops, 27,743 SPECjbb2005 bops/JVM submitted for review; IBM eServer p5 570 (8 chips, 16 cores, 32 Threads 2.2 GHz) 326,651 SPECjbb2005 bops, 40,831 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

Certified Results
	Performance:		332,917 SPECjbb2005 bops
			27,743 SPECjbb2005 bops/JVM
	Reference Date:		May 8, 2007
Systems:			Sun Fire E2900
Processor/GHz:			12 US-IV+ 1.95 GHz
Operating System:			Solaris 10
JVM:			Java HotSpot(TM) 32-Bit Server, Version 6.0_02

Java performance: Sun Fire E25K UltraSPARC IV+ Beats HP Superdome Itanium 2

Tuesday May 08, 2007

Sun leads the way, beating Itanium2 and POWER5+(by a lot):

• Sun Fire E25K with dual-core US-IV+ beats the HP Superdome with dual-core Itanium 2.
• Sun Fire E25K is 6.4 times faster than the fastest IBM POWER5+ p5 570 result (1.9GHz 16 cores) of 326,651 bops. Note: The largest IBM p5 595 only has 4 times as many POWER5+ cores. IBM has not published this benchmark on their largest systems. why does IBM keep avoiding comparison to Sun on accepted standard benchmarks like SPECjbb2005?
• Sun Fire E25K 1.95GHz US-IV+ also beats the Fujitsu PRIMEPOWER 2500 2.08GHz SPARC64 V by 67%.

The Sun Fire E25K with 1.95GHz US-IV+ set a World Record for systems with 72 or fewer chips, achieving 2,105,264 SPECjbb2005 bops and 29,240 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark.

The 6.0_02 version of the Java HotSpot(TM) 32-Bit Server VM showed a 27% improvement of the 6.0 version on the SPECjbb2005 benchmark. The Sun Fire E25K result used Solaris 10.

SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)

System	Date	Processors		Performance
		(Chips, Cores, Threads)	GHz Type	bops	JVMs	bops/JVM
Sun Fire E25K	5/07	72, 144, 144	1.95 US-IV+	2,105,264	72	29,240
HP Superdome	9/06	64, 128, 128	1.6 Itanium 2	2,054,864	32	64,215
Fujitsu PP2500	3/06	128, 128, 128	2.08 SPARC64 V	1,251,024	32	39,095
IBM p5 570	1/06	8, 16, 32	2.2 POWER5+	326,651	8	40,831

Sun results have been submitted to SPEC for review and are on track for publication.

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Disclosure Statement:

SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 2,105,264 SPECjbb2005 bops, 29,240 SPECjbb2005 bops/JVM submitted for review; Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 1,657,274 SPECjbb2005 bops, 23,018 SPECjbb2005 bops/JVM; HP Itanium Superdome (64 chips, 128 cores, 1.6 GHz) 2,054,864 SPECjbb2005 bops, 64,215 SPECjbb2005 bops/JVM; Fujitsu PRIMEPOWER 2500 (128 chips, 128 cores, 2.08 Ghz) 1,251,024 SPECjbb2005 bops, 39,095 SPECjbb2005 bops/JVM; IBM eServer p5 570 (8 chips, 16 cores, 2.2 GHz) 326,651 SPECjbb2005 bops, 40,831 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

Certified Results
	Performance:		2,105,264 SPECjbb2005 bops
			29,240 SPECjbb2005 bops/JVM
	Reference Date:		May 8, 2007
Systems:			Sun Fire E25K
Processor/GHz:			72 US-IV+ 1.95 GHz
Operating System:			Solaris 10
JVM:			Java HotSpot(TM) 32-Bit Server, Version 6.0_02

Power5+ now off the road?

Saturday Mar 31, 2007

IBM lacks Power5+ benchmarks on new & old workloads that everyone else is publishing on. Why no lastest GHz full-system IBM p595 publications on:

• SPECjbb2005?
• SPECint_rate2006?
• SPECfp_rate2006?
• Linpack?
• SPECint_2006?
• SPECfp_2006?
• ....the list goes on...

Don't they want comparisons? I hear IBM bloggers still love TPC-C so is the IBM p595 only suited for that very old (14-year old) test? The press and analysts are overwhelmingly seeing TPC-E the successor to the simplistic 13 year-old TPC-C. 7 years ago when Sun established a World Record TPC-C, Sun told the world the benchmark was too simplistic. It is good the see the rest of the industry is catching up. Sun never quotes 23-year old Dhrystones benchmark anymore either.

For those who may not remember, IBM didn't even end the EOL'ed SPECint_rate2000 on a high note: http://www.spec.org/cpu2000/results/rint2000.html, search for "1644" and "1513" Since we're talking history, I should be clear and state that by "1513" I wasn't talking about the year that Juan Ponce de Leon definitely is known to have sighted what is now the USA and claimed it for Spain.

Promises, promises & IBM

Thursday Feb 15, 2007

IBM POWER6 info in a CNET article.

The say, "The first Power6 systems, lower-end models, are due to arrive midway through 2007."

So in the mean time will IBM start publishing the benchmarks they've avoided on IBM p5 595 POWER5+ any time soon? Or is it just too embarrassing to show SPECjbb2005, SPECint_rate2006, etc. results compared to Sun 1.8GHz US-IV+ systems?

when do the high-end power6 systems start to show? ...late 2007 or 2008?