SPECpower_ssj2008 Sun Netra X4250 - SPECpower issues highlighted

Thursday May 14, 2009

Sun has finally published some results that show the configuration issues with SPECpower_ssj2008, these three results substantiate a just a few of the issues that I've been talking about with SPECpower_ssj. Clearly now is not the time for special power benchmarks but publication of watts on all benchmarks - which Sun has been doing for YEARS! Now, on to information that I saw in an email that went out yesterday...

The Sun Netra X4250 8GB server (two 2.13 GHz Intel L5408 QC) obtained a peak overall ssj_ops/watt metric of 600 (with special BIOS tuning). A more typical 32GB configuration of the same system achieved results of 478 (with special BIOS tuning) and a lower 437 (with standard BIOS settings). Note 8GB is only 0.5GB/core which is much smaller than 32GB (2GB/core) which is used for many 2-socket QC benchmarks that just measure performance).

These results were obtained on the Sun Netra X4250 server using Microsoft Windows Server 2003 R2 Enterprise x64 Edition SP2 and Sun Java HotSpot 32-Bit Server VM on Windows, version 1.6.0_14.

SPECpower_ssj results shows that servers (even those with the industry's best power-management) running at low-utilization levels use many times more watts per unit-of-work than systems running at higher utilization levels. Datacenters can realize the biggest energy savings by running fewer servers at higher utilization levels (50% utilization or above).

Sun's results on the 8GB (or 0.5GB/core) configuration show that running at 10% utilization requires 4.4 times more power per unit of work than running at 50% utilization.

4.4 times = (581 performance-to-power @ 50% utilization /133 performance-to-power @ 10% utilization)

Most SPECpower_ssj2008 are published on small-memory configurations that are much smaller than typical customer deployments. Sun is the only vendor to publish multiple results to clearly show effect of memory configuration.

A more normal-sized memory configuration of 32GB (or 2GB/core) uses 30% more watts than a tiny 8GB (or 0.5GB/core) configuration at 100% load. At active-idle the wattage difference is also 30%. Some competitors use additional configuration differences such as non-redundant fans, non-redundant power supplies, and single slow disk to further reduce the wattages and significantly improve SPECpower_ssj scores.

Most published SPECpower_ssj2008 results make low-level BIOS changes to turn off hardware prefetch. Sun shows that non-default BIOS changes improve Peak Performance ssj_ops by 9%. This non-default BIOS change hurts the performance of other workloads.

Dramatic minimization of memory configuration (0.5GB/core) and the use of a non-standard BIOS provided a 39% improvement to the peak Performance-to-Power Ratio (ssj_ops/watt) on small 8GB configuration Sun Netra X4250 compared to the same server with more-typically configured 32GB (2GB/core) of memory with the default BIOS.

Also when I look at the data, I find a _very_ linear relationship on wattage from active-idle to 100%. In other words if you measure 100% and Idle you can easily and very accurately estimate the watts at any utilization level. If anyone has data please to the contrary please post a comment with data & analysis - Thanks from your friendly BM Seer.

SPECpower_ssj2008 Low-Power Harpertown (QC L5400-series) Performance Chart (ordered by benchmark primary metric, overall ssj_ops/watt) Selected low-power Harpertown leading and major-manufacturer results. Metric: overall ssj_ops/watt (bigger is better)

Some of the competitive results use NON-redundant fans and NON-redundant power supplies and minimize other aspects of the configuration.

The Sun Netra X4250 includes:

• 2x redundant power supplies
• redundant fan modules
• standard I/O expansion mezzanine
• standard Telco dry contact alarm
which are only present in the Netra X4250 among SPECpower_ssj2008 configurations.

System	Processors		Performance
	Model	GHz	Metric overall ssj_ops/watt	Peak Performance ssj_ops	Peak Power watts	Idle Power watts
Powerleader PR2510D2 (8GB non-std BIOS)	L5430	2.66	1135	285970	161	84.6
NEC ECO CENTER (8GB non-std BIOS)	L5420	2.5	1010	288502	175	102
HP ProLiant DL180 G5 (8GB non-std BIOS)	L5430	2.66	930	282281	189	106
Fujitsu PRIMERGY TX300 S4 (8GB non-std BIOS)	L5430	2.66	917	326128	221	136
Dell PowerEdge R300 (8GB non-std BIOS)	L5410	2.33	800	155342	117	75.1
Sun Netra X4250 (8GB non-std BIOS)	L5408	2.13	600	244832	226	174
Sun Netra X4250 (32GB non-std BIOS)	L5408	2.13	478	251555	294	226
Sun Netra X4250 (32GB default BIOS)	L5408	2.13	437	229828	296	225

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org. Results as of May 8, 2009.

Benchmark Description

SPECpower_ssj2008 is the first SPEC benchmark intended to measure the power efficiency of a server. It is based on SPECjbb2005 but the workload has been modified so that the performance portion of the results are not comparable to SPECjbb2005 results. In addition, the workload is varied from unconstrained (ie. maximum) throughput performance to idle (but active) state in 10% decrements, during which the average power consumption is measured. The power and performance is measured, and the ratio of performance to power is computed, for each load point. The overall metric, denoted overall ssj_ops/watt, is the ratio of the sum of performance at each point to the sum of average power at each point, to include the idle point.

Disclosure Statement:

Sun Netra X4250 server 600 overall ssj_ops/watt and (226 watts, 244832 ssj_ops) @ 100% target load, (210 watts, 121828 ssj_ops) @ 50% target load, (181 watts, 24150 ssj_ops) @ 10% target load, (174 watts) @ active idle target load. Sun Netra X4250 server 478 overall ssj_ops/watt and (294 watts, 251555 ssj_ops) @ 100% target load, (226 watts) @ active idle target load. Sun Netra X4250 server 437 overall ssj_ops/watt and (296 watts, 244832 ssj_ops) @ 100% target load, (225 watts) @ active idle target load. SPEC and the benchmark names SPECpower_ssj, SPECweb, SPECjbb, SPECjAppServer, SPEComp are trademarks of the Standard Performance Evaluation Corporation. Benchmark results stated above reflect results published on http://www.spec.org as of March 30, 2009. For the latest SPECpower_ssj2008 benchmark results, visit http://www.spec.org/power_ssj2008. See Also: SPECpower_ssj2008 Benchmark Reports

System Configuration

Sun's three results all used the same software components and processors.

Processor:			2 x Intel L5408 QC 2.13 GHz
Operating System:			Windows Server 2003 R2 Enterprise x64 Edition SP2
JVM:			Java HotSpot(TM) 32-Bit Server VM on Windows, version 1.6.0_14

The following result was produced using a non-typical, small configuration and special BIOS tuning.

Reference Date:			March 30, 2009
Results			600 overall ssj_ops/watt
System:			Sun Netra X4250 (8GB, 4 x 2048MB as PC2-5300F 2Rx8)
BIOS:			non-standard (hardware prefetch off)

• 1 x Sun 146GB 10K RPM SAS drive
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

The following result was produced using a more typical larger configuration including fully configured disk drives and an option NIC card. Special BIOS tuning was retained to boost performance and allow a direct comparison between BIOS tunings and standard BIOS.

Reference Date:			May 6, 2009
Results			478 overall ssj_ops/watt
System:			Sun Netra X4250 (32GB, 8 x 4096MB as PC2-5300F)
BIOS:			non-standard (hardware prefetch off)

• 4 x Sun 146GB 10K RPM SAS drive
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

The following result was produced using a more typical larger configuration including fully configured disk drives and an option NIC card. Standard BIOS tuning was used to demonstrate the advantage obtained by special BIOS tuning which benefits this benchmark.

Reference Date:			May 6, 2009
Results			437 overall ssj_ops/watt
System:			Sun Netra X4250 (32GB, 8 x 4096MB as PC2-5300F)
BIOS:			default (normal prefetch)

• 4 x Sun 146GB 10K RPM SAS drive
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm