InfoWorld weighs in on Energy Star for Servers

Wednesday May 27, 2009

New article on Energy Star for Servers http://www.infoworld.com/print/76258

Idle servers are the devil's tools So what's wrong with the new Energy Star criteria? Perhaps the most significant drawback is that a qualifying server need only demonstrate energy efficiency when it's in idle -- that is, powered on but doing no work.

They continue...

To better illustrate the problem, imagine you're at a used car lot where a shifty salesperson is trying to push you to buy an SUV. His selling point: "This baby uses the same amount of fuel as a hybrid sedan -- when you're not moving."

My view is, as I've stated before, ALL computer vendors need to publish watts on standard benchmarks. Sun has been doing many for years... My view is some vendors only want to hid behind idle watts and argued enough so that in the end the US Gov just put out this.

But what about SPECpower, see these:
http://blogs.sun.com/bmseer/entry/specpower_ssj2008_sun_netra_x4250
http://blogs.sun.com/bmseer/entry/chopped_configs_and_specpowerssj2008

Miles per gallon or GPM, same as perf/watt or watt/perf

Tuesday May 19, 2009

As a reminder: "Miles per gallon is misleading and can play tricks on our intuitions," Jack Soll said.

Take this little test, if you think he is wrong:
http://www.fuqua.duke.edu/news/mpg/mpg.html

In computing the same is true perf/watt is misleading and plays tricks on our intuitions. Sun tries to promote watt/performance. You'll even hear Sun's VPs talking about power-performance!

Today in email, I saw a pointer to this gem of a website: http://www.mpgillusion.com

SPECpower_ssj2008 Sun Netra X4250 - SPECpower issues highlighted

Thursday May 14, 2009

Sun has finally published some results that show the configuration issues with SPECpower_ssj2008, these three results substantiate a just a few of the issues that I've been talking about with SPECpower_ssj. Clearly now is not the time for special power benchmarks but publication of watts on all benchmarks - which Sun has been doing for YEARS! Now, on to information that I saw in an email that went out yesterday...

The Sun Netra X4250 8GB server (two 2.13 GHz Intel L5408 QC) obtained a peak overall ssj_ops/watt metric of 600 (with special BIOS tuning). A more typical 32GB configuration of the same system achieved results of 478 (with special BIOS tuning) and a lower 437 (with standard BIOS settings). Note 8GB is only 0.5GB/core which is much smaller than 32GB (2GB/core) which is used for many 2-socket QC benchmarks that just measure performance).

These results were obtained on the Sun Netra X4250 server using Microsoft Windows Server 2003 R2 Enterprise x64 Edition SP2 and Sun Java HotSpot 32-Bit Server VM on Windows, version 1.6.0_14.

SPECpower_ssj results shows that servers (even those with the industry's best power-management) running at low-utilization levels use many times more watts per unit-of-work than systems running at higher utilization levels. Datacenters can realize the biggest energy savings by running fewer servers at higher utilization levels (50% utilization or above).

Sun's results on the 8GB (or 0.5GB/core) configuration show that running at 10% utilization requires 4.4 times more power per unit of work than running at 50% utilization.

4.4 times = (581 performance-to-power @ 50% utilization /133 performance-to-power @ 10% utilization)

Most SPECpower_ssj2008 are published on small-memory configurations that are much smaller than typical customer deployments. Sun is the only vendor to publish multiple results to clearly show effect of memory configuration.

A more normal-sized memory configuration of 32GB (or 2GB/core) uses 30% more watts than a tiny 8GB (or 0.5GB/core) configuration at 100% load. At active-idle the wattage difference is also 30%. Some competitors use additional configuration differences such as non-redundant fans, non-redundant power supplies, and single slow disk to further reduce the wattages and significantly improve SPECpower_ssj scores.

Most published SPECpower_ssj2008 results make low-level BIOS changes to turn off hardware prefetch. Sun shows that non-default BIOS changes improve Peak Performance ssj_ops by 9%. This non-default BIOS change hurts the performance of other workloads.

Dramatic minimization of memory configuration (0.5GB/core) and the use of a non-standard BIOS provided a 39% improvement to the peak Performance-to-Power Ratio (ssj_ops/watt) on small 8GB configuration Sun Netra X4250 compared to the same server with more-typically configured 32GB (2GB/core) of memory with the default BIOS.

Also when I look at the data, I find a _very_ linear relationship on wattage from active-idle to 100%. In other words if you measure 100% and Idle you can easily and very accurately estimate the watts at any utilization level. If anyone has data please to the contrary please post a comment with data & analysis - Thanks from your friendly BM Seer.

SPECpower_ssj2008 Low-Power Harpertown (QC L5400-series) Performance Chart (ordered by benchmark primary metric, overall ssj_ops/watt) Selected low-power Harpertown leading and major-manufacturer results. Metric: overall ssj_ops/watt (bigger is better)

Some of the competitive results use NON-redundant fans and NON-redundant power supplies and minimize other aspects of the configuration.

The Sun Netra X4250 includes:

• 2x redundant power supplies
• redundant fan modules
• standard I/O expansion mezzanine
• standard Telco dry contact alarm
which are only present in the Netra X4250 among SPECpower_ssj2008 configurations.

System	Processors		Performance
	Model	GHz	Metric overall ssj_ops/watt	Peak Performance ssj_ops	Peak Power watts	Idle Power watts
Powerleader PR2510D2 (8GB non-std BIOS)	L5430	2.66	1135	285970	161	84.6
NEC ECO CENTER (8GB non-std BIOS)	L5420	2.5	1010	288502	175	102
HP ProLiant DL180 G5 (8GB non-std BIOS)	L5430	2.66	930	282281	189	106
Fujitsu PRIMERGY TX300 S4 (8GB non-std BIOS)	L5430	2.66	917	326128	221	136
Dell PowerEdge R300 (8GB non-std BIOS)	L5410	2.33	800	155342	117	75.1
Sun Netra X4250 (8GB non-std BIOS)	L5408	2.13	600	244832	226	174
Sun Netra X4250 (32GB non-std BIOS)	L5408	2.13	478	251555	294	226
Sun Netra X4250 (32GB default BIOS)	L5408	2.13	437	229828	296	225

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org. Results as of May 8, 2009.

Benchmark Description

SPECpower_ssj2008 is the first SPEC benchmark intended to measure the power efficiency of a server. It is based on SPECjbb2005 but the workload has been modified so that the performance portion of the results are not comparable to SPECjbb2005 results. In addition, the workload is varied from unconstrained (ie. maximum) throughput performance to idle (but active) state in 10% decrements, during which the average power consumption is measured. The power and performance is measured, and the ratio of performance to power is computed, for each load point. The overall metric, denoted overall ssj_ops/watt, is the ratio of the sum of performance at each point to the sum of average power at each point, to include the idle point.

Disclosure Statement:

Sun Netra X4250 server 600 overall ssj_ops/watt and (226 watts, 244832 ssj_ops) @ 100% target load, (210 watts, 121828 ssj_ops) @ 50% target load, (181 watts, 24150 ssj_ops) @ 10% target load, (174 watts) @ active idle target load. Sun Netra X4250 server 478 overall ssj_ops/watt and (294 watts, 251555 ssj_ops) @ 100% target load, (226 watts) @ active idle target load. Sun Netra X4250 server 437 overall ssj_ops/watt and (296 watts, 244832 ssj_ops) @ 100% target load, (225 watts) @ active idle target load. SPEC and the benchmark names SPECpower_ssj, SPECweb, SPECjbb, SPECjAppServer, SPEComp are trademarks of the Standard Performance Evaluation Corporation. Benchmark results stated above reflect results published on http://www.spec.org as of March 30, 2009. For the latest SPECpower_ssj2008 benchmark results, visit http://www.spec.org/power_ssj2008. See Also: SPECpower_ssj2008 Benchmark Reports

System Configuration

Sun's three results all used the same software components and processors.

Processor:			2 x Intel L5408 QC 2.13 GHz
Operating System:			Windows Server 2003 R2 Enterprise x64 Edition SP2
JVM:			Java HotSpot(TM) 32-Bit Server VM on Windows, version 1.6.0_14

The following result was produced using a non-typical, small configuration and special BIOS tuning.

Reference Date:			March 30, 2009
Results			600 overall ssj_ops/watt
System:			Sun Netra X4250 (8GB, 4 x 2048MB as PC2-5300F 2Rx8)
BIOS:			non-standard (hardware prefetch off)

• 1 x Sun 146GB 10K RPM SAS drive
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

The following result was produced using a more typical larger configuration including fully configured disk drives and an option NIC card. Special BIOS tuning was retained to boost performance and allow a direct comparison between BIOS tunings and standard BIOS.

Reference Date:			May 6, 2009
Results			478 overall ssj_ops/watt
System:			Sun Netra X4250 (32GB, 8 x 4096MB as PC2-5300F)
BIOS:			non-standard (hardware prefetch off)

• 4 x Sun 146GB 10K RPM SAS drive
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

The following result was produced using a more typical larger configuration including fully configured disk drives and an option NIC card. Standard BIOS tuning was used to demonstrate the advantage obtained by special BIOS tuning which benefits this benchmark.

Reference Date:			May 6, 2009
Results			437 overall ssj_ops/watt
System:			Sun Netra X4250 (32GB, 8 x 4096MB as PC2-5300F)
BIOS:			default (normal prefetch)

• 4 x Sun 146GB 10K RPM SAS drive
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

Green Grid tool assesses free cooling options

Friday Apr 10, 2009

effective ones is to simply run servers at high-utilization which can save energy both to power the servers (2x to 5x more work done per unit of work watt!) and therefore you need less energy to cool your datacenters.

Another way to save on the cooling costs side of the equation is use free cooling.

Free cooling comes in the form of air-side and water-side economizers. With air-side economizers, often called fresh-air cooling, outside air is taken into a data center, usually filtered, and then used to cool the IT equipment. With water-side economizers, the outside air cools water in the outside chiller or water tower, which in turn cools a data center.

The Green Grid's online calculator includes a series of inputs -- location, temperature and humidity thresholds, IT load, and the cost of electricity are a few -- and at the other end spits out how many hours that data center can use air-side and water-side economizers, and how much money it could save.

Both of these quotes were from the SearchDataCenter.com Article: "Green Grid tool assesses free cooling potential", By Mark Fontecchio, News Writer 09 Apr 2009 SearchDataCenter.com

For more info also see: www.thegreengrid.org

SPECpower Issues Watts up? Two views

Wednesday Mar 04, 2009

A couple of postings ago in http://blogs.sun.com/bmseer/entry/specpower_ssj_obscuring_important_factors I mentioned how I thought they had too much detail in the wrong places. I do want critical things listed, but can we please focus on the important issues to power of REALISTIC CONFIGURATIONS. What does spec place in the summary table? Information on the JVM, GREAT, but what about size memory? Look at the results page and see that some vendors used 110+ characters to describe the JVM used in the main summary table. Misplaced details.

Standard bodies need to concentrate focus on important system issues FIRST have extraneous like testing location ruthlessly located in the back of the document.

To me it looks like the SPECpower was designed by people who focus on CPUs rather than the system that's likely to be deployed in a REAL customer environment. I also see this when looking at one of our more frequent commenters, rick jones who wrote:

Still my take on the power draws in a SPECpower_ssj2008 benchmark, at least as they have been published thusfar it would be processors, dimms, what I've seen called alternatively the CEC (Core Electronics Complex) or glue or perhaps "other motherboard components," and either the power supply or the boot disc.

This is a very CPU-centric view, in my opinion Rick's list boils down to:

• processors,
• dimms,
• CEC (Core Electronics Complex) or motherboard components,
• the power supply,
• boot disc.

Rather funny that he considers one of the things you can't configure in a system to be 3rd on the list.

I'd rather take a more system's view on a wide variety of tests! Shouldn't we just show power WHENEVER WE RUN ANY BENCHMARK! Sun does. Are others avoiding this because they can't beat Sun? Likely.

Having looked at REAL customer configurations and lots of SPECpower results, my view of the most important things to report on a system configuration when measuring power are:

• Memory Size (GBs), some use boutique LV-DIMMS
• Fans (need to mention when non-redundant) - most SPECpower results use NON-redundant ?!?
• Power supply (need to mention when non-redundant) - many SPECpower use NON-redundant
• Power management SW - Please tell me why is this NOT on by default for all servers?
• Processor type & GHz
• Disks - most deployed servers have standard RPM good size disk or disks! - most SPEC power results have a single 60GB-160GB?!? 7200RPM antique

These are all things that effect power to a larger extent and things that customers change in their configuration.

SPEC and SPECpower benchmark name are registered trademarks of the Standard Performance Evaluation Corporation. www.spec.org for details.

Dell now understands importance of server utilization

Wednesday Feb 18, 2009

Even Dell is starting to get it. Server utilisation levels in a data centre are critical to advance overall performance, improving productivity, and reducing costs.

In: http://www.dell.com/downloads/global/power/ps1q09-20090176-esser.pdf

They say:

Operational policies designed to increase server utilization and advance overall performance and efficiency can lead to dramatic improvements in data center productivity without increasing power consumption.

Yes indeed, I've been blogging that since 2005, remember this little gem from 2006: http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low

But I ran into several customers last week where a major computer vendor (not Dell or Sun) was telling everyone that 10% server utilisation was the most important utilisation to measure - SHAME ON YOU!

Can a group at SPEC or TPC just get on with adding power measurement to all benchmarks as they exist now at benchmark utilisation levels? ...Or just what is going on behind closed doors?

DISCLOSURE: I'm not on any SPEC or TPC committees, nor do I read any confidential updates from Sun employees who do.

SPECjAppServer Sun T5140/T5440

Friday Feb 13, 2009

Sun SPARC Enterprise T5140 and T5440 Delivers Outstanding Application and Database Performance on SPECjAppServer2004. Yet again, Sun is showing measured watts on another benchmark. I encourage all other vendors to do the same on all benchmarks. We all need this kind of transparency!

Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440 server delivered a result of 9500.76 SPECjAppServer2004 JOPS@Standard. ZFS was also used in this benchmark.

One Sun SPARC Enterprise T5140 server in the application tier, consumed on average 614 Watts of power and the Sun SPARC Enterprise T5440 server in the database tier, consumed on average 1836 Watts of power during the execution of this benchmark.

This benchmark used the Oracle WebLogic 10.3 Application Server and Oracle Database 11g Enterprise Edition. This benchmark result proves that the Sun SPARC Enterprise T5440 server using the UltraSPARC T2 Plus processor performs as an outstanding Oracle 11g OLTP database server.

The Sun SPARC Enterprise T5140, T5440 and M3000 servers used to produce this benchmark result all used the Solaris 10 10/8 Operating Environment.

Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440 server demonstrated better performance compared to the HP result of 9459.19 SPECjAppServer2004 JOPS@Standard which used 11x HP BL860c servers and 2x HP Superdomes.

Each Sun SPARC Enterprise T5140 server used 4 instances of Oracle WebLogic 10.3 and the Sun JVM 1.6.0_06 Performance Release in Solaris Containers. Each Sun SPARC Enterprise T5140 server used ZFS to mirror 2 Solid State Disks to meet the benchmark durable storage requirements for the application server logs and JMS Persistence filestore.

The Sun SPARC Enterprise T5440 server in the database tier used Oracle Automatic Storage Management (ASM) to manage the StorageTek 2540 and 2501 storage arrays for the database files and redo logs.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier used 24 Oracle licenses for the database. The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using one HP Superdome in the database tier used 40 Oracle database licenses. The Sun T5440 delivered 90% of the performance using 60% of the database licenses compared to the HP Superdome.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier used 4 Rack Units of space (H x W x D = 7" x 18" x 25" = 1.8 cu feet). The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome database (a 40 processor partition in 2x A9834A cabinets), have space requirements of (H x W x D = 72" x 48" x 45" = 90 cu feet). The Sun T5440 occupies 1/50 of the datacenter space at 90% of the performance of an HP Superdome.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier consumed on average 1836 watts of power during the execution of this benchmark. The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome database (a 40 processor partition in 2x A9834A cabinets), have power requirements of 13720 watts(1) or 7.5 TIMES more than Sun's T5440. The Sun T5440 consumes 14% of the power at 90% of the performance of an HP Superdome.

Performance Comparisons

SPECjAppServer2004 Performance Chart as of 02/04/2009. Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org. SPECjAppServer2004 JOPS@Standard (bigger is better)

Submitter	SPECjAppServer2004 JOPS@Standard	J2EE Server	DB Server
HP	10519.43	12x HP BL860c 4 cores, 2 chips @ 1.66 GHz Itanium 9100 Oracle OC4J 10.1.3.3.2	1x HP Superdome 40 cores, 20 chips @ 1.6 GHz Itanium 9000 Oracle 10g DB 10.2.0.3
Sun	9500.76	4x Sun SPARC Enterprise T5140 16 cores, 2 chips @ 1.2 GHz US-T2 Plus Oracle WebLogic 10.3	1x Sun SPARC Enterprise T5440 32 cores, 4 chips @ 1.4 GHz US-T2 Plus Oracle 11g DB 11.1.0.7
HP	9459.19	11x HP BL860c 4 cores, 2 chips @ 1.66 GHz Itanium 9100 Oracle OC4J 10.1.3.3.2	2x HP Superdome 80 cores, 40 chips @ 1.6 GHz Itanium 9000 Oracle 10g DB 10.2.0.3 with RAC
Sun	8439.366	6x Sun SPARC Enterprise T5120 8 cores, 1 chip @ 1.4 GHz US-T2 Sun Java System Application Server	1x Sun SPARC Enterprise E6900 48 cores, 24 chips @ 1.95 GHz US-IV+ IBM DB2 V9.1

Benchmark Description

SPECjAppServer2004 (Java Application Server) is a multi-tier benchmark for measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based application servers. SPECjAppServer2004 is an end-to-end application which exercises all major J2EE technologies implemented by compliant application servers as follows:

• The web container, including servelets and JSPs
• The EJB container
• EJB2.0 Container Managed Persistence
• JMS and Message Driven Beans
• Transaction management
• Database connectivity

Moreover, SPECjAppServer2004 also heavily exercises all parts of the underlying infrastructure that make up the application environment, including hardware, JVM software, database software, JDBC drivers, and the system network.

The primary metric of the SPECjAppServer2004 benchmark is jAppServer Operations Per Second (JOPS) which is calculated by adding the metrics of the Dealership Management Application in the Dealer Domain and the Manufacturing Application in the Manufacturing Domain. There is NO price/performance metric in this benchmark.

Disclosure Statement:

SPECjAppServer2004: 4x Sun SPARC Enterprise T5140 (8 chips, 64 cores) 9500.76 SPECjAppServer2004 JOPS@Standard. 12x HP BL860c (24 chips, 48 cores) 10519.43 SPECjAppServer2004 JOPS@Standard. 12x HP BL860c (22 chips, 44 cores) 9459.19 SPECjAppServer2004 JOPS@Standard. 6x Sun T5120 (6 chips, 48 cores) 8439.36 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 1/28/09.

1. HP Integrity Superdome using 2x A9834A cabinets. Taking 70% of Typical Input power of 9800 watts for an 8-cell cabinet and from: http://h18000.www1.hp.com/products/quickspecs/11717_div/11717_div.HTML

See Also: SPECjAppServer2004 Results Page

Results Summary

Published Results			9500.76 SPECjAppServer2004 JOPS@Standard
Reference Date:			Feb 4, 2009
Systems:			4x Sun SPARC Enterprise T5140
			1x Sun SPARC Enterprise T5440
			1x Sun SPARC Enterprise M3000
Total Number Processors:			2, 4, 1
Processor/GHz of Server:			UltraSPARC T2 Plus 1.2 GHz
			UltraSPARC T2 Plus 1.4 GHz
			SPARC64VII 2.52 GHz
Operating System:			Solaris 10 10/08
Software:			Oracle WebLogic 10.3 Application Server, Standard Edition
			Oracle Database Enterprise Edition Release 11.1.0.7
JVM:			JDK 1.6.0_06 Performance Release

note: ...as always you can post comments anonymously, but if you work for a system's vendor you should state who it is.

VMware/VMmark Sun Fire X4450 Best 24 Core Result and watts

Wednesday Jan 14, 2009

The Sun Fire X4450 powered by four 6-core Intel Xeon X7460 processors at 2.66 GHz, has achieved a score of 19.47 @14 tiles, supporting 84 fully fledged VM instances (14 Tiles). This result puts the SUN X4450 on the leading front of 4-socket/24 cores Intel-based VMware servers.

This results beats IBM x3850-M2 result to date of 19.10 @ 14 tiles and all the other Intel based submissions in the 24 cores category with similar architecture while using equal or smaller memory footprint (80GB) then the other systems.

The Sun Fire X4450 server, equipped with four 6C Xeon X7460, provides a highly scalable virtualization platform, that, in combination with the VMware ESX Server 3.5.0 hypervisor, takes full advantage of Intel Virtualization Technology.

To support 14 tiles per server , efficient and reliable Sun StorageTek 2540 Fibre Channel arrays with Sun StorageTek 2501 SAS expansion units were used. This family of arrays offers enterprise-class, reliable RAID-protected functionality in a high-density 2 RU package and is well suited for virtualization environments.

The electrical Power consumption measured for the 4-socket 6C 80GB server during the benchmark runs was on average: 933 Watts (Idle 750 Watts).

The electrical Power consumption for the data storage during the benchmark runs for each disk array and workload type was on average:

• 1x STK2540 (database workloads): 304 Watts (Idle 296 Watts)
• 1x STK2540 (Tile workloads except database): 306 Watts (Idle 296 Watts)
• 1x STK2501 (Tile workloads except database): 261 Watts (Idle 253 Watts)

Sun is the first vendor (and only) to disclose actual power consumption measured while running VMmark, the industry's most popular virtualization workload.

VMmark Performance Results for 24 Cores, Benchmark Score (bigger is better)

System	Socket/ Core/ Thrds	GHz Type	GB	ESX ver	Spindle	Mirr	Tiles	Score	Date Pub
Sun Fire X4450	4/24/24	2.66 Xeon X7460	80GB	3.5.0U2	192	n	14	19.47	13-Jan-09
IBM x3850 M2	4/24/24	2.66 Xeon X7460	80GB	3.5.0U2	166	n	14	19.10	17-Sep-08
Dell PE R900	4/24/24	2.66 Xeon X7460	128GB	3.5.0U2	140	n	14	18.96	02-Dec-08
HP PL DL580 G5	4/24/24	2.66 Xeon X7460	96GB	3.5.0U2	168	n	14	18.56	18-Aug-08

Complete benchmark results may be found at the VMware benchmark website http://www.vmware.com/products/vmmark/results.html.

Benchmark Description

VMmark is the first benchmark that was designed specifically to quantify and measure the performance of virtualized environments. It features a novel tile-based scheme for measuring the scalability of consolidated workloads and provides a consistent methodology that captures both the overall scalability and individual application performance.

The VMmark benchmark is built on VMware's expertise in virtualization performance and incorporates popular workloads from application categories most commonly represented in customer data centers.

The purpose of this benchmark is to measure performance and scalability of a pre-established mix of workloads (a Tile), which allows comparisons among similar configuration platforms.

A Tile consists of 6 fixed workload applications, each running in its own Virtual Machine (VM) (6 VMs per Tile) such as Mail, Java, Web, Database and File Serving plus a standby server which only purpose is to provide a spare Virtual Machine that does not do any work and is not accounted in the score.

VMmark benchmark provides two key performance metrics:

1. The number of tiles supported by a system, which is an indication of how many systems/applications can be consolidated on one platform where the higher the number of tiles supported the higher the number of consolidated systems.
2. The Score, which is an overall measure of the amount of work that is accomplished by all the Tiles in the system and summarizes the level of service of all the workloads during a benchmark run. The score or amount of work is a composition of Actions/minute(Mail server), New Orders/minute(Java server), Access/minute(web server), Commits/minute(Database), MB/second(file server). Thus, among systems with the same number of tiles, the system with the higher score is the system that is capable of producing the greater amount of work.

For detailed description of VMmark, tiles and score definition, please refer to VMmark Features

Disclosure Statement:

VMware(R) VMmark(R) is a product of VMware, an EMC Company. VMmark utilizes SPECjbb2005(TM) and SPECweb2005(TM), which are available from the Standard Performance Evaluation Corporation (SPEC). Results from http://www.vmware.com/products/vmmark/ as of January 13, 2009.

Result Information

Certified Results
	Score:		19.47@14-tiles
	Server:		Sun Fire X4450
	Processors:		4-socket 2.66 GHz 6-core Intel Xeon X7460
	Memory:		80 GB
	VMware ESX Server:		3.5.0 Update 2
	VMmark:		1.1
	Storage:		9 x STK2540, 7x STK2501, each storage array using 12x146GB 15K RPM disks

Sun's 2008 summary of measured watts & watt/performance

Thursday Jan 08, 2009

Sun has shown measured watts with measured performance on UltraSPARC for over four years!:
2008: UltraSPARC T2+ T5240 & UltraSPARC T2+ T5440
2007: UltraSPARC T2
2005: UltraSPARC T1 & T2000 blogs with power-performance

Sun is now showing measured watts with measured performance for Xeon-based and Opteron-based servers as well, Sun is saying we are going see a lot more soon:
2008 Xeon: Virtualization, Web
2008 Opteron: Java, Mail, Web

Real measured watts on a variety of workloads with real-sized memory configurations is critical to truly inform customers. I heard that one vendor was giving out top-secret watt measurements in a 16C datacentre (61F) as a way to lower their watt numbers... but does that make any real sense? But let's look at the big picture: Datacentre managers need to be look at by "power usage effectiveness" (PUE). To keep a datacentre that cold you need to burn LOTS of watts -- so great way to slash your server numbers for marketing, but really net loss for a real customer.

Sun is looking at a variety of ways to save customers money. Take a look at this ZDnet article "Some like it hot: Why waste dough cooling down a data center?"

Expect to see many more measured results from Sun in 2009... its been years will any other system vendor step up and show the same or will they just do slick marketing and give you dozens of reasons why they can't give you a number you can simply measure? ...or ask you just to measure idle?

T5440 World Record SPECjAppServer Single Application Server

Monday Nov 24, 2008

One Sun SPARC Enterprise T5440 server with four UltraSPARC T2 Plus processors at 1.4GHz, delivered a single system World Record result of 6334.86 SPECjAppServer2004 JOPS@Standard. The Sun SPARC Enterprise T5440 consumed an average of 1578 Watts of power to obtain this result for a power-performance rating of 0.25 Watts/JOP.

The Sun SPARC Enterprise T5440 server (four 1.4 GHz UltraSPARC T2 Plus chips) demonstrated 32% better performance over the HP DL580 G5 result of 4410.07 SPECjAppServer2004 JOPS@Standard, which used four 2.66 GHz Intel 6-core Xeon processors.

The Sun SPARC Enterprise T5440 server (four 1.4 GHz UltraSPARC T2 Plus chips) demonstrated 75% better performance over the HP DL580 G5 result of 3339.94 SPECjAppServer2004 JOPS@Standard, which used four 2.93 GHz Intel 4-core Xeon processors.

This benchmark used the Oracle WebLogic 10.3 Application Server and Oracle Database 11g Enterprise Edition. This benchmark result proves that the Sun SPARC Enterprise T5440 server using the UltraSPARC T2 Plus processor performs as an outstanding J2EE application server as well as an Oracle 11g OLTP database server.

This result used a Sun SPARC Enterprise T5440 server (four 1.4 GHz UltraSPARC T2 Plus chips) in the database tier to obtain this World Record result. This Sun SPARC Enterprise T5440 consumed an average of 1463 Watts of power to obtain this result for a power-performance rating of 0.23 Watts/JOP.

These results were obtained using Sun Java SE 6 Update 6 Performance Release on the Sun SPARC Enterprise T5440 server and running the Solaris 10 10/08 Operating Environment.

The power-performance metric is a measure of server efficiency ratio that includes system power & performance consumption on a specific benchmark. (Power-performance = Watts / Performance).

Sun publishes watts on this SPEC benchmark, why won't HP on the 6core systems with this size memory?

Performance Landscape

SPECjAppServer2004 Performance Chart as of 10/13/2008. Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org. SPECjAppServer2004 JOPS@Standard (bigger is better)

Submitter	SPECjApp Server2004 JOPS@Standard	J2EE Server	DB Server
Sun	6334.86	1x Sun SPARC Enterprise T5440 32 cores, 4 chips @ 1.4 GHz US-T2 Plus Oracle WebLogic 10.3	1x Sun SPARC Enterprise T5440 32 cores, 4 chips @ 1.4 GHz US-T2 Plus Oracle 11g DB 11.1.0.7
HP	4410.07	1x ProLiant DL580 G5 24 cores, 4 chips @ 2.66 GHz Xeon X7460 Oracle WebLogic 10.3	1x ProLiant DL580 G5 24 cores, 4 chips @ 2.66 GHz Xeon X7460 Oracle 11g DB 11.1.0.6
HP	3339.94	1x ProLiant DL580 G5 16 cores, 4 chips @ 2.93 GHz Xeon X7350 Oracle WebLogic 10.3	1x ProLiant DL580 G5 16 cores, 4 chips @ 2.93 GHz Xeon X7350 Oracle 11g DB 11.1.0.6

Benchmark Description

SPECjAppServer2004 (Java Application Server) is a multi-tier benchmark for measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based application servers. SPECjAppServer2004 is an end-to-end application which exercises all major J2EE technologies implemented by compliant application servers as follows:

• The web container, including servlets and JSPs
• The EJB container
• EJB2.0 Container Managed Persistence
• JMS and Message Driven Beans
• Transaction management
• Database connectivity

Moreover, SPECjAppServer2004 also heavily exercises all parts of the underlying infrastructure that make up the application environment, including hardware, JVM software, database software, JDBC drivers, and the system network.

The primary metric of the SPECjAppServer2004 benchmark is jAppServer Operations Per Second (JOPS) which is calculated by adding the metrics of the Dealership Management Application in the Dealer Domain and the Manufacturing Application in the Manufacturing Domain. There is NO price/performance metric in this benchmark.

Results Summary

Results			6334.86 SPECjAppServer2004 JOPS@Standard
Reference Date:			Nov 20, 2008
Systems:			2x Sun SPARC Enterprise T5440
Total Number Processors:			4, 4
Processor/GHz of Server:			UltraSPARC T2 Plus 1.4 GHz
Operating System:			Solaris 10 10/08
Software:			Oracle WebLogic 10.3 Application Server, Standard Edition
			Oracle Database Enterprise Edition Release 11.1.0.7
JVM:			JDK 1.6.0_06 Performance Release

Disclosure Statement:

SPECjAppServer2004
Sun SPARC Enterprise T5440 (4 chips, 32 cores) 6334.86 SPECjAppServer2004 JOPS@Standard. HP DL580 G5 (4 chips, 24 cores) 4410.07 SPECjAppServer2004 JOPS@Standard. HP DL580 G5 (4 chips, 16 cores) 3339.94 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 10/13/08

SPECjAppServer2004 Sun SPARC Enterprise T5240 (16 cores, 2 chip) 3331.31 SPECjAppServer2004 JOPS@Standard. IBM p570(4 cores, 2 chips) 1197.51 SPECjAppServer2004 JOPS@Standard. IBM p550(4 cores, 2 chips) 1197.51 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 04/09/2008. Power References: IBM p6 570 power specifications from 80% of maximum report power consumption published here, 06/07/07, posted at ftp://ftp.software.ibm.com/common/ssi/rep_sp/n/PSB01628USEN/PSB01628USEN.PDF IBM p5 power specifications calculated by applying 70% of the power numbers published in ?Facts and Features Report?, 3/10/06, posted at http://www-03.ibm.com/servers/eserver/pseries/hardware/factsfeatures.html

Video games very inefficient idle

Thursday Nov 20, 2008

Here is a clear example why vendors need to show measured watts on their products - hidden waste.

http://green.yahoo.com/blog/the_conscious_consumer/21/stop-wasting-money-video-games-and-energy-efficiency.html

Note that consumer products spend most of their day "idling" so idle watts is CRITICAL for everyone to know.  Servers should be nowhere near idle if you really want to save energy.

Servers are most wasteful if they are NOT being used, so they should be turned off if they are idle or consolidated if they are near idle (less than 20%).  It is an egregious bit of vendor "smoke and mirrors" not to show watts with measured benchmark performance on a wide range of benchmarks.  See the posting earlier today for measured watts on a Sun benchmark.  I've posted before that all servers are 2x to 8x more power efficient at 50% utilisation rather than 10% utilisation.

SPEComp2001 Sun Blade X6440 x86 world Record and measured watts

Thursday Nov 20, 2008

The Sun Blade X6440 server module (2.7 GHz Opteron 8384 'Shanghai') running OpenSolaris 2008.05 and using Sun Studio Express 11/08 compilers delivers a World Record SPECompM2001 x86 16-threads result of 35896.

Sun Blade X6440 server module (2.7 GHz Opteron 8384 'Shanghai') was 19% faster on SPECompM2001 while consuming 9% less power (measured in watts), over the same Sun Blade with AMD Opteron 8356 processors.

very editorial note: Sun continues to measure watts on actual performance benchmark runs on real-sized memory configurations, this number of benchmark expands beyond Sun's CMT platforms this list continues to expand.  The SPEComp  results was run with 32GB on the Sun Blade. If your vendor does not measure watts, then you really need to ask yourself why they avoid publicly showing everyone easy to measure data.

The Sun system beat the Supermicro H8QM8 result by 4%, even though the X6440 uses DDR667 memory and the Supermicro uses DDR800 memory. This was made possible by using OpenSolaris 2008.5 plus the Sun Studio Express 11/08 compiler rather than SuSE Linux 10 SP1 with the PathScale compiler used by Supermicro.

Press Release content on www.sun.com

SPEComp2001 Performance Chart - SPECompM2001 (bigger is better, ordered by peak)

Result		Cores	Chips	Thrds	System
Peak	Base
35896	32843	16	4	16	Sun Blade X6440 (Opteron 8384 2.7GHz)
34415	33086	16	4	16	Supermicro H8QM8 (Opteron 8384 2.7GHz)
30275	29094	16	4	16	AMD Tyan Thunder 425QE (Opteron 8360 2.5GHz)
30228	27568	16	4	16	Sun Blade X6440 (Opteron 8356 2.3GHz)
28283	27001	16	4	16	AMD Tyan Thunder 425QE (Opteron 8356 2.3GHz)

Benchmark Description

The SPEC OMPM2001 Benchmark Suite was released in June 2001 and tests HPC performance using OpenMP for parallelism.

• 11 programs (3 in C and 8 in Fortran) parallelized using OpenMP API

Goals of suite:

• Targeted to mid-range (4-32 processor) parallel systems
• Run rules, tools and reporting similar to SPEC CPU2006
• Programs representative of HPC and Scientific Applications

Power data in watts all results using OpenSolaris 2008.05, watts are measured on a single blade.  As with any vendor multiple blades can be put into a single chassis.

Configuration	System State	Max. Power drawn in watts
Single X6440 Blade 32GB (16x2GB DDR667) 4x Opteron 8384 2.7GHz	During run of SPECompM2001	532 Watts
Single X6440 Blade 32GB (16x2GB DDR667) 4xAMD Opteron 8356 2.3GHz	During run of SPECompM2001	578 Watts
Systemwide Power saving AMD Opteron 8384 vs AMD Opteron 8356	During run of SPECompM2001	46 Watts or 9%
Single X6440 Blade 32GB (16x2GB DDR667) 4x Opteron 8384 2.7GHz	System active idle	195 Watts
Single X6440 Blade 32GB (16x2GB DDR667) 4x Opteron 8356 2.3GHz	System active idle	205 Watts

Disclosure Statement:

SPEC, SPEComp reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of Nov 19, 2008 and this report. Sun Blade X6440 (4 cores, 4 chips, 16 threads, Opteron 8384 2.7GHz), 35,896 SPECompM2001; Supermicro H8QM8 (4 cores, 4 chips, 16 threads, Opteron 8384 2.7GHz), 34415 SPECompM2001; AMD Tyan Thunder 425QE (4 cores, 4 chips, 16 threads, Opteron 8360 2.5GHz), 30275 SPECompM2001; Sun Blade X6440 (4 cores, 4 chips, 16 threads, Opteron 8356 2.3GHz), 30228 SPECompM2001; AMD Tyan Thunder 425QE0 (4 cores, 4 chips, 16 threads, Opteron 8356 2.3GHz), 28283 SPECompM2001.

Result
	X6440 16-threads:		35896 SPECompM2001
Reference Date:			Nov 18, 2008
System:			Sun Blade X6440
Total Number Processors:			4
Total Memory :			32 GB (16x2GB DDR2-667MHz)
Processor/GHz of Server:			AMD Opteron 8384, 2.7 GHz
Operating System:			OpenSolaris 2008.05
Compiler:			Sun Studio Express 11/08

Web 2.0 Consolidation Sun Storage 7410 Unified storage array

Monday Nov 10, 2008

Update: also check out this post: http://blogs.sun.com/mheckel/entry/sun_storage_7410_unified_storage

Web2.0 data centers are filled with racks of x86 servers. Data center architects simply put a single app on a single box, but this can be difficult to manage and inefficient in terms of utilization, power, and space. There is a very easy way however to consolidate many web servers onto a single CMT server using Open Storage.

With the introduction of the Sun Storage 7410 Unified storage array, cost and configuration of storage just got easier. UltraSPARC CMT servers have been proven to be a perfect choice for consolidation of the Web2.0 tiers. The Sun Storage 7410 Unified storage array allows for easy consolidation of storage for the Web 2.0 environment.

Traditional Web 2.0 environments utilize distributed file systems like MogileFS. These solutions use an adhoc approach to maintenance and availability. The Sun Storage 7410 Unified storage array uses inexpensive SAS drives, SSDs, ZFS, and the Fishworks open storage software. As compared to traditional NAS solutions, the Sun Storage 7410 Unified storage array provides high performance solution that is easy to manage, monitor, and deploy.

Proves Open Storage is an excellent choice for Consolidation of Web2.0 based applications.

The test was even run with the benchmark kit "Olio" which is an open-source project for testing Web2.0 environments.

Sun Storage 7000 Unified Storage Systems are quick to install and easy to configure. It is also easy to analyze and optimize your storage when you are in full production.

One can gain up to a 75% cost savings over traditional storage solutions while realizing significant space, power and cooling efficiencies.

The Open Storage Community is driving storage innovation and cost effective storage solutions that foster growth and economic opportunity for your business.

Results Summary

For each UltraSPARC T2 socket, eight older x86 servers can be eliminated. Additionally, by increasing average utilization and expending less watts/user this vastly improves compute density and creating a huge savings in floor space and power.

System	Processors			Results
	Storage	Ch, Cr, Th	GHz Type	users	Util	RU	watts / user	users /RU
Sun Fire T5120	Sun Storage 7410 Unified storage array (NFS)	1, 8, 64	1.4 UltraSPARC T2	2,400	72%	1	0.20	2,400
Sun Fire T5120	LocalFS	1, 8, 64	1.4 UltraSPARC T2	2,400	60%	1	0.20	2,400
Sun Fire v20z	LocalFS	2, 2, 2	2.2 AMD 248	300	40%	1	1.163	300
4 x Sun Fire x4200, Distributed	LocalFS	16, 16,16	2.2 AMD 248	1,900	xx	8	0.73	238

Benchmark Description

The application in the web2.0 kit implements a social events calendar with features such as AJAX, tagging, tag cloud, comments, ratings, feeds, mashups, extensive use of data caching, use of both structured and unstructured data and a high data read:write ratio that is typical of applications in this space. The web2.0 benchmark kit has multiple different flavors. For purposes of this evaluation, we decided to use the following components all running on UltraSPARC CMT servers:

• Solaris
• Apache
• memcached
• MySQL
• PHP

See Also

• Sun Unified Storage Website
• Sun Storage 7410 Unified storage array
• Faban
• Olio open-source project

Consolidation process

For easy mapping from the old environment to the new, we created one zone for each "core" on the UltraSPARC T2. For the 2,400 user run, we had 300 users per zone simulating the consolidation of 8 x v20z servers. We chose 8 zones so that each zone could be mapped to a core. For UltraSPARC T2+ based servers, more zones could easily be added to take advantage of the throughput of this server.

This flexible environment can be scaled up or down based on the needs of the applications running in the zone. For this test, one full "LAMP" stack was created on each of the 8 local Solaris zones.

mpg and perf/watt are misleading

Monday Jun 23, 2008

Last friday I blogged about an article on Duke University's Larrick & Soll's research:

Posting a vehicle’s fuel efficiency in “gallons per mile” (GPM) rather than “miles per gallon” (MPG) would help consumers make better decisions about car purchases and environmental impact, researchers from Duke University’s Fuqua School of Business report in the June 20 issue of Science magazine.

The main issue is that people usually make comparisons by linear improvement in miles/gallon, but this leads most to errors. Switching to gallons/mile (and as I said for servers watt/performance) avoids these problems.

If one does the calculations correctly of course it doesn't matter, but on a quick look one can be mislead. For example, (do this quickly!) if one climbs 10 miles a hill and gets 10mpg and then coasts down the hill for 10 miles getting 100 mpg, how many mpg does one average? If you didn't come up with an answer of 18mpg (or nearly double the uphill rate), then you should consider looking at the reciprocal calculation.

If on that same hill that same car gets 1 gal/10 miles (10mpg) uphill and 0.1gal/10miles (100mpg), then it is easy to that coasting downhill can only come close doubling your fuel efficiency. Even if you doubled your fuel efficiency on the downhill section to 200mpg (0.05gal/10 miiles) you can see that your average fuel efficiency doesn't change much.

As I've said before on servers it is also critical to understand watt/performance on a wide variety of benchmarks, Sun understands this. This way you avoid benchmarks were vendors only highlight small-memory and low-GHz configurations.

Finally increase your server utilisation (even a small amount) and closely look at power-performance (watt/perf).

Eco on the desktop: Thin(Sunray) vs. Thick(big PC)

Wednesday Jun 11, 2008

Thick versus Thin Clients: Today online at 4PM(east)/1PM(pac) there will be a debate to discuss the energy use and TCO of a thin client model versus their thicker alternatives. See: http://blogs.intel.com/technology/2008/06/ecotechnology_great_debates_at.php

I'm sure the "think thin" will have followup thoughts on this afterwards at: blogs.sun.com/ThinkThin

Desktop systems have a very different usage model (for most people's work environment they are mostly IDLE or very low utilisation), so thin usually wins. Servers are a different beast and you really want to run fewer servers at high utilisation)