Power-performance benchmarks show Linear estimates very accurate at different utilisation levels

Friday May 15, 2009

Servers have near-linear power response between Idle and 100% utilisation, that is just knowing idle and 100% utilisation you can come up with very good estimates of watts used at any utilisation level for a given workload.

I might even venture to guess that run to run variation may be greater than the error of this linear estimation?

For more details on Sun's three new SPECpower_ssj2008 results see yesterday's posting where I blogged about SPECpower and used Sun's new results to show directly some of the things vendors do to really boost scores on SPECpower_ssj2008. That blog entry can be found here:
http://blogs.sun.com/bmseer/entry/specpower_ssj2008_sun_netra_x4250

Let's look at the results on typical configuration Sun Netra X4250 (32GB default BIOS). Many companies use 2GB/core on Xeon processors and a good heuristic on memory sizings and that is why I call this a typical configuration for this blog posting.

Relative ops/sec (%util)	Actual watts	watts (linear estimate)	Error
100%	296w	296w est	0.0%
90%	291w	289w est	0.7%
80%	286w	282w est	1.5%
70%	280w	275w est	1.9%
60%	273w	268w est	2.0%
50%	267w	261w est	2.5%
40%	259w	253w est	2.2%
30%	251w	246w est	1.9%
20%	243w	239w est	1.6%
10%	235w	232w est	1.3%
0% (Idle)	225w	225w est	0.0%

OK so the error is less than 2.5%, that is very very close. No need to set up complicated tests if you want to measure you own watts and guess your utilisation-watts curve.

Benchmark Description

SPECpower_ssj2008 is the first SPEC benchmark intended to measure the power efficiency of a server. It is based on SPECjbb2005 but the workload has been modified so that the performance portion of the results are not comparable to SPECjbb2005 results. In addition, the workload is varied from unconstrained (ie. maximum) throughput performance to idle (but active) state in 10% decrements, during which the average power consumption is measured. The power and performance is measured, and the ratio of performance to power is computed, for each load point. The overall metric, denoted overall ssj_ops/watt, is the ratio of the sum of performance at each point to the sum of average power at each point, to include the idle point.

Some of the competitive results use non-redundant fans and non-redundant power supplies and minimize other aspects of the configuration.

The Sun Netra X4250 includes a realistic:

• 2x redundant power supplies
• redundant fan modules
• standard I/O expansion mezzanine
• standard Telco dry contact alarm
which are only present in the Netra X4250 among SPECpower_ssj2008 configurations.

To see the effect of changing configurations and using non-default {HACKED } BIOS see:
http://blogs.sun.com/bmseer/entry/specpower_ssj2008_sun_netra_x4250

Disclosure Statement:

Sun Netra X4250 server 600 overall ssj_ops/watt and (226 watts, 244832 ssj_ops) @ 100% target load, (210 watts, 121828 ssj_ops) @ 50% target load, (181 watts, 24150 ssj_ops) @ 10% target load, (174 watts) @ active idle target load. Sun Netra X4250 server 478 overall ssj_ops/watt and (294 watts, 251555 ssj_ops) @ 100% target load, (226 watts) @ active idle target load. Sun Netra X4250 server 437 overall ssj_ops/watt and (0 ssj_ops, 225 watts) @ 0% target active idle target load, (22866 ssj_ops, 235 watts) @ 10% target load, (45752 ssj_ops, 243 watts) @ 20% target load, (68959 ssj_ops, 251 watts) @ 30% target load, (92768 ssj_ops, 259 watts) @ 40% target load, (115284 ssj_ops, 267 watts) @ 50% target load, (138548 ssj_ops, 273 watts) @ 60% target load, (162384 ssj_ops, 280 watts) @ 70% target load, (184875 ssj_ops, 286 watts) @ 80% target load, (208601 ssj_ops, 291 watts) @ 90% target load, (229828 ssj_ops, 296 watts) @ 100% target load, SPEC and the benchmark names SPECpower_ssj, SPECpower are trademarks of the Standard Performance Evaluation Corporation. Benchmark results stated above reflect results published on http://www.spec.org as of March 30, 2009. For the latest SPECpower_ssj2008 benchmark results, visit http://www.spec.org/power_ssj2008. See Also: SPECpower_ssj2008 Benchmark Reports

System Configuration

Sun's three results all used the same software components and processors.

Processor:			2 x Intel L5408 QC 2.13 GHz
Operating System:			Windows Server 2003 R2 Enterprise x64 Edition SP2
JVM:			Java HotSpot(TM) 32-Bit Server VM on Windows, version 1.6.0_14

The following result was produced using a more typical larger configuration including fully configured disk drives and an option NIC card. Standard BIOS tuning was used to demonstrate the advantage obtained by special BIOS tuning which benefits this benchmark.

Reference Date:			May 6, 2009
Results			437 overall ssj_ops/watt
System:			Sun Netra X4250 (32GB, 8 x 4096MB as PC2-5300F)
BIOS:			default (normal prefetch)

• 4 x Sun 146GB 10K RPM SAS drive
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)
• 2 x 658watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

Green Grid tool assesses free cooling options

Friday Apr 10, 2009

effective ones is to simply run servers at high-utilization which can save energy both to power the servers (2x to 5x more work done per unit of work watt!) and therefore you need less energy to cool your datacenters.

Another way to save on the cooling costs side of the equation is use free cooling.

Free cooling comes in the form of air-side and water-side economizers. With air-side economizers, often called fresh-air cooling, outside air is taken into a data center, usually filtered, and then used to cool the IT equipment. With water-side economizers, the outside air cools water in the outside chiller or water tower, which in turn cools a data center.

The Green Grid's online calculator includes a series of inputs -- location, temperature and humidity thresholds, IT load, and the cost of electricity are a few -- and at the other end spits out how many hours that data center can use air-side and water-side economizers, and how much money it could save.

Both of these quotes were from the SearchDataCenter.com Article: "Green Grid tool assesses free cooling potential", By Mark Fontecchio, News Writer 09 Apr 2009 SearchDataCenter.com

For more info also see: www.thegreengrid.org

Dell now understands importance of server utilization

Wednesday Feb 18, 2009

Even Dell is starting to get it. Server utilisation levels in a data centre are critical to advance overall performance, improving productivity, and reducing costs.

In: http://www.dell.com/downloads/global/power/ps1q09-20090176-esser.pdf

They say:

Operational policies designed to increase server utilization and advance overall performance and efficiency can lead to dramatic improvements in data center productivity without increasing power consumption.

Yes indeed, I've been blogging that since 2005, remember this little gem from 2006: http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low

But I ran into several customers last week where a major computer vendor (not Dell or Sun) was telling everyone that 10% server utilisation was the most important utilisation to measure - SHAME ON YOU!

Can a group at SPEC or TPC just get on with adding power measurement to all benchmarks as they exist now at benchmark utilisation levels? ...Or just what is going on behind closed doors?

DISCLOSURE: I'm not on any SPEC or TPC committees, nor do I read any confidential updates from Sun employees who do.

IBM Power6 - carefully read the fine print

Friday Jun 01, 2007

A new low. IBM has to do a huge amount of questionable math to try to make the POWER6 look good. You now have to go to IBM's website to see their press release, as it seems it is very different than the one on launch date. http://www-03.ibm.com/press/us/en/pressrelease/21580.wss

But the bottom line is IBM uses totally bogus reasoning to try to make POWER6 look good in comparison to Sun. IBM says Sun has lower utilisation so you need to derate Sun systems by a bogus factor of THREE! Bull!

I've already posted last year http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low which stated "a significant number of Sun's large servers run at over 80% utilisation", and then also quoted Computerworld which stated, "Dennis Callahan, CIO at The Guardian Life Insurance Company of America in New York, server utilization has shot up to nearly 50% in the past 18 months, with a goal in coming years of nearly 70%.

OK, on to the footnote with so much bull it stinks up the whole IBM press release. It really makes one understand that IBM really is on shaky ground with the POWER6, expensive, not that fast, and very slowly being released as we've covered in many posts since the IBM POWER6 launch. By the way I still have NOT seen system configuration, GHz, memory size and all details for the POWER6 "published wattage."

(3) This calculation is based on the trend toward consolidation of existing installed systems. Performance comparisons of the system were based on available benchmarks using the Java-based SPECjbb2005 benchmark (results as of 5/22/07: System p 570 (16-core, 8 chips, 2 chips per core, 4.7 GHz) SPECjbb2005 691,975 bops, 86497 bops/JVM; Sun Fire v890 (16-core, 8 chips, 2 chips per core) 1.5 GHz, SPECjbb2005 117,986 bops, 29,497 bops/JVM). A conservative 5 to 1 performance comparison was used. System utilization levels were derived from studies conducted by IBM of currently installed base of UNIX systems (available at www.ibm.com/servers/library/pdf/scorpion.pdf) and the recent trend on System p for utilizations levels well over 60% using advance virtualization technologies. 20% was used for the currently installed base of Sun Fire v890 systems and 60% for a virtualized System p 570. Higher utilization levels provide a 3 to 1 consolidation factor. Power consumption figures of 5600 W for the IBM System p 570 and 3200 W for the Sun Fire v890 were based on the maximum rates published by IBM and Sun Microsystems, respectively. Air conditioning power requirement estimated at 50% of system power requirement. Energy cost of $.092 per kWh is based on 2007 YTD US Average Retail price to commercial customers per US DOE at http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_b.html as of 5/18/2007. For space calculations, two IBM System p 570 servers will fit in a single, standard rack. Assumed rates were 60% for IBM System p 570 and 20% for Sun Fire v890.

Watts a matter with their marketing?

Thursday May 17, 2007

...sorry for the bad pun, but wattage is a huge concern at most datacenters.

• Vendors need to state watts/performance at full performance on benchmark and not watts/performance at low utilisation. Datacenter managers must plan and provision a datacenter knowing the peak watts. There is no way they could count on poor (low) utilisation of servers to keep their datacenters from frying.

• Need to use metrics of Watt/performance so it matches $/performance. I've blogged about this before: http://blogs.sun.com/bmseer/entry/power_performance_metrics

• Stop selling the advantage of brand new latest systems and low utilisation. It matters not that there are lots of sloppy low utilisation datacenters, because anyone buying new stuff if they are not running at 50% or more are just wasteful and not worth taking any advice from. I've blogged about this before: http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low

These are some of thoughts after talking to a dozen customers this week.

Driving to higher utilization plus low wattage systems like T2000

Wednesday Nov 29, 2006

Don't be swayed too much by low-power saving technology hype. Don't look at "wattage vs. utilisation", you really need to look at "perf/watt vs. utilisation". Perf/watt must be completely constant over the utilisation curve if you think you can run a datacenter at 15% utilisation and save money.

Check out my last posting and look at the last graph to understand the issues.

Total Tyranny of low utilization datacenters

Friday Nov 17, 2006

The Total Tyranny of low utilization datacenters

In this blog and other blogs I've commented on, Woodcrest supporters always want to say their servers are better at low utilisation. This is totally the wrong way to go! They first claim typical datacenters are running at low utilisations, example: Xen claims typical datacenters are at 15%. Horrible, HORRIBLE.

So why shouldn't use just add all kinds of techniques to power at lower utilisations, clearly that is the best way to save money? Right? Wrong.

Lets take a simple example of a 400 watt server(@ 100%) that saves 20 watts for each 10% reduction in utilisation. Will show this in a table below and compare equivalent work done compared to 100% so you can see the hyperbolic nature of the curve. Of course I'm only looking at one server so there is some discretisation but when you have a datacenter it will quickly approach these numbers.

%Utilisation	100%	90%	80%	70%	60%	50%	40%	30%	20%	10%	0%
Watts-at-Util	400	380	360	340	320	300	280	260	240	220	220
watts/work	400	422	450	486	533	600	700	867	1200	2200	inf.

Now that I've got you shocked, let's look at a more typical example. Lets compare 5 servers running at 10% utilisation (that is 220 watts each or 1100 watts for the 5 of them). A single server running at 50% utilisation only uses 300 watts! The 10% case almost require 3.7 times more power! OUCH!

Bottom line: It is far too easy to be fooled to think you are saving money if power-saving features at low utilisation is your answer. By the by, a significant number of Sun's large servers run at over 80% utilisation using Solaris, of course.

Here is an example from 2004 of someone on different products who likely understands this math. As reported in Computerworld:

"Dennis Callahan, CIO at The Guardian Life Insurance Company of America in New York, server utilization has shot up to nearly 50% in the past 18 months, with a goal in coming years of nearly 70%.