BM Seer Unofficial thoughts from an anonymous Sun employee

« VMWare's blog on... | Main | Idle servers are the... »

Changing your %util a tiny bit, saves lots of money & energy!

Friday May 30, 2008

The SPECpower_ssj2008 benchmark goes out of its way to measure servers at low-utilisation (5 of 11 datapoints are in the wasteful active-idle to 40% range - the worst range for servers). SPECpower_ssj actually shows this. Customers needs to demand that power measured on all benchmarks, now.

Let's look at HP DL580 G5 4-socket (for SPECpower_ssj2008 HP only used the low-GHz 1.86GHz Xeon, tiny 16 GB with special DIMMs, and a hacked non-standard BIOS). Regardless it can still be used to prove my point.

Increasing utilisation even a small amount provides HUGE improvements in watts-per-unit-of-work. Increasing utilisation a tiny 10% improve your watt/work an amazing 33-163% ! If you increase utilisation more you save even more:

<

%Util Improvement Savings in watts-per-unit-of-work

10 33-163%
20 45-257%
30 62-353%
40 84-437%
50 114-511%
higher% even more!

%Util Improvement	Savings in watts-per-unit-of-work
10	33-163%
20	45-257%
30	62-353%
40	84-437%
50	114-511%
higher%	even more!

The biggest savings occur when you stop running at low utilisation. We need to do everything we can to discourage low-utilisation!

The first column, in the table below, is the %utilisation you start at, the rows than show you the %savings if you increase your utilisation. So for example if you were at 20% utilisation (2nd row from the bottom) and increase the %utilisation to 40% (3rd column from the right) you save 140% per unit of work!

%u Perf/Power 100% 90% 80% 70% 60% 50% 40% 30% 20%
100% 929
90% 867 33%
80% 793 45% 35%
70% 712 62% 50% 37%
60% 626 84% 71% 56% 41%
50% 538 114% 99% 82% 64% 46%
40% 451 155% 138% 117% 96% 75% 54%
30% 357 223% 200% 174% 148% 121% 94% 64%
20% 243 374% 341% 303% 264% 224% 185% 140% 89%
10% 129 793% 731% 659% 586% 511% 437% 353% 257% 163%

%u	Perf/Power	100%	90%	80%	70%	60%	50%	40%	30%	20%
100%	929
90%	867	33%
80%	793	45%	35%
70%	712	62%	50%	37%
60%	626	84%	71%	56%	41%
50%	538	114%	99%	82%	64%	46%
40%	451	155%	138%	117%	96%	75%	54%
30%	357	223%	200%	174%	148%	121%	94%	64%
20%	243	374%	341%	303%	264%	224%	185%	140%	89%
10%	129	793%	731%	659%	586%	511%	437%	353%	257%	163%

IBM bloggers accuse me of many things. LET ME BE CLEAR: These are my personal opinions and NOT the opinions of Sun. This blog is NOT the source for official opinions.

Disclosure statement

SPECpower_ssj2008:HP Proliant DL580 G5 (4-chip QC Xeon L7345 1.86GHz), 546 overall ssj_ops/watt, 359,523 ssj_ops and 387 watt at 100% target load, 325,931 ssj_ops and 376 watt at 90% target load, 291,991 ssj_ops and 368 watt at 80% target load, 255,512 ssj_ops and 359 watt at 70% target load, 217,222 ssj_ops and 347 watt at 60% target load, 180,262 ssj_ops and 335 watt at 50% target load, 145,079 ssj_ops and 322 watt at 40% target load, 110,173 ssj_ops and 309 watt at 30% target load, 71,409 ssj_ops and 294 watt at 20% target load, 36,070 ssj_ops and 280 watt at 10% target load, and Active Idle 271 watts. SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 12/11/07.

In a more realistic configuration the HP DL580 G5, from HP's own power calculators, a HP DL580 G5 with four QC Xeon 2.93GHz Tigerton and 64 GB memory should draw 1,072watts. HP DL580 power consumption from HP Power Calculator system configured with 4 x2.93GHz processors, redundant PSU, 16 x 4GB DIMMs, 8 x 36GB SAS drives,1 x PCI card, 80% utilisation on 9/10/07: http://h30099.www3.hp.com/configurator/powercalcs.asp

[13] Comments

Like this post?

Posted at 10:22AM May 30, 2008 in Wattage-Power
Tags: performance watt/performance wattage watts

Comments:

So, when can we expect to see Sun post some power benchmarks to the SPEC site? I see dell, hp, intel and ibm.... but no Sun.

Posted by John on May 30, 2008 at 11:20 AM PDT #

When will we see HP DL580 G5 *2.93GHz 4-socket* on SPECpower_ssj2008?
With 16GB or better yet 64GB?

When will we see measured power-performance on HP DL580 G5 on any
of these benchmarks:
SPECweb?
SPECjbb?
SPECjAppServer(Appl server)?
SPECjAppServer(DB server)?
SPEComp?
Lotus iNotes?
VMmark?
SugarCRM?
Drupal?

Sun has publicly posted wattage results on *all of these benchmarks*. Sun has publicly posted watts on over 22 benchmarks!

SPEC, SPECweb, SPECjbb, SPECjAppServer, SPEComp, SPECpower reg tm of Standard Performance Evaluation Corporation. VMware, VMmark reg tm of VMware Inc.

Posted by BM Seer on May 30, 2008 at 02:50 PM PDT #

Also now POWER6 on SPECpower_ssj.

John: you also miss the point that a benchmark that measures watts at low utilisation is totally the WRONG THING TO DO!

Posted by BM Seer on May 30, 2008 at 02:52 PM PDT #

I suspect you meant to say "no POWER6" rather than "now POWER6" also, could you add a link sometime that describes just _how_ (the nuts and bolts as it were) Sun was measuring power on those 22 benchmarks? With SPECpower_ssj2008 we know the methodology because the SPEC tells us. And the FDR's include stuff like ambient temp, voltage, what meter was used etc etc etc in addition to the processor frequencies and RAM counts you keep mentioning.

On the flip side one could interpret the data as sugesting that there is a great opportunity to improve power management in servers. :)

Posted by rick jones on May 30, 2008 at 05:46 PM PDT #

Rick, Sun clearly states the methodology and you've been reading it on sun.com (I point to all of the public info all of the time).

Contact your sales person if you need help in finding the info, which I always point to all of the time. Remember this blog is not the official source, I write about things I think are important.

Power management, in the above example, watts at 20%util with the power management only drops the system watts 24% from the 100%util. But as the example shows 20%-util *even with HP's best power-per-unit-of-work* is 3.7 TIMES more wasteful!

Low utilisation is the worst thing to do if you want to save power. Why buy the latest server and run it at 20%util? As a world we could do more by raising utilisation to 50-100%. We should all develop all of our strategies around high utilisation.

Posted by BM Seer on May 31, 2008 at 09:02 AM PDT #

"Rick, Sun clearly states the methodology and you've been reading it on sun.com (I point to all of the public info all of the time)."

So, I took the UltraSPARC T2+ link in your post "Sun's summary of measured watts & watt/performance, and followed it to:

http://www.sun.com/servers/coolthreads/t5240/benchmarks.jsp

The SAP stuff didn't discuss power, only rack units and such so I skipped to the Lotus iNotes entry. It discussed "SWaP" but all it had to say about how power was measured (ie _method_) was "Sun SPARC Enterprise T5240 power consumption taken from measurements made during the benchmark run." It didn't say anything about whether it was some automated mechanism, what meter was used, what was the voltage, the air inlet temp or anything. For all we know the system could have been in a refrigerator with some poor schmuck in a parka writing down numbers on a clipboard.

So I moved on to the SPECjAppServ2004 entry. All it had to say about methodology was " SPARC Enterprise T5240 power consumption of 720 watts reported during benchmark run."

All the SPECjbb2005 entry had to say about methodology was "Sun SPARC Enterprise T5240 power consumption taken from measurements made during the benchmark run." Again, rather vague.

The SPECint and SPECfp rate entry had nothing to say about power, nor did the OMP2001 entry.

So, where do I go to find a description of just _how_ power was measured on the systems?

Posted by rick jones on June 02, 2008 at 05:38 PM PDT #

Sun measures watts for benchmarks in datacentres only above 20C. Standard accurate meters (with power factor accounted for), standard voltage, no games are used to lower watts. There are internal docs that say all of this, and I've heard one of them was supposed to be externally published, I'll find that external URL and post it when I do. Remember this is a Sun-employee blog, I try my best to point at the right info - don't put your lawyers on me.

Rick this posting was about low-utilisation being completely wasteful. I even used SPEC results to show this so you won't argue with the source. Although that benchmark has weaknesses, for example...

In addition, Sun doesn't hack the BIOS except to "keep up with the Joneses" on Xeon SPECjbb2005 results. (note: hack BIOS = non-standard BIOS to turn off prefetch for a benchmark, this was used in nearly all Xeon SPECpower_ssj results? Can someone confirm?) I wish SPEC would change their rules and only allow shipping BIOS to be used on tests, but I'm "shouting into the wind" I guess.

Note: "keeping up with the Joneses" is an expression, not any reference to Mr. Jones.

http://blogs.sun.com/bmseer/entry/dell_games_with_little_info
http://blogs.sun.com/bmseer/entry/intel_defaults_and_judging_performance

SPEC, SPECjbb, SPECpower reg tm of Standard Performance Evaluation Corporation.

Posted by BM Seer on June 04, 2008 at 08:47 AM PDT #

"IBM bloggers accuse me of many things. LET ME BE CLEAR: These are my personal opinions and NOT the opinions of Sun. This blog is NOT the source for official opinions." seems rather at odds with:

http://blogs.sun.com/bmseer/entry/back_to_blogging

"My director came back from a sun internal VP/director meeting last week and is telling all of us engineers to really get the word out in public on everything. I'll be doing my part. Now with renewed vigor."

Posted by rick jones on June 04, 2008 at 01:21 PM PDT #

not at odds at all. Sun encourages Sun's employees to blog. ...but having said that: *ALL postings on this BM Seer blog solely reflect my personal views and do _not_ necessarily represent the views, positions, strategies or opinions of Sun or Sun management."

above it said: "get the word out in public on everything." I take this as my opinions, questions, and I also like posting benchmark results.

If you want official opinions see http://sun.com for that.

Posted by BM Seer on June 04, 2008 at 01:48 PM PDT #

Why then is Sun Systemnews quoting "unofficial" blogs? Is there that great a dearth of "official" word from Sun?

Posted by rick jones on June 05, 2008 at 07:39 AM PDT #

I'm clearly stating who I work for, what I do, and *MY* opinions (no one tells me what to write here). ....But remember this post was about low-utilisation being extremely wasteful, even looking at SPECpower_ssj results, this seems very clear.

Sun bloggers are sharing all kinds of things:
http://blogs.sun.com/richb/date/20080604

Rick Jones, by the by, Who do you work for?

SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation.

Posted by BM Seer on June 05, 2008 at 10:16 AM PDT #

Most of the time it feels as though I work for my wife, two kids and mortgage lender :) All kidding aside, interesting situation isn't it? Those reading our back and forth know for whom you work, but not who you are. They know who I am, but not necessarily for whom I work. Well, those except our friends, co-workers and closest adversaries I suppose :)

I've put my name rather than a pseudonym on virtually everything I have posted to/on the Internet since ca 1988. Much of that has been posted from my place of work. So, while I suspect you already know for whom I work, even if you don't you and/or the readers can do so without too much trouble if you/they think it makes any difference. If you like, think of it as a game of "Out the BM Seer critic" :)

Whom, by the way, if it wasn't already clear, is speaking for himself alone, sans any exhortations from his management chain.

Posted by rick jones on June 06, 2008 at 08:05 AM PDT #

Rick, there is a bit of a difference here. Who writes your paycheck may inform us about how your opinions are formed, or if you have an agenda.
Rick, I don't know who you work for and I haven't tried to search.

I have always been very clear about who I work for, let me say that again *SUN* (see the title of this blog as well). I speak for myself, as I've said many times in this comment chain. (and even edited it into the posting you mentioned. Also, that I don't belong to any of the TPC, SPEC, etc. organizations that I sometimes criticize. My name doesn't change any of that. Again I work for Sun, but these are my opinions.

So feel free to attack my opinions, ...arggh, now I'm late for lunch.

Posted by BM Seer on June 06, 2008 at 09:06 AM PDT #