BM Seer Unofficial thoughts from an anonymous Sun employee

« back to blogging | Main | Lack of proper infor... »

HP watts up with no specification?

Friday Mar 21, 2008

HP DL 580 G5 = 387 watts at 100% target load

Reading "HP ProLiant DL580 G5 server posts highest 4P result on the new SPECpower_ssj2008(TM) benchmark" brochure leaves one quite confused. HP does not specify processor GHz or Memory size, why?. Take note SPEC members: you guys need to force that be clearly specified in the future, or you will just encourage confusion. An HP DL580 G5 4P QC 1.86GHz Xeon 16GB uses 387 watts at 100% target load!

...or... HP DL 580 G5 = 942 watts at average (uncomparable test)

IBM points to a Principled technologies paper (by the way, who commissioned that paper?). Looking at that paper they clearly specify the memory size, processor GHz, and measured average wattage. An HP DL580 G5 4P QC 2.93GHz Xeon 64GB uses 942 average watts!

I do not present these wattage numbers for comparison, as they are different tests. But what are the real watts of a DL580 G5? Clearly HP isn't telling us what we need to know.

I'd suggest SPEC require better disclosure of information and clearly show effects of processor GHz and memory size. MEMORY SIZE makes a HUGE difference in watts. Wake up world! Again my plea to add power measurements and power-performance metrics to all performance benchmarks at full utilization.

SPEC Disclosure statement

SPECpower_ssj2008:HP Proliant DL580 G5 (4-chip QC Xeon L7345 1.86GHz, 16GB), 546 overall ssj_ops/watt, 359523 ssj_ops and 387 watt at 100% target load, 255512 ssj_ops and 359 watt at 70% target load, and 71409 ssj_ops and 294 watt at 20% target load. SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 12/11/07.

note: if anyone has corrections to the above disclosure, let me know in comment below as quickly as you can and I will correct it immediately. I don't attend SPEC meetings so I don't know all of the rules, but I try my best to write disclosures correctly.

[10] Comments

Like this post?

Posted at 03:16PM Mar 21, 2008 in Competitor Follies
Tags: dl580 hp performance specpower ssj

Comments:

You left-off a leading 'h' in the URL for the paper about which you were asking (rhetorically I presume) the sponsorship.

As for the rest, presumably that is what the FDR's on www.spec.org are all about.

Posted by rick jones on March 24, 2008 at 09:05 AM PDT #

Thanks, I fixed the bad URL specification.

Sorry Rick, I still think it is misleading to not clearly reference important data. 1.86GHz & 16GB... why did they not mention that in the brochure?

only 4 links away from spec.org, yep easy :)

Posted by BM Seer on March 24, 2008 at 03:05 PM PDT #

Only four?-) Misleading or no, it isn't yet clear to me that it is something for SPEC to address - it is at best a slippery slope starting to add what must be disclosed, and may or may not be sufficient.

For example, is it that 16 vs 64 GB of memory was used, or is it the number of DIMMs that is more important? Is it as important if DDRmumble DIMMs are used vs FDB? Is it that the CPU was N GHz versus M GHz or might the fab process (45 mn, 65, 32 etc) as important as the frequency? Where is the "cut line" for importance? 50% of SUT power consumption? 15% 5%?

Ultimately it comes-down to being able to find the FDR.

And, if someone/entity feels the story with the current crop of SPECpower_ssj2008 results is incomplete, unless Windows or BEA JRockit have "thou shalt not disclose benchmarks without our sayso" clauses like clause 5F for Solaris 10 or Studio 12 it would seem quite possible for that someone/entity to go ahead and submit results for variations on the configs submitted. At least I see nothing in the SPECpower_ssj2008 run and reporting rules to preclude it.

Posted by rick jones on March 24, 2008 at 07:23 PM PDT #

Rick, see today's (3/25/08) posting. Why don't you want the critical information up front? Why do you want vendors to confuse customers?

Posted by BM Seer on March 25, 2008 at 11:02 AM PDT #

You seem to have inferred an opinion on my part on that HP paper that I did not (mean to imply) imply. I am not taking a position on that paper, only stating that currently I believe what you present as issues with it are not *necessarily* ones that _SPEC_ must address and suggesting that a cut line on what you consider critical information may not be arrived at as facilely as you appear to believe.

Posted by rick jones on March 25, 2008 at 06:21 PM PDT #

From http://www.spec.org/spec/ "(SPEC), was founded in 1988 by a small number of workstation vendors who realized that the marketplace was in desperate need of realistic, standardized performance tests. The key realization was that an ounce of honest data was worth more than a pound of marketing hype."

I believe requiring specification of CPU GHz and memory size is an ounce of honest data to fight the pounds of marketing hype around SPECssj_2008.
I think SPEC should change the SPECssj_2008 disclosure statement From (example):
SPECpower_ssj2008: HP Proliant DL580 G5, 546 overall ssj_ops/watt, 359523 ssj_ops SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 12/11/07.

To:
SPECpower_ssj2008: HP Proliant DL580 G5 (four 1.86GHz Xeon L7345, 16GB), 546 overall ssj_ops/watt, 359523 ssj_ops SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 12/11/07.

only 5 additional words. If the CPU's 1.86GHz doesn't make a big difference from the 2.93GHz Xeon then HP should publish and prove me wrong. If 32GB doesn't change the power draw from 16GB, then again I'd suggest HP publish to prove me wrong, then I'd shut up about this issue.

I've see measured watts on other workloads, so I don't think I'm wrong.

5 words prevents the problem I talked about here:
http://blogs.sun.com/bmseer/entry/lack_of_proper_information_hurts

I'm not the only one who thinks that marketing is taking advantage of SPEC: IBM's benchmarking blogger Stahl says, "And that's exactly what Dell is counting on with their press release on new PowerEdge R300 and T300 servers this week. In the release, Dell makes several SPEC benchmark performance claims. The footnotes for these claims tell you to go to the SPEC website to see any data. Which means you need to actually figure out which systems, configurations, and which data they are referring to and do your own compare. How many of us actually have the time and the inclination to do this ? And that is exactly what Dell is betting on.

If you do the homework, you see that in many cases Dell compares their new results with other vendor's results from over a year ago with older technology. So in this case, it pays not to be lazy."

Posted by BM Seer on March 26, 2008 at 10:49 AM PDT #

It would seem then that Sun has an excellent opportunity to demonstrate how SPECpower results should be presented when they release their first set.

Posted by rick jones on March 27, 2008 at 03:15 PM PDT #

Actually I've already done that, you can see it in every disclosure statement I put with every SPECpower_ssj_2008 result.

I look forward to results on 4-socket HP 2.93GHz and 32GB. Seems HP has no problem publishing at 1.86GHz, so they need to just upgrade the CPUs, use default BIOS, and full regular memory, and say what they used in every brochure.

SPEC Disclosure statement

Posted by BM Seer on March 27, 2008 at 04:00 PM PDT #

No, I meant when Sun decides to finally publish some SPECpower results of their own.

Posted by rick jones on March 27, 2008 at 06:00 PM PDT #

I look forward to seeing HP benchmark four 2.93GHz QC with 64GB of memory with default (normal) BIOS settings. So the industry can see the wattage difference vs 1.86GHz QC with 16GB memory.

Since 2.93GHz QC 4-socket results (with 32GB to 64GB with normal BIOS) exist on performance benchmarks they should also exist on power-performance benchmarks.

Posted by BM Seer on March 28, 2008 at 02:02 PM PDT #