"listed as "4" Not even 4GB" - I wear bifocals and even I can easily see the label just to the left of those "4s" (or other values) and see the field label which says "Memory Amount (GB):" Rather like how it does CPU frequency with "CPU Frequency (MHz):" and a "bare" number. Doesn't seem obscure at all.

"I see several server submission on SPECpower_ssj with ONLY 4GB of total server memory -- that is 0.5GB/core. TINY!!!" You may want to look again, more closely, and check your math. I went to:

http://www.spec.org/cgi-bin/osgresults?conf=power_ssj2008&op=form and asked to have it display core count and memory amount, and looking at the results I did not find a single one at 0.5GB/core. They were all 1.0GB/core or 2.0GB/core. Just going back and double checking, I see that I actually missed one that was 4GB/core.

"ok the form shows this as 1x or 2x, but all of them seem to be 1x, meaning non-redundant." Again using SPEC's supplied search form, it is possible to see that while indeed _almost_ all are single power supply, there are 6 results with 2 power supplies and 2 results with 3 power supplies. Of course two or even three power supplies does not a priori mean they were redundant.

WRT calling-out redundant power supplies or fans, I've tried to go back through your blog posts using the "power" and "watts" links, and thus far haven't found that called-out. Sometimes it seems that details on memory config were sparse as well - sometimes they were there, sometimes they were not. I did find one instance of an ambient temperature being mentioned, (another important item that SPECpower does _not_ seem to leave-out) but that was only where you were chastising (correctly IMPO) some unnamed competitor for quoting numbers from an ambient temp of 61F.

So, while you _may_ have found a couple things that SPECpower_ssj2008 might improve in its disclosure reports in the future - perhaps calling-out if the power supply configuration is redundant, maybe something about fans (although if a system has neither, I'm sure marketing departments will be able to FUD those on their own without having to have a disclosure report telling them), one very significant virtue SPECpower_ssj2008 seems to have is that everything SPECpower_ssj2008 has disclosed is always there in every disclosure. That definitely seems _not_ to be the case with ad-hoc vendor disclosures.

Posted by rick jones on February 25, 2009 at 09:15 PM PST #

"But how come other benchmarks are 4GB/core with some more and some less?" It seems reasonable to guess that the benchmarks using lots of RAM are doing so either to get the last bits of performance, or because they were leveraging the same hardware across multiple benchmarks and the extra RAM didn't hurt on a given benchmark.

Once more benchmarks include power as part of their metrics, and I'm sure that SPEC and others are working on such things - with defined rather than ad-hoc disclosure requirements - you may see a change in configs. So, you should definitely work with Sun's SPEC and other reps to support that. That will have greater long-term benefits than calling for an expansion of the ad-hoc. After all, it was to bring some sense to the ad-hoc that SPEC et al were created right?

For SPECweb2005 you left-out a 72 GB result from FujitsuSiemens and some 64 GB results from Sun, and even some recent 128 GB results from HP, but folks who would like to see them all and more can use: http://www.spec.org/cgi-bin/osgresults?conf=web2005&op=form to include Memory config in the list and sort - seems it does an "alphabetical" sort on Memory rather than a Numeric one though. The disclosure for SPECweb2005 doesn't seem to give thread counts so while one can concoct a mem/core or mem/chip derivative it will take a bit more work to do so for mem/thread.

There are similar search forms for the rest of the SPEC benchmarks.

As for what should or should not be in the first line, from my perusal, the header of a SPECpower_ssj2008 disclosure doesn't look remarkably different from any other SPEC benchmark's. And, since full details are important, not just sound bites, folks should be reading the full disclosure anyway - it really isn't that long for most SPEC benchmarks.

Posted by rick jones on February 27, 2009 at 07:49 AM PST #

The only question I have outstanding to you in this thread was a semi-rhetorical one about the impetus behind the creation of SPEC for which I wasn't necessarily expecting an answer. Still my take on the power draws in a SPECpower_ssj2008 benchmark, at least as they have been published thusfar it would be processors, dimms, what I've seen called alternatively the CEC (Core Electronics Complex) or glue or perhaps "other motherboard components," and either the power supply or the boot disc.

Posted by rick jones on March 03, 2009 at 08:48 AM PST #