BM Seer Facts & Questions from an Anonymous Sun Source

Intel defaults and judging performance

Tuesday Sep 18, 2007

Intel non-default BIOS change results by 25%? Sure turning off prefetch is a technique but if you don't know if a priori if you should, then should you use it to judge performance?

Always interesting when you have more information. I guess our friends at AMD wanted everyone to see what our friends at Intel were doing so they submitted two SPEC results for them.

Case in point on Clovertown there are two AMD results on the same hardware that gives 25% difference.

Point: Normal mode = prefetch on
Gives 163,080 SPECjbb2005 bops
www.spec.org/jbb2005/results/res2007q2/jbb2005-20070326-00276.txt

Counter-point: Disable HW prefetcher in BIOS for benchmark imprv
Gives 203,754 SPECjbb2005 bops
www.spec.org/jbb2005/results/res2007q2/jbb2005-20070326-00275.txt

...both on the same hardware:
same: 2-socket SuperMicro X7DBE (Intel 2.66GHz Xeon quad-core X5355), 16 GB

Disclosure statement

SPECjbb2005 SuperMicro X7DBE (2 chips, 8 cores, 2.66 GHz) SPECjbb2005 bops=163080, SPECjbb2005 bops/JVM=81540 submitted by AMD; SuperMicro X7DBE (2 chips, 8 cores, 2.66 GHz) SPECjbb2005 bops=203754, SPECjbb2005 bops/JVM=101877 submitted by AMD; SPEC, SPECjbb are registered trademarks of Standard Performance Evaluation Corporation. Results 3/7/07 on www.spec.org.

[5] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 07:50AM Sep 18, 2007 by BM Seer in Benchmark Issues
Tags:
Comments:

If there were already Intel-based results for a SPEC benchmark where prefetching was disabled, and that wasn't the default setting, that would (should) have already been part of the disclosure. Not that it would have told people by how much it changed things, but it would have suggested that were one's workload similar to the benchmark, that disabling prefetching might be a worthwhile thing to try.

Posted by rick jones on September 19, 2007 at 05:28 PM PDT #

Agreed, but with such a difference, how does one know if one's workload is similar to the benchmark? Or is their some official statement that for Java applications prefetch should always be turned off?

Do we really want users administering BIOS changes? Shouldn't the chip
or the OS reconfigure itself to minimize complexity?

Posted by BM Seer on September 20, 2007 at 09:34 AM PDT #

The degree of performance difference doesn't come into play when deciding whether one's own workload is similar to any given benchmark. It is a matter of familiarity with one's own workload and that of the benchmark.

Posted by rick jones on September 21, 2007 at 09:13 AM PDT #

OK so how would a user know what characteristic of their workload
makes them want to change the default BIOS setting? If Intel recommends pre-fetch "on" then they should benchmark it that way, if they recommend turning off pre-fetch then they should say what circumstances (beyond just wanting the best SPECjbb2005).

Posted by BM Seer on September 21, 2007 at 01:22 PM PDT #

Why the characteristics of their workload that make it similar to the benchmark of course :)

Posted by rick jones on September 21, 2007 at 03:35 PM PDT #

Post a Comment:
Comments are closed for this entry.