BM Seer Unofficial thoughts from an anonymous Sun employee

World Record SAP-SD 2-Tier: Sun SPARC Enterprise M9000 SPARC64 VII SAP-SD 2-Tier ERP 6.0 (2005)

Monday Jul 14, 2008

The Sun SPARC Enterprise M9000 (64 processors, 256 cores, 512 threads) set a World Record for the SAP-SD 2-Tier Standard Application benchmark. World Record SAP-SD 2-Tier: Sun SPARC Enterprise M9000 SPARC64 VII SAP-SD 2-Tier ERP 6.0 (2005) outperforms largest IBM Power 595 / 5GHz POWER6.

The 64-way Sun SPARC Enterprise M9000 with 2.52 GHz SPARC64 VII processors achieved 39,100 users on the two-tier SAP Sales and Distribution (SD) standard SAP ERP 2005 application benchmark.

The 64-way Sun SPARC Enterprise M9000 beat the 32-way IBM Power 595 (5GHz 64-core Power6) by 10%. This is the largest configuration that IBM makes. IBM has a very different and very complicated core. Users should compare hardware system costs for these two systems. The IBM p595 achieved 35,400 users on SAP-SD 2005 6.0 (177,950 SAPS, 5,561 SAPS/proc, 08-Apr-08).

The 64-way Sun SPARC Enterprise M9000 beat the 64-way HP Integrity Superdome by 30%. The IBM p595 achieved 30,000 users on SAP-SD 2005 6.0 (152,530 SAPS, 2,383 SAPS/proc, 18-Dec-06).

SAP-SD 2-Tier ERP 6.0 (2005) Benchmark Description

The SAP Standard Application SD (Sales and Distribution) Benchmark is a two-tier ERP business test that is indicative of full business workloads of complete order processing and invoice processing, and demonstrates the ability to run both the application and database software on a single system. The SAP Standard Application SD Benchmark represents the critical tasks performed in real-world ERP business environments.

SAP is one of the premier world-wide ERP application providers, and maintains systems on the various SAP products.

SAP-SD 2-Tier Performance Table (in decreasing performance order).

System OS
Database		 Users SAP
ERP/ECC
Release		 SAPS SAPS/
Proc		 Date
Sun SPARC Enterprise M9000
64xSPARC64 VII @2.52GHz
1024 GB		 Solaris 10
Oracle 10g		 39,100 2005
6.0		 196,564 3,071 14-Jul-08
IBM Power 595
32xPOWER6 @5.0GHz
64 cores, 512 GB		 AIX 6.1
DB2 9.5		 35,400 2005
6.0		 177,950 5,561 08-Apr-08
HP Integrity SD64B
64xItanium2 @1.6GHz
128 cores, 512 GB		 HP-UX 11iV3
Oracle 10g		 30,000 2005
6.0		 152,530 2,383 18-Dec-06
Sun SPARC Enterprise M9000
64xSPARC64 VI @2.4GHz
1024 GB		 Solaris 10
Oracle 10g		 25,130 2005
6.0		 129,420 2,022 11-Jul-08
IBM p5 595
64xPOWER5+ @2.3GHz
64 cores, 512 GB		 AIX 5.3
DB2 9		 23,456 2004
5.0		 117,520 1,836 25-Jul-06
Sun SPARC Enterprise M8000
16xSPARC64 VI @2.4GHz
256 GB		 Solaris 10
Oracle 10g		 7,300 2005
6.0		 36,570 2,285 17-Apr-07

Complete benchmark results may be found at the SAP benchmark website http://www.sap.com/benchmark.

Disclosure Statement:

Two-tier SAP Sales and Distribution (SD) standard SAP ERP 2004/2005 application benchmark as of 07/14/08: Sun SPARC Enterprise M9000 (64 processors, 256 cores, 512 threads) 64 x 2.52 GHz SPARC64 VII, 1024GB memory, 39,100 SD benchmark users, 1.93 sec. avg. response time, Cert#2008042, Oracle 10g, Solaris 10, SAP ECC Release 6.0; Sun SPARC Enterprise M9000 (64 processors, 128 cores, 256 threads) 64 x 2.4 GHz SPARC64 VI, 1024GB memory, 25,130 SD benchmark users, 1.65 sec. avg. response time, Cert#2008040, Oracle 10g, Solaris 10, SAP ECC Release 6.0; Sun SPARC Enterprise M8000 (16 processors, 32 cores, 64 threads) 16 x 2.4 GHz SPARC64 VI, 256GB memory, 7,300 SD benchmark users, 1.98 sec. avg. response time, Cert#2007026, Oracle 10g, Solaris 10, SAP ECC Release 6.0; IBM Power 595 (32 processors, 64 cores, 128 threads), 35,400 SD benchmark users, 32 x 5.0 GHz POWER6, 512 GB, DB2 9.5, AIX 6.1, Cert. 2008019, SAP ECC Release 6.0; IBM System p5 595 (64 processors, 64 cores, 128 threads), 23,456 SD benchmark users, 64 x 2.3 GHz POWER5+, 512 GB, DB2 9, AIX 5.3, Cert. 2006045, SAP ECC Release 5.0; HP Integrity SD64B (64 processors, 128 cores, 256 threads), 30,000 SD benchmark users, 64 x 1.6 GHz Dual-Core Intel Itanium 2, 512 GB, Oracle 10g, HP-UX 11iV3, Cert#2006089, SAP ECC Release 6.0; SAP, R/3, mySAP reg TM of SAP AG in Germany and other countries. More info www.sap.com/benchmark.

Sun's submitted results for the SAP-SD 2-Tier benchmark
Certified Results
Performance: 39,100 benchmark users
Server: Sun SPARC Enterprise M9000
Processors: 64 x 2.52 GHz SPARC64 VII
Memory: 1024 GB
Operating system: Solaris 10
Database S/W: Oracle 10g
SAP S/W: SAP ECC 6.0
SAP Certification: #2008042
Storage: 1 x Internal System Disk
8 x Sun StorageTek(tm) 6140 Arrays

[17] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 11:48AM Jul 14, 2008 in Benchmark Issues
Tags:
Comments:

So, am i reading this right.

The sun system has compared with the IBM P6-P595 2 x memory, 2 x CPU's, and 4 x cores, yet yields a 10% increase ?

hmmmm.....

Posted by Alex on July 14, 2008 at 02:05 PM PDT #

SAP won't let vendors do price comparisons. Otherwise you'd see that core count is as useless as comparing performance/transistor.

Posted by BM Seer on July 14, 2008 at 02:48 PM PDT #

and licensing ?

Posted by 90.192.154.235 on July 14, 2008 at 02:56 PM PDT #

Alas, Sun's PR folks haven't yet bought-in to your assertions about core counts since with the VII announcement they were talking-up 44% less power per-core compared with the VI. I suppose that sounds more positive than 32% more power per-socket :)

What sort of scaling is this SAP result relative to the previous M9000 result?

Posted by rick jones on July 14, 2008 at 06:55 PM PDT #

That is unfortunate that SAP forbids that. The question people should ask is 'how much does it cost to achieve business metric X' (such as supported users, transactions/sec, etc (such numbers should of course account for any anticipated growth and such).

How that sum of money is divided into buying memory, cpus, etc. should only be relevant in as much as power, space, and cooling can end up adjusting the total cost.

To use an extreme hypothetical, if a 1,000 core system could deliver double the performance of a 32-core platform of a different vendor at half the total cost. What's more important -- the 1,000 cores, or getting twice as much total work done at half the cost?

Now if sales could only present it so simply...

Posted by Jason on July 15, 2008 at 09:03 AM PDT #

Well, being an applied math major I of course messed-up my subtraction. Assume 100 power units for the VI core. The VII core at 44% less is then not 66 power units but 56 power units. Twice the number of cores per socket is then (112 - 100)/100 or 12% more not 32%.

Posted by rick jones on July 15, 2008 at 02:02 PM PDT #

You Rick are doing all kinds of funny core estimates based on things stated in PR. Since you are an applied math major then you know the problems with doing math on top of average % increases. So your conclusions are not worth very much.

I've seen some workloads double performance with doubling of cores on the SPARC64 VII over SPARC64 VI. Mileage will vary. Why not share some of your experience on scaling between DC and QC on X64?

Why are you avoiding system performance discussions. The fastest thing IBM makes was beaten by the fastest new server from Sun.

Price the 5GHz Power6 to see the $. (be sure to add in all of the memory and CPU activation fees, they are NOT trivial, yes hardware licensing is a big deal on IBM).

We've covered the huge variability in core pricing in many earlier postings. You need to compare on system price and not the misleading IBM PR love of perf/core.

I don't know any Sun PR people, but I'm glad they don't focus on perf/core messages on benchmarks -- that is just IBM FUD.

Posted by BM Seer on July 16, 2008 at 10:16 AM PDT #

Few more thoughts:

Sun can mix SPARC64 VII and SPARC64 VI in the same chassis (different speeds). Which can save in upgrade $. IBM can not do that on power6, ouch get the fork lift.

What is the overheads of IBM LPARS/mPARS micro-partitioning? I've seen IBM documents show 25% overhead for 5-6 partitions, I need to go find that doc again, I think it was in a redbook.

Can IBM hot-swap memory & procs? no. Mseries can.

Posted by BM Seer on July 16, 2008 at 11:02 AM PDT #

So how much does this Sun M9000 with 12TB of memory cost ? Care to divulge ?

25 % overhead ? Yes, please find that doc again and post the link here.

And seeing as your brought up virtualization, it seems to me that the M9000 is absolutely no match for the Power systems in this department. Years behind in fact.

Posted by Alex on July 16, 2008 at 12:59 PM PDT #

My notes say...
http://www-03.ibm.com/systems/p/os/aix/whitepapers/pdf/aix_support.pdf
on page 50 (figure 55), written by the international Technology Group.

Also have this note...
http://www.redbooks.ibm.com/redbooks/pdf/sg245768.pdf, page 126.

But I can't get to those right now on the web. Sometimes IBM pulls information if it exposes warts so they may be gone for good :(
I'm usually good about keeping pdf to avoid such issues but I can't find
my local copy :(

There has to be power6 data out there, but again, IBM avoids showing overheads of partitioning so there may be no public data.

Posted by BM Seer on July 16, 2008 at 01:57 PM PDT #

Your reference aix_support.pdf is dated 2004 - in case you haven't noticed, IBM have let you far behind since then.

The overhead of running LPARs and micropartitions? Well yes there is an overhead, so small that we can ignore it. The important thing to remember is the flexibility you get from POWER servers, PowerVM and AIX makes anything Sun has to offer look second class. IBM is clearly years ahead on virtualization and continuous running of systems. Sun is merely a seller of unreliable and inflexible tin boxes with an operating system trying to keep up with the big boys. Oh, and the Sun website has a few penguins here and there because a few Sun machines in the hideous spectrum available can run Linux (but Sun would rather you didn't).

Posted by Investa Ixnotsun on July 16, 2008 at 02:28 PM PDT #

OK, then point to DATA yes, measured overhead, you can't say years ahead with no data. If IBM virtualization is so far ahead, why can't one hot-swap CPUs, boards, memory???? Why can't you point to IBM documents in public that show _measured_ overheads? You can't just vaguely claim "so small that we can ignore it" Prove that things have changed with real data!

I show data, you give me FUD. As wikipedia says under FUD: "FUD is generally a strategic attempt to influence public perception by disseminating negative (and vague) information."

IBM bested again by Sun on measured system performance on a variety of benchmarks. IBM avoids power6 benchmarks, measured data and then you provide _vagueness_.

Goes back to when FUD was defined:

"FUD was first defined by Gene Amdahl after he left IBM to found his own company, Amdahl Corp.: "FUD is the fear, uncertainty, and doubt that IBM sales people instill in the minds of potential customers who might be considering Amdahl products."
http://en.wikipedia.org/wiki/Fear%2C_uncertainty_and_doubt

Posted by BM Seer on July 16, 2008 at 03:03 PM PDT #

The other doc I pointed to sg245768.pdf was posted in Nov 2005 it showed no change in overheads after working on it 1.5 years!

That sg245768.pdf is only 2.5 years ago.

IBM has since removed this doc ...maybe to avoid showing warts?

Show me data!

Posted by BM Seer on July 16, 2008 at 03:13 PM PDT #

We all know what FUD is Mr Seer. We read enough of it here on your blog, and quite amusing it is too. However, you are the one who has stated these so called facts, so YOU show us the data to back it up. Either that, or pipe down ! :)

As someone who has benchmarked multi-partitioned Power 5 and Power 6 system, i have my own data on hypervisor overhead, and i can assure that 25% is very unlikely. Hypervisor overhead is directly related to workload, so if your workload is creating consistent hypervisor overhead of 25%, then its very likely that your system is hugely under-resourced.

Also, you may want to read up on virtual processor folding. A feature IBM introduced in Power 5 a few years back, which resolved the performance penalty associated with configuring too many virtual processors, which could cause increased context switching.

If your relying on documents from 2005 to back up your case, then perhaps this explains a few things.

Posted by Alex on July 16, 2008 at 11:20 PM PDT #

"If IBM virtualization is so far ahead, why can't one hot-swap CPUs, boards, memory???

With Sun servers being able to do these things is a life saver - with so many failures of CPUs, boards and memory we would contantly have to take servers down to replace these parts. We waste *so* much time handling Sun engineer calls.

Contrast this with the IBM approach: build highly reliable systems and components that don't fail and on the rare occasion they give errors, supply predictive warnings. What a great idea! If your service is so critical you can not take a rare downtime to replace non-concurrent hardware then you have a cluster. This is how an IBM device called a 'mainframe' has worked for many years and I hear they are quite popular.

Anyway, the few remaining components that do require downtime to replace on System p, will be hot swappable in the not too distant future.

Posted by Investa Ixnotsun on July 17, 2008 at 02:49 AM PDT #

I concur. In many years of supporting IBM Power systems, I have never seen a cpu failure. The occasional memory dimm yes, but with Powers fleixbility in being able to dynamically move memory around, this usually means that no immediate action is required.

Also, dont forget that Power systems support dark resources via CUoD which can be activated at any time to cater for growth, and failure. This is an online, dynamic process which requires no downtime or impact. Much better to do that than have a Sun engineer poking around in the back of a production system fitting new processor/memory imho.

of course, if you absolutely must take a system down, then Power 6 supports mobilization of running partitions. Not something you guys at Sun can do yet i beleive ?

Posted by Alex on July 17, 2008 at 03:09 AM PDT #

OK, well I had a different experience, seen a Power CPU fail in a datacenter. So that was a total pain. Too bad IBM didn't have hotswap.
I wish I could still have access and could measure performance on that
IBM called a server which was claimed as fast but was slow on real workloads.

I show data all of the time, the IBM-loving commentors are the ones who
put vague comments like "I have my own data" --(well show it dude!
Why does IBM avoid showing measured data on IO rates, virtualization overheads all kinds of thing, then the IBMers pull docs when Sun finds
real information that IBM published.

"A feature IBM introduced in Power 5 a few years back, which resolved the performance penalty associated with configuring too many virtual processors, which could cause increased context switching."
OK, show data. You can't just make baseless claims. The IBM docs showed
the overhead, if someone fixed it why are there NO quantified docs that show it fixed. Alex, all you ever post in comments is baseless FUD claims.

Posted by BM Seer on July 17, 2008 at 10:30 AM PDT #

Post a Comment:
Comments are closed for this entry.