Siebel CRM 8.0 PSPP UltraSPARC T2 beats POWER6 and sets World Record

Thursday Jan 10, 2008

arrgghhh... I've been asked to show only Sun's results. You must now do your own math with the information posted on Oracle's website: http://www.oracle.com/apps_benchmark/doc/Sun_Siebel8_10000_PSPP_On_Solaris.pdf
http://www.oracle.com/apps_benchmark/doc/IBM_Siebel8_7000_PSPP_On_AIX_POWER6%20Final.pdf

IBM now longer holds the world record and really needs to post a correction on:
http://www-03.ibm.com/systems/p/hardware/benchmarks/erp.html

The Sun results using the UltraSPARC T2 supported 30% higher Siebel benchmark concurrent users compared to other results on the Siebel CRM Applications Release 8.0.

Sun again shows the UltraSPARC T2 servers are ideally suited for Oracle database applications. The database server ran Oracle 10g R2 on this Siebel benchmark.

{ Stuff deleted }

Sun's Solaris and Coolthreads based servers proves once again to be the best combination for scalability and resource utilization in the datacenter, giving users a consistent response time on critical applications as shown 10,000 users benchmark on Siebel CRM 8.0.

The 10,000 Siebel benchmark users performance results on 4 Sun SPARC Enterprise T5120/T5220 servers running Solaris 10 delivers a scalable and cost-effective platform for deploying Siebel CRM Application and Oracle 10g R2 deployment.

The result of 10,000 active concurrent Siebel user benchmark was run end to end on the new generation of Sun SPARC Enterprise servers using coolthreads technology with the highest level of space and energy efficiency.

See Also: http://www.oracle.com/apps_benchmark/html/white-papers-siebel.html

Siebel CRM 8.0 PSPP Performance Chart as of 01/04/2008 (bigger is better)

Vendor	Users	Web Server	Application Servers	Database Server
Sun	10,000	1 x Sun SPARC Enterprise T5120 4 cores, 1 chip @1.2 GHz US-T2     8 GB RAM Siebel CRM 8.0 SIA [20204] ENU Sun Java System Web     Server 6.1 SP8 Solaris 10 8/07	1 x Sun SPARC Enterprise T5220 8 cores, 1 chip @1.4 GHz US-T2     32 GB RAM 1 x Sun SPARC Enterprise T5220 8 cores, 1 chip @1.2 GHz US-T2     32 GB RAM Siebel CRM 8.0 SIA [20204] ENU Solaris 10 8/07	1 x Sun SPARC Enterprise T5120 8 cores, 1 chip @1.2 GHz US-T2     32 GB RAM Oracle 10gR2 Database     Server v10.2.0.1.0 Solaris 10 8/07
.	.	.	.	.

As noted on the official benchmark report: "Siebel CRM Release 8.0 Industry Application Platform Sizing and Performance benchmarks are based on Siebel CRM Release 8.0 customized industry applications and reflect a heavier scenario mix and more-aggressive think times than earlier version. Results of this benchmark are not comparable with those of prior Siebel CRM Release 7 benchmarks."

Benchmark Description

Siebel CRM 8.0 Platform Sizing and Performance Program (PSPP) is a multi-tier benchmark designed to stress the Siebel CRM Release 8.0 architecture and to demonstrate that large customers can successfully deploy many thousands of concurrent users. Among the Siebel CRM Release 8.0 architecture features exercised are the following:

• Smart Web Architecture: Takes advantage of the newest Web browser technology to deliver a highly interactive experience. The interaction model, which is similar to Windows-based applications, also improves productivity. Utilization rates on the web server are low, allowing customers to retain existing Web server infrastructure.
• Smart Network Architecture: Allows Siebel CRM Release 8.0 customers to leverage their existing network infrastructure by compressing and caching user interface components, so that browser/Web server interaction occurs only when the application requests data. This allows customers to avoid expensive network upgrades that can be necessary with competing products.

• Server Connection Broker: The Siebel Connection Broker (SCBroker) is a server component that provides intraserver loadbalancing. SCBroker distributes server requests across multiple instances of Application Object Managers (AOMs) running on a Siebel server.
• Smart Database Connection Pooling and Multiplexing: Allows customers to scale their database without intrducing expensive and complex transaction-processing monitors.
• Server Request Broker: Server Request Broker (SRBroker) processes synchronous server requests - reuqests that must be run immediately, and for which the calling process waits for completion.
• Enterprise Application Integration: Allows customers to integrate their existing systems with Siebel CRM applications.
• eScript: eScript is a scripting or programming language that application developers use to write simple scripts to extend Siebel applications. Javascript, a popular scripting language used primarily on Web sites, is its core language.

The test simulated real-world requirements of a large organization, consisting of 10,000 concurrent, active users from multiple departments accessing a call center. Test conditions simulated service representatives running Siebel Financial Services Call Center and partner organizations running Siebel Partner Relationship Management (Web sales and Web service). Siebel Workflow and the Siebel Scripting Engine were used to incorporate business-process-management customizations. The application also simulated integration with Web systems, using the Siebel Enterprise Application Integration component and Siebel Web Services.

Disclosure Statement:

Siebel CRM 8.0 Platform Sizing and Performance Program (PSPP) benchmark as of 01/04/08. Sun Microsystems: 10,000 users, 1 x Sun SPARC Enterprise T5120 web server (4 cores, 1 chip @1.2 GHz US-T2, 8 GB RAM), Siebel CRM 8.0 SIA [20204] ENU, Sun Java System Web Server 6.1 SP8, Solaris 10 8/07, 1 x Sun SPARC Enterprise T5220 application server (8 cores, 1 chip @1.4 GHz US-T2, 32 GB RAM), 1 x Sun SPARC Enterprise T5220 application server (8 cores, 1 chip @1.2 GHz US-T2, 32 GB RAM) Siebel CRM 8.0 SIA [20204] ENU, Solaris 10 8/07, 1 x Sun SPARC Enterprise T5120 database server (8 cores, 1 chip @1.2 GHz US-T2, 32 GB RAM), Oracle 10gR2 Database Server v10.2.0.1.0, Solaris 10 8/07 Oracle, Siebel, registered trademarks of Oracle Corporation and/or its affiliates. More info www.oracle.com/apps_benchmark/html/white-papers-siebel.html

Power Reference:

Sun measured: Database Server (1.2 GHz T5120, 8 core, 32G memory): 291W, Gateway/Application Server #1 (1.4 GHz T5220, 8 core, 32G memory): 323W, Application Server #2 (1.2 GHz T5220, 8 core, 32G memory): 376W, Web Server (1.2 GHz T5120, 4 core, 8G memory): 212W.

IBM power calculation based on the following: The p570 is supplied in building blocks with 2 chips, 4 cores per chassis called a CEC. Up to 4 CECs can be connected together to create a single 16 chip, 32 core SMP system. Each CEC is 4 RU, and each CE is estimatedC to consume 1,040 watts when configured with 2 processors, based on the following: IBM p6 570 power specifications from 80% of maximum report power consumption published here, 06/07/07, posted at ftp://ftp.software.ibm.com/common/ssi/rep_sp/n/PSB01628USEN/PSB01628USEN.PDF

System Configuration

Certified Results			10,000 Users
Reference Date:			January 4, 2008
Systems:			1 x Sun SPARC Enterprise T5120, web server (one 1.2GHz UltraSPARC T2)
			1 x Sun SPARC Enterprise T5220, gateway/application server (one 1.4GHz UltraSPARC T2)
			1 x Sun SPARC Enterprise T5220, application server(one 1.2GHz UltraSPARC T2)
			1 x Sun SPARC Enterprise T5120, database server (one 1.2GHz UltraSPARC T2)
Operating System:			Solaris 10 8/07
Software:			Sun Java System Web Server 6.1 SP8
			Siebel CRM 8.0 SIA [20204] ENU
			Oracle 10gR2 Database Server v10.2.0.1.0

Like this post? | | | |

Posted at 06:58AM Jan 10, 2008 in Benchmark Issues
Tags: cmt performance power power6 siebel t2 ultrasparc

UltraSPARC T2 servers better/lower-price than IBM high-end Power6

Thursday Dec 06, 2007

In the past posting there was a comment that one can't compare high-end Power6 systems with what they call low-end T2 systems. Quite simply UltraSPARC T2 systems can out perform Power6 on important datacenter workloads and much lower cost. Little need to buy from the ultra-expensive-IBM.

• SAP-SD, clearly a high-end ERP benchmark with SAP and database. Sun T5120 Beats ultra-expensive 4-core IBM p570 4.7 GHz POWER6 by 7%. Oracle Database used on Sun's SAP-SD.
• SPECjAppServer, another application tier and database-tier benchmark. Good for high-end servers, clearly UltraSPARC T2 is in that class and much less expensive the Power6. UltraSPARC T2 used as Oracle Database server. Sun T5120 67% faster 4-core IBM p570 4.7GHz Power6
  App Server: 3.8 better power-perf & 7x better SWaP
  Oracle Server: 3.4 better power-perf & 14x better SWaP
  
• SPECjbb, another application benchmark with lots of big servers. Sun T5220 9% faster than 4-core IBM p570 4.7GHz POWER6, 2.5x better power-performance & 5x better SWaP
• SPECint_rate2006: UltraSPARC T2 1.4GHz beats best 1-chip(2-core) IBM POWER6 4.7GHz by 29%
• No SPECweb2005 on power6, OK web is not an application for Power6 otherwise they would have published. However IBM did publish on the IBM p550 Power5 (an embarrassingly low result). Evidently the even more expensive are reputably faster Power6 couldn't get to the level of performance.

Pricing dataquoted to a customer!
IBM p570 POWER6 4.7GHz 4-core, 32GB $252K USD
IBM p570 POWER6 4.7GHz 4-core, 64GB $313K USD ($78K/core!!!)

So I'm left with 4-core Power6 is way to expensive and doesn't have any more redundancy features that UltraSPARC T2 which outperforms it.

The only public info that lists IBM component prices that I know is at: http://www.tpc.org/results/individual_results/IBM/IBM_570_10000GB_20071015_ES.pdf

Disclosure statement

SPEC, SPECint reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 11/13/07. Sun Blade T6320 (UltraSPARC T2, 1 chip, 8 cores), 78.6 SPECint_rate2006. Sun SPARC Enterprise T5220 (UltraSPARC T2, 1 chip, 8 cores), 78.5 SPECint_rate2006.Sun SPARC Enterprise T5220 gccfss (UltraSPARC T2, 1 chip, 8 cores), 78.0 SPECint_rate2006. IBM p570 (POWER6, 1 chip, 2 cores), 60.9 SPECint_rate2006. Sun SPARC Enterprise T5220 (UltraSPARC T2, 1 chip, 8 cores), 62.3 SPECfp_rate2006. IBM p570 (POWER6, 1 chip, 2 cores), 58.0 SPECfp_rate2006. SPEC, SPEComp reg tm of Standard Performance Evaluation Corporation. Other results from www.spec.org as of 11/13/07. Sun SPARC Enterprise T5220 (1 chip, 8 cores, 64 threads, 1.4 GHz) 16208 SPECompM2001. Sun SPARC Enterprise T5220 (1 chip, 8 cores, 64 threads, 1.4 GHz) 16208 SPECompM2001. IBM p5 520 (1 chip, 2 cores, 4 threads, 1.9 GHz) 8174 SPECompM2001. Sun SPARC Enterprise T5220 (1 chip, 8 cores, 64 threads, 1.4 GHz) 16208 SPECompM2001. Sun SPARC Enterprise T5220 (8 cores, 1 chip) 37001 SPECweb2005. IBM p5 550 (4 cores, 2 chips) 7881 SPECweb2005. SPEC, SPECweb reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of Nov 13, 2007. Sun SPARC Enterprise T5220 server power consumption taken from measurements made during the benchmark run. Power is average measured watts during benchmark run.IBM 550 power specifications calculated by applying 70% of the Maximum Watts published in “Facts and Features Report”, 11/14/06, posted at ftp://ftp.software.ibm.com/common/ssi/rep_sp/n/PSB01628USEN/PSB01628USEN.PDF. SPECjbb2005 Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192055 SPECjbb2005 bops, 24007 SPECjbb2005 bops/JVM, Sun SPARC Enterprise T5220 (1 chip, 8 cores) 192055 SPECjbb2005 bops, 24007 SPECjbb2005 bops/JVM, Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170153 SPECjbb2005 bops, 170153 SPECjbb2005 bops/JVM, Sun SPARC Enterprise T5220 (1 chip, 8 cores) 170153 SPECjbb2005 bops, 170153 SPECjbb2005 bops/JVM, Sun SPARC Enterprise T5120/T5220 results submitted to SPEC, IBM p570 (1 chip, 2 cores) 88089 SPECjbb2005 bops, 88089 SPECjbb2005 bops/JVM, IBM p570 (2 chips, 4 cores) 175474 SPECjbb2005 bops, 87737 SPECjbb2005 bops/JVM, IBM p505Q (2 chips, 4 cores) 63544 SPECjbb2005 bops, 31772 SPECjbb2005 bops/JVM, SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 10/08/2007 on www.spec.org. The 2-core IBM p570 POWER6 system requires 4 RU or 4 times the rack space of a Sun T5120 and consumes on average 1040 Watts of power. The 4-core IBM p570 POWER6 system requires 4 RU or twice the rack space of a Sun T5220 and consumes on average 1040 Watts of power. IBM p6 570 2-core & 4-core power specifications from 80% of maximum report power consumption published here, 06/07/07, posted here. The IBM p505Q POWER5+ system requires 1 RU of rack space and consumes on average 320 Watts of power. IBM p505 power specifications from 80% of maximum report power consumption published in ?Facts and Features Report?, 03/27/06, posted here. 1 IBM p570 (4 cores, 2 chips) and 1 IBM p550 (4 cores, 2 chips) 1197.51 SPECjAppServer2004 JOPS@Standard. 1 IBM p505Q (4 cores, 2 chips) and 1 IBM p550(4 cores, 2 chips) 618.38 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 10/10/2007. The IBM p570 POWER6 system requires 4RU or twice the rack space of a Sun T5220 and consumes on average 1040 Watts of power. The IBM p6 570 power specifications from 80% of maximum report power consumption published 06/07/07, posted here. The IBM p505Q requires 1 RU of rack space and consumes on average 320* Watts of power. The IBM p505 power specifications from 80% of maximum report power consumption published in "Facts and Features Report", 03/27/07, posted here. The IBM p550 requires 4RU of rack space and consumes on average 770* Watts of power. The IBM p5 power specifications calculated by applying 70% of the power numbers published in ?Facts and Features Report?, 3/10/06, posted < href=http://www-03.ibm.com/servers/eserver/pseries/hardware/factsfeatures.html> here. Two-tier SAP Standard Sales and Distribution (SD) standard SAP ERP 2005 application benchmark: SPARC Enterprise Model T5120 (1-way, 1 proc, 8 cores, 64 threads) 1 x 1.4 GHz UltraSPARC T2, 64GB memory, 2175 SD Benchmark users, 1.91 sec avg response time, Cert#20070xx, Oracle 10g, Solaris 10; SPARC Enterprise Model T2000 | Sun Fire T2000 (1-way, 1 proc, 8 cores, 32 threads) 1 x 1.4 GHz UltraSPARC T1, 64GB memory, 1100 SD Benchmark users, 1.91 sec avg response time, Cert#2007051, Oracle 10g, Solaris 10; IBM System p 570 (2-way, 2 processors, 4 cores, 8 threads) 2 x 4.7 GHz POWER6, 32GB memory, 2035 SD Benchmark users, 1.99s avg resp time, Cert#2007037, Oracle 10g, AIX 5L Version 5.3; SAP, R/3, mySAP reg TM of SAP AG in Germany and other countries. More info www.sap.com/solutions/benchmark.

Like this post? | | | |

Posted at 02:30PM Dec 06, 2007 in IBM Follies
Tags: cmt ibm power6 ultrasparc

Best use of your money for effective performance

Wednesday Dec 05, 2007

IBM makes many of its world records claims based on core count. IBM is relying on everyone to think that every vendors cores roughly cost the same. However, cost per core varies widely, with IBM leading the pack in the wrong way. IBM cores are 2x to 10x more expensive than other vendors. IBM simply has very expensive systems that have been out-performed by Sun's UltraSPARC T2 based servers. The UltraSPARC T2 servers are faster and have better price-performance than IBM's Power6 systems.

Sun has always advocated that one needs to look at system price as benchmarked. IBM tries to say that low-cost = "cheap junk". One needs to look at performance, price-performance, and power-performance of the configurations benchmarked. The only problem is IBM hides actual prices and actual power. I guess the truth hurts IBM, so they stick to their marketing drone..."fastest core".

Let's get to the real costs from a customer quote!
IBM p570 POWER6 4.7GHz 4-core, 32GB $252K USD
IBM p570 POWER6 4.7GHz 4-core, 64GB $313K USD ($78K/core!!!)

The only public info that lists IBM component prices that I know is at: http://www.tpc.org/results/individual_results/IBM/IBM_570_10000GB_20071015_ES.pdf

You can do your own comparisons to 2-socket Quad-core Opteron, 2-socket Quad-core Xeon, or the 1 chip 8-core UltraSPARC T2 all with reasonable amount of memory 32GB to 64GB.

Clearly you can't just say "best 4-core" or "best 8-core" in performance and hide prices without being rotten to the core

Like this post? | | | |

Posted at 02:09PM Dec 05, 2007 in IBM Follies
Tags: ibm power6

Some IBM POWER6 actual prices and why per-core-performance doesn't say much

Friday Nov 30, 2007

Here are some quick IBM POWER6 prices that have been quoted to a customer!
IBM p570 POWER6 4.7GHz 4-core, 32GB $252K USD
IBM p570 POWER6 4.7GHz 4-core, 64GB $313K USD
...it would be so much easier if IBM just posted their prices on these small servers.

I'll have to check how these compare to TPC-C pricing at some point... http://www.tpc.org/results/individual_results/IBM/IBM_570_10000GB_20071015_ES.pdf

Also how does that compare to 2-socket Quad-core Opteron, 2-socket Quad-core Xeon, or the 1 chip 8-core UltraSPARC T2 all with reasonable amount of memory.

Clearly you can't just say "best 4-core" or "best 8-core" in performance and hide prices without being rotten to the core

...ready, steady, go. ...to the weekend.

Like this post? | | | |

Posted at 03:53PM Nov 30, 2007 in IBM Follies
Tags: ibm power6

Good review of IBM's weak POWER6

Wednesday Nov 28, 2007

This is really good information on IBM POWER6 http://blogs.sun.com/jmeyer/entry/power6_goes_thud_part_vii

Also shows why you can't believe IBM at all when they talk about best cores.

Tomorrow I'll review *IBM's actual pricing of cores*, IBM leads the industry in the very doubious category of worst price per core by a HUGE factor. While enjoying the american holiday of thanksgiving a friend's neighbor walked me over to his house and showed me actual price quotes on IBM power6 OUCH! I totally nailed the pricing correctly http://blogs.sun.com/bmseer/entry/part2_ibm_power6_cores_extremely!

Yes if you take the price IBM charges you for a Power6 system with reasonable memory and divide that system price by the number of cores in a configuration you find the price is $70K/core to 150K/core - I'm talking a 4-core or 8-core system with only medium size memory!).

Postscript To answer the last comment
Actually what I write about most is peeling away the IBM marketing veneer to reveal the true facts about performance and IBM's outrageous pricing claims. Whistle blowers like me are always attacked, I realize that. There have been very few corrections to my factual postings, and I have posted LOTS of factual corrections to IBM's marketing errors. I can't believe the things IBM has told customers as well! Really when the facts come out IBM loses business.

Interesting that as you say any IBM customer knows the future direction, well I'm pointing out what they actually deliver now. I'm sure everything gets fixed in the future.

I spend my work life working very hard to make things even better for Sun. I write in my blog about correcting the insanity I see others say in public.

Like this post? | | | |

Posted at 10:36AM Nov 28, 2007 in IBM Follies
Tags: ibm power6

They tried to make IBM show perf, IBM said no, no, no...

Thursday Nov 15, 2007

no STREAM Bandwidth p570 power6 results
no TPC-H single-system Power6 results
no SPECweb2005 Power6 results
<16-Nov: line deleted due to correction, see note below> no SPECjAppServer2004 Power6 as database server
No SPECmail Power6 results
no SPECompL2001 Power6 results
no Lotus Domino R6iNotes Power6 results
no cryptography performance on Power6 results
no power-performance on any published benchmarks
no consolidation overheads on Power6, (Solaris Zones=~0%)
etc.

What does the industry get - the most expensive core now built and a bogus press release chip bandwidth number: http://blogs.sun.com/bmseer/entry/finally_proof_power6_has_meaningless

I don't even think the Power6 has a single-chip world record on anything that everyone publishes?

...without even going into HPC ISV results (err, lack thereof)
no Fluent Power6 results
no StarCD Power6 results
no LSDyna Power6 results
no MSC.Nastran Power6 results
etc.

If it was as fast and IBM claims in marketing wouldn't IBM just do the ones they haven't done? Also why can't they beat the UltraSPARC T2 world records on the ones they publish?

"You won't know, know, know..." -Amy Winehouse

SPEC, SPECint, SPECweb, SPECfp, SPECjAppServer, SPECmail, SPEComp, reg tm of Standard Performance Evaluation Corporation. TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info http://www.tpc.org. Lotus Domino more info: www.notesbench.org

Postscript Note:
The original posting had an error. IBM did publish a SPECjAppserver2004 benchmark on Power6. The comparison is: One Sun T5220 server (single UltraSPARC T2 chip) demonstrated 67% better performance over the IBM result of 1197.51 SPECjAppServer2004 JOPS@Standard which used 4-core IBM p570 with 4.7GHz POWER6 processors. The Sun SPARC Enterprise T5220 used as Application server has 3.8x better power-performance and has 7.3x better SWaP than the IBM p570 power6. Enterprise T5120 used as database servrer has 3.4 better power-performance and has 13.5x better SWaP as the IBM p550.

Disclosure Statement:

SPECjAppServer2004 1 Sun SPARC Enterprise T5220 (8 cores, 1 chip) and 1 Sun SPARC Enterprise T5120 (8 cores, 1 chip) 2000.92 SPECjAppServer2004 JOPS@Standard. 1 IBM p570 (4 cores, 2 chips) and 1 IBM p550 (4 cores, 2 chips) 1197.51 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 10/10/2007.

Like this post? | | | |

Posted at 08:08AM Nov 15, 2007 in IBM Follies
Tags: blogwatch ibm performance power6

IBM wrong at top of their voice and IBM JS22 power6 blade

Thursday Nov 08, 2007

Even IBM expert blogs continue to trumpet bad data. In the previous posting I pointed out the problems, please see BM Seer's "IBM JS22 Power6 blades not the performance you think"

While, I realize that over-excited IBM marketing types may tend to overstate and make bad claims, but IBM's performance blog expert too?

IBM's Stahl writes: Equally compelling is the analysis that one rack of IBM's new POWER6 processor-based blades is so powerful when virtualized that it can replace many non-virtualized racks of Sun's latest V490 servers, potentially saving a ton in energy costs.

I guess IBM uses something learned from Lucy in Peanuts cartoon: "If you can't be right, .... be wrong at the top of your voice!"

So in how many more blogs, ads, and other marketing material will we see this comparison? My guess is LOTS at the top of their voice!

Like this post? | | | |

Posted at 01:51PM Nov 08, 2007 in IBM Follies
Tags: blogwatch ibm performance power6 specint_rate2006

IBM JS22 Power6 blades not the performance you think

Thursday Nov 08, 2007

Why do I attack IBM alot, it is not because they are a competitor (I don't attack all competitors), it is because IBM continues to pull such funny games to confuse the marketplace.

Here is a prime example of an un-fair/stilted comparisons from IBM press release:

Calculations show that one rack of IBM's new POWER6 processor-based blades is so powerful when virtualized that it can replace 23 non-virtualized racks of Sun's latest V490 servers, potentially saving more than $200,000 per year in energy costs alone. (3)

Why didn't IBM compare against the Sun Blade T6320 (UltraSPARC T2) blades? IBM would have lost if they properly compared to Sun.

IBM gamed again by comparing new servers to older Sun servers. IBM claimed one rack of JS22 BladeCenter Hs to replace 180 V490s. IBM based its JS22 claim by 'pulling a a *3X* utilization rate" out of the air' when compared to the V490. The JS22 utilization was 60%, while the V490 was 20% utilized-- a report from Alinean Consulting was cited as the source for the utilization comparisons(note:I guess you can pay for anything). This was the same fake 3x difference in utilization that IBM tryed to pull in the POWER6 announcement that BM Seer shot down in "IBM rewrites history, OK footnotes to clearly show bogus calculations"

As a reminder as always with IBM read written material very carefully: http://blogs.sun.com/bmseer/entry/careful_reading_shows_a_lot

Here is the IBM footnote:

(3) The number of IBM BladeCenter JS22 servers required to replace 180 Sun Fire V490 was calculated based on SPECint_rate2006 results. The V490 SPECint_rate2006 result is for a 2.1GHz system with 4 chips and 2 cores per chip. It has a result of 78.0. The V490 result can be found at www.spec.org. It is current as of October 23, 2007. The JS22 result for the same benchmark is for a 4.0GHz system with 2 chips and 2 cores per chip. It has a result of 84.7. That result was submitted on November 6, 2007. It will also be posted on www.spec.org. The cumulative capacity of these servers is estimated to be the SPECint_rate2006 result for one server multiplied by the number of servers. A virtualization factor of 3X was applied to the JS22 virtualization scenario using utilizations derived from studies conducted by Alinean available at http://www-935.ibm.com/services/us/cio/optimize/opt_wp_ibm_systemp.pdf. That is the utilization rate for the non-virtualized V490 is estimated to be 20% and the utilization rate for the virtualized JS22 is estimated to be 60%. Using these assumptions, the cumulative capacity of the 56 JS22 servers at 60% is greater than the cumulative capacity of the 180 V490 servers at 20% utilization.

Like this post? | | | |

Posted at 10:18AM Nov 08, 2007 in IBM Follies
Tags: blogwatch ibm performance power6 specint_rate2006

careful reading shows a lot

Wednesday Oct 24, 2007

You have to read some things carefully

"...And the good news is that about 40-70% of the stuff we do in performance tuning actually ends up helping end users," -- Bruce Lindsay(IBM Fellow), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf

"This is feasible in the TPC-C benchmark because there are only five tables and only ten to fifteen columns in each table. In a more realistic application, where there are many more queries to be considered, the tables are typically much, much wider, in the 80 to 100 column range; and there are dozens if not thousands of tables. Then this kind of analysis(ed note: tuning) is no longer practical." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf

"The idea is to get the numbers by hook and by crook." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf

The TPC-C benchmark is an industry standard for measuring the ability of a system to process complex online transactions and large volumes of business data. The TPC-C benchmark is unique in the way it exercises all components of a system, including processors, memory, networking, storage, operating system and database software, demonstrating total system performance in a way that many of the other benchmarks touted by some competitors do not. -- Bruce Lindsay(IBM Fellow since '96), July 25, 2006, http://www-03.ibm.com/solutions/sap/doc/content/news/pressrelease/1623288130.html

stated "processor bandwidth", even though "delivered" system bandwidth would actually be required to move the data (not address ).

Next:

"IBM calculates that 30 SunFire v890s can be consolidated into a single rack of the new IBM machine, saving more than $100,000 per year on energy costs (3)." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss

Issues:

claimed Sun at 20% utilisation and IBM at 60% utilisation, that is one way to get 3x over your competition

never showed exactly what power was drawn by a 4.7GHz, 64GB memory system, at ??MHz DDR2 used in the comparison, etc.

This was a bit of a repeat, but some things should not be forgotten.

I've never been about popularity or names. You don't need my expertise to see funny things in IBM's statements. Don't attack me, attack the facts. Anonymously yours, Sun's BM Seer.

Disclosure statement:

TPC-C is a trademark of Transaction Processing Performance Council (TPC). More info www.tpc.org.

[1] Comments

• Oh boy you make me laugh with all the crap that yo...

• Post a comment

Like this post? | | | |

Posted at 03:14PM Oct 24, 2007 in Benchmark Issues
Tags: cmt ibm power6 t1 t2 tpc-c ultrasparc

IBM continues funny configurations on benchmarks.

Tuesday Oct 16, 2007

IBM just published a somewhat funny TPC-H results. IBM used 32 (4-core Power6 p570) systems in a clustered TPC-H. Some funny things:

• Why 32 4-core systems instead of clustering eight 16-core systems? Both configurations are built from the same 4RU 4-core unit.
• Why no IBM Power6 single-system TPC-H results? No stand-alone 16-core, 8-core, 4-core POWER6 results? Maybe they just don't stand on their own when you look at the performance?
• Why did they use 96 x 36.4GB drives on 31 systems and 96 x 73.4GB drives only on server14?
• Why did they use the smallest 36.4GB drives for most drives in the system?
• If the un-discounted server hardware price for thirty-two 4-core systems is $7,042,378usd, what is the price of one 4-core system?
• If the un-discounted server hardware price of $7,042,378usd, and 128 cores, what is the un-discounted price per core when configured in a 4-core system with 32GB?

Disclosure statement

IBM TPC-H 10000GB result on the IBM System p6 570 of 343,551.2 QphH@10000GB ($32.89usdd $/QphH@10000GB, avail. 4/15/2008) on a 32-node cluster of 4-core p570 (each with 2 POWER6 4.7 GHz processor chips, 4 cores, 8 threads) and 32GB of memory per node running DB2 Warehouse 9.5 on AIX 5L V5.3. Total disk capacity was 110,489.27 GB in a IBM Totalstorage DS4800 storage subsystem (using 36.4GB drives on 31 nodes and 73.4 GB drives on server 14) and 10Gigabit Ethernet for cluster interconnect. Source: http://www.tpc.org; Results current as of 10/15/07.

Like this post? | | | |

Posted at 10:48AM Oct 16, 2007 in Competitor Follies
Tags: ibm performance power6 tpc-h

a while before more Power6

Friday Oct 05, 2007

Looks like it is February 2008 before we see Power6 blades, the Register writes: http://www.theregister.co.uk/2007/10/05/ibm_power6_blade_blue_business_platform/

Right now IBM sells 3.5GHz, 4.2GHz, and the 4.7GHz 2-core to 16-core IBM p570 (power6). The 4.7GHz which according to benchmarks must be Generally Available now but I've asked several times if anyone has actually has one? If not one has one in the field is it generally available? Please add a comment if you datacenter has one, I'd like to verify this.

From what I've been hearing in the rumor mill, IBM p550/p560 is sometime next spring and the IBM p590/p595 in the summer. If one eyeballs IBM public slides this seems to match up. This is a long time to roll-out a processor across a product line. This long time is rather atypical for a major vendor.

Given IBM's confusing "marketing speak" talked about in today's earlier posting "Critical reading absolutely required for IBM", read carefully around IBM's launch dates.

Like this post? | | | |

Posted at 01:06PM Oct 05, 2007 in IBM Follies
Tags: ibm power6

Critical reading absolutely required for IBM

Friday Oct 05, 2007

First:

"First of all, the math is really simple. 4.7 is greater than 1.4. IBM's POWER6 4.7 GHz chip is faster than Sun's 1.4 GHz UltraSPARC T1 chip. And second of all, the IBM System p 570 remains the #1 SPECjbb2005 2-core result (1)." - Elisabeth Stahl, program manager of IBM performance marketing, 20 years experience, Jun07

stated "processor bandwidth", even though "delivered" system bandwidth would actually be required to move the data (not address ).

Next:

"IBM calculates that 30 SunFire v890s can be consolidated into a single rack of the new IBM machine, saving more than $100,000 per year on energy costs (3)." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss

Issues:

An equally invalid statement: x86 do operate at 80-100% and mainframes may run as low as 5%.

Moral: Be VERY VERY CAREFUL when you read big blue.

Disclosure statement:

TPC-C is a trademark of Transaction Processing Performance Council (TPC). More info www.tpc.org.

Like this post? | | | |

Posted at 06:56AM Oct 05, 2007 in IBM Follies
Tags: ibm power6 t1 tpc-c ultrasparc

POWER6: per-core silliness

Wednesday Sep 05, 2007

IBM needs to get back comparing systems that are similar, this "best core" business is a disservice to the industry. It is what the system can do - not what a core does. A core does not a system make.

This is now all over the internet...

"A high-end, system begins at $2 million before IBM discounts. This is a 16-core version running at 4.7 GHz"

So doing the math: $2,000,000/16-cores = $125,000/core (when configured in a system)

IBM definitely loses on $/perf when you look at systems. ... please post a comment if your company actually has a 4.7GHz system.

http://blogs.sun.com/bmseer/entry/power6_latency_data_warehousing_and

Like this post? | | | |

Posted at 02:23PM Sep 05, 2007 in IBM Follies
Tags: ibm power6

Sun UltraSPARC T2 & IBM Power6 comparison blogged about

Wednesday Aug 29, 2007

There is more preliminary UltraSPARC T2 performance is blogged about at: http://blogs.sun.com/jmeyer/entry/power6_goes_thud_part_v

Where John states:

And IBM knows that next quarter, Sun will be introducing systems based on the new UltraSPARC T2, the world's first true system-on-a-chip and the world's fastest microprocessor. Preliminary estimates on one popular benchmark show that a single rack of UltraSPARC T2-based systems will outperform four racks of 4.7GHz POWER6-based p5 570s (more on that as we get closer to system announcement). No kidding.

...John also talks more about the lagging IBM POWER6 rollout.

Like this post? | | | |

Posted at 01:35PM Aug 29, 2007 in Benchmark Issues
Tags: cmt ibm performance power6 sparc t2 t5120 t5250 ultrasparc

Finally proof POWER6 has meaningless 300 GB/s peak bandwidth

Wednesday Aug 29, 2007

An IBM Hot chips presentation finally revealed what I posted 3 months ago... IBM's iTunes POWER6 comparison was misleading. In their press release IBM added things that can't be added to create a big number that means nothing.

IBM Press Release said:
"Even more impressive, the processor bandwidth of the POWER6 chip -- 300 gigabytes per second -- could download the entire iTunes catalog in about 60 seconds -- 30 times faster than HP's Itanium."

IBM MIXED maximum theoretical peaks and added them together to compare in an unfair way to HP Itanium. By the way all of these peaks are guaranteed not to reached by real applications (and certainly not iTunes. like IBM states. Here is my posting from 24-May as written, I've used strikes to correct the posting (not many corrections). Below "B"=bytes. "wr" is write, "rd" is read. in Bold I've added the numbers they revealed in the presentation. IBM only shows this for 4.7GHz, expect less on 3.5GHz and 4.2GHz (the POWER6 GHz it seems customers are getting).

• L3 cache: 16B wr + 16B rd, for a total of 32B x 2.5GHz = 80GB/s
  this is a max of 40GB/s in each direction
• memory: 8B wr + 16B rd, for a total of 24B x 2.5GHz = 60GB/s (75GB/s when even adding in address lines)
  peak no more than 40GB/s memory read bandwidth
  peak no more than 20GB/s copy bandwidth (limited by write)
• IO: 4B wr + 4B rd, for a total 8B x 2.5GHz = 20GB/s (but for addr and data)
• chip to chip: 3x links of 8B in + 8B out, clock rate? 50GB/s between 4 chips(8 cores)
• node to node: 2x links of 8B in + 8B out, clock rate? 80GB/s for configurations with more than 4 chips(8 cores)

It is funny that on slide 7 of their hot chips paper that the numbers add to 305GB/s but they say "Total=300 GB/s", ah well they probably can't even meet some of their peaks.
Lots of problems with quoting peak bandwidth. Since STREAM is so easy to run, why hide results? are the true numbers so bad?

An analogy:
One can't just add peak rates to say it is the bandwidth. It is like saying I can run at 10 miles per hour, my car has gone 100 miles per hour, and the plane travels at 650 miles per hour. Does that mean I traveled at 760 Miles per hour (650+100+10=760). No it just doesn't work that way.

The slides also indicate that global snoop limits memory bandwidth above 32 cores on IBM POWER6 on cache-intense workloads. But again not enough detail to tell what this really means yet. Currently IBM is only shipping 16-core Power6 p570?

http://blogs.sun.com/bmseer/entry/power6_err_try_10gb_s

Here is the problem with marketing peaks see the "Beware the Ides..." posting: http://blogs.sun.com/bmseer/entry/beware_the_ides_of_may

Like this post? | | | |

Posted at 10:52AM Aug 29, 2007 in Benchmark Issues
Tags: ibm performance power6