Wednesday Oct 24, 2007
You have to read some things carefully
"...And the good news is that about 40-70% of the
stuff we do in performance tuning actually ends up helping end users,"
-- Bruce Lindsay(IBM Fellow), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
"This is feasible in the TPC-C benchmark because there
are only five tables and only ten to fifteen columns in each table.
In a more realistic application, where there are many more queries
to be considered, the tables are typically much, much wider, in
the 80 to 100 column range; and there are dozens if not thousands
of tables. Then this kind of analysis(ed note: tuning) is no longer
practical." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
"The idea is to get the numbers by hook and by crook." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
The TPC-C benchmark is an industry standard for measuring the ability of a system to process complex online transactions and large volumes of business data. The TPC-C benchmark is unique in the way it exercises all components of a system, including processors, memory, networking, storage, operating system and database software, demonstrating total system performance in a way that many of the other benchmarks touted by some competitors do not. -- Bruce Lindsay(IBM Fellow since '96), July 25, 2006, http://www-03.ibm.com/solutions/sap/doc/content/news/pressrelease/1623288130.html
Issues:
This means that 30% to 60% of IBM's TPC-C tuning is useless for customers.
IBM clearly over-hyped TPC-C, just 2-3 months after they publicly showed all of its problems and "optimizations" they used.
Next:
"Significantly, the high utilization rate of the System z9 mainframes -- systems can and do operate at 80 to 100 percent utilization -- combined with its ability to "virtualize" workloads, can enable a single mainframe processor to perform far more work than a single x86 processor running Microsoft Windows. The latter may run as low as 5 percent utilization." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/19577.wss
Issues:
used different work for mainframe and for its competitor.
"do" and "may" mean very different things
"mainframes do operate at 80-100%", "x86 processor running Microsoft Windows. The latter may run as low as 5%". So it is a valid but totally useless statement.
An equally invalid statement: x86 do operate at 80-100% and
mainframes may run as low as 5%.
Next:
"First of all, the math is really simple. 4.7 is greater than 1.4. IBM's POWER6 4.7 GHz chip is faster than Sun's 1.4 GHz UltraSPARC T1 chip. And second of all, the IBM System p 570 remains the #1 SPECjbb2005 2-core result (1)."
Marketing Program manager of IBM performance blog, Jun07
Issues:
Did not compare system or chip performance but only quoted the GHz of a chip?
Made a true statement about core count but ignored that that IBM cores cost much more than Sun UltraSPARC T1 and/or UltraSPARC T2 on a per core basis, I know this
is hard to verify since IBM isn't public about pricing, so you'll have to ask your IBM people to price specific configurations for you, be specific so you understand exactly what is priced.
Next:
"Even more impressive, the processor bandwidth of the POWER6 chip – 300 gigabytes per second -- could download the entire iTunes catalog in about 60 seconds" - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss
Issues:
Added every bandwidth (L3 cache, address bandwidth?!?,...) in a chip,
even though peak memory bandwidth is limited to at least a 10th of that, delivered is a lot less.
stated "processor bandwidth", even though "delivered" system bandwidth would actually be required to move the data (not address 
).
Next:
"IBM calculates that 30 SunFire v890s can be consolidated into a single rack of the new IBM machine, saving more than $100,000 per year on energy costs (3)." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss
Issues:
used 2 year old sun result compared to power6 yet to be shipped as of may press release
said V890, so that people think it is a current comparison, had to read in the footnotes that it was 1.5 GHz slower CPU. Sun has introduced 1.8GHz, and 2.1GHz since.
made a "conservative" comparisons by giving IBM another 15% in performance
claimed Sun at 20% utilisation and IBM at 60% utilisation, that is one way to get 3x
over your competition
never showed exactly what power was drawn by a 4.7GHz, 64GB memory system,
at ??MHz DDR2 used in the comparison, etc.
This was a bit of a repeat, but some things should not be forgotten.
I've never been about popularity or names. You don't need my expertise
to see funny things in IBM's statements. Don't attack me, attack the facts.
Anonymously yours, Sun's BM Seer.
Disclosure statement:
TPC-C is a trademark of Transaction Processing Performance Council (TPC). More info www.tpc.org.
Tuesday Oct 09, 2007
Today, Sun submitted the SPECint_rate2006 and SPECfp_rate2006
Single-Chip World Records on the Sun SPARC Enterprise T5120/T5220.
What are these servers? UltraSPARC T2 1.4GHz servers that you will
hear loads more on today.
The Sun SPARC Enterprise T5120 is the 1RU version, and the
Sun SPARC Enterprise T5220 is the 2RU version, both of these
servers are electronically equivalent with the 2RU having a bit more
connectivity and storage if you need.
The Sun SPARC Enterprise T5220 server, running at 1.4 GHz, beat all single-chip results running SPECint_rate2006 with a result of 78.5.
The Sun SPARC Enterprise T5220 server beats the best single IBM 4.7 GHz dual-core POWER6 processor result by 29% and beat the best published single
3 GHz Xeon quad-core by 28% on SPECint_rate2006. There are no single quad-core Opteron results published for SPECint_rate2006.
"but I've heard there is no floating point on Niagara processors
Nay, the 1.4GHz UltraSPARC T2 in the Sun SPARC Enterprise T5220 server, beat
all single-chip results running SPECfp_rate2006 with a result of 62.3.
The Sun SPARC Enterprise T5220 server beat the best single IBM 4.7 GHz
POWER6 processor based system result by 7% and beats the best published
single 3 GHz quad-core Intel Xeon by 61% for SPECfp_rate2006.
There are no single quad-core Opteron results published for SPECfp_rate2006.
SPEC CPU2006 Performance Charts -
bigger is better, selected recent results,
please see
www.spec.org for complete results.
SPECint_rate2006
| System |
Procs |
Perf Results |
| Type |
GHz |
Chips
Cores |
Threads |
Peak |
Base |
| T5120/T5220 |
UltraSPARC T2 |
1.4 |
1, 8 |
64 |
78.5 |
73.0 |
| HP DL380 G5 |
Intel X5365 |
3.0 |
1, 4 |
4 |
61.3 |
53.8 |
| IBM p 570 |
Power6 |
4.7 |
1, 2 |
4 |
60.9 |
53.2 |
| Fujitsu RX300 |
Intel X5355 |
2.66 |
1,4 |
4 |
52.8 |
50.5 |
SPECfp_rate2006
| System |
Processors |
Performance Results |
| Type |
GHz |
Chips, Cores |
Threads |
Peak |
Base |
| T5120/T5220 |
UltraSPARC T2 |
1.4 |
1, 8 |
64 |
62.3 |
57.9 |
| IBM p 570 |
Power6 |
4.7 |
1, 2 |
4 |
58.0 |
51.5 |
| HP DL380 G5 |
Intel X5365 |
3.0 |
1, 4 |
4 |
38.8 |
36.4 |
| Fujitsu RX300 |
Intel X5355 |
2.66 |
1, 4 |
4 |
37.5 |
36.2 |
Results as of 27 Sep 2007 from www.spec.org.
Benchmark Description
SPEC CPU2006 is made up of two suites of benchmarks, CFP2006 and
CINT2006. CFP2006 targets floating-point performance, while CINT2006
targets integer performance.
Each suite has two different measures. First is the CPU measure, which
is the performance on the suite as a single stream. This can be either
a single thread or automatic compiled parallel run. This measure is
further defined by base and optimized runs. Base uses the same compiler
flags for all kernels, where optimized is allowed to use different
compiler flags for each kernel. Results are compared against a baseline
system run that was standardized by SPEC.
The second measure is Rate. It is a measure of how many CPU measures
can be run at a time. Typically, it is run as n processes on n
processors. It shows how well the same job mix can run on a system
under some load. It also is run as a base and optimized set of
results.
Disclosure Statement:
SPEC, SPECint reg tm of Standard Performance Evaluation Corporation.
Sun result submitted to SPEC, other results from www.spec.org as of 9/27/07.
Sun SPARC Enterprise T5220/T5120 (UltraSPARC T2, 1 chip, 8 cores),
78.5 SPECint_rate2006, IBM p570 (POWER6, 1 chip, 2 cores), 60.9 SPECint_rate2006, HP DL380 G5 (X5365, 1 chip, 4 cores), 61.3 SPECint_rate2006,
Sun SPARC Enterprise T5220 (UltraSPARC T2, 1 chip, 8 cores),
62.3 SPECfp_rate2006.
SPEC, SPECfp reg tm of Standard Performance Evaluation Corporation.
Sun result submitted to SPEC, other results from www.spec.org as of 9/27/07.
Sun SPARC Enterprise T5220/T5120 (UltraSPARC T2, 1 chip, 8 cores),
62.3 SPECfp_rate2006.
IBM p570 (POWER6, 1 chip, 2 cores), 58.0 SPECfp_rate2006,
Sun SPARC Enterprise T5220 (UltraSPARC T2, 1 chip, 8 cores),
62.3 SPECfp_rate2006.
HP DL380 G5 (X5365, 1 chip, 4 cores), 38.8 SPECfp_rate2006.
System Configuration
| Results |
| Reference Date: |
|
Oct 09, 2007 |
| System: |
|
Sun SPARC Enterprise T5120/T5220 |
| Processor: |
|
Sun UltraSPARC T2, 1.4 GHz |
|
|
|
78.5 SPECint_rate2006 |
|
|
|
62.3 SPECfp_rate2006 |
| Software: |
|
Solaris 10, Sun Studio 12 Compiler |
Friday Oct 05, 2007
First:
"First of all, the math is really simple. 4.7 is greater than 1.4. IBM's POWER6 4.7 GHz chip is faster than Sun's 1.4 GHz UltraSPARC T1 chip. And second of all, the IBM System p 570 remains the #1 SPECjbb2005 2-core result (1)."
- Elisabeth Stahl, program manager of IBM performance marketing, 20 years experience, Jun07
Issues:
Did not compare system or chip performance but only quoted GHz of a chip.
Made a true statement about core count but ignored that that IBM core costs an order of magnitude more than Sun T1 on a per core basis
Next:
"Even more impressive, the processor bandwidth of the POWER6 chip – 300 gigabytes per second -- could download the entire iTunes catalog in about 60 seconds" - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss
Issues:
Added every bandwidth (L3 cache, address bandwidth?!?,...) in a chip,
even though peak memory bandwidth is limited to at least a 10th of that, delivered is a lot less.
stated "processor bandwidth", even though "delivered" system bandwidth would actually be required to move the data (not address ).
Next:
"IBM calculates that 30 SunFire v890s can be consolidated into a single rack of the new IBM machine, saving more than $100,000 per year on energy costs (3)." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/21580.wss
Issues:
used 2 year old sun result compared to power6 yet to be shipped as of may press release
implied V890, so that people think it is a current comparison, had to read in the footnotes that it was 1.5 GHz slower CPU. Sun has introduced 1.8GHz, and 2.1GHz since.
made a "conservative" comparisons by giving IBM another 15% in performance
claimed Sun at 20% util and IBM at 60% util to get bogus 3x factor
never showed exactly what power was drawn by a 4.7GHz, 64GB memory system,
at ??MHz DDR2 used in the comparison, etc.
...and many more.
Next:
"...And the good news is that about 40-70% of the
stuff we do in performance tuning actually ends up helping end users,"
-- Bruce Lindsay(IBM Fellow), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
"This is feasible in the TPC-C benchmark because there
are only five tables and only ten to fifteen columns in each table.
In a more realistic application, where there are many more queries
to be considered, the tables are typically much, much wider, in
the 80 to 100 column range; and there are dozens if not thousands
of tables. Then this kind of analysis(ed note: tuning) is no longer
practical." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
"The idea is to get the numbers by hook and by crook." -- Bruce Lindsay(IBM Fellow since '96), May 06, http://www.sigmod.org/sigmod/record/issues/0506/p71-column-winslet.pdf
The TPC-C benchmark is an industry standard for measuring the ability of a system to process complex online transactions and large volumes of business data. The TPC-C benchmark is unique in the way it exercises all components of a system, including processors, memory, networking, storage, operating system and database software, demonstrating total system performance in a way that many of the other benchmarks touted by some competitors do not. -- Bruce Lindsay(IBM Fellow since '96), July 25, 2006, http://www-03.ibm.com/solutions/sap/doc/content/news/pressrelease/1623288130.html
Issues:
This means that 30% to 60% of IBM's TPC-C tuning is useless for customers.
IBM clearly over-hyped TPC-C, just 2-3 months after they publicly showed all of its problems and "optimizations" they used.
Next:
"Significantly, the high utilization rate of the System z9 mainframes -- systems can and do operate at 80 to 100 percent utilization -- combined with its ability to "virtualize" workloads, can enable a single mainframe processor to perform far more work than a single x86 processor running Microsoft Windows. The latter may run as low as 5 percent utilization." - IBM Press Release http://www-03.ibm.com/press/us/en/pressrelease/19577.wss
Issues:
very convenient to use different work for mainframe and for its competitor.
"mainframes do operate at 80-100%", "x86 processor running Microsoft Windows. The latter may run as low as 5%". So it is a valid but totally useless statement.
"do" and "may" mean very different things
An equally invalid statement: x86 do operate at 80-100% and
mainframes may run as low as 5%.
Moral: Be VERY VERY CAREFUL when you read big blue.
Disclosure statement:
TPC-C is a trademark of Transaction Processing Performance Council (TPC). More info www.tpc.org.
Thursday Aug 23, 2007
In the last posting we showed Oracle Database with SAP-SD benchmarks all
running on a Sun Fire T2000. As Sun has been saying since Day one of CMT.
Major databases are perfectly matched for UltraSPARC T1. By the way Sun
has also used Open source databases on benchmarks as well.
We have lots of customers deploying RDBMS on UltraSPARC T1 and planning
on UltraSPARC T2 servers. It really works well even though competitors
and doubters want to try to say it is special purpose, sorry it isn't.
Here is an opinion:
"Now Sun's T2 is out and it's pretty much the world beater they promised -
30% faster on SPEC throughput than IBM's 4.7 Ghz Dual core Power6 and,
more significantly, one third the cost and somewhere between two and three
times the throughput of the Itanium. ... anyone still buying HP-UX and
Itanium after Rock comes out will be doing it because they hate Sun and are
quietly hoping for a miracle, just as DEC's partisans (and HP's own MPE
customer base) did before them." -- zdnet's Paul Murphy
Source: "A Dumb prediction: IBM will Buy HP's Unix Customers," By Paul Murphy, zdnet, 08/17/07,
http://blogs.zdnet.com/Murphy/?p=941
Thursday Aug 23, 2007
The SPARC Enterprise Model T2000 | Sun Fire Model T2000 is the performance leader in Two-Tier SAP-SD Standard Application Benchmarks on single processor systems as of August 22nd, 2007. This result used the Oracle database on the UltraSPARC T1. Again as Sun has always maintained the UltraSPARC T1 is good at database-tier, application tier, and web tier!
- Sun Fire Model T2000 supported 1100 SD Benchmark Users, 5530 SAPS, using Oracle 10g is the fastest single-processor systems.
- Sun Fire Model T2000 beats a 2-chip dual-core Itanium2-based HP Integrity rx2660.
- Sun Fire Model T2000 beats a 2-chip dual-core Opteron-based HP ProLiant DL365.
- Sun Fire Model T2000 beats a 2-chip dual-core Xeon-based Fujitsu BFi20 S2 (Unicode).
- The Fujitsu BX620 S4 that uses two-chip 3GHz Xeon Quad-cores is only 1.8x faster than a single chip Sun Fire Model T2000 using UltraSPARC T1.
- The IBM p570 that uses two-chip 4.7GHz POWER6 is only 1.8x faster than a single chip Sun Fire Model T2000 using UltraSPARC T1.
- The just-announced UltraSPARC T2 has twice the thread count of the
UltraSPARC T1.
SAP-SD 2-Tier Performance, Benchmark Users (bigger is better)
| Sys |
Users |
# / GHz / Type |
Mem |
OS |
DB |
LI/Hr |
SAPS |
BM rev |
Date |
| IBM p570 |
2035 |
two 4.7 POWER6+ DC |
32 GB |
AIX 5L 5.3 |
Oracle 10g |
203,670 |
10,180 |
6.0 |
5/21/07 |
| Fujitsu BX620 S4 |
1940 |
two 3.0 Xeon QC |
32 GB |
Windows Srvr 2003 EE |
SQL Server 2005 |
194,000 |
9,700 |
6.0 |
8/13/07 |
| Sun Fire T2000 |
1100 |
one 1.4 US T1 |
64 GB |
Solaris 10 |
Oracle 10g |
110,670 |
5,530 |
6.0 |
8/22/07 |
| HP Integrity rx2660 |
1090 |
two 1.6 Itan2 DC |
32 GB |
HP-UX 11iV3 |
DB2 9 |
109,670 |
5,480 |
6.0 |
3/20/07 |
| HP ProLiant DL365 |
1083 |
two 2.8 Opt DC |
32 GB |
Windows Srvr 2003 EE |
SQL Srvr 2005 |
108,670 |
5,430 |
6.0 |
2/9/07 |
| Fujitsu BFi20 S2 Unicode |
1020 |
two 3 Xeon 5160 DC |
16 GB |
Solaris 10 |
Oracle 10g |
102,330 |
5,120 |
6.0 |
5/4/07 |
| IBM p550 |
1000 |
four 1.9 POWER5+ DC |
32 GB |
SuSE Linux ES9 |
DB2 UDB 8.2.2 |
100,330 |
5,020 |
5.0 |
10/04/05 |
| Sun Fire T2000 |
950 |
one 1.2 US T1 |
32 GB |
Solaris 10 |
MaxDB 7.5 |
95,670 |
4,780 |
5.0 |
11/17/05 |
| IBM x3250 |
850 |
one 2.13 Xeon |
8 GB |
Windows SrVr 2003 EE |
DB2 9 |
88,000 |
4,400 |
6.0 |
5/11/07 |
Complete benchmark results may be found at the SAP benchmark website http://www.sap.com/benchmark.
Benchmark Description
The SAP Standard Application SD (Sales and Distribution) Benchmark is a
two-tier ERP business test that is indicative of full business workloads
of complete order processing and invoice processing, and demonstrates the
ability to run both the application and database software on a single
system. The SAP Standard Application SD Benchmark represents the critical
tasks performed in real-world ERP business environments.
SAP is one of the premier world-wide ERP application providers, and maintains
a suite of benchmark tests to demonstrate the performance of competetive
systems on the various SAP products.
SAP has specified that the Benchmark Users metric is the only metric to be used
for public comparisons.
However, Benchmark Users can be traded off with response time in performance
tuning, and so comparing Line Items per Hour or SAPS
may be a different way to compare the actual power of systems.
Funny that Sun compares against current IBM results, IBM bloggers
decide to do funny comparisons on a different SAP benchmark, but
compared their latest system to a 16-month old result on a US-IV system
that is 2 processor GHz upgrades behind. I guess that is one way to win...
Disclosure Statement:
Two-tier SAP Standard Sales and Distribution (SD) standard SAP ERP 2004/2005 application benchmark:
SPARC Enterprise Model T2000 | Sun Fire T2000 (1-way, 1 proc, 8 cores, 32 threads) 1 x 1.4 GHz
UltraSPARC T1, 64GB memory, 1100 SD Benchmark users, 1.91 sec avg response time,
Cert#2007051, Oracle 10g, Solaris 10;
Sun Fire T2000 (1-way, 1 proc, 8 cores, 32 threads) 1 x 1.2 GHz
UltraSPARC T1, 32GB memory, 950 SD Benchmark users, 1.91 sec avg response time,
Cert#2005047., MaxDB 7.5 database, Solaris 10;
Fujitsu Siemens Computers PRIMERGY MOdel BX620 S4
(2-way, 2 procs, 8 cores, 8 threads), 2 x 3.0 GHz Quad-Core Intel Xeon,
32 GB memory, 1940 SD Benchmark users, 1.99 sec avg response time,
Cert#2007049, SQL Server 2005, Windows Server 2003 Enterprise Edition;
HP ProLiant DL365 (2-way, 2 procs, 4 cores, 4 threads)
2 x 2.8 GHz Opteron, 32GB memory, 1083 SAP SD Benchmark users,
1.98 sec avg response time, Cert#2007006, SQL Server 2005,
Windows Server 2003 Enterprise Edition;
HP Integrity rx2660 (2-way, 2 procs, 4 cores, 8 threads)
2 x 1.6 GHz Itanium, 32GB memory, 1090 SAP SD Benchmark users,
1.93 sec avg response time, Cert#2007016, DB2 9, HP-UX 11iV3;
IBM System p 570 (2-way, 2 procs, 4 cores, 8 threads) 2 x 4.7 GHz
POWER6+, 32GB memory, 2035 SD Benchmark users, 1.99s avg resp time, Cert#2007037, Oracle 10g, AIX 5L Version 5.3;
Fujitsu Siemens Computers PRIMERGY Model BFi20 S2 (2-way, 2 procs, 4 cores, 4 threads)
2 x 3GHz Intel Xeon 5160 dual-core, 16GB memory,(Unicode) 1020 SAP SD Benchmark users,
1.94 sec avg response time, Cert#2007031, Oracle 10g, Solaris 10;
IBM System x3250 (1-way, 1 proc, 4 cores, 4 threads) 1 x 2.13 GHz
Xeon, 8GB memory, 850 SD Benchmark users, 1.59s avg resp time, Cert#2007036,
DB2 9, Windows Server 2003 Enterprise Edition;
IBM System eServer p5 550 (4-way, 4 procs, 4 cores, 8 threads) 4 x 1.9 GHz
POWER5+, 32GB memory, 1000 SD Benchmark users, 1.97s avg resp time, Cert#2005040,
IBM DB2 Universal Database 8.2.2, SuSE Linux Enterprise Server 9;
SAP, R/3, mySAP reg TM of SAP AG in Germany and other countries.
More info http://www.sap.com/benchmark.
| Certified Results |
|
Performance: |
|
1100 benchmark users |
|
Server: |
|
Sun Fire |
|
Processors: |
|
1 1.4 GHz UltraSPARC T1 |
|
Memory: |
|
64 GB |
|
Operating system: |
|
Solaris 10 |
|
Database S/W: |
|
Oracle 10g |
|
SAP S/W: |
|
SAP ECC 6.0 |
|
SAP Certification: |
|
2007051 |
|
Storage: |
|
Sun StorEdge 6020 |
Thursday Aug 09, 2007
Postscript:
Be careful when comparing performance results, as an example look at
a comment in yesterday's
"Can I use 64 threads in a chip?" posting. At
least this comment pointed out that you can use 4-8 threads in 2 chip Intel-based systems, but it was really trying to
be a stab at UltraSPARC Performance. Here was the comment:
One really needs to look at the complete data on those .pdf's
to make a fair comparison (also in the disclosure statement
below).
First: The T2000 SAP-SD used a 1.2GHz UltraSPARC T1, Sun now ships faster 1.4GHz UltraSPARC T1, and has announced 1.4GHz UltraSPARC T2. The 1.4GHz T2 has double the threads of that 1.4GHz (double the computational power).
Second: The T2000 SAP-SD result was submitted in Dec 2005, at that time it
was near the performance of the expensive 4-way POWER5 IBM p550.
Third: The 2-chip Dual-core Xeon SAP-SD result above was
submitted 18 months after the T2000 SAP-SD result.
Fourth: Different versions of the benchmark. The 2-chip
Dual-core Xeon was run with ECC 6.0 (not SAP 5.0). The a newer version
of the benchmark takes more computational work to produce the same results.
Dual-core SAP-SD result was also run with Solaris 10 on Xeon, how cool is that!
Fifth: The 2-chip quad-core Xeon SAP-SD result above was
submitted 19 months after the T2000 SAP-SD result.
Sixth: The Sun result used open-source MySQL MaxDB database,
how cool is that! The Xeon results used Oracle or MicroSoft SQL Server.
postscript:
Sun latter used Oracle, others suggested US T1 has some sort of silly database limitation - NOT TRUE!
You'll see more results soon.
Triffids, as a reminder if you work for a partner company of SAP you must
put the following disclosures when you post results. If you are not
they you don't need to put this in, but as you can see the data in
it would have allowed you to make a better comparison of systems.
Don't worry I'm not asking you to identify yourself at all.
Disclosure Statement:
Two-tier SAP ECC 5.0 Standard Sales and Distribution (SD) benchmark Sun Fire T2000 (1-way, 1 proc, 8 cores, 32 threads) 1x 1.2 GHz UltraSPARC T1, 32 GB mem, 950 SD benchmark users, 1.91 sec avg response time, Cert#2005047., MaxDB 7.5 database, Solaris 10; Two-tier SAP ECC 5.0 Standard Sales and Distribution (SD) benchmark IBM System eServer p5 550 (4-way, 4 procs, 4 cores, 8 threads) 4x 1.9 GHz POWER5+, 32GB mem, 1,000 SD benchmark users, 1.97s avg resp time, Cert#2005040, IBM DB2 Universal Database 8.2.2, SuSE Linux Enterprise Server 9;
Two-tier SAP ECC 6.0 Standard Sales and Distribution (SD) benchmark Fujitsu Siemens Computers PRIMERGY Model BFi20 S2 (2 procs, 4 cores, 4 threads) 2x Intel Xeon 5160, 3.0 GHz, 16GB mem, 1,020 SD benchmark users, 1.94s avg resp time, Cert#2007031, Oracle 10g, Solaris 10;
Two-tier SAP ECC 6.0 Standard Sales and Distribution (SD) benchmark Fujitsu Siemens Computers PRIMERGY Model TX300 S3 (2 procs, 8 cores, 8 threads) 4x Quad-Core Intel Xeon Processor X5355 2.66 GHz, 32GB mem, 1865 SD benchmark users, 1.99s avg resp time, Cert#2007025, SQL Server 2005, Windows Server 2003 Enterprise Edition; SAP, R/3, mySAP reg TM of SAP AG in Germany and other countries. More info www.sap.com/benchmark.
I edited in:
2 processors into Quad-Core Intel Xeon Processor X5355 2.66 GHz
...and..
32 threads to the Sun Fire T2000, 1 processor / 8 cores
...in order to make the comparisons more consistent.
Friday Aug 03, 2007
News is starting to trickle out on Niagara2. Sun continues to do revolutionary
things that will have huge effect on how we compute. For what has leaked so
far see: http://www.informationweek.com/blog/main/archives/2007/08/suns_niagara_2.html
At OpenSPARC.net you will
find a in-depth presentation of the upcoming Niagara 2 processor: "An 8-core, 64-thread, 64-bit, power efficient SPARC SoC (Niagara2)".
The presentation was held at the International Solid State Circuits Conference 2007.
So the question becomes how do we now judge performance of chips with different
architectures? One thing that hasn't changed is system performance and system cost. Look at what
a system costs compare it against other systems in the same price range.
The computer industry can no longer determine performance based on old dinosaur habits. Remember when vendors determined all performance by GHz. Then remember back
in '04 when Intel quite naming processors by GHz... they realized that
GHz is only part of the overall equation for processor performance.
What does that equation look like in very rough terms?
performance = GHz * number-of-cores * number-of-threads-per-core * efficiency-of-design * other * software efficiency
...so if you compare performance between different designs by any of these factors
alone (example: perf/core, perf/Ghz, perf/thread) you are being very misleading!
Case in point, IBM loves to talk about perf/core, but they avoid telling everyone
that they lead the industry, by several factors in price/core!
The IT jungle says: "340,000. It costs $17,700 to activate a core, so pushing it to eight cores costs another $141,600." So on a per core basis that means POWER6 costs
$60,000/core. Actually I've estimated that in a system with memory an IBM power6
system costs $65K/core to $130K/core! Any customer needs to do their own pricing.
So comparing systems of dramatically different cost-per-core on a per-core basis
NO LONGER MAKES ANY SENSE! No wonder IBM wants to stilt comparisons and avoid
pricing! The only other clues on list price on power6 is
here.
Friday Jun 15, 2007
IBM statement no longer true. IBM press release proudly stated 3 weeks ago
that "IBM Unleashes World's Fastest Chip in Powerful New Computer". Now the 1.4 GHz UltraSPARC T1 chip is 10% faster than the IBM POWER6 chip.
The IBM p570 Power6 4.7GHz 1 chip/2core/4threads had a score of 88,089
SPECjbb2005 bops. It is also very clear that you can not compare systems
performance on a per core basis. You'd have to do it by system price, go ahead
and price out a 4RU p570 and a 2RU T2000, or check the wattage. Also as you
can see IBM's faster GHz and fewer very expensive cores end up delivering
slower system performance.
SPECjbb2005 (ordered by perf, bops : SPECjbb2005 Business Operations per Second, bigger is better)
|
System
|
Date
|
Processors
|
Performance
|
|
(Chips,
Cores,
Threads)
|
GHz /
Type
|
SPECjbb
2005
bops
|
JVMs
|
SPECjbb
2005
bops/JVM
|
|
Dell PowerEdge 860
|
1/07
|
(1, 4, 4)
|
2.4 Xeon
|
112092
|
1
|
112092
|
|
Sun Blade T6300
|
6/07
|
(1, 8, 32)
|
1.4 US-T1
|
96523
|
4
|
24121
|
|
IBM p570
|
6/07
|
(1, 2, 4)
|
4.7 POWER6
|
88089
|
1
|
88089
|
|
Fujitsu TX150
|
6/07
|
(1, 2, 2)
|
2.66 Xeon
|
70324
|
1
|
70324
|
|
Dell PowerEdge 840
|
10/06
|
(1, 2, 2)
|
2.66 Xeon
|
52002
|
1
|
52002
|
|
Fujitsu RX100
|
10/06
|
(1, 2, 2)
|
2.4 Xeon
|
49892
|
1
|
49892
|
Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.
Benchmark Description
SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the
database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).
Disclosure Statement:
SPECjbb2005
Sun Fire T6300 (1 chip, 8 cores) 96523 SPECjbb2005 bops, 24131 SPECjbb2005 bops/JVM,
IBM p570 (1 chip, 2 cores) 88089 SPECjbb2005 bops, 88089 SPECjbb2005 bops/JVM,
Fujitsu TX150 (1 chip, 2 cores) 70324 SPECjbb2005 bops, 70324 SPECjbb2005 bops/JVM,
Dell PowerEdge 860 (1 chip, 4 cores) 112092 SPECjbb2005 bops, 112092 SPECjbb2005 bops/JVM,
Dell PowerEdge 840 (1 chip, 2 cores) 52052 SPECjbb2005 bops, 52052 SPECjbb2005 bops/JVM,
Fujitsu RX100 (1 chip, 2 cores) 49892 SPECjbb2005 bops, 49892 SPECjbb2005 bops/JVM.
IBM p570 Power6 (1 chip, 2 cores, 4 threads) 88089 SPECjbb2005 bops, 88089 SPECjbb2005 bops/JVM.
SPEC, SPECjbb reg tm of Standard Performance Evaluation
Corporation. Results as of 6/14/2007 on http://www.spec.org.
|
Results
|
| |
Sun Blade T6300: |
|
96523 SPECjbb2005 bops |
| |
|
|
24131 SPECjbb2005 bops/JVM |
| |
Reference Date: |
|
June 6, 2007 |
| Systems: |
|
Sun Blade T6300, 32GB |
| Total Number Processors: |
|
1 |
| Processor/GHz of Server: |
|
US-T1 1.4 GHz |
| Operating System: |
|
Solaris 10 8/07 |
| JVM: |
|
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_02 |
See Also
Sun Press Release
Wednesday Feb 07, 2007
CNET news has some press about Sun's next CMT technology.
"We have silicon back (from manufacturer Texas Instruments) and
Victoria Falls systems running in the labs at Sun at full speed,
full performance," John Fowler, executive vice president of systems,
said via Webcast Tuesday. "In the first half of calendar 2008, we
will introduce products based on Victoria Falls."
You can read more at:
http://news.com.com/Sun+dual-Niagara+servers+due+in+2008/2100-1010_3-6156883.html
...all I can say is fast to production, and fast in the lab!
Wednesday Jan 31, 2007
Sun Blueprints are technical best practices, derived from the real-world experience
of Sun experts. The whitepapers are available for free! downloads. You can
buy blueprint books on other topics.
A new one of note is the updated: "Developing and Tuning Applications on
UltraSPARC T1 Chip Multithreading Systems" written by Denis Sheehan. For more
info see:
http://www.sun.com/blueprints/0107/819-5144.html
Wednesday Dec 13, 2006
Sun Fire T2000 Sun Java Systems Application Server World Record Two-Node Price/Performance on SPECjAppserver2004. The HP Itanium2-based server is over
3.4x times most costly than the Sun Fire T2000 Server using Sun Java EE application. Sun's solutions is the lowest overall application
tier deployment and maintenance cost. The Sun Fire T2000 server also
requires only 2RU vs. 4RU for the more-costly HP Itanium 2 server.
Sun's dual-node configuration delivered throughput in excess of 26,000
operations per minute with absolutely no Java EE Application Server licensing
constraints. This submission is fully Java EE 5 compatible and fully
supported by Sun Microsystems.
Sun Microsystems is the only vendor to provide robust performance on the SPECjAppServer benchmark using a Java EE 5 certified application server.
One Sun Fire T2000 server equipped with 1 UltraSPARC T1 processor (8 cores)
at 1.2 GHz running Sun Java System Application Server 9.0 UR 1 PE and a
Sun Fire T2000 equipped with 1 UltraSPARC T1 processor (6 cores) at
1.0 GHz running Oracle 10g Database Standard Edition delivered a
result of 521.42 SPECjAppServer2004
JOPS@Standard for the SPECjAppServer2004 benchmark for
the best application tier price/performance result for a two-node server.
Competitive Landscape
SPECjAppServer2004 Performance Chart - Ordered by Application Tier Cost per
SPECjAppServer2004 JOPS@Standard (lower is better),
CPJ column is the Cost Per SPECjAppServer2004 JOPS@Standard,
Score column is the
SPECjAppServer2004 JOPS@Standard result,
Cost column is the application cost of the hardware and software.
| Submitter |
CPJ |
Score |
Cost |
J2EE Server |
DB Server |
| Sun |
$51.81 |
521.42 |
$27,016.95 |
1 x Sun Fire T2000
8 cores, 1 chip (8 cores/chip) @1.2GHz
Sun Java System AS 9.0 PE |
1 x Sun Fire T2000
6 cores, 1 chip (6 cores/chip) @1.0GHz
Oracle 10g 10.2.0.2 |
| Sun/BEA |
$76.37 |
615.64 |
$47,016.95 |
1 x Sun Fire T2000
8 cores, 1 chip (8 cores/chip) @1.2GHz
WebLogic |
1 x Sun Fire V490
8 cores, 4 chips (2 cores/chip) @1.5GHz
Oracle 10g 10.1.0.4 |
| Sun |
$122.13 |
616.22 |
$75,234.92 |
1 x Sun Fire T2000
8 cores, 1 chip (8 cores/chip) @1.2GHz
WebSphere Application Server Network Deployment, Version 6.1 |
1 x Sun Fire X4200
4 cores, 2 chip (2 cores/chip) @2.2GHz
DB2 Universal Database v8.2.4 |
| HP |
$298.20 |
471.28 |
$140,537.88 |
1 x rx4640
4 cores, 4 chips (1 core/chip) Itanium 2 @1.6GHz
BEA WebLogic 9.0 |
1 x rx4640
4 cores, 4 chips (1 core/chip) Itanium 2 @1.6GHz
Oracle 10g 10.1.0.4 |
SPECjAppServer2004 Results Page
Benchmark Description
SPECjAppServer2004 (Java Application Server) is a multi-tier benchmark for
measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based
application servers. SPECjAppServer2004 is an end-to-end application which
exercises all major J2EE technologies implemented by compliant application
servers as follows:
-
The web container, including servlets and JSPs
-
The EJB container
-
EJB2.0 Container Managed Persistence
-
JMS and Message Driven Beans
-
Transaction management
-
Database connectivity
Moreover, SPECjAppServer2004 also heavily exercises all parts of the underlying
infrastructure that make up the application environment, including hardware,
JVM software, database software, JDBC drivers, and the system network.
The primary metric of the SPECjAppServer2004 benchmark is jAppServer Operations
Per Second (JOPS) which is calculated by adding the metrics of the
Dealership Management Application in the Dealer Domain and the Manufacturing
Application in the Manufacturing Domain. There is NO price/performance
metric in this benchmark.
Disclosure Statement:
SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of Dec 6, 2005.
Sun Fire T2000 (8 cores, 1 chip) 521.42 SPECjAppServer2004 JOPS@Standard.
HP rx4640 (4 cores, 4 chips) 471.28 SPECjAppServer2004 JOPS@Standard.
Sun HW+SW application tier cost = $27,016.95, appl cost per JOP = $51.81.
HP HW+SW application tier cost = $140,537.88, appl cost per JOP = $298.20
HP Bill of Material from
http://www.spec.org/jAppServer2004/results/res2005q3/jAppServer2004-20050913-00016.html.
BEA pricing from http://www.awaretechnologies.com/BEA/index.html.
IBM pricing from http://www.cdw.com/
System pricing dated 12/06/06
HP rx4640 server specifications 10/19/05 from http://www.hp.com/products1/servers/integrity/entry_level/rx4640/index.html
| Certified Results |
|
521.42 SPECjAppServer2004
JOPS@Standard |
| Reference Date: |
|
Dec 6, 2006 |
| Systems: |
|
1 x Sun Fire T2000, UltraSPARC T1 1.2 GHz (8 cores, 1 proc), 32 GB Memory |
|
|
1 x Sun Fire T2000, UltraSPARC T1 1.0 GHz (6 cores, 1 proc), 8GB Memory |
| Operating System: |
|
Solaris 10 6/06 (same for both) |
| Software: |
|
Sun Java System Application Server 9.0 Platform Edition UR1 Patch 1 |
|
|
Oracle 10g Database Enterprise Edition v10.2.0.2 |
| JVM: |
|
J2SE 6.0 |
|
|
|
|
Oh boy you make me laugh with all the crap that yo...