BM Seer Facts & Questions from an Anonymous Sun Source

They tried to make IBM show perf, IBM said no, no, no...

Thursday Nov 15, 2007

no STREAM Bandwidth p570 power6 results
no TPC-H single-system Power6 results
no SPECweb2005 Power6 results
<16-Nov: line deleted due to correction, see note below> no SPECjAppServer2004 Power6 as database server
No SPECmail Power6 results
no SPECompL2001 Power6 results
no Lotus Domino R6iNotes Power6 results
no cryptography performance on Power6 results
no power-performance on any published benchmarks
no consolidation overheads on Power6, (Solaris Zones=~0%)
etc.

What does the industry get - the most expensive core now built and a bogus press release chip bandwidth number: http://blogs.sun.com/bmseer/entry/finally_proof_power6_has_meaningless

I don't even think the Power6 has a single-chip world record on anything that everyone publishes?

...without even going into HPC ISV results (err, lack thereof)
no Fluent Power6 results
no StarCD Power6 results
no LSDyna Power6 results
no MSC.Nastran Power6 results
etc.

If it was as fast and IBM claims in marketing wouldn't IBM just do the ones they haven't done? Also why can't they beat the UltraSPARC T2 world records on the ones they publish?

"You won't know, know, know..." -Amy Winehouse

SPEC, SPECint, SPECweb, SPECfp, SPECjAppServer, SPECmail, SPEComp, reg tm of Standard Performance Evaluation Corporation. TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info http://www.tpc.org. Lotus Domino more info: www.notesbench.org

Postscript Note:
The original posting had an error. IBM did publish a SPECjAppserver2004 benchmark on Power6. The comparison is: One Sun T5220 server (single UltraSPARC T2 chip) demonstrated 67% better performance over the IBM result of 1197.51 SPECjAppServer2004 JOPS@Standard which used 4-core IBM p570 with 4.7GHz POWER6 processors. The Sun SPARC Enterprise T5220 used as Application server has 3.8x better power-performance and has 7.3x better SWaP than the IBM p570 power6. Enterprise T5120 used as database servrer has 3.4 better power-performance and has 13.5x better SWaP as the IBM p550.

Disclosure Statement:

SPECjAppServer2004 1 Sun SPARC Enterprise T5220 (8 cores, 1 chip) and 1 Sun SPARC Enterprise T5120 (8 cores, 1 chip) 2000.92 SPECjAppServer2004 JOPS@Standard. 1 IBM p570 (4 cores, 2 chips) and 1 IBM p550 (4 cores, 2 chips) 1197.51 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 10/10/2007.

[4] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

IBM wrong at top of their voice and IBM JS22 power6 blade

Thursday Nov 08, 2007

Even IBM expert blogs continue to trumpet bad data. In the previous posting I pointed out the problems, please see BM Seer's "IBM JS22 Power6 blades not the performance you think"

While, I realize that over-excited IBM marketing types may tend to overstate and make bad claims, but IBM's performance blog expert too?

    IBM's Stahl writes: Equally compelling is the analysis that one rack of IBM's new POWER6 processor-based blades is so powerful when virtualized that it can replace many non-virtualized racks of Sun's latest V490 servers, potentially saving a ton in energy costs.
Interesting that in a previous posting she complained of an HP comparison because they used their own test. In the IBM JS22 comparison, IBM comes up with their own baseless 'Sun is three times worse' derating factor.

I guess IBM uses something learned from Lucy in Peanuts cartoon: "If you can't be right, .... be wrong at the top of your voice!"

So in how many more blogs, ads, and other marketing material will we see this comparison? My guess is LOTS at the top of their voice!

Like this post? del.icio.us | furl | slashdot | technorati | digg

IBM JS22 Power6 blades not the performance you think

Thursday Nov 08, 2007

Why do I attack IBM alot, it is not because they are a competitor (I don't attack all competitors), it is because IBM continues to pull such funny games to confuse the marketplace.

Here is a prime example of an un-fair/stilted comparisons from IBM press release:

    Calculations show that one rack of IBM's new POWER6 processor-based blades is so powerful when virtualized that it can replace 23 non-virtualized racks of Sun's latest V490 servers, potentially saving more than $200,000 per year in energy costs alone. (3)

Why didn't IBM compare against the Sun Blade T6320 (UltraSPARC T2) blades? IBM would have lost if they properly compared to Sun.

IBM gamed again by comparing new servers to older Sun servers. IBM claimed one rack of JS22 BladeCenter Hs to replace 180 V490s. IBM based its JS22 claim by 'pulling a a *3X* utilization rate" out of the air' when compared to the V490. The JS22 utilization was 60%, while the V490 was 20% utilized-- a report from Alinean Consulting was cited as the source for the utilization comparisons(note:I guess you can pay for anything). This was the same fake 3x difference in utilization that IBM tryed to pull in the POWER6 announcement that BM Seer shot down in "IBM rewrites history, OK footnotes to clearly show bogus calculations"

As a reminder as always with IBM read written material very carefully: http://blogs.sun.com/bmseer/entry/careful_reading_shows_a_lot

Here is the IBM footnote:

    (3) The number of IBM BladeCenter JS22 servers required to replace 180 Sun Fire V490 was calculated based on SPECint_rate2006 results. The V490 SPECint_rate2006 result is for a 2.1GHz system with 4 chips and 2 cores per chip. It has a result of 78.0. The V490 result can be found at www.spec.org. It is current as of October 23, 2007. The JS22 result for the same benchmark is for a 4.0GHz system with 2 chips and 2 cores per chip. It has a result of 84.7. That result was submitted on November 6, 2007. It will also be posted on www.spec.org. The cumulative capacity of these servers is estimated to be the SPECint_rate2006 result for one server multiplied by the number of servers. A virtualization factor of 3X was applied to the JS22 virtualization scenario using utilizations derived from studies conducted by Alinean available at http://www-935.ibm.com/services/us/cio/optimize/opt_wp_ibm_systemp.pdf. That is the utilization rate for the non-virtualized V490 is estimated to be 20% and the utilization rate for the virtualized JS22 is estimated to be 60%. Using these assumptions, the cumulative capacity of the 56 JS22 servers at 60% is greater than the cumulative capacity of the 180 V490 servers at 20% utilization.

Like this post? del.icio.us | furl | slashdot | technorati | digg

gcc, SPEC CPU2006, & Sun SPARC Enterprise T5220

Friday Nov 02, 2007

Sun has released benchmarks results on SPEC CPU with GCCfss. GCCfss is a GCC compatible frontend with Sun Studio backend. If you have codes developed with GCC you can now just use it to run really fast on UltraSPARC T2, with all kinds of great optimizations.

For more on GCCfss see: http://cooltools.sunsource.net/gcc/

The Sun SPARC Enterprise T5220 server, running at 1.4 GHz, delivered a result 78.0 SPECint_rate2006 which is slightly lower (1%) when compared with the full Sun Studio 12 compiler.

The Sun SPARC Enterprise T5220 using the GCC for SPARC Systems (gccfss) compiler topped all competitor's single chip results including the 4.7 GHZ POWER6 result from IBM by over 28% which used a proprietary compiler.

The gccfss compiler allows one to use the optimal Sun SPARC optimization tools along with the popular gcc coding conventions and deliver performance that has not been possible before without time consuming code changes.

SPEC CPU2006 Performance Charts: bigger is better, selected recent results

SPECint_rate2006

System Processors Performance Results
Type GHz Chips Cores Threads Peak Base
T5120/T5220 UltraSPARC T2 1.4 1 8 64 78.5 73.0
T5220 (gccfss) UltraSPARC T2 1.4 1 8 64 78.0 71.6
HP DL360 G5 Intel X5365 3.0 1 4 4 61.3 53.8
IBM p 570 Power6 4.7 1 2 4 60.9 53.2
Fujitsu RX300 Intel X5355 2.66 1 4 4 52.8 50.5

Results as of 30 Oct 2007 from www.spec.org.

Benchmark Description

SPEC CPU2006 is made up of two suites of benchmarks, CFP2006 and CINT2006. CFP2006 targets floating-point performance, while CINT2006 targets integer performance.

Each suite has two different measures. First is the CPU measure, which is the performance on the suite as a single stream. This can be either a single thread or automatic compiled parallel run. This measure is further defined by base and optimized runs. Base uses the same compiler flags for all kernels, where optimized is allowed to use different compiler flags for each kernel. Results are compared against a baseline system run that was standardized by SPEC.

The second measure is Rate. It is a measure of how many CPU measures can be run at a time. Typically, it is run as n processes on n processors. It shows how well the same job mix can run on a system under some load. It also is run as a base and optimized set of results.

Disclosure Statement:

SPEC, SPECint reg tm of Standard Performance Evaluation Corporation. Sun result submitted to SPEC, other results from www.spec.org as of 10/30/07. Sun SPARC Enterprise T5220 gccfss (UltraSPARC T2, 1 chip, 8 cores), 78.0 SPECint_rate2006; IBM p570 (POWER6, 1 chip, 2 cores), 60.9 SPECint_rate2006; HP DL360 G5 (Intel X5365 1chip 4-core), 61.3 SPECint_rate2006; Fujitsu RX300 (Intel X5355, 1-chip, 4-core) 52.8 SPECint_rate2006; Sun SPARC Enterprise T5220 (UltraSPARC T2, 1 chip, 8 cores), 78.5 SPECint_rate2006.

Results Summary

Results
Reference Date: Oct 30, 2007
System: Sun SPARC Enterprise T5220
Processor: Sun UltraSPARC T2, 1.4 GHz
  78.0 SPECint_rate2006
Software: Solaris 10, Sun Studio 12 Compiler gccfss

[2] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

Outstanding TPC-H results: Sun & Paraccel

Wednesday Oct 31, 2007

Yesterday I posted several Sun Fire X4100 world record performance & $/performance results on the TPC-H benchmarks. Sun leads the pack with Paraccel's innovative software and even the software pricing model. I wanted to try to give even more insights.

ParAccel has a very efficient data compression techniques so using the storage associated with Sun's hardware design, the Sun TPC-H cluster submissions did not need ANY external storage. The data could be entirely contained on the internal drives of each node. Once again, Sun is the ONLY vendor that has been able to do so. Other vendors, using traditional RDBMS, are forced to submit results with a disproportionate number of spindles, in order to achieve competitive performance, even at small scale factors. As a consequence, the Sun ParAccel cluster solution greatly reduces costs and requirements for space, power/cooling, maintenance and complex hardware administration.

Sun and ParAccel are in the remarkably unique position of having simultaneously the best performance and price/performance by far at multiple TPC-H scale factors. As I said yesterday, innovators seek out and find innovators.

The ParAccel Analytic Database is an innovative RDBMS designed from the ground up as Decision Support and Business Intelligence database.

ParAccel has a parallel cluster architecture, "shared-nothing", with node redundancy, which relies on a column-based table structure and heavy data compression. Compression provides for a significantly smaller data footprint, higher speed in data processing and transmission. The significant reduction in the size of the data allows for even large databases to be fully contained in the memory of a cluster of multiple systems. The database can also operate by accessing data on permanent storage, although with a tradeoff in performance.

ParAccel doesn't make use of complex index structures, further reducing database size, administration complexity and simplifying data updates.

Data partitioning and load balancing is provided automatically by the ParAccel product and doesn't require manual DB admin intervention, either to optimize the data layout or to create additional indexes and structures, traditionally needed to speedup query performance.

Results detailed at:

http://blogs.sun.com/bmseer/entry/innovation_setting_world_records_in

http://blogs.sun.com/bmseer/entry/tpc_h_100gb_sun_fire

http://blogs.sun.com/bmseer/entry/sun_paraccel_in_tpc_h

Disclosure Statement:

TPC-H @1000GB Sun Fire X4100 cluster 315,842.9 QphH@1000GB, $4.57/QphH@1000GB, avail 10/29/07; TPC-H @300GB Sun Fire X4100 cluster 198,578.1 QphH@300GB, $3.15/QphH@300GB, avail 10/29/07; TPC-H @100GB Sun Fire X4100 cluster 98,857.0 QphH@100GB, $2.65/QphH@100GB, avail 10/29/07; TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info http://www.tpc.org.

Like this post? del.icio.us | furl | slashdot | technorati | digg

Sun/Paraccel in TPC-H 300GB World Records

Tuesday Oct 30, 2007

This benchmark result demonstrates that a shared-nothing cluster of Sun Fire X4100s achieves the best performance and, simulatenously, the best price/performance among all TPC-H@300GB submissions. Each of the 30 node in the cluster was powered by 2 dual-core 2.8GHz Opteron processors. Specifically, the Sun Fire X4100 cluster achieved a QphH@300GB of 198,578 together with $/QphH@300GB of $3.15. The performance is almost 5 times the previously reported best performance and the price/performance is more than twice as good as the previously reported best price/performance.

For other Other Innovations see: http://blogs.sun.com/bmseer/entry/innovation_setting_world_records_in

TPC-H @300GB Performance:
$/QphH = TPC-H Price/Performance metric (smaller is better)
QphH = TPC-H Composite Metric (bigger is better)

System Socket
/Core/
Thread
CPU Type GHz QphH $/
QphH
total
price
DBMS
Avail # Clust Nodes
Sun Fire X4100
60/ 120/ 120
Opt 2.8GHz 198,578 3.15
624,770
Paraccel
10/29/07 30
HP BL480c
16/ 32/ 32
Opt 3.0GHz 40411
18.67
754,232
Oracle
12/18/06 8
HP BL460c
16/ 32/ 32 Xeon 3.0GHz 39614
12.57
497,869
Oracle
9/15/07
 8
Dell PE6800
8/ 16/ 32 Xeon 3.0GHz 18881
24.37
460,004
Oracle
4/24/06
 2
IBM eSvr 325
16/ 16/ 16 Opt 2.0GHz 13195
65.44
863,410
DB2
11/08/03
 8

TPC-H @300GB Price-Performance:
$/QphH = TPC-H Price/Performance metric (smaller is better)
QphH = TPC-H Composite Metric (bigger is better)

System Socket/
Core/
Thread
CPU Type GHz QphH Price/
QphH
Price
in
US dollars
DBMS
Available #
Clust
Nodes
Sun Fire X4100
60/120/120
Opteron 2.8 GHz 198578
3.15
624,770
Paracell
10/29/07 30
Sun Fire X4100M2
2/ 4/ 4 Opt 3.0 GHz 7641
5.89
45,001
SybIQ
5/25/07
 1
Sun Fire X4200
2/ 2/ 2
Opt 3.0GHz 4936
6.29
31,033
SybIQ
6/23/06 1
HP DL585 G1
4/ 4 /4 Opt 2.4GHz 12226
11.71
143,041
SQLS
1/26/06  1
HP BL460c
16/ 32/ 32 xeon 3.0GHz 39614
12.57
497,869
Oracle
9/15/07  8
HP DL585 G2
4/ 8/ 8 Opt 2.8GHz 18299
13.67
250,057
SQLS
4/19/07
 1
IBM x3650
2/ 4/ 4 Xeon 3.0GHz 10165
15.40
156,535
DB2
10/06/06
 1
Dell 6800
8/ 8/ 8 Xeon 3.33GHz 11743
21.84
256,383
Oracle
1/08/06  2

Benchmark Description

The TPC-H benchmark is a performance benchmark established by the Transaction Processing Council (TPC) to demonstrate Data Warehousing/Decision Support Systems (DSS). TPC-H measurements are produced for customers to evaluate the performance of various DSS systems. These queries and updates are executed against a standard database under controlled conditions. Performance projections and comparisons between different TPC-H Database sizes (100GB, 300GB, 1000GB, 3000GB and 10000GB) are not allowed by the TPC.

TPC-H is a data warehousing-oriented, non-industry-specific benchmark that consists of a large number of complex queries typical of decision support applications. It also includes some insert and delete activity that is intended to simulate loading and purging data from a warehouse. TPC-H measures the combined performance of a particular database manager on a specific computer system.

The main performance metric reported by TPC-H is called the TPC-H Composite Query-per-Hour Performance Metric (QphH@SF, where SF is the number of GB of raw data, referred to as the scale factor). QphH@SF is intended to summarize the ability of the system to process queries in both single and multi user modes. The benchmark requires reporting of price/performance, which is the ratio of QphH to total HW/SW cost plus 3 years maintenance. A secondary metric is the storage efficiency, which is the ratio of total configured disk space in GB to the scale factor.

Disclosure Statement:

TPC-H @300GB Sun Fire X4100 cluster 198,578.1 QphH@300GB, $3.15/QphH@300GB, avail 10/29/07; TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info http://www.tpc.org.

See Also:

Results Summary TPC-H SF300 - 300GB benchmark

    Audited Results

    Database Size: 300 GB (Scale Factor 300)

    TPC-H Composite: 198,578 QphH@300GB

    Price/performance: $3.15 / QphH@300GB

    Available Oct 29, 2007
    Number of Systems: 30 x Sun Fire X4100
    each X4100:
    16GB memory each,
    2 x 146GB (10k RPM) internal SAS each
    Total Number Processors: 60
    Total Number Cores: 120
    Total Number Threads: 120
    Processor/GHz of Server: Opteron/2.8 GHz Dual-core
    Storage: 8158 Gigabytes of disk
    Database: Paraccel Analystic Database
    Operating System: RedHat Enterprise Linux 4.4
    Total 3 year Cost: $624,770.12
    Other Performance Metrics

    TPC-H Power: 145,219.3

    TPC-H Throughput: 271,542.9

    Database Load Time  15 minutes 5 seconds

    Storage Ratio: 27.19

    Like this post? del.icio.us | furl | slashdot | technorati | digg

Innovation: Setting World Records in TPC-H Paraccel & Sun Fire X4100 SF1000

Tuesday Oct 30, 2007

Innovation is an infectious thing in a very good way. Innovators seek out innovators to get even greater things. This is a story that is all about that. Customer wanted innovative hardware, they came to Sun, customer was interested in new database software (Paraccel). We saw each other's innovations, did some test, made some plans, and became quick friends. Wait till we apply a few more innovative things and make things even better.

Sun Fire X4100 using Paraccel Analytic Database sets World Record Performance and Price/Performance for the TPC-H @1000GB. This benchmark result demonstrates that a shared-nothing cluster of Sun Fire X4100s achieves the best performance and at the same time the best price/performance amongst all TPC-H@1000GB submissions. Each of the 48 node in the cluster was powered by 2 dual-core 2.8GHz Opteron processors.

The best thing:The Paraccel Analytic Database isn't about setting benchmark world records. It SETS world records because it is designed to do industrial strength everyday DSS. If you need this kind of database work for mission critical activities you should get to know Paraccel.

The benchmark details: the Sun Fire X4100 cluster achieved a QphH@1000GB of 315,842 together with $/QphH@1000GB of $4.57. The performance is almost 5 times the previously reported best performance and the price/performance is almost twice as good as the previously reported best price/performance.

Other Innovations: Another key area of innovation is business model. I've long complained about pricing a gallon of automotive gasoline by the number of cylinders in each particular car (wow that sounds silly) - whoops, I mean pricing software by # of cores. Paraccel Analytic Database innovates here as well. Databases manage data, pricing is tied to the amount of data managed. Fair and easy to measure and related to what the software does. The majors could learn from Paraccel in this area as well.

TPC-H @SF1000GB Performance Table:

$/QphH = TPC-H Price/Performance metric (smaller is better)
QphH = TPC-H Composite Metric (bigger is better)
System Socket
/Core/
Thread
CPU Type GHz QphH $/
QphH
Total Price DBMS
Avail # Clust node
Sun Fire X4100
96/ 192/ 192
Opt 2.8GHz
315,842
4.57
624,770
Paraccel
 10/29/07 48
HP Super- dome
32/ 64/ 64 Itan2 1.6GHz 69,999
28.69
2,008,168
SQLS
6/18/07  1
HP Super- dome
64/ 64/ 64 Itan2 1.6GHz 68,101
59.00
4,008,065
Oracle
1/18/06  1
IBM eSvr 346
64/ 64/ 128 Xeon 3.6GHz 53,451
32.80
1,753,144
DB2
2/14/05  64
HP DL585
48/ 48/ 48 Opt 2.2GHz
35,141
59.93
2,106,123
Oracle
10/21/04  12
Bull 6320
32/ 32/ 32 Itan2 1.6GHz 34,988
38.41
1,343,811
Oracle
2/02/06  1



TPC-H @1000GB Price-Performance Table:

$/QphH = TPC-H Price/Performance metric (smaller is better)
QphH = TPC-H Composite Metric (bigger is better)

System Socket/
Core/
Thread
CPU Type GHz QphH $/
QphH
Price
in
US dollars
DBMS
Available #
Cluster
Node
Sun Fire X4100
96/ 192/ 192
Opt 2.8GHz 315842
4.57
624,770
Paraccel
 10/29/07 48
Sun Fire X4500 G2
2/ 4/ 4
Opt 2.8 GHz 5604
8.11
45,439
SybIQ
 10/15/07 1
HP DL585 G2
4/ 8/ 8
Opt 2.8GHz 14772
9.73
143,736
SQLS
 04/25/07 1
Bull 3045
4/ 8/ 16 Itan2 1.6GHz 12087
12.56
151,870
SQLS
03/06/07
 1
HP DL585 G1
4/ 4/ 4 Opt 2.4GH 10493
13.85
145,264
SQLS
03/02/06
 1
Bull 5651
16/ 16/ 16 Itan2 1.6GHz 17059
25.48
434,553
SQLS
05/08/06
 1
IBM eSvr 346
64/ 64/ 128 Xeon 3.6GHz 53451
32.80
1,753,144
DB2
02/14/05
 64

Benchmark Description

The TPC-H benchmark is a performance benchmark established by the Transaction Processing Council (TPC) to demonstrate Data Warehousing/Decision Support Systems (DSS). TPC-H measurements are produced for customers to evaluate the performance of various DSS systems. These queries and updates are executed against a standard database under controlled conditions. Performance projections and comparisons between different TPC-H Database sizes (100GB, 300GB, 1000GB, 3000GB and 10000GB) are not allowed by the TPC.

TPC-H is a data warehousing-oriented, non-industry-specific benchmark that consists of a large number of complex queries typical of decision support applications. It also includes some insert and delete activity that is intended to simulate loading and purging data from a warehouse. TPC-H measures the combined performance of a particular database manager on a specific computer system.

The main performance metric reported by TPC-H is called the TPC-H Composite Query-per-Hour Performance Metric (QphH@SF, where SF is the number of GB of raw data, referred to as the scale factor). QphH@SF is intended to summarize the ability of the system to process queries in both single and multi user modes. The benchmark requires reporting of price/performance, which is the ratio of QphH to total HW/SW cost plus 3 years maintenance. A secondary metric is the storage efficiency, which is the ratio of total configured disk space in GB to the scale factor.

Disclosure Statement:

TPC-H @1000GB Sun Fire X4100 cluster 315,842.9 QphH@1000GB, $4.57/QphH@1000GB, avail 10/29/07; TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info http://www.tpc.org.

See Also:

Results Summary
Audited Results

Database Size: 1000 GB (Scale Factor 1000)

TPC-H Composite: 315,842 QphH@1000GB

Price/performance: $4.57 / QphH@1000GB

Available Oct 29, 2007
Number of Systems: 48 x Sun Fire X4100,
each X4100:
16GB memory each,
2 x 146GB (10k RPM) internal SAS each
Total # Procs: 96
Total # Cores: 192
Total # Threads: 192
Processor GHz: Opteron/2.8 GHz Dual-core
Storage: 13053 Total Gbytes of disk
Database: Paraccel Analytic Database
Operating System: RedHat Enterprise Linux 4.4
Total 3 year Cost: $1,442,050.52
Other Performance Metrics

TPC-H Power: 289,877.5

TPC-H Throughput: 344,134.1

Database Load Time  41 minutes 44 seconds

Storage Ratio: 13.05

[1] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg