Tuesday Jul 10, 2007
The Sun Blade X6250 outperfoms all posted ANSYS V11.0 (MCAE) results at www.ansys.com website. A single Sun Blade X6250 beats a single Intel S5000 XAL (same 3GHZ Xeon 5160) by as much as 40% at
each of the three "cpu" levels tested (1-, 2-, and all 4 cores available
on both 2 socket platforms equipped with dual core processors).
Sun Wins at these processor configurations in 6 of
the total 7 cases in the benchmark test suite. Overall, on the geometric mean,
Sun was 10% higher.
The only case "bm-2" where the Sun X6250 looses has an exceptionally high I/O component, and even so Sun was only 3-4% slower.
The Sun X6250 had 10K rpm internal disk drives where
the Intel S5000 XAL had 15K rpm drives.
The Sun Blade X6250 with 3.0GHz Xeon EM64T 5160 (Woodcrest)
and under 64-bit Linux SuSE SLES 10 beats all of the following
platforms with results posted at the ANSYS website
for all 7 test cases in the ANSYS "Standard" benchmark test suite (1-, 2- & 4-cpu).
Yes this result was run with Linux, Sun wants to show that we can
win with every OS. There now is an officially certified, supported and maintained version of a Solaris build of ANSYS V11.0
for X86-64 platform architectures compiled with recent Sun Studio 11 compilers.
This is the first SX64 version that has become available.
Competitive Landscape
ANSYS V 11.0 "Standard" Benchmark Test Suite on X2200 M2 & Constellation Blades
(run times in seconds, smaller is better; for % bigger is better)
| System |
Cores |
bm-1 |
bm-2 |
bm-3 |
bm-4 |
bm-5 |
bm-6 |
bm-7 |
| |
| Sun X6250/5160 |
4 |
100 |
1362 |
343 |
164 |
181 |
131 |
752 |
| Intel S5000XAL/5160 |
4 |
109 |
1312 |
369 |
169 |
187 |
161 |
1048 |
| Sun % better |
|
9% |
-4% |
8% |
3% |
3% |
23% |
39% |
| |
| Sun X6250/5160 |
2 |
118 |
1398 |
385 |
183 |
223 |
169 |
1064 |
| Intel S5000XAL/5160 |
2 |
128 |
1356 |
417 |
186 |
244 |
211 |
1437 |
| Sun % better |
|
9% |
-3% |
8% |
2% |
9% |
25% |
35% |
| |
| Sun X6250/5160 |
1 |
150 |
1455 |
456 |
211 |
339 |
253 |
1770 |
| Intel S5000XAL/5160 |
1 |
164 |
1416 |
489 |
215 |
340 |
314 |
2330 |
| Sun % better |
|
9% |
-3% |
7% |
2% |
1% |
24% |
32% |
(please note: per core performance isn't the right metric for comparing different CPUs, as system costs vary greatly, but they are used here to identify configuration)
It is "SYSTEM" performance not 'core' performance that matters!)
Key Technical Points
- The test cases from the ANSYS standard benchmark test suite all have a substantial I/O
component where 15% to 20% of the total run times are associated with I/O activity
(primarily scratch files).
Performance will be enhanced by using the fastest available drives and striping
together more than one of them or using a high performance disk storage system with high performance interconnects. When running with the SX64 build a ZFS system might be a good idea to employ.
ANSYS 11.0 Standard Test Cases
bm-1
Name:Exhaust Elbow Manifold
Description:Static structural analysis. Solved for equivalent stresses.
Statistics:~850,000 DOF Model
bm-2
Name:Floor Panel
Description:Surface body geometry. Harmonic analysis with mode superposition.
Statistics:~765,000 DOF Model
bm-3
Name:Engine Assembly - Piston and Crank
Description:Assembly with contact. Nonlinear structural DOF solution.
Statistics:~250,000 DOF Model
bm-4
Name:Electric Motor
Description:Electromagnetic analysis. Solved for magnetic field intensities.
Statistics:~250,000 DOF Model
bm-5
Name:Brake Rotor
Description:Thermal transient analysis. Solved for temperature DOF?s.
Statistics:~230,000 DOF Model
bm-6
Name:Wing Section
Description:Static structural analysis.
Statistics:~250,000 DOF Model
Comparing bm-6 and bm-7 is a good indication of performance characteristics for systems as larger problems are attempted. These problems will differentiate hardware performance most accurately for users expecting to solve problems approaching 1 million degrees of freedom or more.
bm-7
Name:Wing Section
Description:Static structural analysis.
Statistics:~800,000 DOF Model
Notes:bm-6 and bm-7 are designed to demonstrate ability of systems to handle larger memory demands and increased I/O. bm-6 should run well on any system. Bm-7 will be substantially impared in performance on a 32-bit machine limited to 2 or 3 Gbytes of memory. The model used for these runs selects Solid95 20-node brick elements. The cost of matrix factorization for these elements is much higher than the shell dominated model in bm-1 Bm-7 generates a large 12.8 Gb file containing the factored matrix. It requires aver 1 Gbyte of solver memory to run in optimal out-of-core mode. On PC workstations the solver will run using less than optimal out-of-core memory requiring excessive I/O during factorization. Comparing bm-6 and bm-7 is a good indication of performance characteristics for systems as larger problems are attempted. These problems will differentiate hardware performance most accurately for users expecting to solve problems approaching 1 million degrees of freedom or more.
Disclosure Statement:
The following are trademarks or registered trademarks of ANSYS, Inc. : ANSYS Multiphysics TM All information on the ANSYS website is Copyrighted 2007 by ANSYS, Inc. Results at
http://www.ansys.com/services/hardware-support-db.htm, July 2, 2007.
Hardware Configuration:
Sun Blade X6250
4 2-socket Sun Blade X6250's
2x3.0 GHz DC Intel Xeon EM64T 5160 (Woodcrest) processors
32 GB memory
Software Configuration:
64-bit Linux SuSE SLES 10
(note: Sun works great with Linux, that is why we show all kinds of benchmarks! )
ANSYS V11.0
ANSYS 11 "Standard" Benchmark Test Suite
Wednesday Jun 20, 2007
The Sun Fire V890 with 8x 2.1 GHz UltraSPARC IV+ obtained a result of
244846 SPECjbb2005 bops, 30606 SPECjbb2005/JVM on the server-side Java benchmark.
The Sun Fire V890 is 40% faster than the expensive 4-core IBM p570 (4.7 GHz power6). I'll leave it to the reader
to check the system prices to see the "per core performance" comparisons don't work at best, and are in fact are disingenuous because they make you compare systems of very different costs to you. You'll spend more for IBM.
At the high-end the IBM p570 16-core ($Megabucks) is only 2.8 times faster than the Sun Fire V890 2.1GHz (16-core). Again check the prices of the servers to really understand what you are getting.
Postscript:IBM will drone on every time about performance/core
or trying to equate systems on a per-core basis. The reality
is IBM's cores cost so much more than anyone elses cores. Get
very suspicious any time you see IBM saying things like "8-core IBM system vs. 8-core Sun system" or "perf/core IBM wins". Price out one of these comparison and you WILL BE TRULY AMAZED at the smoke & mirrors they are using. Also note how many times they will mention performance on 16-core and they do a price comparison on 8-core with
lots of memory on the Sun system and a sparse config on the IBM system.
If you want to see a chip to chip comparison see
BM Seer's posting of UltraSPARC T1 result. Now that box rocks.
SPECjbb2005 Performance Chart (ordered by performance,
bops : SPECjbb2005 Business Operations per Second (bigger is better)
|
System
|
Date
|
Processors
|
Performance
|
|
(Chips,
Cores,
Threads)
|
GHz/
Type
|
SPECjbb2005
bops
|
JVMs
|
SPECjbb2005
bops/JVM
|
|
IBM p570 power6
|
6/07
|
(8,
16,
32)
|
4.7 power6
|
691,975
|
8
|
86,497
|
|
Sun Fire V890
|
6/07
|
(8,
16,
16)
|
2.1 US-IV+
|
244,846
|
8
|
30,606
|
|
IBM p570 power6
|
6/07
|
(2,
4,
16)
|
4.7 power6
|
175,474
|
2
|
87,737
|
|
Sun Fire V890
|
10/05
|
(8,
16,
16)
|
1.5 US-IV+
|
117,986
|
4
|
29,497
|
Benchmark Description
SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the
database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).
Disclosure Statement:
SPECjbb2005
Sun Fire V890 (8 chip, 16 cores 16 threads) 244846 SPECjbb2005 bops, 30606 SPECjbb2005 bops/JVM,
Sun Fire V890 (8 chip, 16 cores, 16 threads) 117986 SPECjbb2005 bops, 29497 SPECjbb2005 bops/JVM,
IBM System p 570 (4.7 GHz) running AIX 5L V5.3 175,474 SPECjbb2005 bops, 87,737 SPECjbb2005 bops/JVM, 2 chips, 4 cores, 8 threads,
IBM System p 570 (4.7 GHz) running AIX 5L V5.3, 691,975 SPECjbb2005 bops, 86,497 SPECjbb2005 bops/JVM, 8 chips, 16 cores, 32 threads,
SPEC, SPECjbb reg tm of Standard Performance Evaluation
Corporation. Results as of 06/20/2007 on www.spec.org.
Results Summary
|
Results
|
| |
Sun Fire V890: |
|
244846 SPECjbb2005 bops |
| |
|
|
30606 SPECjbb2005 bops/JVM |
| |
Reference Date: |
|
June 20, 2007 |
| Systems: |
|
Sun Fire V890, 64 GB |
| Total Number Processors: |
|
8 |
| Processor/GHz of Server: |
|
US-IV+ 2.1 GHz |
| Operating System: |
|
Solaris 10 6/06 |
| JVM: |
|
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_02 |
See Also
SPECjbb2005 Benchmark Reports
IBM Consolidation Press Release
Wednesday Jun 13, 2007
InfoWorld has published a very positive review of the Sun Fire x4500 server. The combination X4500 running Solaris 10 with Sun's ZFS scored an great 8.8 rating with "Excellent" recommendation.
Paul Venezia, author/reviewer, started with a description X4500 (code name "Thumper"), highlighting it design and unprecedented hard drive capactiy.
He evaluated x4500 running Solaris -- mentioning that other OSes simply did not have the file system capabilities to take full advantage of the huge number
of X4500's drives.
Paul writes: "ZFS and the X4500 go hand in hand, seemingly created for each other in a love story rivaling anything that’s come out of Hollywood in the past 10 years."
"Thumper is aptly named and is a truly unique product from a company that seems to be pulling away from a faltering reputation in the server market. Recent studies have shown that within a few short years, the world will generate more data than it can store. It would seem that Sun is doing its part to bridge that gap."
Read all about it at the full link:
www.infoworld.com/infoworld/article/07/06/07/23TCthumper_1.html
Wednesday Jun 13, 2007
Sun Blade X6250 Delivers a pair of x86 SPEC CPU2006 integer performance World Records:
Sun Blade X6250 (Dual-Core Intel Xeon 5160)
and running Solaris 10 and using Sun Studio 12 compiler delivered the
best x86 result for the SPECint2006 benchmark.
Sun Blade X6250 (Dual-Core Intel Xeon 5160) using Solaris 10 and
Studio 12, delivered x86 4-core world record on
SPECint_rate2006.
Sun Blade X6250 server had a SPECint2006 result of 21.0 and SPECint_rate2006 result of 65.0. The advanced features of freely available
Sun Studio 12 complier were critical for getting this level of
performance on the Sun Blade 6250.
The Sun Blade X6250 is only 3% slower than the peak score of the very-expensive
new IBM POWER6 p570, which was recently announced. SPECint2006 is a single
job stream. So let's now turn to comparing 4 thread results, in this case
the Sun Blade X6250 is 7% faster than the peak SPECint_rate2006 score of
he very-expensive new IBM POWER6 p570 (both IBM and Sun at 4 threads). Oh, and remember that anymore clock
rate is not how you compare systems the Sun Blade X6250 is at 3GHz and the
IBM POWER6 is at 4.7GHz. CPU frequency is basically irrelevant, it is CPU and system architecture that matters!
SPEC CPU2006 Landscape - bigger is better, selected recent results
SPECint2006
| System |
Processors |
Performance Results |
| Type |
GHz |
Chips |
Cores |
Peak |
Base |
| IBM p570 (power6) |
Power6 |
4.7 |
1 |
1 |
21.6 |
17.8 |
| Sun Blade X6250 |
Intel Xeon 5160 |
3.0 |
2 |
4 |
21.0 |
|
| Supermicro X7DB8+ board |
Intel Xeon 5160 |
3.0 |
2 |
4 |
20.8 |
18.9 |
| Sun Ultra 40 M2 |
AMD Opteron 2222SE |
3.0 |
2 |
4 |
16.1 |
|
SPECint_rate2006
| System |
Processors |
Performance Results |
| Type |
GHz |
Chips |
Cores |
Threads
/ Copies |
Peak |
Base |
| Sun Blade X6250 |
Intel Xeon 5160 |
3.0 |
2 |
4 |
4 |
65.0 |
|
| Supermicro X7DB8+ |
Intel Xeon 5160 |
3.0 |
2 |
4 |
4 |
64.9 |
60.0 |
| IBM p570 (Power6) |
Power6 |
4.7 |
1 |
2 |
4 |
60.9 |
53.2 |
| Sun Ultra 40 M2 |
AMD Opteron 2222SE |
3.0 |
2 |
4 |
4 |
60.4 |
|
| Fujitsu BX620 S3 |
Xeon 5160 (Woodcrest) |
3.0 |
2 |
4 |
4 |
59.4 |
56.7 |
Results as of 06 Jun 2007 from www.spec.org.
Benchmark Description
SPEC CPU2006 is made up of two suites of benchmarks, CFP2006 and
CINT2006. CFP2006 targets floating-point performance, while CINT2006
targets integer performance.
Each suite has two different measures. First is the CPU measure, which
is the performance on the suite as a single stream. This can be either
a single thread or automatic compiled parallel run. This measure is
further defined by base and optimized runs. Base uses the same compiler
flags for all kernels, where optimized is allowed to use different
compiler flags for each kernel. Results are compared against a baseline
system run that was standardized by SPEC.
The second measure is Rate. It is a measure of how many CPU measures
can be run at a time. Typically, it is run as n processes on n
processors. It shows how well the same job mix can run on a system
under some load. It also is run as a base and optimized set of
results.
Disclosure Statement:
SPEC, SPECint, SPECfp reg tm of Standard Performance Evaluation Corporation.
Results from
www.spec.org or from IBM public websites as of 6/06/07.
Sun Blade X6250 (Intel Xeon 5160, 2chips/4cores, Solaris 10) 65.0 SPECint_rate2006;
Sun Blade X6250 (Intel Xeon 5160, 2chips/4cores, Solaris 10) 21.0 SPECint2006;
IBM System p 570 (POWER6, 1chip/1core, AIX 5L v5.3) 21.6 SPECint2006;
IBM System p 570 (POWER6, 4 theads, 1chip/2cores, AIX 5L v5.3) 60.9 SPECint_rate2006.
System Configuration
| Results |
| Reference Date: |
|
Jun 06, 2007 |
| System: |
|
Sun Blade X6250
SPEED: 16GB memory 8x2GB
RATE : 32GB memory 8x4GB |
|
X6250 |
|
21.0 SPECint2006 |
|
X6250 |
|
65.0 SPECint_rate2006 |
| Total Number Processors: |
|
2 x Intel Xeon 5160 |
| Software: |
|
Solaris 10 11/06, Sun Studio 12 Compiler, MicroQuill's SmartHeap Library v7.4 |
See Also
All Benchmark results on Sun Blade 6000 Blade Server
Tuesday Jun 05, 2007
A podcast of Scientific American, covers container history. It is an
interview with Professor Arthur Donovan who wrote a book about cargo
containerization.
Go to: http://www.sciam.com/podcast/
Look for May 30, 2007: Science Talk "How Cargo Containers Shrank the World and Transformed Trade"
Important 2006 Anniversaries
50th Anniversary of container ship voyage
50th Anniversary of federal interstate highway act
Announcement of Sun's "Project Blackbox" (datacenter container)
Will Sun's Project Blackbox create a "common culture of the datacenter."
...or are we still going to waste money with customized datacenters that
all basically look alike?
Monday Jun 04, 2007
The Sun Fire X4100 M2 has 50% better price performance than the HP DL585.
This benchmark result demonstrates that the Sun Fire X4100 M2, powered by 2 dual-core 3.0GHz Opteron, improves upon Sun's previously published world-record
$/performance result at 300GB. The Sun Fire X4100 M2 is the only 1U system ever submitted for a TPC benchmark at the 300GB scale-factor.
The Sun Fire X4100 M2 achieved the best price-performance among all systems
at 300GB. It improved upon Sun's previous world-record price-performance, achieved by the Sun Fire X4200, by 6%.
Note all of this detail, and the very different ways in which results are marketed with the IBM POWER6 post.
The Sun Fire X4100 M2 achieved a 55% QphH@300GB improvement upon previously
published 2-socket Single-core RevE Sun Fire X4200 result, i.e., 7641 QphH@300GB versus 4936 QphH@300GB.
Specifically, Sun, using its Sun Fire X4100 M2 server achieved a
$/QphH@300GB of $5.89, whereas the Sun Fire X4200 achieved a $/QphH@300GB
of $6.29. The latter result was submitted on June 23, 2006.
TPC-H @300GB Performance Results (sorted by $/QphH
for single (non-clustered) systems:
$/QphH = $/QphH@300GB TPC-H Price/Performance metric (smaller is better)
QphH = QphH@300GB TPC-H Composite Metric (bigger is better)
Disk Data Ratio is the ratio of the total number of gigabytes of
configured storage to the scale factor
number of gigabytes (smaller is better)
| System |
Sockets/
Cores/
Threads CPU GHz |
QphH |
Price/
QphH
|
Price
in
currency
|
DBMS
|
Available |
Disk
Data
Ratio
|
Cluster |
Sun X4100 M2
|
2/4/4 Opteron 3.0
|
7641
|
$5.89
|
45,001 $US
|
SybIQ
|
6/23/06 |
3.17 |
N
|
Sun X4100
|
2/2/2 3.0 Opteron |
4936
|
$6.29
|
31,033 $US
|
SybIQ
|
6/23/06 |
2.9 |
N
|
HP DL585 G1
|
4/8/8 Opteron 2.4
|
12225 |
$11.71
|
143,041 $US
|
SQLS
|
01/26/06
|
19.9 |
N
|
HP DL585 G2
|
4/8/8 Opteron 2.8
|
18298 |
$13.67
|
250,057 $US
|
SQLS
|
04/19/07
|
24.96 |
N
|
| IBM x3650 |
2/4/4 WoodC 3.0
|
10165
|
$15.40
|
156,535 $US
|
DB2
|
10/06/06
|
12.8
|
N
|
Sun V440
|
4/4/4 US IIIi 1.6
|
2501
|
$22.09
|
55,245 $US
|
SybIQ
|
05/09/05
|
1.81
|
N
|
| HP DL585 G1 |
4/8/8 Opteron 2.4
|
11915
|
$22.78
|
271,379 $US
|
DB2
|
10/05/05 |
19.7
|
N
|
| HP DL585 G1 |
4/4/4 Opteron 2.6
|
8434
|
$30.18
|
255,586 $US
|
DB2
|
05/17/05 |
13.8
|
N
|
IBM eServer 366
|
4/4/8 Xeon 3.6
|
7762
|
$32.94
|
255,702 $US
|
DB2
|
05/02/05
|
18.5
|
N
|
The results reported here were performed on a Sun Fire X4100 M2 system
running the Sybase IQ database manager. Sybase IQ is a special product
designed specifically
for data warehousing applications. Sybase IQ was developed as a totally
separate product from the more widely known Sybase database management
system (Sybase Adaptive Server).
Sun achieved this result using only 14 disks. Other vendors used anywhere
from 104 disks (the IBM x3650 result) to 208 disks (the HP DL585 G2 result).
The
significance of being able to house a data warehouse with
fewer disks provides numerous advantages far beyond the
scope of the TPC-H metrics. These include, ease of management, lower
probability of admin errors, a
much lower probability of disk failures and a true
reduction in the total cost of ownership over the life of a
system.
All Sun/SybaseIQ submissions, including this one, RAID protect their storage.
Only a few, of the almost 30 existing non-Sun submissions, at 300GB RAID
protect their storage. The lack of RAID protection results in artificially
cheaper configurations, which no production shop would ever deploy.
Benchmark Description
The TPC-H benchmark
is a performance benchmark established by the Transaction Processing
Council
(TPC) to demonstrate Data Warehousing/Decision Support Systems (DSS).
TPC-H
measurements are produced for customers to evaluate the performance of
various DSS systems. These queries and updates are executed against a
standard
database under controlled conditions. Performance projections and
comparisons
between different TPC-H Database sizes (100GB, 300GB, 1000GB, 3000GB
and
10000GB) are not allowed by the TPC.
TPC-H is a
data warehousing-oriented,
non-industry-specific benchmark that consists of a large number of
complex
queries typical of decision support applications. It also includes some
insert and delete activity that is intended to simulate loading and
purging
data from a warehouse. TPC-H measures the combined performance of a
particular
database manager on a specific computer system.
The main performance
metric reported by TPC-H is called the TPC-H Composite Query-per-Hour
Performance
Metric (QphH@SF, where SF is the number of GB of raw data, referred to
as the scale factor). QphH@SF is intended to summarize the ability of
the
system to process queries in both single and multi user modes. The
benchmark
requires reporting of price/performance, which is the ratio of QphH to
total HW/SW cost plus 3 years maintenance. A secondary metric is the
storage
efficiency, which is the ratio of total configured disk space in GB to
the scale factor.
Disclosure Statement:
TPC-H @300GB Sun Fire X4100 M2 7641 QphH@300GB, $5.89/QphH@300GB, avail 5/25/07;
TPC-H, QphH, $/QphH tm of
Transaction Processing Performance Council (TPC).
More info www.tpc.org.
| Audited Results |
|
Database Size: |
300 GB (Scale Factor 300) |
|
TPC-H Composite: |
7641 QphH@300GB |
|
Price/performance: |
$5.89 / QphH@300GB |
|
Available |
May 25, 2007 |
| Number of Systems: |
Sun Fire X4100 M2
|
| Total Processors, cores, Threads: |
2,2,2
|
| Processor |
Dual-core Opteron 3.0GHz
|
| Storage: |
951 Gigabytes of disk |
| Database: |
Sybase IQ 12.6 |
| Operating System: |
Solaris 10 |
| Total 3 year Cost: |
$45,001.35 |
| Other Performance Metrics |
|
TPC-H Power: |
7847
|
|
TPC-H Throughput: |
7440.5
|
|
Database Load Time |
4 hours 22 minutes 53 seconds
|
|
Storage Ratio/type: |
3.17 ratio/ two STK3320 SCSI JBOD array
|
|
See Also
Sun Fire X4100 M2 TPC-H Executive Summary Report Acrobat PDF (68K)
Complete Sun Fire X4100 M2 TPC-H Full Disclosure Report Acrobat PDF (590K)
Transaction
Processing Performance Council (TPC) Home Page
Ideas International Benchmark page
I'll even show my math, I challenge other vendors to show it too!
6% claim from: (6.29-5.89)/6.29 = 0.0635
50% claim from: (11.71-5.89)/11.71 = 0.4970 (49.7 rounds to 50)
55% claim from: (7641-4936)/4936 = 0.5480 (54.8 rounds to 55)
Thursday May 24, 2007
Sun continues to grow, lots of great new products and even more coming soon.
http://www.sun.com/aboutsun/pr/2007-05/sunflash.20070523.1.xml
Tuesday May 15, 2007
Careful reading often required when vendors make claims.
I've lumped some of the bad comparisons in the industry
into some general classifications. I'll perfect these
classifications at a later date.
"Pay no attention to the man behind the curtain."
"Don't accept analysis rotten to the core."
Here vendors avoid pointing at delivered performance results on
modern benchmarks (you know the ones developed in the past 5 years)
and looking at complete systems. They instead construct ratios
to things such as cores, threads etc -- AND ignore that everyone has
cores and threads that are implemented completely differently and
at very different cost structures. Everyone's cores cost completely
differently.
Note: My comments on the pre-jurassic TPC-C benchmark, if you
think that Sun is avoiding current tests, TPC-C isn't current.
Action: Ask for system performance and system cost, for example: ignore
all performance per core comparisons.
"Don't believe in paper tigers."
Rush out the prototype then deliver the world (with un-promised date).
Take a long time to avoid showing on current projects
Action: Ask for certified delivery dates, if they get dodgy during
the discussion then the don't have it.
"Don't think things in magnifing glass are as big as real things."
This comes in a couple of forms:
First, use small systems to deliver
some performance and then imply undelivered bigger systems to be
world beaters.
Second, claim a new feature that has huge benefit, show it on one
test -- then make it sound like it is also a huge benefits for everything.
Action: Read carefully, don't make assumptions,
ask questions, share your analysis with others. If you see a low level
performance measurement asks what how it really effects real
workloads -- get nervous if they say "your mileage will very." Press
for numbers.
"Don't believe 'A' implies 'C'. 'Sea' implies 'B', therefore 'A' implies 'B'."
Might be fun to use this "Sieve of BM Seer" to rate any product announcement.
I'm thinking of evolving this and having a regular "Beware the Ides of xxxx"
article and see how it develops. Check back on the ides of each month
for the next installment.
(ides = 15th)
Wednesday May 09, 2007
David Yen is in an article about Sun's CPU in EEtimes.
http://eetimes.com/ in the search box
type "Yen Sun" to get to the article "Reborn Sun Micro plots server CPU push"
David had a lot to do with the systems and chips that lead to this world record
blogged a couple of days ago:
http://blogs.sun.com/bmseer/entry/java_performance_sun_fire_e25k
Tuesday May 08, 2007
Sun US-IV+ vs. POWER5+:
- Sun Fire E2900 with dual-core US-IV+ (24 threads) beats the the fastest
IBM POWER5+ p5 570 result (2.2 GHz 32 threads) of 326,651 bops.
-
How do they compare on pricing? Well IBM doesn't seem to post pricing online for expensive products over about $500,000 or 32 threads, so I can't get the public
pricing for the IBM p570.
The only online pricing (a bit old) is from
IBM p570 TPC-C disclosure report.
IBM TPC-C result of 1,025,169 tpmC at $4.42/tpmC on a 16-core (8 processors, 32 threads) 2.2 GHz IBM System p5 570 (configuration planned to be available 05/31/06).
look carefully at the line items, just turning on the processors is expensive: "MODEL 570 PERMANENT PROCESSOR ACTIVATION FEATURE *16 = $343,040" (note: that does NOT count any memory costs or actually getting the processors any of the 48 other p570 required line items). My comments on the TPC-C benchmark.
-
Also see this for IBM hardware per core pricing on the high end - wow!
The Sun Fire E2900 with 1.95GHz US-IV+ achieved 332,917 SPECjbb2005 bops and 27.743 SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark. The Sun Fire E2900
used Solaris 10.
SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)
|
System
|
Date
|
Processors
|
Performance
|
|
(Chips, Cores, Threads)
|
GHz Type
|
bops
|
JVMs
|
bops/JVM
|
|
Sun Fire E2900
|
5/07
|
12, 24, 24
|
1.95 US-IV+
|
332,917
|
12
|
27,743
|
|
IBM p5 570
|
1/06
|
8, 16, 32
|
2.2 POWER5+
|
326,651
|
8
|
40,831
|
Sun results have been submitted to SPEC for review and are on track for publication.
Benchmark Description
SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the
database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).
Disclosure Statement:
SPECjbb2005 Sun Fire E2900 (12 chips, 24 cores, 24 threads1.95 GHz) 332,917
SPECjbb2005 bops, 27,743 SPECjbb2005 bops/JVM submitted for review;
IBM eServer p5 570 (8 chips, 16 cores, 32 Threads 2.2 GHz) 326,651 SPECjbb2005 bops,
40,831 SPECjbb2005 bops/JVM.
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation.
Results as of 5/8/07 on www.spec.org
|
Certified Results
|
|
|
Performance:
|
|
332,917 SPECjbb2005 bops
|
|
|
|
|
27,743 SPECjbb2005 bops/JVM
|
|
|
Reference Date:
|
|
May 8, 2007
|
|
Systems:
|
|
Sun Fire E2900
|
|
|
|
Processor/GHz:
|
|
12 US-IV+ 1.95 GHz
|
|
Operating System:
|
|
Solaris 10
|
|
JVM:
|
|
Java HotSpot(TM) 32-Bit Server, Version 6.0_02
|
Tuesday May 08, 2007
Sun leads the way, beating Itanium2 and POWER5+(by a lot):
- Sun Fire E25K with dual-core US-IV+ beats the HP Superdome with dual-core Itanium 2.
-
Sun Fire E25K is 6.4 times faster than the fastest IBM POWER5+
p5 570 result (1.9GHz 16 cores) of 326,651 bops. Note: The largest IBM p5
595 only has 4 times as many POWER5+ cores. IBM has not published this
benchmark on their largest systems. why does IBM keep avoiding comparison to Sun
on accepted standard benchmarks like SPECjbb2005?
-
Sun Fire E25K 1.95GHz US-IV+ also beats the Fujitsu PRIMEPOWER 2500 2.08GHz SPARC64 V
by 67%.
The Sun Fire E25K with 1.95GHz US-IV+ set a World Record for
systems with 72 or fewer chips, achieving 2,105,264 SPECjbb2005 bops and 29,240
SPECjbb2005 bops/JVM on the SPECjbb2005 benchmark.
The 6.0_02 version of the Java HotSpot(TM) 32-Bit Server VM showed a 27%
improvement of the 6.0 version on the SPECjbb2005 benchmark. The Sun Fire E25K result used Solaris 10.
SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)
|
System
|
Date
|
Processors
|
Performance
|
|
(Chips, Cores, Threads)
|
GHz Type
|
bops
|
JVMs
|
bops/JVM
|
|
Sun Fire E25K
|
5/07
|
72, 144, 144
|
1.95 US-IV+
|
2,105,264
|
72
|
29,240
|
|
HP Superdome
|
9/06
|
64, 128, 128
|
1.6 Itanium 2
|
2,054,864
|
32
|
64,215
|
|
Fujitsu PP2500
|
3/06
|
128, 128, 128
|
2.08 SPARC64 V
|
1,251,024
|
32
|
39,095
|
|
IBM p5 570
|
1/06
|
8, 16, 32
|
2.2 POWER5+
|
326,651
|
8
|
40,831
|
Sun results have been submitted to SPEC for review and are on track for publication.
Benchmark Description
SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the
database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).
Disclosure Statement:
SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 2,105,264
SPECjbb2005 bops, 29,240 SPECjbb2005 bops/JVM submitted for review;
Sun Fire E25K (72 chips, 144 cores, 1.95 GHz) 1,657,274
SPECjbb2005 bops, 23,018 SPECjbb2005 bops/JVM;
HP Itanium Superdome (64 chips, 128 cores, 1.6 GHz) 2,054,864
SPECjbb2005 bops, 64,215 SPECjbb2005 bops/JVM;
Fujitsu PRIMEPOWER 2500 (128 chips, 128 cores, 2.08 Ghz) 1,251,024 SPECjbb2005 bops,
39,095 SPECjbb2005 bops/JVM;
IBM eServer p5 570 (8 chips, 16 cores, 2.2 GHz) 326,651 SPECjbb2005 bops,
40,831 SPECjbb2005 bops/JVM.
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation.
Results as of 5/8/07 on www.spec.org
|
Certified Results
|
|
|
Performance:
|
|
2,105,264 SPECjbb2005 bops
|
|
|
|
|
29,240 SPECjbb2005 bops/JVM
|
|
|
Reference Date:
|
|
May 8, 2007
|
|
Systems:
|
|
Sun Fire E25K
|
|
|
|
Processor/GHz:
|
|
72 US-IV+ 1.95 GHz
|
|
Operating System:
|
|
Solaris 10
|
|
JVM:
|
|
Java HotSpot(TM) 32-Bit Server, Version 6.0_02
|
Thursday Apr 26, 2007
Here we show the first Mirroring FC/SAS 4Gb result with world
record $/Perf beating HP by 2.5x in performance. The Sun StorageTek
2540 mid-range product coupled with
Sun's 4Gb HBAs has demonstrated industry leading SPC-2 benchmarking.
The Sun StorageTek 2540 has World Record $/performance of $46.26
$/SPC-2 MBPS and best-in-class performance of 730.04 SPC-2 MBPS.
Sun achieved this result using a RAID 5 storage configuration to
protect all data against disk failures. Outside of Sun's prior
submissions all other competitors used the same data protection showing
an apples-to-apples comparison. Sun's StorageTek 2540 submissions can
suffer a disk failure without incurring a lengthy service outage,
during which time a full rollforward recovery from a backup must be
performed.
SPC-2 Performance Data Protection Level: Mirroring
(in increasing $/performance order, only results to date)
| System |
SPC-2 MBPS |
$/SPC-2 MBPS |
ASU Capacity (GB) |
TSC Price |
Date |
Results Identifier |
| Sun ST2540 |
730.04 |
$46.26 |
1282.048 |
$33,772 |
4/10/07 |
B00022 |
| HP SW1000 |
285.39 |
$73.12 |
362.925 |
$20,868 |
3/09/07 |
B00020 |
SPC-2 MBPS = the Performance Metric
$/SPC-2 MBPS = the Price/Performance Metric
ASU Capacity = the Capacity Metric
Data Protection = Data Protection Metric
TSC Price = Total Cost of Ownership Metric
Results Identifier = A unique identification of the result Metric
Benchmark Description
The SPC Benchmark-2 (SPC-2) is a series of related benchmark
performance tests that simulate the sequential component of demands
placed upon on-line, non-volatile storage in server class computer
systems. SPC-2 provides measurements in support of real world
environments characterized by:
Large numbers of concurrent sequential transfers.
Demanding data rate requirements, including requirements for real time
processing.
Diverse application techniques for sequential processing.
Substantial storage capacity requirements.
Data persistence requirements to ensure preservation of data without
corruption or loss.
Disclosure Statement:
Sun StorageTek 2540 730.04 SPC-2 MBPS, $/SPC-2 MBPS $46.26, ASU
Capacity 1,282.048GB, Protect Mirroring, Cost $33,772.00, Ident.
B00022. SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of
Storage Performance Council (SPC). More info http://storageperformance.org.
| Certified Results |
|
System: |
|
Sun StorageTek 2540
2 Controllers
36 73GB 15K RPM SAS drives
2x4Gb front-end ports
2x4Gb back-end ports
512MB controller cache (per controller)
|
|
Performance: |
|
730.04 SPC-2 MBPS |
|
Price/Performance: |
|
$46.26 $/SPC-2 MBPS |
|
ASU Capacity: |
|
1,282.048 GB |
|
Data Protection Level: |
|
Mirroring |
|
TSC Price: |
|
$33,772.00 |
|
Results Identifier: |
|
B00022 |
|
Server: |
|
Sun Fire X4600, 8x2.6GHz Opteron 885, 47.5GB
|
|
Operating System: |
|
Solaris 10 11/06 |
Tuesday Apr 24, 2007
I saw this being passed around internally and thought more of you might
be intereseted:
Project Blackbox information sources on the web:
Sun.com:
http://www.sun.com/emrkt/blackbox
Discover Sun Tour:
http://www.sun.com/events/st/index.jsp
The "Where is Project Blackbox?" blog:
http://blogs.sun.com/blackbox/
3 D Interactive tour/video:
http://frsun.downloads.edgesuite.net/sun/07C00868/
Youtube Video Tour:
http://www.youtube.com/watch?v=Hp3QxlSK9Kc
Technology Evangelist Video Tour:
http://www.technologyevangelist.com/2007/03/sun_microsystems_pro.html
Friday Apr 20, 2007
Customers are seeing that IBM costs 3x more per core - that means that Sun
could be half the performance per core and STILL make Sun a much better value
and be a lot faster since Sun can have twice the number of cores. IBM seems to live in a core-fantasy world.
So if you get confused by IBM's over-built and vastly over-priced
cores, you should just go back to comparing system versus system, because
that is what you buy. I don't look at the number of valves (cores) when
I'm buying a car (system).
Fully-configured Sun E25K are less expensive and faster than
fully-configured IBM p5 595 - included all hardware & software costs.
Check out these benchmarks:
http://www.sun.com/servers/sparc_benchmarks/
...and this reality posting (check pricing and calculate the per core price): http://blogs.sun.com/bmseer/entry/not_comparing_e25k_p595
IBM's tuning of benchmarks.
http://blogs.sun.com/bmseer/entry/ibm_making_the_simplistic_sound
In other news, Goldman Sachs downgraded IBM.
http://www.investors.com/editorial/IBDArticles.asp?artsec=1&artnum=1&issue=20070418
|
|
|
|
Prices:
8-way Opt...