BM Seer Facts & Questions from an Anonymous Sun Source

IBM, do they always compare benchmarks in wierd ways?

Wednesday Oct 03, 2007

IBM bloggers seem to talk about any marketing comparison where they look good even if it they compare things they shouldn't.

Recently Sun released SPC-1 results, and because the Sun StorageTek® 9990 System can be hosted by so many systems Sun even used the IBM p595 as the host. How cool is that! Let me shout it out, "Sun used IBM servers on with Sun Storage in a benchmark!" Not that the IBM server provided an particular advantage in performance, in fact it was an older 24-core 1.65GHz power5. Clearly Sun wants everyone to notice you can put our fast and cost-effective storage on your IBM box or whatever.

Did the IBM blogger compare Sun's high-end storage to IBM's high-end storage, of course not, IBM would have lost in that case. Here is Sun's high-end compared to IBM's high-end storage...

    The StorageTek 9990V competes with the IBM DS8300 (both Monolithic technologies). The StorageTek 9990V (200,245.73 @ $17.31 $/SPC-1 IOPS) truly beat IBM DS8300 (123,033.40 SPC-1 IOPS @ $18.99 $/SPC-1 IOPS) by over 60% in performance with SPC-1 and still had better price/performance about 10%.

...but the IBM blogger compared different points on the scale. The mid-range SAN Volume Controller is a different technology (clustering of IBM DS4800s and switches) does not compete with the StorageTek 9990V. Anybody can put together a bunch of mid-range storage, switch them up, and put a common interface on it for a management nightmare.

Disclosure Statement:
SPC Benchmark-1 and SPC Benchmark-2 are trademarks of the Storage Performance Council.

High-end comparison:
Sun StoreTek 9900 system SPC-1 storage benchmark result of 200,245.73 IOPS over 26,000 GB ASU Capacity (mirrored) on the Sun StorageTek® 9990 System ($17.31 $/IOPS). IBM DS8300 (123,033.40 SPC-1 IOPS @ $18.99 $/SPC-1 IOPS, mirrored, ASU (9,103GB) Source: http://www.storageperformance.org/results; Results as of 10/02/07.

Mid-range comparison:
IBM TotalStorage SAN Volume Controller SPC-1 result of 272,505 IOPS (22,433 GB, $12.05/IOP, Mirroring).

...makes me think that this Thursday I should post how IBM says two things at one all of the time. Google, get warmed up :)

[1] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 08:00AM Oct 03, 2007 by BM Seer in IBM Follies
Tags:

Sun StorageTek 6540 Best-in-Class SPC-1 Performance and $/Performance

Thursday Sep 06, 2007

Sun beats IBM by Over 2X in Price/Performance in SPC-1 benchmark. The power of the Sun StorageTek 6540, coupled with Sun's 4Gb HBAs demonstrates industry best-in-class SPC-1 benchmark performance and price/performance.

Sun, using its StorageTek 6540, achieved a price/performance of $6.62 $/SPC-1 IOPS. In so doing, Sun also lowered the total cost of ownership, among higher midrange systems to $307,710.

Over 2x better price/performance than IBM at $16.03 $/SPC-1 IOPS.

Sun achieved this result using a MIRRORED storage configuration to protect all data against disk failures. Outside of Sun's prior submissions all other competitors used the same data protection showing an apples-to-apples comparison. Sun's StorageTek 6540 submissions can suffer a disk failure without incurring a lengthy service outage, during which time a full roll-forward recovery from a backup must be performed just like the competitors.

Competitive Landscape

SPC-1 Performance Chart (in increasing price-performance order)

System SPC-1 IOPS $/SPC-1 IOPS ASU Capacity (GB) TSC Price Data Protect Level Date Result Id
Sun ST6540 46,491.72 $6.62 6890.000 $307,710 Mirroring 6/15/07 A00051
IBM DS4800 45,014.81 $16.03 6871.277 $721,618 Mirroring 4/12/07 A00050

Benchmark Description

SPC Benchmark-1 (SPC-1): is the first industry standard storage benchmark and is the most comprehensive performance analysis environment ever constructed for storage subsystems. The I/O workload in SPC-1 is characterized by predominately random I/O operations as typified by multi-user OLTP, database, and email servers environments. SPC-1 uses a highly efficient multi-threaded workload generator to thoroughly analyze direct attach or network storage subsystems. The SPC-1 benchmark enables companies to rapidly produce valid performance and price/performance results using a variety of host platforms and storage network topologies.

SPC1 is built to:

  • Provide a level playing field for test sponsors.
  • Produce results that are powerful and yet simple to use.
  • Provide value for engineers as well as IT consumers and solution integrators.
  • Is easy to run, easy to audit/verify, and easy to use to report official results.

SPC-1 IOPS = the Performance Metric
$/SPC-1 IOPS = the Price/Performance Metric
ASU Capacity = the Capacity Metric
Data Protection = Data Protection Metric
TSC Price = Total Cost of Ownership Metric
Results Identifier = A unique identification of the result Metric

See Also

Disclosure Statement:

SPC-1, SPC-1 IOPS, $/SPC-1 IOPS reg tm of Storage Performance Council (SPC). More info www.storageperformance.org Sun StorageTek 6540 46,491.72 SPC-1 IOPS, ASU Capacity 6,890.000GB, $/SPC-1 IOPS $6.62, Protection Mirroring, Cost $307,710, Ident. A00051. IBM DS4800 45,014.81 SPC-1 IOPs, ASU Capacity 6,871.227GB, $/SPC-1 IOPS $16.03, Protection Mirroring, Cost $721,618, Ident. A00050.

Results Summary

    Certified Results
    System: Sun StorageTek 6540 - ST6540
    2 Controllers
    224 73GB 15K 4Gb RPM FC drives
    4Gb front- and back-end
    Performance: 46,491.72 SPC-1 IOPS
    Price/Performance: $6.62 $/SPC-1 IOPS
    ASU Capacity: 6,890.00GB
    Data Protection Level: Mirroring
    TSC Price: $307,710.00
    Results Identifier: A00051
    Server: Sun Fire 6900
    24x1.2GHz UltraSPARC IV
    48GB memory
    8 HBAs
    Operating System: Solaris 10 11/06

[1] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 08:00AM Sep 06, 2007 by BM Seer in Benchmarks
Tags:

New storage performance results

Friday Aug 24, 2007

New storage benchmark results sparked a story in Computer Business Review. They say they show, "twice the bang per buck of rival HP hardware."

    Sun's Sun's storage benchmark manager Leah Schoeb said, "The trays, the software, the packaging we do ourselves. That's why we're consistently half the price of the competition. That's what OEM'ing is about," she said.
You can see the full article "Sun brags of benchmarks", By Tim Stammers (Aug 23, 07) at... http://www.computer-business-review.com/article_news.asp?guid=12D86EB7-96A8-4DD9-9446-989568BBFB81

Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 09:00AM Aug 24, 2007 by BM Seer in Benchmarks
Tags:

Storage Performance: first Mirroring FC/SAS 4Gb result

Thursday Apr 26, 2007

Here we show the first Mirroring FC/SAS 4Gb result with world record $/Perf beating HP by 2.5x in performance. The Sun StorageTek 2540 mid-range product coupled with Sun's 4Gb HBAs has demonstrated industry leading SPC-2 benchmarking. The Sun StorageTek 2540 has World Record $/performance of $46.26 $/SPC-2 MBPS and best-in-class performance of 730.04 SPC-2 MBPS.

Sun achieved this result using a RAID 5 storage configuration to protect all data against disk failures. Outside of Sun's prior submissions all other competitors used the same data protection showing an apples-to-apples comparison. Sun's StorageTek 2540 submissions can suffer a disk failure without incurring a lengthy service outage, during which time a full rollforward recovery from a backup must be performed.

SPC-2 Performance Data Protection Level: Mirroring
(in increasing $/performance order, only results to date)

System SPC-2 MBPS $/SPC-2 MBPS ASU Capacity (GB) TSC Price Date Results Identifier
Sun ST2540 730.04 $46.26 1282.048 $33,772 4/10/07 B00022
HP SW1000 285.39 $73.12 362.925 $20,868 3/09/07 B00020

SPC-2 MBPS = the Performance Metric
$/SPC-2 MBPS = the Price/Performance Metric
ASU Capacity = the Capacity Metric
Data Protection = Data Protection Metric
TSC Price = Total Cost of Ownership Metric
Results Identifier = A unique identification of the result Metric

Benchmark Description

The SPC Benchmark-2 (SPC-2) is a series of related benchmark performance tests that simulate the sequential component of demands placed upon on-line, non-volatile storage in server class computer systems. SPC-2 provides measurements in support of real world environments characterized by:

    Large numbers of concurrent sequential transfers.

    Demanding data rate requirements, including requirements for real time processing.

    Diverse application techniques for sequential processing.

    Substantial storage capacity requirements.

    Data persistence requirements to ensure preservation of data without corruption or loss.

Disclosure Statement:

Sun StorageTek 2540 730.04 SPC-2 MBPS, $/SPC-2 MBPS $46.26, ASU Capacity 1,282.048GB, Protect Mirroring, Cost $33,772.00, Ident. B00022. SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of Storage Performance Council (SPC). More info http://storageperformance.org.

Certified Results
System: Sun StorageTek 2540
2 Controllers
36 73GB 15K RPM SAS drives
2x4Gb front-end ports
2x4Gb back-end ports
512MB controller cache (per controller)
Performance: 730.04 SPC-2 MBPS
Price/Performance: $46.26 $/SPC-2 MBPS
ASU Capacity: 1,282.048 GB
Data Protection Level: Mirroring
TSC Price: $33,772.00
Results Identifier: B00022
Server: Sun Fire X4600, 8x2.6GHz Opteron 885, 47.5GB
Operating System: Solaris 10 11/06

Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 10:13AM Apr 26, 2007 by BM Seer in Benchmarks
Tags:

Sun StorageTek 6140 $/Perf leadership

Wednesday Feb 14, 2007

he Sun StorageTek 6140 has demonstrated $/perf industry-leading results on the SPC-2 benchmark. Sun maintains the world record price/performance with new SPC-2 ST6140 results. These results beat the nearest competitor (Fujitsu) by 40% in price/performance and beat IBM by more than 2x in price/performance.

  • Sun StorageTek 6140 (mirroring) has World Record $/performance of $70.25/SPC-2 MBPS™
  • Sun StorageTek 6140 (RAID-5) has World Record $/performance of $67.82/SPC-2 MBPS™
  • Sun achieved this result using a RAID 5 storage configuration to protect all data against disk failures.

Sun holds 7 out of the top 10 in price/performance.

Another example of Sun using standard components (like controllers) and then innovating to differentiate the products and make Sun's products price/performance leaders!

SPC-2 Performance

    SPC-2 MBPS = Performance Metric
    $/SPC-2 MBPS = $/Perf
    ASU Capacity = storage capacity
    TSC Price = Total Cost of Ownership
    Data Protection = Protection (RAID 0, Mirroring, RAID 5, other)
    Results Identifier = unique result identification

Sponsor System SPC-2 MBPS $/SPC-2 MBPS ASU Cap. (GB) TSC Price Data Protection Level Pub. Date Results Ident.
Sun StorageTek 790.67 $67.82 1,675.037 $53,622 RAID 5 2/14/07 B00017
Sun StorageTek 763.28 $70.25 1,116.691 $53,622 Mirroring 2/14/07 B00016
Fujitsu ETERNUS4000MODEL500 1985.63 $114.72 2,284.923 $227,783 Mirroring 11/2/06 B00013
IBM DS4700EXPRESSMODEL70 804.29 $169.47 3,510.683 $136,305 RAID 5 10/27/06 B00012

Complete SPC-2 benchmark results at http://www.storageperformance.org/results/benchmark_results_spc2.

http://www.sun.com/storagetek/disk_systems/.

Benchmark Description

The SPC Benchmark-2™ (SPC-2) is a series of related benchmark performance tests that simulate the sequential component of demands placed upon on-line, non-volatile storage in server class computer systems. SPC-2 provides measurements in support of real world environments characterized by:

  • Large numbers of concurrent sequential transfers.

  • Demanding data rate requirements, including requirements for real time processing.

  • Diverse application techniques for sequential processing.

  • Substantial storage capacity requirements.

  • Data persistence requirements to ensure preservation of data without corruption or loss.

See Also

Sun StorageTek 6140 (mirroring SPC-2 Executive Summary (6 pages) Acrobat PDF

Sun StorageTek 6140 (RAID 5) SPC-2 Executive Summary (6 pages) Acrobat PDF

Complete SPC-2 Full Disclosure Report Acrobat PDF

Disclosure Statement:

Sun StorageTek 6140 763.28 SPC-2 MBPS, $/SPC-2 MBPS $70.25, ASU Capacity 1,116.691GB, Protect Mirroring, Cost $53,622.00, Ident. B00016. Sun StorageTek 6140 790.67 SPC-2 MBPS, $/SPC-2 MBPS $67.82, ASU Capacity 1,675.037GB, Protect RAID5, Cost $53,622.00, Ident. B00017. Fujitsu ETERNUS4000MODEL500 1985.63 SPC-2 MBPS, $114.72/SPC-2 MBPS, ASU Capacity 2,284.923GB, Protect Mirroring, Cost $227,783.00, Ident. B00013. IBM DS4700EXPRESSMODEL70, 804.29SPC-2 MBPS, $169.47/SPC-2 MBPS, ASU Capacity 3,510.683GB, Protect RAID 5, Cost $136,305.00, Ident. B00012. SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of Storage Performance Council (SPC). More info www.storageperformance.org

Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 11:07AM Feb 14, 2007 by BM Seer in Benchmarks
Tags:

SPC-2 Sun StorageTek 6540 (RAID 5) World Record $/performance

Tuesday Jan 23, 2007

The power of the Sun StorageTek 6540 mid-range product coupled with Sun's 4Gb HBAs has demonstrated industry leading SPC-2 benchmarking capabilities.

  • Sun StorageTek 6540 has World Record $/performance of $92.41
  • Sun StorageTek 6540 has best-in-class performance of 1,360.40 SPC-2 MBPS.
  • First product to break the $100 price/performance barrier.
  • Sun achieved this result using a RAID 5 storage configuration to protect all data against disk failures. Outside of Sun's prior submissions all other competitors used the same data protection showing an apples-to-apples comparison. Sun's StorageTek 6540 submissions can suffer a disk failure without incurring a lengthy service outage, during which time a full rollforward recovery from a backup must be performed just like the.

SPC-2 Performance Chart (in increasing price-performance order)

    SPC-2 MBPS = the Performance Metric
    $/SPC-2 MBPS = the Price/Performance Metric
    ASU Capacity = the Capacity Metric
    TSC Price = Total Cost of Ownership Metric
    Data Protection = Data Protection Metric (RAID 0, Mirroring, RAID 5, other)
    Results Identifier = A unique identification of the result metric

Sponsor System SPC-2 MBPS $/SPC-2 MBPS ASU Cap. (GB) TSC Price Data Protection Level Pub. Date Results Ident.
Sun StorageTek 1360.40 $92.41 3,350.074 $125,719.00 RAID 5 1/18/06 B00015
HP EVA 8000 1137.78 $346.43 6,571.275 $394,152.00 RAID 5 12/5/06 B00004

Complete SPC-2 benchmark results may be found at http://www.storageperformance.org.

Benchmark Description

The SPC Benchmark-2™ (SPC-2) is a series of related benchmark performance tests that simulate the sequential component of demands placed upon on-line, non-volatile storage in server class computer systems. SPC-2 provides measurements in support of real world environments characterized by:

  • Large numbers of concurrent sequential transfers.

  • Demanding data rate requirements, including requirements for real time processing.

  • Diverse application techniques for sequential processing.

  • Substantial storage capacity requirements.

  • Data persistence requirements to ensure preservation of data without corruption or loss.

See Also

Sun StorageTek 6540 SPC-2 Executive Summary (6 pages) Acrob at PDF

Complete Sun StorageTek 6540 SPC-2 Full Disclosure Report Acrob at PDF

Disclosure Statement:

Sun StorageTek 6540 1360.40 SPC-2 MBPS, $/SPC-2 MBPS $92.41, ASU Capacity 3,350.074GB, Protect RAID 5, Cost $125,719.00, Ident. B00015. SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of Storage Performance Council (SPC). More info www.storageperformance.org

    Certified Results
      System:   Sun StorageTek 6540, 2 controllers (4GB cache total, 2GB each), 64 72GB 15K RPM FC drives, 8 4Gb front-end ports, 4 4Gb back-end ports
      Performance:   1360.40 SPC-2 MBPS
      Price/performance:   $92.41 $/SPC-2 MBPS
      ASU Capacity:   3,350.074 GB
      Data Protection Level:   RAID 5
      TSC Price:   $125,719.00
      Results Identifier:   B00015
      Server:   Sun Fire E6900, 24 1.2GHz processors, 48GB memory, 8 4Gb HBAs
      Operating System:   Solaris 10, update 2

Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 01:00PM Jan 23, 2007 by BM Seer in Benchmarks
Tags: