Thursday December 15, 2005
Thursday December 15, 2005 Sun T2000 vs Dell 6850: LDAP AuthRate
As I mentioned in my last post, at the Sun Fire™ T2000 launch last week, we had our own demo at the Austin campus to help show it off. We pitted the T2000 against the Dell PowerEdge™ 6850 server, which we believe to be the best system that Dell has to offer. Here's a side-by-side comparison of the system specs:| Dell PowerEdge™ 6850 | Sun Fire™ T2000 | |
| CPU Type | Intel® Xeon® EM64T | UltraSPARC®
T1 with CoolThreads™ technology |
| Total CPU Sockets | 4 | 1 |
| Total CPU Cores | 8 | 8 |
| Total Hardware Threads | 8 | 32 |
| CPU Clock Rate (GHz) | 3.2 | 1.0 |
| DDR2 Memory Available (GB) | 32 | 32 |
| System Height (Rack Units) | 4 | 2 |
| Approx. Idle Power Draw (Watts) | 320 | 220 |
| Approx. Loaded Power Draw (Watts) | 600 | 267 |
| Operating System (Pre-Installed) |
Microsoft® Windows® Server 2003 Standard x64 Edition |
Sun Solaris™ 10 3/05 HW 2 |
| List Price (US Dollars) | $33,652 | $25,995 |
A few notes on the information in this table:
- The Sun Fire™ T2000 server usually ships with a 1.2 GHz UltraSPARC T1 processor. However, as this was a pre-release system we only had a 1.0 GHz processor.
- The Dell PowerEdge™ 6850 was actually observed to draw in excess of 640 Watts under load. However, for the purpose of the SWaP calculation, a value of 600 Watts was used.
- The Dell PowerEdge™ 6850 was actually observed to draw in excess of 640 Watts under load. However, for the purpose of the SWaP calculation, a value of 600 Watts was used. It should also be noted that this system requires 200-240VAC power.
- The Sun Fire™ T2000 was actually observed to draw around 240 Watts under load. However, the system spec sheet lists a maximum consumption of 267 Watts, and therefore that value was used for our SWaP calculations.
- The provided list price for the Sun Fire™ T2000 server is for a system with a 1.2 GHz CPU whereas the system we were testing had only a 1.0 GHz processor.
In order to compare the performance of these two systems, we measured LDAP authentication performance running the Sun Java™ System Directory Server 5.2 patch 4. The workload for this test was very similar to that used by the SLAMD LDAP Weighted AuthRate job, but because this was a public demo, we used a custom client to show the relative performance of these systems in real time (I may be able to post a Shockwave Flash recording of this demo later this week). In this workload, each authentication consists of a subtree equality LDAP search operation on an indexed attribute (in order to locate the user entry based on a login ID) followed by a bind as that user. The login ID value for each authentication was selected at random from the entire data set, with a weighted access pattern such that 80% of the search operations were targeted at a set of 20% of the user entries, which reflects many measured real-world access patterns. Two Sun Fire™ V20z servers were used to generate the load against the Directory Server instances (1 V20z for each server).
The Directory Server instances were installed and optimally tuned for each system. A total of 250,000 user entries (generated using the MakeLDIF tool provided with SLAMD using the default example.template template file) were loaded into each directory. We would have used a much larger number of entries, but this is near the maximum cacheable amount for our Directory Server on a Windows system, and exceeding that would have given the T2000 system a large unfair advantage. In fact, tests with a data set of 1 million users showed that the server running on the T2000 system was able to achieve even higher authentication performance than with a set of 250,000 users, whereas the server on the Dell system exhibited severely degraded performance compared with that measured with a set of 250,000 users.
Each server was asked to process a total of 250,000 user authentications as quickly as possible. Both total length of time required to process these operations and the average number of authentications per second were measured. The average number of authentications per second was used as the "performance" component of the SWaP (space, Watts, and performance) metric. The SWaP value for each system was calculated by dividing the average number of authentications per second by the product of the space consumed (in rack units) and the power consumption under load (in Watts).
The maximum LDAP authentication performance that we were able to achieve from each system is as follows:
| Dell PowerEdge™ 6850 | Sun Fire™ T2000 | |
| Performance (LDAP auths/second) | 2853.34 | 8067.58 |
| Total Processing Time (seconds) | 87.65 | 31.00 |
| SWaP | 1.19 | 15.11 |
As can be seen from this information, when it comes to LDAP authentication performance the Sun Fire™ T2000 server beats the Dell PowerEdge™ 6850 server in all areas that we compared:
- The Sun Fire™ T2000 server delivers over 2.8 times better LDAP authentication performance than the Dell PowerEdge™ 6850, even when you consider that the Sun system only had a 1.0 GHz processor whereas the T2000 normally comes with a 1.2 GHz processor.
- The list price for the Sun Fire™ T2000 server is $7657 less than the list price for the Dell PowerEdge™ 6850 server, even when you consider that the price for the Sun system includes a faster CPU than the one that we were able to test.
- The Dell PowerEdge™ 6850 requires twice as much rack space as the Sun Fire™ T2000 server.
- The Dell PowerEdge™ 6850 consumes nearly 1.5 times as much power (measured in Watts) than the Sun Fire™ T2000 server when both systems are idle.
- The Dell PowerEdge™ 6850 consumes 2.25 times as much power (measured in Watts) under load as the Sun Fire™ T2000 server, even with an optimistic estimate for the Dell system and a pessimistic estimate for the Sun system.
- The Dell PowerEdge™ 6850 system requires 200-240VAC power, which is less commonly-available than 110V power, particularly in environments that typically run on x86/x64 systems. The Sun Fire™ T2000 server accepts 100-240VAC power and therefore can use standard 110V circuits.
When you look at any one of these metrics, the Sun Fire™ T2000 certainly looks attractive. However, the real value of this system is even more apparent if you compare what you would need in order to meet a given level of performance. For example, if your directory environment needs to be able to handle 10,000 authentications per second under peak load, then a solution with two Sun Fire™ T2000 systems would be over $82000 cheaper to buy up front than the four Dell PowerEdge™ 6850 systems that would be required, and you'd also save more than 1.8 kiloWatts of power and 12 rack units of space with the Sun solution.
Posted by cn_equals_directory_manager ( Dec 15 2005, 11:46:33 PM CST ) Permalink Comments [7]
Post a Comment:
Comments are closed for this entry.
| Archives | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Language | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Links | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Referrers | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Today's Page Hits: 148 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Posted by W. Wayne Liauh on December 16, 2005 at 01:40 PM CST #
While the naive user might run with whatever the "standard" option is, I'd expect most technical purchasers would be considering running Solaris (or Linux) on the Dell. I'd love to know what the 6850 could achieve under those circumstances.
Incidentally, does the quoted cost of the Dell include the Windows Server license? If so, how much is that worth?
The T2000 (and T1000) are clearly awesome machines, but none of the comparative benchmarks Sun has produced have been truly even-handed. I'll certainly be looking to get one in to do my own benchmarking, but I wish I wasn't forced to do so. Sun isn't the worst when it comes to flagrant benchmarkering, but they could do a whole lot better.
Nonetheless, thankyou for presenting this information - it might fall down as a T2000 vs PE6850 comparison, but it still has merit in the Sun/Solaris vs Dell/Windows stakes.
Posted by Zac Stevens on December 16, 2005 at 11:45 PM CST #
I do believe that the cost of the Windows license was included in the quoted list price for the Dell system, just as the cost for the Solaris license and support contract was included in the price for the T2000. According to the Dell site, the cost of the Windows Server 2003 Standard x64 Edition with 5 client access licenses adds $799 to the purchase price for the system as compared with ordering it with no operating system. Conversely, Solaris 10 has a free right-to-use license and you only need to pay for support if you need it. Solaris support costs for a system like the T2000 range from $120-$360 per year (although support is already included in the price of the system).
We are also being shipped a couple of Sun's new dual-core Opteron-based Galaxy systems and will be running Solaris, Linux, and Windows tests on them as well.
Posted by Neil Wilson on December 17, 2005 at 01:20 AM CST #
Posted by Zac Stevens on December 17, 2005 at 07:20 AM CST #
I must apologise, as my response to you wasn't a fair reflection on the test you posted here. Perhaps the worst example from Sun in recent days has been the Blueprint on Apache webserver consolidation, which pitted the T1000 against a 1RU, dual-Xeon box (presumably a Dell 1850). The tested specs differed in two important ways:
1) The T1000 had four times the memory of the competition.
2) The Dell had two SCSI drives in a RAID1 configuration, versus a single SATA drive in the T1000.
While the first point is unlikely to have had any impact in the performance test (only static page delivery was being benchmarked), the second point would surely have skewed the SWaP scores.
The disappointment here, for me, is that Sun has finally returned as a force to be reckoned with - both in software and hardware - yet the marketing doesn't seem to have caught up. There's still too much spin, even though it's now accompanied by serious substance. :)
While world-beating SPEC results (and the like) are nice, I put a lot more weight in the sort of tests you're doing. I hope we see a lot more (from you and your colleagues), proving beyond any doubt that Sun is the winning team.
Posted by Zac Stevens on December 17, 2005 at 07:45 AM CST #
Posted by Bernd Eckenfels on December 21, 2005 at 04:24 AM CST #
Note that we have done testing in the past on other systems with CPUs that do have HyperThreading capability (e.g., older Sun Fire V65x systems, which have two HyperThreaded Xeon CPUs at 3.2 GHz) but we have seen mixed results. In some cases, HyperThreading does help, but most of the time it actually degrades performance. As it has been explained to me, there are two key problems with Intel's implementation of HyperThreading that prevent it from achieving maximum performance:
Our experience has been that you need to test the system with and without HyperThreading under the load that you expect to see in production to see whether it helps or hurts performance.
Posted by Neil Wilson on December 21, 2005 at 08:14 AM CST #