Chi-Chang Lin's Weblog
RSA Performance of Sun Fire T2000
You might have heard that UltraSPARC T1 has special hardware circuitry to accelerate certain crypto operations. In this blog I will show you what operations it is good at, and how good.
UltraSPARC T1 comes with Modular Arithmetic Unit (MAU) per core which can accelerate expensive modular arithmetic operations found in public key crypto algorithms such as RSA, DSA and DH. In Solaris, the utilization of MAU has to go through Niagara Cryptographic Provider (NCP) within Solaris Cryptographic Framework (SCF). Currently only RSA (up to 2048 bit) and DSA (up to 1024 bit) are supported by NCP.
On the Sun Fire T2000/T1000 with the UltraSPARC T1 processor, you can readily get a glimpse of the fast RSA operations performed by MAU. Here is an example for the popular 1024-bit and 2048-bit RSA on a Sun Fire T2000 with 1.2 GHz UltraSPARC T1 with 8 cores:
[watercloset]~> /usr/sfw/bin/openssl speed rsa1024 rsa2048 -engine pkcs11
engine "pkcs11" set.
Doing 1024 bit private rsa's for 10s: 10332 1024 bit private RSA's in 0.45s
Doing 1024 bit public rsa's for 10s: 25550 1024 bit public RSA's in 0.89s
Doing 2048 bit private rsa's for 10s: 2371 2048 bit private RSA's in 0.11s
Doing 2048 bit public rsa's for 10s: 10308 2048 bit public RSA's in 0.37s
OpenSSL 0.9.7d 17 Mar 2004
built on: date not available
options:bn(64,32) md2(int) rc4(ptr,char) des(ptr,risc1,16,long)
aes(partial) blowfish(ptr)
compiler: information not available
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 1024 bits 0.0000s 0.0000s 22960.0 28707.9
rsa 2048 bits 0.0000s 0.0000s 21554.5 27859.5
This invokes the OpenSSL speed test bundled with Solaris. The OpenSSL bundled
with Solaris has
PKCS#11 engine built-in which is necessary to access SCF (and thus MAU); if
you download OpenSSL package and build it yourself, you will not be able to
take advantage of MAU because it does not have PKCS#11 engine.
Let's examine the performance numbers above. What we just did was to test
the single-threaded RSA performance. Each RSA operation is run for
10 seconds. However, due to the timing errors in OpenSSL speed test in the
single-threaded case,
the throughput numbers at the bottom cannot be trusted when the operations are
done in hardware. After some re-calculations we get:
sign verify sign/s verify/s
rsa 1024 bits 0.0000s 0.0000s 1033.2 2550.0
rsa 2048 bits 0.0000s 0.0000s 237.1 1030.8
Are these numbers good? They are actually very good. Take 1024-bit RSA sign operation number, 1033.2, and compare it with the number on 3.6 GHz Xeon Dell box - 843.0. UltraSPARC T1 offers 20% more RSA performance at 1/3 clock rate and uses less power. Note that this is single-threaded test. As shown below, UltraSPARC T1 really dwarfs others in the multi-process test.
Now, let's look at multi-process RSA performance. This is where UltraSPARC T1 really shines. Do an OpenSSL speed test again, this time with the "-multi" option to invoke multiple processes to conduct RSA operations concurrently:
[watercloset]~> /usr/sfw/bin/openssl speed rsa1024 rsa2048 -engine pkcs11 -multi 32
[ intermediate output snipped..... ]
sign verify sign/s verify/s
rsa 1024 bits 0.0001s 0.0000s 12871.3 45148.1
rsa 2048 bits 0.0004s 0.0000s 2299.6 20425.3
We have used 32 processes to fully saturate the 32 hardware threads on UltraSPARC T1 to get the maximum throughput. Compare this with the results on the 2-way 3.6 GHz Xeon Dell PowerEdge 2850 (with hyperthreading on):
wgs93-187:~ openssl speed rsa1024 rsa2048 -multi 4
[ intermediate output snipped..... ]
sign verify sign/s verify/s
rsa 1024 bits 0.0005s 0.0000s 1943.2 34632.7
rsa 2048 bits 0.0031s 0.0001s 327.4 10891.9
For 1024-bit RSA sign operation (as commonly used in web server SSL handshaking), Sun Fire T2000 outperforms Dell PowerEdge 2850 by a whopping 6x! UltraSPARC T1 also excels when compared with the Sun Crypto Accelerator 4000, which can do 8000 1024-RSA signs/s. And remember, all this comes with just the Sun Fire T2000/T1000 box, no extra crypto accelerator card is needed.
In summary, if RSA/DSA operations consume a certain amount of CPU cycles in your application (e.g. HTTPS), Sun Fire T2000/T1000 with UltraSPARC T1 will offer you the biggest bang for the bucks with its per-core MAU and unique 8-core CMT architecture.
Posted at 10:24AM Dec 06, 2005 by chichang_lin in Sun | Comments[12]
Tuesday Dec 06, 2005
Posted by Kenneth on December 06, 2005 at 01:05 PM PST #
Posted by bbr on December 07, 2005 at 05:07 AM PST #
Posted by chichang_lin on December 07, 2005 at 10:43 AM PST #
Those 1.2 GHz UltraSPARC T1 performance numbers are very interesting, and I have compared them with those of produced by a 1.8 GHz Opteron (running openssl 0.9.7d in 64-bit mode).
In the single-threaded case, the T1 is able to do 1033.2 1024-bit RSA signs/s, which is very good and similar to what is accomplished by the Opteron: about 1100 signs/s. However the T1 is only able to do 2550.0 verifies/s, whereas the Opteron does about 18400 verifies/s.
chichang_lin: why is the T1 so slow on the RSA verify operations ? Is it still using the special hardware circuitry in this case ? My 2048-bit RSA tests exhibit the same performance pattern (same sign speed, worse verify speed).
PS: of course in multi-threaded cases the 8-core T1 largely outperforms the single-core Opteron.
Posted by marc on December 14, 2005 at 11:37 PM PST #
Posted by rick on December 15, 2005 at 01:19 PM PST #
Posted by chichang_lin on December 15, 2005 at 06:32 PM PST #
Posted by chichang_lin on December 15, 2005 at 06:44 PM PST #
chichang_lin: thanks for the explanation, it makes sense. But then I am wondering, if setting up the MAU is too much overhead, would it be faster to use a software implementation for RSA verify operations ? In other word, does speed(1) reports better verifies/s numbers _whithout_ "-engine pkcs11" ? Of course the disadvantage of not using the MAU is that RSA operations would use traditional CPU execution units which could be more useful for other processes, but I am just asking because I am curious.
Oh and another thing, regarding the timing errors, my guess is that these are due to the fact that by default speed(1) takes the "user time" and not the "real time" (as reported by the kernel) to compute the performance numbers; and since your 'pkcs11' openssl engine actually spends most of its time waiting for the MAU to complete its computations (I think you guys implemented it this way, didn't you ?) then the whole openssl process ends up consuming very little "user time", which is why openssl reports 0.45s instead of 10s. In order to fix this you can instruct openssl to take the "real time" by appending the "-elapsed" argument to your speed(1) command line. Can you try it ?
Damn the more I look at it, the more I think Niagara is the greatest general purpose processor released of this decade...
Posted by marc on December 16, 2005 at 12:45 AM PST #
marc: You have a good point. Actually during the design/implementation phase the engineers had considered this and found that doing RSA verify in MAU still offers better performance: without "-engine pkcs11" 1024-bit RSA verify only has 1068 ops/s compared to 2550 ops/s with MAU.
The reason for the timing error is indeed as you said. With the "-elapsed" option the timings are all correct now.
Thanks for all the feedbacks and comments! Are you planning to purchase a Sun Fire T2000 soon? :-)
Posted by chichang_lin on December 16, 2005 at 02:31 PM PST #
Posted by marc on December 16, 2005 at 03:46 PM PST #
Hi,
just run the code, you provided here on a T5140......
I mean, your input is from 2005!
Are there special things with regards to T2 processors?
Thanks
Helmut
Posted by Helmut Kirrmaier on April 02, 2009 at 09:15 AM PDT #
To Helmut,
Everything should be the same on T2-based system. You should see even higher numbers on T2.
Posted by chichang_lin on May 03, 2009 at 12:17 PM PDT #