What Was The Best Performance You Ever Had With OpenDS ?
Rationale
After discussing the article I posted yesterday with someone, they asked me: "What was the best performance you ever had with OpenDS?" and though I couldn't really answer off the top of my head, I dug in my archives from the last benchmark and found what I think was my best run so far.
Bird's Eye View
To put it bluntly, about 120,000 operations per second @ <2ms. As this was done while I was tuning OpenDS for the 10 Million entries benchmark on Intel Nehalem-based Sun Blade x6270, I therefore had the whole setup, 10M entries, searches span across the entire DB and some of the Java tunings are bleeding edge, as I will detail in the next section.
The Meat
Environment
As I said earlier, this is the same environment as described in my previous entry except for Java.
Java
The JVM arguments are as follows: -d64 -server -Xbootclasspath/p:/path/to/SecretSauce.jar -XX:+UseNUMA -XX:LargePageSizeInBytes=2m -XX:+UseCompressedOops -XX:+AggressiveOpts -XX:+UseBiasedLocking -Xms6g -Xmx6g -Xmn4g -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=85 -XX:MaxTenuringThreshold=1
It's all pretty much business as usual but some of them call for explanation:
- -Xbootclasspath/p:/path/to/SecretSauce.jar: One of our engineers, our lead for OpenDS core actually, has found a significant performance improvement in one of the JVM's core classes. This SecretSauce.jar contains his patched and improved version that overrides the JVM's own at run time. This makes a big difference in lowering GC pause times.
- -XX:+UseNUMA: this is simply because the Sun Blade x6270 is a NUMA architecture and using this switch tells the JVM to be clever about memory and cache locality.
- -XX:+UseCompressedOops: This allows to benefit of the 64-bit JVM larger Heap size, actually not quite as big but bigger than that of the 32-bit JVM while retaining 32-bit like performance. The best of both worlds. Works beautifully. And it is being improved ...
Results
| Searches Completed | |
|
||||||||||||
| Search Time (ms) | |
|
Caveats
So, now that I told you all my secrets, you're wondering why I didn't use those settings for the benchmark? Because the benchmark is supposed to give me numbers on what could be achieved in a production environment, and in this case, using our patched JVM core class and a somewhat experimental or at least relatively new memory addressing mode of the JVM isn't what I would advise to a customer about to go live.
All these bleeding edge settings only give us a 12% boost overall, I don't think it is worth the risk. But this shows that we are paving the way for an ever increasing performance on OpenDS. Tomorrow, these settings will all be well proven and safe for production. Just hang in there.
