The Good, the Blog & the Ugly - Tim Cook's Weblog

Scalability and Stability for SysBench on Solaris

My mind is playing "Suffering Succotash..."

I have been working on MySQL performance for a while now, and the team I am in have discovered that SysBench could do with a couple of tweaks for Solaris.

Sidebar - sysbench is a simple "OLTP" benchmark which can test multiple databases, including MySQL. Find out all about it here , but go to the download page to get the latest version.

To simulate multiple users sending requests to a database, sysbench uses multiple threads. This leads to two issues we have identified with SysBench on Solaris, namely:

• The implementation of random() is explicitly identified as unsafe in multi-threaded applications on Solaris. My team has found this is a real issue, with occasional core-dumps happening to our multi-threaded SysBench runs.
• SysBench does quite a bit of memory allocation, and could do with a more scalable memory allocator.

Neither of these issues are necessarily relevant only to Solaris, by the way.

Luckily there are simple solutions. We can fix the random() issue by using lrand48() - in effect a drop-in replacement. Then we can fix the memory allocator by simply choosing to link with a better allocator on Solaris.

To help with a decision on memory allocator, I ran a few simple tests to check the performance of the two best-known scalable allocators available in Solaris. Here are the results ("libc" is the default memory allocator):

To see the differences more clearly, lets do a relative comparison, using "umem" (A.K.A. libumem) as the reference:

So - around 20% less throughput at 16 or 32 threads. Very little difference at 1 thread, too (where the default memory allocator should be the one with the lowest synchronization overhead).

Where you see another big difference is CPU cost per transaction:

I will just point out two other reasons why I would recommend libumem:

• It's portable - available for at least Linux, OS X and Windows. This means any cross-OS comparisons could eliminate the memory allocator as a possible difference.
• You get debugging and profiling for free. See these links:
  • Identifying Memory Management Bugs Within Applications Using the libumem Library
  • Debugging with libumem and MDB
  • Adam Leventhal's Weblog - Number 11 of 20: libumem

I have logged these two issues as sysbench bugs:

• 2422927 use of random() & srandom() is not MT-Safe on Solaris
• 2422935 not using scalable memory allocator on Solaris

However, if you can't wait for the fixes to be released, try these:

• Patch to make SysBench 0.4.8 use lrand48() instead of random()
• Script to configure SysBench 0.4.8 so that it will be linked with libumem

Posted at 03:20PM Dec 12, 2008 by timc in MySQL  |  Comments[2]