cn=Directory Manager
All about Directory Server
All | Personal | Sun

20051208 Thursday December 08, 2005

Little-known performance enhancements #2: Solaris tuning

This set of tips is kind of a cop-out since I'm not really talking about tuning the Directory Server itself all that much. However, if you're running the Directory Server on Solaris (and you should be), then there are a couple of simple things that you might not know about that can provide notable performance improvements. And unlike my last post, which was around write performance, this one should provide an all-around benefit. However these improvements will be most noticeable for CPU-intensive operations like searches, binds, and compares.

The first thing that you can do is to make the server run under the fixed-priority scheduler rather than the default time-sharing scheduler. In my experience, this generally gives you about a 5% increase in maximum search performance. And it's really simple. Just add the following near the top of your start-slapd script (and I also usually put in ldif2db as well, since it can help out there too):
# Use the fixed-priority scheduler.
priocntl -s -c FX $$

In this case, the "$$" is a special shell variable that expands to the PID of the current process (i.e., the start-slapd script itself), and the scheduler for that process will be inherited by anything that it spawns, including the Directory Server. Note that while you can provide additional configuration like setting a specific priority, you often need special privileges for doing that and even scheduling with the maximum priority doesn't seem to make much difference versus the default.

The second tip applies primarily if you're running the server on Solaris x86 (which is an excellent choice for our Opteron-based systems). By default, the Directory Server will use the mtmalloc memory manager, which was intended to help provide better memory allocation performance than the standard single-threaded malloc. Unfortunately, it doesn't quite live up to its intended purpose. However, Solaris 10 introduces the new libumem memory manager, and it has been backported to most versions of Solaris 9 (I believe update 3 or later). It is far better than mtmalloc in virtually every respect, and it will generally allow the server to go faster and consume less memory. To configure the server to use libumem rather than mtmalloc, edit the start-slapd script to add the following:
# Use the libumem memory manager.
LD_PRELOAD=/usr/lib/libumem.so
LD_PRELOAD_64=/usr/lib/64/libumem.so
export LD_PRELOAD LD_PRELOAD_64

Note that you can do this with Solaris on SPARC-based systems as well, but in general this is not necessary because on that platform we use a third-party allocator that accomplishes the same thing.

For those of you that may still be running Solaris 8, there's another potential boost that you can take advantage of (until, of course, you decide to upgrade to Solaris 10 for even better performance and tons of great new features like least privilege and zones). You can configure the server to use the alternate thread library, which became the default in Solaris 9. To do this, add the following to your start-slapd:
# Use the alternate thread library.
LD_LIBRARY_PATH=/usr/lib/lwp
LD_LIBRARY_PATH_64=/usr/lib/lwp/64
export LD_LIBRARY_PATH LD_LIBRARY_PATH_64

Note that in this case, though, you'll also need to remove another line already in the start-slapd script that unsets the LD_LIBRARY_PATH so they don't get undone before they can be of any use.

Posted by cn_equals_directory_manager ( Dec 08 2005, 12:08:57 AM CST ) Permalink Comments [2]

Comments:

Let me share my experience with mtmalloc/umem/libgc

> .. it has been backported to most versions of Solaris 9 (I believe update 3 or later) ..

Yes; it was made available from Solaris 9 U3.

> .. It (libumem) is far better than mtmalloc in virtually every respect, ..

May be umem is performing better with Directory Server. But for certain CPU intensive workloads with hundreds of lwps per process, mtmalloc might be a good choice.

> .. and it (umem) will generally allow the server to go faster and consume less memory ..

At first look, a normal user may think that umem actually reduces the memory footprint. In this context, both mtmalloc and umem, increase the per process memory footprint; but the increase in memory with umem is much less compared to mtmalloc's.

Now that Sun Studio 11 compiler collection is freely downloadable, I guess experimenting with libgc would produce some interesting results. The application may not run as fast as it would with umem, but the increase in memory footprint is almost negligible. Unfortunately libgc has no support from Sun.

Posted by Giri Mandalika on December 08, 2005 at 03:05 AM CST #

Thanks for your comments. First, I didn't mean to imply that mtmalloc wasn't useful at all. Rather, I was speaking in the context of Directory Server, in which extensive testing has shown libumem to provide much better performance and a smaller overall process size. And in this case, the process size is very important because at least until we have a 64-bit build of Directory Server on Solaris x86, the amount of memory that you can use is constrained (to either 3GB for 32-bit kernels and some 64-bit kernels on older x64 systems, or 4GB for 64-bit kernels on newer x64 systems). That can be a big deal for Directory Server, which often reaps huge benefits from being able to use memory for caching.

In the vast majority of installations, the number of threads used by Directory Server is in the 35-50 range, depending on the configuration. We don't generally recommend using huge numbers of worker threads because it will usually degrade performance.

Posted by Neil Wilson on December 08, 2005 at 08:45 AM CST #

Post a Comment:

Comments are closed for this entry.

Archives
Language
Links
Referrers