cn=Directory Manager
All about Directory Server
All | Personal | Sun

20060306 Monday March 06, 2006

Directory Server Denial of Service Protection

The Directory Proxy Server (which is included in the Directory Server Enterprise Edition) offers a number of features that can help protect your Directory Server instances from malicious or poorly-designed client applications that might otherwise consume lots of resources or otherwise interfere with the most efficient operation of the Directory Server. However, there are a lot of features built into the Directory Server itself that can also help. If you don't feel a need to use DPS, or if you have cases in which some clients might directly access the server without going through a proxy, then it's probably a good idea to make use of one or more of these.

Search Time Limit
Inefficient searches can take a long time to process. Depending on the amount of data involved, along with other factors, it can range from a few milliseconds to several hours or possibly even longer. This is certainly something that you'll want to prevent in most cases, and one way to limit that is to define a time limit for search operations. This can be done by setting the nsslapd-timelimit attribute in the cn=config entry to the maximum length of time in seconds that you want searches to be allowed to take. Note that this is a global setting, and it will automatically apply for all users (except the root DN, who is exempt from this limit). However, if there are special users for which you want to have a higher or lower limit, you can achieve this by putting the nsTimeLimit operational attribute in that user's entry so that limit will be enforced rather than the global limit. You should also note that it is also possible to specify a time limit in the search request itself, and if there is a conflict between the value configured in the Directory Server and the value included in the search request then the lower of the two will be the one that is enforced.

Search Result Size Limit
Another way that you can prevent clients from consuming large amounts of server resources is to place a limit on the number of entries that may be returned from any given search request. This is called the size limit, and the server-wide limit may be configured using the nsslapd-sizelimit attribute of the cn=config entry. As with the time limit, the size limit can also be overridden on a per-user basis by putting the nsSizeLimit operational attribute in that user's entry and setting it to the size limit that should be used instead of the server-wide value. And also like the time limit, the size limit can be specified in the search request itself, and the lower of the requested and server-enforced limits will be used in the event of a conflict.

Search Candidate Size Limit
The search size limit is a good way to limit the number of entries that may be returned to the client for a given operation, but it isn't necessarily a panacea because it only applies to entries that are actually returned to the client. It doesn't take into account entries that the server has to examine in the course of processing the search but that aren't returned for some reason (e.g., the user doesn't have permission to see them, or the server identified them as candidates during the index processing phase but it turned out that they didn't really match the criteria). In some cases, this could be a lot of entries, and the server has to expend effort on each of them while processing the search. However, as a means of protecting against situations like this, the Directory Server does have a configuration attribute that will limit the number of entries that the server will "look through" in the context of processing a search, regardless of whether or not they are actually sent to the client. This is controlled on a server-wide basis by the nsslapd-lookthoughlimit attribute in the cn=config,cn=ldbm database,cn=plugins,cn=config entry, but can be overridden on a per-user basis with the nsLookThroughLimit attribute in the user's entry.

Reject Unindexed Searches
The size limit, time limit, and lookthrough limit settings listed above can help significantly limit the amount of processing effort that the server expends while processing inefficient searches. Some of the most inefficient searches are those for which the server does not have appropriate indexes defined and therefore must look at a potentially very large result set (e.g., all entries in the database) in order to identify which ones match the criteria provided by the client. If this occurs, then the size/time/lookthrough limits will treat the symptoms of the problem, but you can also treat the root of the problem by configuring the server to reject unindexed searches. This can be done at a per-database level using the nsslapd-require-index attribute. If this setting is turned on, then any search request for which the server cannot use the defined indexes to identify a finite candidate list will be rejected. It is important to note that this cannot be overridden on a per user basis, so it should not be used if you have specific clients that might have a legitimate need to perform unindexed searches from time to time (although in those cases, it might be a good idea to restrict them to a single server instance and have all others configured to reject unindexed searches).

Idle Time Limit
In most cases, a well-behaved client should maintain one or more persistent connections to the server that can be used repeatedly to send many requests to the server. Less efficient clients will establish a separate connection for each request, although this may be acceptable for clients that don't need to communicate with the server all that often (i.e., several minutes or hours between requests). But the worst kind of client is one that establishes connections and then forgets about them after a while, establishing new connections for subsequent reqeusts without closing the existing ones. This is bad because clients like this can evenutally consume all the available file descriptors, which will interfere with the server's ability to accept new connections. To deal with this problem, the best approach is to set an idle timelimit (in seconds) using the nsslapd-idletimeout configuration attribute in the cn=config entry. If a client connection remains established for longer than this period of time without issuing any requests, then the server will automatically close it. If you do have individual applications that have legitimate need to have connections that may be idle for long periods of time, then you can override this setting on a per-user basis (in this case the "user" would be the account that the client application uses to bind to the server) with the nsIdleTimeout operational attribute in the user's entry.

Unresponsive Client Time Limit
Sometimes, certain denial-of-service conditions can occur through no fault of the client. For example, if a network failure occurs, or if a poorly-designed firewall or other network device closes drops connections without properly informing either end of the connection, then the server can have no way of knowing that a network connection is invalid. If the connection was idle at the time, then this isn't really a problem since the server won't try to use it and it will eventually be closed either by the idle timeout or when the underlying OS discovers that the connection is no longer valid. However, if this happens while the server is in the process of sending data to the client, or if the client simply stops reading data from the server when there is a lot more information to be sent, then the server can enter a condition in which further attempts to write to the client will block (i.e., "hang") until the data is read. This will tie up the associated worker thread, making it unavailable to process other requests. To prevent that from becoming a significant problem, the I/O block timeout may be used. This is controlled by the nsslapd-ioblocktimeout configuration attribute in the cn=config entry, and its value is the maximum length of time in milliseconds that attempts to send data to a client should be allowed to block before giving up and terminating the client connection. The current default value of 1800000 milliseconds (30 minutes) is far too high to be effective, and in future releases the default will be lowered to 30 seconds. It is recommended that existing deployments also define a more reasonable limit (e.g., something in the 30-120 second range).

Maximum Request Size
Because of the way that LDAP requests are encoded, it is necessary for the server to be able to hold the entire request in memory when it is decoded. Although this is generally not a problem, it does introduce an opportunity for malicious clients to try to send huge requests to the Directory Server in order for it to allocate large amounts of memory and to increase network load. To protect against this, the nsslapd-maxBERSize configuration attribute of the cn=config entry may be used to specify the maximum request size in bytes that the Directory Server wil allow. Any attempt to send a larger request than this size will cause the connection to be terminated. The default setting for this configuration parameter is two megabytes, which will be sufficient for most cases, but may be too small for directories that need to store large entries (e.g., those with static groups consisting of hundreds of thousands of members or more, or those that need to store large blobs of data like the configuration directory used by SLAMD).

Maximum Concurrent Operations per Client Connection
Because LDAP uses an asynchronous communication mechanism, there is no requirement for a client to wait on the response for one request before sending another. It is entirely possible for the Directory Server to be concurrently processing multiple requests for the same client using separate worker threads. A somewhat evil client may attempt to exploit this behavior by sending lots of requests to the server in an attempt to tie up all the worker threads and/or interfere with the ability of the server to process requests from other clients. However, this can be restricted using the nsslapd-maxThreadsPerConn configuration attribute in the cn=config entry. The value of this attribute may be used to specify the maximum number of worker threads that may be used to process requests from a single client at any given time. If a client attempts to exceed this limit, then its requests will be sent to the back of the work queue and they won't be eligible for processing until other processing for other requests on the same connection has completed.

Maximum Number of Persistent Searches
The persistent search control can be a useful way to receive notification of changes to entries in the Directory Server, but it can also be somewhat expensive because the server will create a separate thread for each persistent search operation. In order to prevent this from becoming a problem, the server can limit the number of persistent searches that may be registered at any given time with the nsslapd-maxpsearch configuration attribute in cn=config. If the maximum number of persistent search operations are already registered against the server, then subsequent attempts to create new ones will be rejected.

Disk Space Thresholds
A somewhat indirect form of denial-of-service may occur if one or more of the disk volumes used to hold Directory Server information become full. If a disk containing database contents should become full, then certain database operations will fail. If a disk holding log files fills up, then it won't be possible to write any more log information about the operations being processed. Fortunately, there are a few configuration parameters that can help prevent this from becoming a problem. In particular, the nsslapd-disk-low-threshold attribute in cn=config,cn=ldbm database,cn=plugins,cn=config will prevent write operations by all users except the root DN if free disk space drops below the specified amount, and the nsslapd-disk-full-threshold attribute in the same entry specifies the point at which even write operations by the root DN will begin to be rejected. In addition, the access, error, and audit log rotation mechanisms may be configured to automatically remove old log files if the amount of free disk space drops below a configured threshold (e.g., for the access log, this is the nsslapd-accesslog-logMinFreeDiskSpace attribute in cn=config), and it is also possible to specify the maximum amount of space that each type of log may consume with configuration attributes like nsslapd-accesslog-maxLogSize for the maximum size of a single access log file and nsslapd-accesslog-logMaxDiskSpace for the total space consumed by all access logs.

Posted by cn_equals_directory_manager ( Mar 06 2006, 08:15:15 AM CST ) Permalink Comments [3]

Comments:

One thing I did not see above and was wondering if the DPS can do it is limit the number of searchs a client does. For example if the client does more than 20 searches in an hour to block them from searching the LDAP. (For example on a public address book, in order to avoid harvesting of addresses by people doing queries like "(sn=a*)" to get all people with last name beginning with an A. Currently I am just looking at the access log and pulling out the number of connects and if it is more than x, blocking the IP with ipfilter. Is there an easier way? thanks jason

Posted by Jason Grove on March 06, 2006 at 10:29 PM CST #

One thing I did not see above and was wondering if the DPS can do it is limit the number of searchs a client does. For example if the client does more than 20 searches in an hour to block them from searching the LDAP. (For example on a public address book, in order to avoid harvesting of addresses by people doing queries like "(sn=a*)" to get all people with last name beginning with an A. Currently I am just looking at the access log and pulling out the number of connects and if it is more than x, blocking the IP with ipfilter. Is there an easier way? thanks jason

Posted by Jason Grove on March 06, 2006 at 10:33 PM CST #

Directory Proxy Server does not currently have a mechanism for rate limiting in the form that you requested. It can do things like limit the number of simultaneous operations per connection, total operations per connection, total concurrent connections (altogether, per network group, or per client), but it does not have a way to restrict the number of operations that can be performed in a given period of time.

It does have features to help prevent trawling kinds of searches that could be used by clients to retrieve all of your data with searches like "(sn=a*)". For example, you can require a minimum number of non-wildcard characters in the filter, or you can define a maximum size limit to control the number of entries that can be returned from a search.

See http://docs.sun.com/source/817-7615/AppFaq.html#wp17384 for more information on the kinds of constraints that you can configure with the Directory Proxy Server. In the future, we do hope to improve these kinds of capabilities and make it possible to better tailor the installation to meet your needs, so we can keep requests like this in mind for future versions.

Posted by Neil Wilson on March 06, 2006 at 11:22 PM CST #

Post a Comment:

Comments are closed for this entry.

Archives
Language
Links
Referrers