cn=Directory Manager
All about Directory Server
All | Personal | Sun

« Previous month (Mar 2007) | Main | Next month (May 2007) »
20070518 Friday May 18, 2007

OpenDS 0.9.0-build001 is now available

I have just uploaded OpenDS 0.9.0-build001, built from revision 1918 of our source tree, to our weekly builds folder. The direct link to download the core server is: https://opends.dev.java.net/files/documents/4926/58055/OpenDS-0.9.0-build001.zip. The direct link to download the DSML gateway is: https://opends.dev.java.net/files/documents/4926/58056/OpenDS-0.9.0-build001-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Note that starting with this build, we have changed our build numbering system to be more consistent with other projects. The base build number is the build of the next official release we are working toward (in our case 0.9.0, since 0.8.0 was released last week), rather than what we had been using in the past, which was the number of the last official release.

Some of the changes that have been incorporated since the 0.8.0 release build include:
  • Revision 1835 -- Update the QuickSetup code to eliminate references to classes in OpenDS.jar, which could force QuickSetup to have to download that JAR file before displaying the setup box instead of being able to download it in the background.
  • Revision 1837 -- Update the Windows batch files so that they will pause if an error occurs because no suitable JVM was found. This makes it possible for users to see the error message if the batch file was launched in a graphical mode (e.g., by double-clicking the icon in Windows Explorer).
  • Revision 1839 -- Update the status panel so that it will resize itself whenever the user first authenticates. This will allow it to be big enough for the user to see all of its content.
  • Revision 1841 -- Update the status panel so that it does not display replication monitoring information if replication is not enabled.
  • Revision 1842 (Issue #1585) -- Update the server so that the audit logger is disabled by default. Also, change the format of the audit log messages so that they use standard LDIF change syntax and therefore can be easily replayed if necessary.
  • Revision 1843 (Issue #1584) -- Update the DSML gateway so that it properly treats the request ID as optional rather than required. Also, update the DSML search processing so that it is more forgiving when parsing scope and deref policy strings.
  • Revision 1850 (Issue #1597) -- Fix a typo in the name of the "pilotPerson" object class.
  • Revision 1856 (Issue #612) -- Update the replication code so that it can replay operations in parallel using multiple threads while still preserving dependencies between operations (e.g., ensuring that a child is always added after its parent).
  • Revision 1861 (Issue #1604) -- Update the status command so that it allows reading the user password from standard input.
  • Revision 1868 (Issue #1507) -- Update the search processing code of the Berkeley DB JE backend so that any unindexed searches will be checked against the virtual attribute subsystem to see if any of the virtual attribute providers may be used to process the search.
  • Revision 1873 (Issue #1502) -- Fix an issue with the override severity configuration for the error logger so that it works as described in the documentation. Also, rename the error and debug logging levels so that they use all lowercase characters and dashes rather than underscores.
  • Revision 1875 (Issue #1430) -- Update the LDAP protocol tools so that they provide a "--version" argument to display information about the version of the tool.
  • Revision 1889 (Issue #1283) -- Change the filename for backups created by the Berkeley DB JE backend so that they no longer contain a ".zip" extension. Even though some forms of the backups do use zip format, others (e.g., when encryption is enable) do not.
  • Revision 1890 (Issues #1479, 1587, 1606) -- Update the access control processing code so that operational attributes do not get automatically included by clauses like (targetattr="*").
  • Revision 1897 (Issue #1615) -- Fix a problem that could prevent the server from starting under certain network configurations.
  • Revision 1907 -- Perform miscellaneous cleanup and bugfixes identified by the FindBugs utility.
  • Revision 1908 (Issue #1614) -- Add a stub to preserve the previous HistoricalCsnOrderingMatchingRule in its original package to allow older databases to continue to be used with upgraded versions of the server.
  • Revision 1918 (Issue #1622) -- Add global ACIs that allow anonymous read access to certain key operational attributes, like those in the root DSE and schema subentries, as well as server-wide attributes like entryDN, modifiersName, and modifyTimestamp.
Posted by cn_equals_directory_manager ( May 18 2007, 07:28:00 PM CDT ) Permalink

OpenDS and Other Sun-Sponsored Open Source Projects

Last night, I gave a talk at CACTUS (the Capital Area Central Texas UNIX Society, http://www.cactus.org/). Since it was a UNIX-focused group, the first part of the talk was about general open source projects that Sun is involved with, including OpenSolaris, OpenSPARC, and OpenJDK. The second part of the talk was more specific to OpenDS, including general information, information about its current state and what still needs to be done, and how people can get involved.

As requested, I have uploaded the slides for my presentation.

[UPDATE] -- We have now posted the slides and an MP3 recording of the presentation to the OpenDS documentation wiki. You can find them at https://www.opends.org/wiki/page/OpenDSPresentations.

Posted by cn_equals_directory_manager ( May 18 2007, 11:17:47 AM CDT ) Permalink

20070514 Monday May 14, 2007

The OpenDS Virtual Attribute Subsystem

One of the key OpenDS components that makes virtual static groups possible is the virtual attribute subsystem. Virtual attributes are those attributes whose values are computed on the fly rather than actually being stored in the database. There are a number of uses for virtual attributes in the server, and there is an API (org.opends.server.api.VirtualAttributeProvider) that can be used to create new types of virtual attributes.

Some of the virtual attribute providers we have defined in OpenDS include:
  • The entryDN provider -- This is used to compute the entryDN operational attribute, which simply contains the DN of the entry. (as defined in draft-zeilenga-ldap-entrydn).
  • The subschemaSubentry provider -- This is used to compute the subschemaSubentry operational attribute, which is used to specify the location of the schema governing the associated entry (as defined in RFC 4512).
  • The isMemberOf provider -- This is used to compute the isMemberOf attribute, which lists the DNs of the groups in which the associated user is a member.
  • The member provider -- This is used to compute the member or uniqueMember attribute for virtual static groups.
  • The user-defined provider -- This is used to allow users to define their own virtual attributes that will appear in entries based on a given set of criteria. More information about user-defined virtual attributes is provided below.

Virtual Attribute Configuration

Virtual attributes are configured below "cn=Virtual Attributes,cn=config". These entries need to have the ds-cfg-virtual-attribute object class, which requires the following attributes:
  • ds-cfg-virtual-attribute-class -- This specifies the class providing the virtual attribute logic.
  • ds-cfg-virtual-attribute-enabled -- This indicates whether the virtual attribute should be enabled so that it can generate values for the target entries.
  • ds-cfg-virtual-attribute-type -- This specifies the name of the attribute type for which the values will be generated
  • ds-cfg-virtual-attribute-conflict-behavior -- This specifies how the server should behave if an entry already has one or more real values for an attribute that could be virtually generated. Allowed values are "real-overrides-virtual" (to indicate that only the real values should be used), "virtual-overrides-real" (to indicate that the real values should be ignored and only the virtual values should be used), and "merge-real-and-virtual" (in which both the real and virtual values will be used).

With only the above configuration attributes, the virtual attribute may be generated for all entries. If you wish to pare down the set of entries in which the virtual attribute may be present, you can use one or more of the additional configuration attributes (all of which are multivalued):
  • ds-cfg-virtual-attribute-base-dn -- This specifies the base DN(s) for the branches below which the virtual attribute may be used. If this is present, then only entries below one of the specified base DNs may include the virtual attribute.
  • ds-cfg-virtual-attribute-filter -- This specifies a search filter that may be used to control the entries in which the virtual attribute may be used. If this is present, then only entries matching at least one of the specified filters may include the virtual attribute.
  • ds-cfg-virtual-attribute-group-dn -- This specifies the DN(s) for the groups whose members will be allowed to have this virtual attribute. If this is present, then only user entries belonging to one of the specified groups may include the virtual attribute.

User-Defined Virtual Attributes

User-defined virtual attributes can be used to supply specific administrator-supplied values to entries matching the virtual attribute criteria. The net effect is essentially the same as what you can get using the Class of Service (CoS) capabilities of the Sun Java System Directory Server, but I think that the implementation and configuration is much more straightforward (although I may be a bit biased since I wrote the code).

In order to create a user-defined virtual attribute, add a new entry to the server configuration. It should contain the ds-cfg-user-defined-virtual-attribute object class (which extends the ds-cfg-virtual-attribute class and therefore takes all of the configuration attributes that it uses as described above), and it should also have at least one value for the ds-cfg-virtual-attribute-value attribute to specify the value that entries matching the criteria should be given. The ds-cfg-virtual-attribute-class should be set to "org.opends.server.extensions.UserDefinedVirtualAttributeProvider".

For example, the following configuration entry assigns a default postalCode value for everyone in the Austin office (although if they already have a postalCode value in their entry, it will be used instead of the virtual value): 
dn: cn=Austin postalCode,cn=Virtual Attributes,cn=config
objectClass: top
objectClass: ds-cfg-virtual-attribute
objectClass: ds-cfg-user-defined-virtual-attribute
cn: Austin postalCode
ds-cfg-virtual-attribute-class: org.opends.server.extensions.UserDefinedVirtualAttributeProvider
ds-cfg-virtual-attribute-enabled: true
ds-cfg-virtual-attribute-type: postalCode
ds-cfg-virtual-attribute-value: 78727
ds-cfg-virtual-attribute-conflict-behavior: real-overrides-virtual
ds-cfg-virtual-attribute-base-dn: ou=People,dc=example,dc=com
ds-cfg-virtual-attribute-filter: (&(l=Austin)(st=Texas))

Note that because of the way that virtual attributes are implemented in OpenDS, you can use them to supply values for pretty much any kind of attribute, including operational attributes. For example, you could use it to set the ds-pwp-password-policy-dn operational attribute to give users a custom password policy, ds-rlim-size-limit to define a custom size limit, or ds-privilege-name to assign one or more privileges. For example, the following virtual attribute configuration entry gives a special set of privileges to everyone on the "Administrators" group: 
dn: cn=Administrator Privileges,cn=Virtual Attributes,cn=config
objectClass: top
objectClass: ds-cfg-virtual-attribute
objectClass: ds-cfg-user-defined-virtual-attribute
cn: Administrator Privileges
ds-cfg-virtual-attribute-class: org.opends.server.extensions.UserDefinedVirtualAttributeProvider
ds-cfg-virtual-attribute-enabled: true
ds-cfg-virtual-attribute-type: ds-privilege-name
ds-cfg-virtual-attribute-value: modify-acl
ds-cfg-virtual-attribute-value: config-read
ds-cfg-virtual-attribute-value: config-write
ds-cfg-virtual-attribute-value: ldif-import
ds-cfg-virtual-attribute-value: ldif-export
ds-cfg-virtual-attribute-value: backend-backup
ds-cfg-virtual-attribute-value: backend-restore
ds-cfg-virtual-attribute-value: password-reset
ds-cfg-virtual-attribute-value: update-schema
ds-cfg-virtual-attribute-conflict-behavior: merge-real-and-virtual
ds-cfg-virtual-attribute-group-dn: cn=Administrators,ou=Groups,dc=example,dc=com
Posted by cn_equals_directory_manager ( May 14 2007, 01:31:47 PM CDT ) Permalink

20070511 Friday May 11, 2007

Virtual Static Groups in OpenDS

Big static groups (with tens or hundreds of thousands of members, or more) are a problem in many large enterprise directories. Since a static group contains an explicit list of the DNs of its members, the more members it contains, the larger the entry will become. Maintaining these groups can become a management problem and isn't very efficient, and some types of searches involving them can be slow as well. Dynamic groups are much better when the groups contain thousands or millions of members, but the problem is that many client applications don't support them. It's easy to understand why, since the client does have a significant amount of work to do in order to determine whether a given user is a member of a dynamic group, but it's also unfortunate because it leads to a lot of cases in which directories are forced to end up with large static groups just to suit those applications.

OpenDS provides an interesting solution to this problem in the form of virtual static groups. It's a special type of entry that behaves like a static group, but all operations which attempt to determine membership are passed through behind the scenes to another group. In many cases, virtual static groups can give you the management and scalability benefits that dynamic groups provide while still maintaining compatibility with clients that only support static groups.

In order to use virtual static groups, you first need a dynamic group that will provide the membership criteria. For the purposes of this example, let's say that we have the following entry: 
dn: cn=Austin Users,ou=Groups,dc=example,dc=com
objectClass: top
objectClass: groupOfURLs
cn: Austin Users
memberURL: ldap:///ou=People,dc=example,dc=com??sub?(&(l=Austin)(st=Texas))
This group will automatically include any user with a location of Austin and a state of Texas. It's a much better choice for a dynamic group than a static group because the set of members will be automatically adjusted as new users are added, existing users are removed, or if someone moves from one place to another.

To create a virtual static group that allows clients to interact with the Austin Users group in a static manner, add the following entry: 
dn: cn=Virtual Static Austin Users,ou=Groups,dc=example,dc=com
objectClass: top
objectClass: groupOfUniqueNames
objectClass: ds-virtual-static-group
cn: Virtual Static Austin Users
ds-target-group-dn: cn=Austin Users,ou=Groups,dc=example,dc=com
With this group, uniqueMember will be treated as a virtual attribute (if we had used the groupOfNames instead of groupOfUniqueNames, then the member attribute would have been used instead). The key here is the ds-virtual-static-group auxiliary object class and the corresponding ds-target-group-dn attribute. OpenDS sees this and knows that it should treat the entry like a virtual static group.

Now, consider that the following users exist in the directory: 
dn: uid=nawilson,ou=People,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: nawilson
givenName: Neil
sn: Wilson
cn: Neil Wilson
l: Austin
st: Texas

dn: uid=bowendk,ou=People,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: bowendk
givenName: Don
sn: Bowen
cn: Don Bowen
l: Peoria
st: Illinois
With thiese entries, the user "nawilson" will be considered a member of the group because he is located in Austin, Texas, whereas user "bowendk" will not. A quick test with ldapsearch confirms this: 
$ bin/ldapsearch -b 'cn=Virtual Static Austin Users,ou=Groups,dc=example,dc=com' \
  -s base --countEntries '(uniqueMember=uid=nawilson,ou=People,dc=example,dc=com)' 1.1
dn: cn=Virtual Static Austin Users,ou=Groups,dc=example,dc=com

# Total number of matching entries:  1

$ bin/ldapsearch -b 'cn=Virtual Static Austin Users,ou=Groups,dc=example,dc=com' \
  -s base --countEntries '(uniqueMember=uid=bowendk,ou=People,dc=example,dc=com)' 1.1
# Total number of matching entries:  0
Note that while dynamic groups are very efficient for determining whether a given user is a member, they can be very inefficient when it comes to retrieving the entire set of members. This is also the case with virtual static groups that use dynamic groups to get their membership information. If you were to actually retrieve the uniqueMember attribute to list all of the members, that could be a very expensive operation. OpenDS provides a way to deal with this in the form of the ds-cfg-allow-retrieving-membership configuration attribute. It defaults to "false", which means that queries like those above which try to determine whether a given user is a member of the group will succeed, but the uniqueMember attribute won't be included in the resulting entry even if it's requested by the client. Most well-behaved clients won't ask for the membership attribute anyway, and of those that do many of them don't use it in any way, so this doesn't cause a problem. However, if you do have an application that expects to retrieve the membership attribute and won't behave properly if it isn't returned, then you're probably stuck with a traditional static group.

Also note that there's no requirement that virtual static groups be used only with dynamic groups. You can actually use it with any type of group (other than another virtual static group, to avoid circular references) that OpenDS supports (right now, we've only got static and dynamic groups, but we may add other types in the future). For example, if you've got a static group based on the groupOfUniqueNames object class but your client only supports groups with the groupOfNames class, then you could create a virtual static group with the groupOfNames structural object class and point the ds-target-group-dn attribute at the static group with the groupOfUniqueNames class.

Posted by cn_equals_directory_manager ( May 11 2007, 11:45:17 AM CDT ) Permalink

20070508 Tuesday May 08, 2007

OpenDS builds on OpenJDK on OpenSolaris

Sun announced today that OpenJDK is now fully-buildable. Aside from a few third-party components that Sun doesn't have the right to open source (which are currently available in binary-only form until that can be resolved), the entire JDK codebase is GPLv2 with the classpath exception. I decided to take a shot and see what it takes to build it. It turns out, it's a pretty simple process.

There are instructions available for building with NetBeans, but honestly I don't use NetBeans and didn't particularly want to install it just for this. Plus, if you really do need to have NetBeans to build it, then that somehow doesn't seem fair. At any rate, I didn't find not using NetBeans to be a problem.

The system that I was using was an Ultra 40 Workstation running Solaris Nevada (Sun's OpenSolaris distribution -- see http://www.opensolaris.org/os/community/on/ and http://www.opensolaris.org/os/project/onnv/ for details) build 61. I already had the Sun Studio 11 compilers installed, and the Subversion client is included as part of Nevada.

Here's what I did to build OpenJDK and then use that to build OpenDS:
  1. I started in /export/home/nawilson/sandbox, which is the directory that I use to hold all of the source code I check out from various repositories.

  2. I checked out the OpenJDK source code with the command:
    svn checkout --username guest https://openjdk.dev.java.net/svn/openjdk
    When prompted for a password, I just pressed ENTER without typing anything.

  3. Once the checkout completed, I went into the openjdk/hotspot/trunk directory.

  4. I found the page http://openjdk.java.net/groups/hotspot/faq.html, which is the HotSpot FAQ, and on that page I found a link to the osse-build-solaris-i586 file that can be used as a simple build script. I downloaded that file and copied it into the current working directory. I then edited it to point both ALT_BOOTDIR and ALT_JDK_IMPORT_PATH to "/usr/java".

  5. I edited my path so that /opt/SUNWspro/bin was the first directory contained in it. This was necessary to make sure that the Sun Studio 11 compiler was getting used instead of /usr/ucb/cc which fails right away.

  6. I made the edited osse-build-solaris-i586 file executable and ran it without any arguments. About two hours later, I had a full JDK 1.6.0_01-b06 build in the build/solaris/jdk-solaris-i586 subdirectory.

  7. I returned to the /export/home/nawilson/sandbox directory and checked out OpenDS with the command:
    svn checkout --username guest https://opends.dev.java.net/svn/opends/trunk/opends
    Once the checkout completed, I went into the opends subdirectory.

  8. I set the JAVA_HOME environment variable to /export/home/nawilson/sandbox/openjdk/hotspot/trunk/build/solaris/jdk-solaris-i586 and ran "./build.sh" with no arguments. About 33 seconds later, I had a build/package/OpenDS-0.8.zip file containing the core OpenDS server.

  9. I went into the build/package/OpenDS-0.8 directory and ran ./setup to launch our QuickSetup utility to configure the server, populate it with sample data, start it up, and use our graphical status panel to verify that everything looks fine.
I also later used "./build.sh all" to perform a full build of OpenDS including the DSML gateway, Javadoc documentation, and running all 34,000+ unit tests. Everything passed with flying colors. As far as OpenDS is concerned, there is no distinguishable difference between the OpenJDK build that I just created and the real Java 1.6.0_01-b06 build that you can download from http://java.sun.com/.

Posted by cn_equals_directory_manager ( May 08 2007, 05:02:19 PM CDT ) Permalink

OpenDS 0.8 Is Now Available

OpenDS has been public for about nine months now, and for all of that time we have tagged it with a 0.1. With our latest build, we're bumping it up to 0.8. In a few months, we'll go to 0.9, and then 1.0 a couple after that.

The easiest way to get OpenDS 0.8 is to use our Java Web Start installer (read more about it at https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool). Alternately, you can download the full server zip file at https://opends.dev.java.net/files/documents/4926/57426/OpenDS-0.8.zip. We also have a DSML gateway available as a WAR file at https://opends.dev.java.net/files/documents/4926/57427/OpenDS-0.8-DSML.war.

If you haven't looked at OpenDS recently, here are some of the things we've added in the last few months:
  • We have added support for updating the schema with the server online. You can do this using modify operations, or using an "add schema file" task to add a new schema file. The schema file structure will be preserved, even when schema changes are replicated between servers.

  • We have added support for subordinate attribute types. Specifying a superior attribute will target all of its subordinate types (e.g., referencing attribute "name", will include attributes like cn, sn, givenName, initials, c, l, st, o, ou, title, and generationQualifier).

  • We have added a lastmod plugin that maintains the creatorsName, createTimestamp, modifiersName, and modifyTimestamp attributes.

  • We have implemented a DSEE-compatible access control handler. This uses the aci attribute with a syntax that is handled like that in the Sun Java System Directory Server. See Managing Access Control for information on OpenDS support for access control.

  • We have implemented a privileges subsystem to make it possible to configure capabilities on a fine-grained level. You can use this to remove privileges from root users, or grant additional privileges to non-root users. See Root Users and Privileges for more information.

  • We have added support for the proxied authorization control, which makes it possible to perform an operation as one user while authenticated as another. We have also added support for using an alternate authorization identity when using SASL DIGEST-MD5 or PLAIN mechanisms.

  • We have added support for the get effective rights control, which can be used to determine what rights a user has for a particular entry.

  • We have added support for the server-side sort control, which can be used to sort entries before returning them to the client. We have also added support for the virtual list view control, which can be used in conjunction with the server-side sort control to retrieve the result set in pages rather than all at once.

  • We have added support for rejecting requests from unauthenticated clients. When operating in this mode, the server will require that clients authenticate before requesting any operations other than bind and StartTLS.

  • We have added support for groups. This includes static groups (both groupOfNames and groupOfUniqueNames variants) and dynamic groups (based on groupOfURLs). We have also added support for virtual static groups, which makes it possible to mirror a dynamic group as a static group.

  • We have improved support for SSL and StartTLS, including making it possible to configure them through the QuickSetup utility, and providing the ability to use different certificates for different listeners. We have added a number of new certificate mappers that can be used to map a certificate to a user entry when performing SASL EXTERNAL authentication.

  • We have added a number of password validators to the server. One looks at how similar the new password is to the user's current password. One looks at whether the new password matches the value of other attributes in the user entry. One looks at whether the new password matches a value in a dictionary. One looks at the sets of characters included in the new password. One looks at the number of unique characters in the password. One looks at whether there are strings of repeated characters in the password.

  • We have implemented support for the CRYPT password storage scheme, which may be needed by some UNIX clients to use the OpenDS for authenticating users.

  • We have added a virtual attribute subsystem, including implementing support for the standard entryDN and subschemaSubentry attributes. We also have added support for isMemberOf, which will include the DNs of all groups in which the associated user is a member. We also have support for user-defined virtual attributes, much like the Class of Service functionality in the Sun Java System Directory Server.

  • We have added support for rebuilding indexes, both as an offline operation or with the server online using the tasks interface.

  • We have improved support for configuration archiving in the server, so that it includes changes made with the server offline. We have also implemented a mechanism for detecting external changes to the configuration file with the server online.

  • We have improved support for Windows systems, including the ability to run as a service.

Posted by cn_equals_directory_manager ( May 08 2007, 09:52:57 AM CDT ) Permalink

20070507 Monday May 07, 2007

OpenDS Documentation Wiki

It seems that I've gone through another span of not writing much here, but as usual I've been busy with other things. OpenDS is coming along quite nicely, and we've added lots of new features and done a lot of cleanup work. One of the things that we've been focusing on recently is documentation. We've just recently moved all of our documentation to a wiki, and in the process we have updated things that were out of date and added lots of new content. You can find it at https://www.opends.org/wiki.

Some of the notable content that on the wiki includes: Posted by cn_equals_directory_manager ( May 07 2007, 09:04:39 AM CDT ) Permalink

Archives
« May 2007 »
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today

XML


Language

Links


blogs.sun.com
Weblog
Login


Referrers

Today's Page Hits: 64