Geertjan Wielenga interviewed Ludo about OpenDS for Javalobby, see http://java.dzone.com/news/opends-open-source-directory.
OpenDS has been open source now for 3 years. Ludo and Geertjan cover some of the capabilities of the LDAPv3 compliant directory service now that 2.0 is out, as well as where OpenDS came from, and where it's going.
Lee Trujillo has published a webcast on DirTracer, a great tool for gathering info around Directory support. Also, as Lee mentions in his blog entry on the subject, DT 6.0.7 and Pkgapp 3.2 (Linux) Available! this update supports Solaris (SPARC and x86), Red Hat, and SuSE.
Lee lists new features and enhancements in http://blogs.sun.com/Dirtracer/entry/dt_6_0_7_coming. Folks outside Sun's network can get DirTracer though Sun support until it becomes available through BigAdmin.
Installing Sun OpenDS Standard Edition 2.0 on a Windows XP laptop for evaluation is a cinch. Here is what I did.
D:\>cd SunOpenDS_SE2.0 D:\SunOpenDS_SE2.0>⁞java -version java version "1.6.0_13" Java(TM) SE Runtime Environment (build 1.6.0_13-b03) Java HotSpot(TM) Client VM (build 11.3-b02, mixed mode, sharing) D:\SunOpenDS_SE2.0>setup.bat
Full install instructions can be found in the official documentation at http://docs.sun.com/source/820-6168/installing-the-ds.html.
As Ludo wrote, OpenDS 2.0 got the Go this week. You can download it here, http://www.opends.org/promoted-builds/2.0.0/.
OpenDS 2.0 includes major improvements over OpenDS 1.0, with a much expanded server control panel, task support for recurring jobs like backup, assured replication, enhanced access control, extensible matching rules functionality, SASL support, schema files for LDAP-based naming on Solaris and OpenSolaris, improved security support in the installer, interop with the IBM JVM, wicked perfs, and more. Docs are at https://docs.opends.org/2.0.
Also, Sun OpenDS Standard Edition 2.0 offers the full support option when you go to deploy with 2.0. Sun documentation is at http://docs.sun.com/coll/2508.1.
Let us know what you are doing with OpenDS. Join the OpenDS community, https://opends.dev.java.net/servlets/ProjectMembershipRequest.
"Join this free webinar to learn how Sun offers
the only high-performance directory server with essential data services
including proxy, virtual directory and data distribution. The webinar
will be lead by Nick Wooler, Product Line Manager for Sun's directory
products, Directory Server Enterprise Edition (DSEE) and OpenDS."
Reserve your seat at https://dct.sun.com/dct/forms/reg_us_2904_330_0.jsp.
Date: Wednesday, May 20, 2009
Time: 10:00 am PDT / 1:00 pm EDT / 18:00 GMT
Interesting reading recently around Directory Server Enterprise Edition:
Benoit Chaffanjon did a benchmark with a new Nehalem system, Sun Blade X6270 & INTEL XEON X5560 on OpenSolaris create the ultimate Directory Server. Benoit estimates that you might get, "459,800 LDAP Operations per second on a fully loaded modular system."
Wajih Ahmed and Abdi Mohammadi wrote a BigAdmin feature article A Planning Guide for Moving to LDAP as Naming Service in the Solaris OS⁞.
Michael Teger wrote, Creating an OpenSSO User Data Store Using Sun Directory Server is Like Riding a Bicycle. Just 2 short procedures.
Arnaud Lacour has written a couple of entries recently on using Directory Proxy Server data distribution to replay the same operations against multiple data sources. The data distribution algorithm in question is referred to as the replication distribution algorithm.
Although the Directory Proxy Server replication distribution algorithm does not handle all the sort of conflict resolution corner cases that Directory Server and OpenDS replication can manage, it does let you perform best-effort replication across supported data sources.
Check out Arnaud's two entries:
Setting DPS As Replication Hub - Part 1: A Simple Tutorial
Setting DPS As Replication Hub - Part 2: Replication to SQL and LDAP
Masood Kalali published an interview today over at DZone with Ludovic Poitou who is community lead for the OpenDS project. Masood and Ludo look at what the OpenDS project is aimed to accomplish, and why Sun moved to chose to invest effort in the the Java-based OpenDS LDAP server to build the next generation of directory services, instead of jumping on board with an existing open source project.
Pierre and Gilles have mostly rewired our central servers and labs to the new switch in Sun's Grenoble engineering center.
Here is the before picture:
And here is what it looks like now:
Why should anyone care? Well, now we have a 10Gbit backbone between labs, with 800Gbit throughput in the central switch. Pierre ran tests showing that a machine with 1Gbit Ethernet now actually can get 1Gbit throughput to a machine in another lab and network. That means we no longer have to have all equipment for a big test physically located in the same place.
That freedom is a good thing. My expectation is that we will soon be trunking 4 x 1Gbit Ethernet for some of our heavy load tests with OpenDS, for example.
Sun engineer Matt Swift recently started blogging about noteworthy development in the core of the OpenDS code base. Worth reading...
In one entry, Matt explains how much he and fellow engineer Bo Li managed to improve response times, especially for large entries, by refactoring OpenDS ASN.1 APIs. (See New ASN1 library brings performance boost to OpenDS.) Because all LDAP messages are encoded in ASN.1, optimal ASN.1 handling makes a significant difference.
In Garbage First - the G1 garbage collector, Matt describes what has been happening between the OpenDS folks and Java engineers like Java HotSpot expert Tony Printezis, and VM wiz Laurent Daynes. In a nutshell, OpenDS is built not only to serve lots of LDAP clients very quickly, but also to serve all requests quickly. The trick is finding the way to get all the benefits of garbage collection without having LDAP clients waiting while the JVM takes out its trash. Looks like they are indeed coming up with some promising new tricks.
Thanks for taking time out to write, Matt.
Michael Ströder's web2ldap provides browser-based access to an LDAP server.
Once installed, web2ldap is simple to use. Make sure you have an LDAP URL in etc/web2ldap/web2ldapcnf/hosts.py, that points to OpenDS such as http://localhost:389 or http://localhost:1389, and start web2ldap.
As web2ldap can also discover the schema, you can use it to add and modify entries.
On Friday, lead QA engineer Gary Williams declared OpenDS 1.2 GO for release. 1.2 is the version of OpenDS going into the next release of OpenSolaris. See Ludo's entry on the release of 1.2 for highlights.
You can download 1.2 at http://www.opends.org/promoted-builds/1.2.0/, where you will find the build and the change log for the build.
The release notes are on the wiki at https://docs.opends.org/1.2/page/ReleaseNotes, with the full documentation set for the release at https://docs.opends.org/1.2/.
Join the OpenDS ProjectLuke Donnelly sent this around a little while ago. Interesting for somebody living and working here in Grenoble, France. In a blog entry from last fall about Yahoo! doing engineering in Grenoble, you can read this:
"We operate in a very competitive, multinational world," said Linwood. "We do development all over the world, but it's important to find locations where we can find talent, work with the government and grow our business in an environment that is friendly to that. These changes have made it much more attractive as a company [to invest in France]."
This year the incentive for tech companies to do R&D in France has increased. The French are giving companies credit for a big part of their R&D costs, whether or not those companies have made enough to have to pay tax on revenues in France. Maybe this sort of busts the myth of old Europe having higher costs in the end.
BTW, despite all the show this year, the 35-hour week is still a myth, too. (Sometimes I wish it weren't. ;-)
Gary Williams published an article with Marina Sum on Sun Developer Network just this week about Perspectives on Quality Assurance for OpenDS. Good to see an article focused on quality. Gary is definitely the subject matter expert as well.
One aspect they mentioned particularly catches the attention:
"Make no mistake, interest and involvement from the community directly affect the quality of the software. It's a huge plus to have members who frequently test, evaluate, and improve the code, and who also reject enhancements that introduce too many bugs. As a result, quality and success of the community are interlinked."
In other words you do not have to be an LDAP expert to get involved in the OpenDS community. As Ludo wrote, "Feel free to register to Java.net and join the OpenDS project as a user. To register, choose a login name, enter a password and your email address. Nothing else is required. Joining the OpenDS project as a user just indicates your interest in the project and will allow you to subscribe to the mailing lists if you want to."
The Java.net page is https://opends.dev.java.net/. There you find the mailing lists, the issue tracker, and so forth. The OpenDS project home page is at http://www.opends.org. You will find a wealth of both developer and user documentation at https://www.opends.org/wiki/.
Years ago LDAP replication technology removed the limit to the number of LDAP reads your directory service can support. Need more read capacity? Simple, add another LDAP server instance. What many might not know is that since March 2007, the data distribution capability in Sun's Directory Proxy Server has made it possible to scale LDAP write operations, too.
As described in the Deployment Planning Guide section on data distribution, simply adding more Directory Server instances to a replicated topology lets you scale reads, but not writes. Why?
Replication brings you high availability by copying data to each server in the replication topology. If one server goes down, others that remain up continue providing LDAP directory services. With a bit of load balancing, your LDAP applications hardly need notice that a server instance has failed. Multi-master replication means this high availability technology works for writes, too. As with other high availability technologies, you gain availability by adding redundancy.
Replication provides its function by replaying changes from each write operation onto every server involved in a replication topology. As a result, the number of operations to perform for each write (add, delete, modify, modDN) is equal to the number of servers involved. So you may have N servers to do the work, but for each write there are N writes to replicate! The writes are only fully completed across the replicated topology as fast as the slowest replica can finish its updates.
When you need go beyond hundreds into thousands of LDAP write operations sustained every second, the way to do so today is not by adding more servers to your existing topology. Instead you split your topology up into chunks, distributing a part of your data set onto each chunk. For high availability, each chunk is its own topology of replicated Directory Servers. Since version 6.0, Directory Proxy Server allows you to split entries from your directory service across chunks. The example shown in the documentation depicts three chunks, the first with surnames (attribute: SN) starting with A through H, the second for I through P, and the third for Q through Z.
The data view shown in the diagram is a configuration concept from Directory Proxy Server. Notice there is a special chunk for the top of the directory information tree.
Now, with three separate chunks, you can achieve as much as three times the write throughput. How? Assuming the write operations are equally distributed, each write has only one chance in three of involving a particular chunk. Each of the three chunks can process write operations in parallel up to the maximum number of writes each chunk can handle. The number of writes you can perform is limited only by the number of truckloads of hardware you can afford. (Maybe I should have mentioned this before you got everything you wanted for Christmas. ;-)
Mark Craig writes about Sun's LDAP directory technology.
