Joe Mocker's Weblog: How'd you know I was blogging at you if you weren't blogging at me?

Quite some time ago when a team of us worked on rewriting the code which ran the Java Developer Connection, a co-worker came up with the idea of a simple instrumentation interface.  It was almost a pre-cursor to JMX. I'm sure other groups inside and outside of Sun had done the same thing. The nice thing about the framework we developed is that it was extremely simple, 'course this was around the time of JDK 1.2 so keeping the code simple was a good thing.

The instrumentation framework was invaluable for us as we worked through code iterations and daily support on the JDC. It provided a multi-level view of all of the components of the system, and helped us monitor things such as how many sessions and users are logged into the JDC, as well as help us resolve problems such as the culprit of a JDBC Connection Pool leaks.

Since then, I have always been keen on pushing fellow engineers to instrument their applications. Of course, now days I've been pushing JMX as the preferred instrumentation solution. It provides many of the same features that our home grown solution provided. And what's better is that because its the blessed Monitoring and Management standard for Java, more and more applications are being instrumented with JMX, and even the newer JDKs.

The only problem that had been unresolved for us is remote monitoring of JMX enabled applications. When I first looked into it a year or two ago, the stage was sort of dim. Although JMX does have remote monitoring capabilities, its based on RMI, which was unfortunate.

Its unfortunate because RMI is somewhat of a dynamic protocol, similar to SunRPC, where ports which services reside are dynamically allocated at runtime by the RMI Registry. This poses a problem when trying to connect to a remote data center, with umpteen firewalls to get through. Security didn't appear to be of much concern to the original RMI developers.

So I decided to abandon the use of RMI back then, and settled on a plain old HTTP based console embedded as part of the application. That sounds like a lot, however, looking around JMX implementations I found MX4J had this built into their framework. And in fact, the HTTP interface was quite reasonable. And that's what we went with.

Recently, I took another look at the whole problem of remote management. After some poking around I found a method of using RMI with JMX in a more "secure" way. That is, configuring RMI and JMX to use well defined ports for communication, which makes me happy, and my Operations people (who manage the firewalls) happy as well.

So here's the deal. There's really only two ports involved. The RMI Registry, which is already on a well-known port, typically 1099, and the JMX RMI Connector, which is the problem.

The solution lies in the fact that when creating a JMXServiceURL to define the connection parameters, you can, in fact, specify a port number for the service. This is not clear at all from the documentation. Here's an example

String svc = 
  "service:jmx:rmi://apphost:9994/jndi/rmi://apphost:1099/connector";

JMXServiceURL url = new JMXServiceURL(svc);
RMIConnectorServer rmiServer = new RMIConnectorServer(url, null, mbeanServer);
rmiServer.start();

The key is really the first two lines. What its saying is, the use RMI Registry is on  "apphost:1099" and register a service called "connector" which is located at "apphost:9994".Now, All I really have to do is tell my Operations folks to open the firewall to ports 1099 and 9994. After that's done, remotely monitoring the application is as simple as running the jconsole command:

jconsole service:jmx:rmi://apphost:9994/jndi/rmi://apphost:1099/connector

How cool is that. And all of a sudden this becomes just the tip of the iceberg. If I don't want to use the standalone RMI Registry, or I want to run an embedded registry within the application, well then all I have to do is instantiate one with the desired port number:

rmiRegistry = LocateRegistry.createRegistry(4099);

One slightly more complete final example to wrap it all up. The nice thing about JMX is that you can start out simple. For example, at the very least, just enabling JMX to monitor what the JVM is doing.  That can be accomplished with just a couple more lines of code. The key is getting a handle to the Platform MBean Server:

Registry rmiRegistry = LocateRegistry.createRegistry(4099);
MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();
String svc =
  "service:jmx:rmi://apphost:9994/jndi/rmi://apphost:4099/connector";

JMXServiceURL url = new JMXServiceURL(svc);
RMIConnectorServer rmiServer = new RMIConnectorServer(url, null, mbeanServer);
rmiServer.start();

Doesn't get much easier than this.