JMX : extending the MBeanServer to support Virtual MBeans

Java Management eXtensions (JMX) provide a means for management interfaces to be exposed as MBeans allowing remote monitoring and management of your application.

This blog article shows how to extend the MBeanServer implementation with your own logic. In particular, it shows how to implement VirtualMBeans - MBeans that don't really exist - but the same technique can be used to do other interesting things, including MBeanServer cascading, fine-grained MBean access controls, auditing, or logging, to mention a few of them.

Background on MBeans and Virtual MBeans

An MBean is a managed Java object, similar to a JavaBeans component. An MBean can represent a device, an application, or any resource that needs to be managed.

Since an MBean is a managed Java object that typically represents a device, application or a resource, one might expect a clean object-oriented approach where there is one MBean for each thing that needs managing.

Now sometimes this is true, and sometimes it isn't, and the reason behind the choice is often down to implementation details rather than architecture. Having one MBean per managed resource means that the person writing the management logic needs to be able to easily manage the lifecycle of the MBeans, and the lifecycle of any state associated to the MBeans, tracking the state of the underlying managed system. This is not always easy or appropriate to do.

As a result, one ends up with a functional programming interface on an MBean, allowing access to individual resources through operations that take a resource identifier as a parameter. There are examples of this kind of approach in the Platform MBeans -  one example is the ThreadMXBean, which carefully describes itself as the management interface of the threading subsystem, but it isn't exposing data just about a (singleton) threading subsystem, but also about individual threads via a decidedly functional-programming interface.

I've nothing against functional programming, but it does make viewing the thread data from the MBean particularly difficult in JConsole (we're lucky here, JConsole has a dedicated panel to viewing threading information), and such a functional interface does rather go against the grain of object-oriented programming in Java.

Also, my experience has shown that whenever I start writing a functional MBean interface I end up needing queries of the form "what are all the instance names I can query" and "what are the attributes for instance X" and "perform operation O on instance X" and "what instances have state Y"... these are of course the types of queries that the MBeanServer interface was designed for!

This blog article describes a way of extending the MBeanServer to expose individual managed resources as separate MBeans, whilst allowing the resource lifecycle state to remain where it belongs - in the underlying application.

What is a Virtual MBean?

A Virtual MBean is an MBean that doesn't really exist - it's virtual. Hence its name. A Virtual MBean can, however, be seen by clients connecting to the MBeanServer, and can be manipulated through the MBeanServer just like any other MBean.

This blog article shows how to extend the MBeanServer to support Virtual MBeans. In fact, this implementation will cause MBean instances to be created and released on-demand, just alive for the duration of a request, rather than being created and registered with the MBeanServer a-priori.

There are basically two families of calls that can be performed on an MBeanServer:

1. Queries across MBeans specifying an ObjectName pattern
2. Actions concerning a specific MBean, specifying the MBean's ObjectName

Our DispatchMBeanServer will use the ObjectName parameter on MBeanServer calls to delegate the implementation of the functionality to different MBeanServer implementations, allowing Virtual MBeans to have their very own MBeanServer implementation.

Limitations of Virtual MBeans

Before we get into the nuts and bolts of the implementation, Virtual MBeans do have some limitations - which might make them inappropriate for your everyday use.

MBeanServer Notifications

The biggest limitation with Virtual MBeans is related to the exact problem that it is trying to solve... that of lifecycle management. JMX 1.2's specification requires that each MBean registration and MBean unregistration causes an MBeanServerNotification to be emitted.

Given that the very problem that we wanted to avoid is that of having to deal with the lifecycle of the underlying objects, this is a big limitation, since we want to avoid having to execute any management code at the time of resource creation/deletion, and want to avoid having to maintain any state related to these resources.

One side-effect of this is that JConsole will not auto-update as Virtual MBeans come and go.

Fortunately, JMX 2.0 is talking about relaxing these constraints within the scope of a specific sub-namespace, in their support for Virtual MBeans. I personally would like to see additional meta-data about a namespace made available through a special MBean or MBeans, including information about the expected notification behavior of the namespace, caching characteristics, recommended polling intervals, and anything else the user may wish to expose. In addition, it'd be nice to have a new notification defined that allowed the namespace to indicate that all names in a given ObjectName pattern should be requeried... allowing things like JConsole to refresh only sub-parts of a namespace upon state-change.

NotificationEmitters

Since supporting the NotificationEmitter interface requires the implementer to maintain per-MBean state (who is subscribed to the MBean), we don't support Virtual MBeans that support this interface. Again, JMX 2.0 will remove the need for this per-MBean state since MBeans will emit notifications through the Event Service.

Security implications

The implementation in this blog article assumes that if you have access to the MBeanServer, then you are trusted. This is the case for the huge proportion of MBeanServer uses. JMX does, however, support a stronger security model, which we don't implement here (but it could be added).

How do you extend the MBeanServer to support Virtual MBeans?

DispatchMBeanServerBuilder

One can replace the MBeanServerBuilder implementation used to construct the default Platform MBeanServer implementation by setting an option on the Java command-line:

java -Djavax.management.builder.initial=dispatch.DispatchMBeanServerBuilder

By writing our own MBeanServerBuilder implementation, we can provide our own MBeanServer implementation which knows how to delegate requests to the appropriate
sub-MBeanServers, thereby allowing VirtualMBeanServers to be plugged in.

Our MBeanServerBuilder implentation creates a DispatchMBeanServer instance which wraps a standard MBeanServer instance.

DispatchMBeanServer

A DispatchMBeanServer does exactly as its name describes - it dispatches the incoming call to the appropriate underlying MBeanServer or MBeanServers, by examining the ObjectName parameter provided to the call.

In order to decide which MBeanServer is responsible for a given MBean, we divide up the JMX ObjectName namespace.

We dispatch requests to different MBeanServer implementations based upon the domain name.

Our DispatchMBeanServer implementation provides support for adding and removing underlying MBeanServer implementations responsible for individual domains:

Infrastructure to support the dispatching of requests

public class DispatchMBeanServer implements MBeanServer {     private Map<String, MBeanServer> mbeanServerMap =             new ConcurrentHashMap<String, MBeanServer>();        private MBeanServer defaultMBeanServer;                 /** Creates a new instance of DispatchMBeanServer */     public DispatchMBeanServer(MBeanServer defaultMBeanServer) {         this.defaultMBeanServer = defaultMBeanServer;         addMBeanServerDomain(             defaultMBeanServer.getDefaultDomain(),             defaultMBeanServer);     }     /** mark a domain for separate management */     public MBeanServer addMBeanServerDomain(String domain,                                             MBeanServer mbs) {         return mbeanServerMap.put(domain, mbs);     }     /** remove a domain from separate management */     public MBeanServer removeMBeanServerDomain(String domain) {         return mbeanServerMap.remove(domain);     }         /** return collection of all underlying MBeanServers */     public Collection<MBeanServer> getMBeanServerCollection() {         return Collections.unmodifiableCollection(mbeanServerMap.values());     }     /** return MBeanServer specific to a given ObjectName */     public MBeanServer getMBeanServer(ObjectName on) {         MBeanServer result = null;                 if (on != null) {             /* We cannot return a single MBeanServer for                an ObjectName pattern */             if (on.isDomainPattern()) {                 throw new IllegalArgumentException(                     "Illegal ObjectName pattern in request : "+                     on);             }             String domain = on.getDomain();             result = mbeanServerMap.get(domain);         }         /* If called with a null ObjectName or an            unknown domain, return default */         if (result == null)             result = defaultMBeanServer;         return result;     }          ...

As has been described above, the DispatchMBeanServer implementation of the MBeanServer interface has to deal with two families of request, here are a couple of code-snippets showing examples from each family.

Dispatching to a single MBeanServer

public Object getAttribute(ObjectName name, String attribute)         throws MBeanException, AttributeNotFoundException,                InstanceNotFoundException, ReflectionException {         return getMBeanServer(name).getAttribute(name, attribute);     }

Querying across multiple MBeanServers

public Set queryNames(ObjectName pattern,                           QueryExp query) {         Set resultSet = new HashSet();                             for (MBeanServer mbs : getMBeanServerCollection()) {             resultSet.addAll(mbs.queryNames(pattern, query));         }         return resultSet;     }

VirtualMBeanServer

Now that we've got a DispatchMBeanServer able to dispatch to more than one underlying MBeanServer, we need to implent the VirtualMBeanServer.

The VirtualMBeanServer is very much like the DispatchMBeanServer, but instead of delegating its requests to an underlying MBeanServer, it answers query requests itself, and delegates requests on individual MBeans to the MBean implementation class, creating an MBean instance just for the lifetime of the request.

Typically one might also do a secondary dispatch using the 'type' key in an ObjectName to allow more than one VirtualMBean type to be implemented in the same domain namespace. The example has been simplified not to do this.

 The VirtualMBeanServer assumes that the ObjectNames of all the VirtualMBeans it manages are of a specific form:


You might recognise this form: it's the ObjectName convention described in JMX Best Practices. Note the quoting of the instance name... I've added this since, in the general case, we cannot be sure that the instance name won't contain illegal characters for a JMX ObjectName, such as the colon or a space character... fortunately, JMX's ObjectName already has support for this via its quote() and unquote() methods.

In order to facilitate the mapping between the name of an underlying resource and this ObjectName form, we've introduced a helper class, called an ObjectNameFactory (onf). This class maps between a simple instance name (String) and an ObjectName of this form:

The ObjectNameFactory class

public class ObjectNameFactory {     /** Constructor - manage ObjectNames for a specific domain and type */     public ObjectNameFactory(String domain, String type) {         ...     }     /** given an instance name, map to an ObjectName */     public ObjectName getObjectName(String instanceName) {         ...     }       /** given an ObjectName, map to an instance name */     public String getInstanceName(ObjectName objectName) {         ...     }     ... }

The VirtualMBeanServer needs to be able to answer the same two families of request as the DispatchMBeanServer:

1. Queries across MBeans specifying an ObjectName pattern
2. Actions concerning a specific MBean, specifying the MBean's ObjectName

In order to be able to support these two families of request, our VirtualMBeanManager needs to have MBean-specific support for three things:

MBean-specific abstract methods in the VirtualMBeanManager

/** Return a Set of all active instance ids of this type */     protected abstract Set<String> getInstanceSet();       /** Return a reference to an MBean given its instance id */     protected abstract DynamicMBean getMBean(String instanceName)         throws InstanceNotFoundException;     /** Return the MBean class name used for all MBeans of this type */     protected abstract String getMBeanClassName();

Here again are the two methods we saw in the DispatchMBeanServer implementation above, reimplemented in the VirtualMBeanServer

Dispatching to a Virtual MBean

public Object getAttribute(ObjectName name, String attribute)         throws MBeanException, AttributeNotFoundException,                InstanceNotFoundException, ReflectionException {         return getMBean(name).getAttribute(attribute);     }

Querying VirtualMBean instance information

public Set queryNames(ObjectName pattern,                           QueryExp query) {         Set resultSet = new HashSet();         if (query != null)             query.setMBeanServer(this);         for (String instance : getInstanceSet()) {             ObjectName on = onf.getObjectName(instance);             try {                 if (pattern == null || pattern.apply(on)) {                     if (query == null || query.apply(on)) {                         resultSet.add(on);                     }                 }             } catch (Exception e) {                 // ignore query exceptions             }         }         return resultSet;     }

An example using VirtualMBeans

Now that we've described all the infrastructure, here's a little example that exposes the individual threads in a JVM as separate JVMThreadMBeans, using the ThreadMXBean as the underlying data source.

Let's first of all define our JVMThreadMBean interface and its implementation:

JVMThreadMBean

The interface provides the same information as in the underlying data source:

JVMThreadMBean interface

package thread; import dispatch.*; /**  * Interface JVMThreadMBean  */ public interface JVMThreadMBean {     /**      * Get blockedCount      */     public long getBlockedCount();     /**      * Get blockedTime      */     public long getBlockedTime();     ... }

Its implementation is pretty straight-forward:

JVMThread MBean implementation

package thread; import dispatch.*; import java.lang.management.ManagementFactory; import java.lang.management.ThreadInfo; import java.lang.management.ThreadMXBean; import javax.management.InstanceNotFoundException; /**  * Class JVMThread  * JVMThread Description  *  * @author nstephen  */ public class JVMThread implements JVMThreadMBean {         private int id;         private ThreadInfo getThreadInfo() {         return ManagementFactory.getThreadMXBean().getThreadInfo(id);     }     public JVMThread(String instance) throws InstanceNotFoundException {         Exception cause = null;         try {             // initialize thread id and get info to see if exists             id = Integer.parseInt(instance);             if (getThreadInfo() != null) {                 return;             }         } catch (Exception e) {             cause = e;         }         InstanceNotFoundException e =                 new InstanceNotFoundException("No such thread id "+id);         e.initCause(cause);         throw e;     }         /**      * Get blockedCount      */     public long getBlockedCount() {         return getThreadInfo().getBlockedCount();     }         ... }

Now let's worry about the JVMThreadMBeanManager, responsible for responding to queries and MBean instance requests. It subclasses the VirtualMBeanManager and implements the missing abstract methods:

JVMThreadMBeanManager class

package thread; import dispatch.*; import java.lang.management.ManagementFactory; import java.util.HashSet; import java.util.Set; import javax.management.DynamicMBean; import javax.management.InstanceNotFoundException; import javax.management.NotCompliantMBeanException; import javax.management.StandardMBean; /**  * JVMThreadMBeanManager.java  */  public class JVMThreadMBeanManager extends VirtualMBeanServer {         /**      * Creates a new instance of JVMThreadMBeanManager      */     public JVMThreadMBeanManager(String domain) {         super(domain);     }         protected String getMBeanClassName() {         return JVMThread.class.getName();     }         protected DynamicMBean getMBean(String instance) throws InstanceNotFoundException {         try {             JVMThread impl = new JVMThread(instance);             return new StandardMBean(impl, JVMThreadMBean.class);         } catch (NotCompliantMBeanException ex) {             // should never happen             throw new RuntimeException("Implementation error", ex);         }     }         protected Set<String> getInstanceSet() {         Set<String> resultSet = new HashSet();         long[] ids = ManagementFactory.getThreadMXBean().getAllThreadIds();         for (int i = 0 ; i < ids.length; i++) {             resultSet.add(Long.toString(ids[i]));         }         return resultSet;     }     }

Our example agent

The example agent is responsible for creating a DispatchMBeanServer and then creating and registering a JVMThreadMBeanManager in a given domain... it then just needs to answer MBeanServer requests.

ExampleAgent

package example; import dispatch.DispatchMBeanServer; import thread.ThreadMBeanManager; import javax.management.MBeanServer; import java.lang.management.ManagementFactory; public class ExampleAgent {     public static void main(String[] args) throws Exception {                  ExampleAgent agent = ExampleAgent.getDefault();                 System.out.println("ExampleAgent started. Waiting...");         System.in.read();     }     public void init() throws Exception {                 if (!(getMBeanServer() instanceof DispatchMBeanServer)) {             throw new RuntimeException("Requires setting\n" +               "-Djavax.management.builder.initial=dispatch.DispatchMBeanServerBuilder");         }                 DispatchMBeanServer mbsd = (DispatchMBeanServer)(getMBeanServer());                 mbsd.addMBeanServerDomain("virtual", new JVMThreadMBeanManager("virtual"));     }         public synchronized static ExampleAgent getDefault() throws Exception {         if(singleton == null) {             singleton = new ExampleAgent();             singleton.init();         }         return singleton;     }         public MBeanServer getMBeanServer() {         return mbs;     }         // Platform MBeanServer used to register your MBeans     private final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();         // Singleton instance     private static ExampleAgent singleton; }

Now that we've put that all together, let's compile and execute it, not forgetting to specify our new DispatchMBeanServerBuilder class on the command-line:

java -Djavax.management.builder.initial=dispatch.DispatchMBeanServerBuilder example.ExampleAgent

... and here's the ob-screen-shot showing the Virtual JVMThreadMBeans all showing up in JConsole:


The above, whilst being a lot of code, just shows basic extensions to an MBeanServer to support Virtual MBean implementations. Improvements might include:

• Supporting more than one VirtualMBean type in the same domain
• Moving the MBeanManager logic to a separate class so that it can be used to instantiate classic MBeans too
• Adding security checks to MBeanServer accesses
• Providing MRU caching of Virtual MBeans and a VirtualMBean lifecycle interface for callbacks
• Providing support for registration/unregistration notifications through listening to underlying events
• ...

Materials

You can find all the sources to this example implementation here.

( Jan 16 2007, 06:00:00 PM CET ) Permalink Comments [4]