Thursday March 13, 2008

As you perhaps know from reading my posts, I am involved in the definition of JSR 262, Web Services Connector for JMX Agents. JSR 262 offers an access to Web Services that is very far removed from Representational State Transfer (RESTful) Web Services. So, this blog is not about JSR 262, but instead it describes some of the investigations and development I have performed related to the Java API for RESTful Web Services (JAX-RS API) and the JMX API.

Having simple HTTP-based access to a Java Management Extensions (JMX^TM) MBean is an idea that I like a lot and one for which I think that there are thousands of real use cases. For example, when developping a Web application, integrating a piece of information obtained from JMX instrumentation, such as the status of a service, should be as simple as loading an HTML page.

My first attempt was to define and implement a merge of JSR 262 and a RESTful approach... All in vain! The two technologies cannot co-habit, so you are forced to choose between them. You either have to go down the WS-* path or you have to go down the RESTful path. There's no way to make one leverage the other to offer a tidy solution. In this case, keeping the two parts separate is much better than trying to bundle them together. In this investigation, I have limited the scope just to retrieving information from JMX instrumentation. Setting MBean attributes or invoking operations has not been addressed (yet).

If you want to play with this prototype for RESTful access to JMX instrumentation (based on Jersey, which you will need to download) before reading the blog entry, you can jump straight to Running the Adaptor. This RestFul JMX Adaptor is hosted by the OpenDMK java.net project.

But first, let's start with a bit of background. The next sections describe how the information from JMX instrumentation is accessed, and how this information can then be mapped into a format that is acceptable to the RESTful approach.

JMX Instrumentation Information Tree

So, how does JMX instrumentation present its information? First, we have an MBean server in which MBeans are registered. Each MBean contains a set of attributes that represent an actual piece of information. MBean attributes are not always exactly actual pieces of data, though, as this is dependent upon the Java class of the attribute. If the attribute is a primitive type, then yes, the attribute is indeed an actual piece of data. If it is a complex type (or a collection), then it is not. Complex types need to be traversed to discover the actual data.

So, we can represent the whole set of information exposed by the JMX instrumentation as a tree. The root of the tree is the MBean server, the MBeans are the root's child nodes and the attributes are the MBeans' child nodes. Some attributes are terminal nodes (primitive nodes), some other attributes (complex types) are intermediary nodes.

The following figure presents the JMX instrumentation information tree:

In this tree, we can see the data path as leaves.

Here, we can identify three elements:

1. The leaf node, which is a data container.

2. The intermediary node, which is a child container.

3. The data path, which is composed of the set of parent/child relationships from the root to the leaf.

Traversing Complex Types

We just saw that you need to traverse complex types to access the actual data. However, how to deal with every sort of complex type that can be defined is outside the scope of these investigations. So, we have limited ourselves to Array, List, Set and the generic JMX types CompositeData (which is very similar to Map) and TabularData (which is a table of CompositeData).

The above list of types complies with MXBean usage, so we are able to go quite far, even with this limited set. These types are traversed in different ways, as follows:

• To traverse a List or an Array, you need to iterate the structure.

• To traverse a CompositeData, you need to find out each key and retrieve the associated value.

• To traverse a TabularData, you need to iterate the structure and then traverse each CompositeData (as described above).

Mapping the Information Tree to RESTful Web Services

I am not making a revolutionary statement by saying that nodes are resources and data paths are URIs. In our information tree, we have two kinds of resources, container resources that are referenced by other resources, and the content resources. These resources can be identified by URIs, as explained in the next sections.

Resource URIs

The different resources in our information tree are identified by the following URIs:

• MBean server: http://<host>:<port>/<root context>

• MBean: http://<host>:<port>/<root context>/<MBean ObjectName>

• MBean attribute: http://<host>:<port>/<root context>/<MBean ObjectName>/<attributeName>

• Collection (list or array) element: http://<host>:<port>/<path to complex type>/<index>

• Map element: http://<host>:<port>/<path to complex type>/<URL encoded(key.toString)>

• CompositeData entry: http://<host>:<port>/<path to complex type>/<key>

• TabularData element: http://<host>:<port>/<path to complex type>/<index>

Container Resources

MBean servers, MBeans, arrays, lists, sets, CompositeData entries and TabularData elements are all container resources. A simple XML definition is enough to represent these resource types. To avoid defining a new XML element, we can reuse the XHTML list element (<ul><li></li></ul>).

References to a child can either be relative to the current resource or absolute. I chose to make them absolute. Below are some examples of representations of different container resources.

• MBean server containing two MBeans:

  <ul>
  <li>http://<host>:<port>/<root context>/java.lang:type=Memory</li>
  <li>http://<host>:<port>/<root context>/java.lang:type=Thread</li>
  </ul>

• MBean with two attributes, NonHeapMemoryUsage and HeapMemoryUsage:

  <ul>
  <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage</li>
  <li>http://<host>:<port>/<root context>/java.lang:type=Memory/HeapMemoryUsage</li>
  </ul>

• CompositeData, composed of commited and init keys:

  <ul>
  <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage/commited</li>
  <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage/init</li>
  </ul

Note: You will notice that I am not using the HTML link tag (<a href="..."></a>). This is because I am not defining HTML access (such as that provided by the JMX technology's HTML adaptor), but rather XML access.

Container resources like those above are not of particularly great interest to us. They help with navigation when you are having difficulties finding your way through the information tree, but when you know where you want to go, they are almost useless. When you know where you want to go you don't need to navigate at all, you just jump at light speed to the end of the path by providing the full URI.

However, in some cases where the information is dynamic, container resources can be really helpful.

Content Resources

In this prototype, actual data is represented by straightforward plain text. For example, Hello world, is plain text...

A Restful Adaptor for JMX Instrumentation

I have developed a prototypal implementation of the above mapping using the JAX-RS Reference Implementation (a.k.a the Jersey project). The adaptor attaches to the platform MBean server that is located inside any Java Virtual Machine (JVM^TM) from Java Development Kit (JDK) version 5.0 or above. The adaptor offers URI-based navigation, and the returned pages are in either HTML or XML format, according to the receiving client's "Accept Headers" setting.

The current prototype covers all the types exposed by the JVM software's MXBean interfaces (used for monitoring memory, threads, classes, etc.). If your MBeans are also MXBeans, they you should be able to navigate to them without problems. I re-used the HTML <ul><li> tags to handle lists in XML. In HTML, the <a href="..."> tag is used in conjonction with the <li> tag in order to make clickable URLs.

The adaptor provides an API (JMXWSMBeanRestResource#setListXMLTags) that allows you to change these list tags and provide your own.

If your MBeans expose unsupported types, you can provide your own resources by calling JMXWSMBeanRestResource#registerResourceHandler. Look at the source code for JMXWSMBeanRestResource.CollectionResource to understand how the adaptor's JAX-RS resources are defined.

JAX-RS Resources

JAX-RS allows you to define “top level” resources that return sub-resources. I defined a top level resource named JMXWSMBeanRestResource, which serves as the JMX adaptor's entry point and handles the MBean server and the MBean resource types. It also delegates the navigation of attribute values to sub-resources. The current sub-resources are the following:

• PlainResource

• ArrayResource

• CollectionResource

• MapResource

• CompositeResource

• TabularResource

Each resource handles a data type and, in turn, delegates it to other sub-resources to reach the leaf (the plain text resource).

The JAX-RS resource / sub-resource pattern allows for very simple tree navigation. Each resource is annotated using JAX-RS annotations. The following ArrayResource shows what a sub-resource actually does:

public class ArrayResource { 

/** Returns the list of URI that point to array items. **/

@GET

@ProduceMime("text/xml")

public Response getXMLItemURIs() throws Exception {

return getURIs();

}

/** Returns the Sub resource pointed by the array index. ResourceFinder is a reference to the JMXWSMBeanRestResource that manages Sub Resources. **/

@Path("collection/{index}")

public Object getItem(@UriParam("index") String index) throws Exception {

int idx = Integer.valueOf(index).intValue();

return resourceFinder.findResource(Array.get(array, idx));

}

...

}

That's about it for the introduction. Now you can see the adaptor in action.

Running the adaptor

1. You need first to do a CVS checkout of the Restful Adaptor sources from OpenDMK java.net project

   1. export CVSROOT=:pserver:guest@cvs.dev.java.net:/cvs

   2. cvs co opendmk/contributions/rest-adaptor

   3. Open the NetBeans project rest-adaptor/RestAdaptor.

2. Follow the steps documented in Restful Adaptor readme file.

3. Run the adaptor (this requires a minimum of JDK version 5.0, although JDK 6 is recommended).

4. The adaptor will display the URL on which it is bound, as well as some other information, such as the composition of the URIs and the names of the MBeans.

5. In your browser type the following URL : http://localhost:9999/rest/jmx.

6. If your browser is set to accept XML headers by default over HTML headers (as is the case for Firefox), you will receive the resources in XML form, which is not really the best form for a browser. However, you can still navigate by copying and pasting the URIs until you reach a leaf. The best thing to do is to change your Firefox settings to accept HTML headers by default over XML headers, as follows:

   1. In your browser, type about:config

   2. Go to network.http.accept.default

   3. Prefix the value with text/html.

   You can now navigate from the root to the leaves.

But, as I said previously, navigation is really not the best way. What you really want to do is to jump to a value using a well-known URL. But where are these well known URLs? They are there already, waiting for you. If you want to see them all, go to the following URI:

http://localhost:9999/rest/jmx/all

This URI references a special resource that lists ALL the leaves of a JMX information tree for a given MBean server. All listed URLs identify an actual piece of data.

This list of URLs (that can be very long) can be filtered by using a URI parameter that contains the following JMX specification ObjectName pattern: http://localhost:9999/rest/jmx/all?pattern={your jmx ObjectName pattern}

Well Known URLs

In a system being managed by JMX instrumentation, the set of exposed MBeans is generally well defined. The set of MBean ObjectName instances is well known and documented so that clients can access the exposed information. We can take advantage of this and select a set of URLs that will never change for a given system. For example, the URIs that identify the commited heap memory of the JVM software from JDK version 5.0 or version 6 will never change, except for the host and port deployment information, as shown below :

http://<host>:<port>/rest/jmx/java.lang:type=Memory/NonHeapMemoryUsage/commited

This file contains the URIs you expect to find in a JVM implementation from JDK version 6, regardless of your context. For monitoring and management of the JDK software, the URLs that change are related to Array and TabularData (i.e. thread IDs, system properties, logger names, etc.) and to memory managers that are not identical as far as the hardware architecture is concerned.

You can easily discover these URLs without navigating through the tree, by starting at index 0 and incrementing by 1 until you reach a 404 error. For example, to retrieve all the thread IDs, you just have to increment the {index} :

http://<host>:<port>/rest/jmx/java.lang:type=Threading/AllThreadIds/{index 0-N}

The example of thread IDs shows up a major limitation of this approach when the information is very dynamic (as threads are). The array size and elements can change in such a way as makes the returned values invalid. In this case we would need a snapshot of the array.

Conclusion

This RESTful adaptor provides a possible way of implementing a RESTful access to JMX instrumentation. I have a strong feeling that it could be very helpful when aggregating loosely coupled pieces of information. The current adaptor does not address all data structures (although MXBeans are fully handled), but it is provided in Java source code and offers an API for you to add your resource handlers.

Possible usages:

• Develop AJAX based dashboads to monitor Java VM and/or your applications.

• Quickly construct HTML pages to offer human readable access to the internals of your systems.

What are the next steps:

• Add a support for Notifications.

• Add a support for POST/PUT.

• Add a support for snapshot of values.

I would like to thank Paul Sandoz for is great help. REST philisophy is Paul's second nature. Expose him your problem and he will Restify it. Really smart!

I would be very happy to receive any feedback about this approach. Feel free to contact me and thank you for reading.

Regards.

Jean-François Denise

Monday February 18, 2008

The JSR 262 has just entered the Public Review phase. The Early Access 3 of the Connector, that implements the Public Review, can be downloaded from http://ws-jmx-connector.dev.java.net.

Specification changes since Early Access 2:

• PUSH mode supported for notifications. This is specified in the client by including JMXWSDefinitions.JMX_WS_NOTIFICATION_PUSH_MODE_URL in the Map given to JMXConnectorFactory.connect. Some related Map entries on the client and server configure security. A new interface JMXWSCredentialsProvider specifies how the connector server gets the credentials it needs to connect to any given client and push notifications.

• New JMX types supported for MBean operations: NotificationResult, NotificationFilterSupport, AttributeChangeNotificationFilter, MBeanServerNotificationFilter. This allows the JSR 262 connector to support the new Event Service being defined by JSR 255, which has MBean operations that use those types. We also added support for java.lang.Enum and java.util.Set.

• Connection identifiers supported consistently with the JMX Remote API (JSR 160) specification. This was already alluded to in the Early Draft 2 but is now formalized consistently.

• Removed the interface JMXWSManResourceHandler. This was a halfway house between the Connector and Adaptor models. The model now is that for a Connector (client sees the JMX Remote API), the protocol is completely specified and invariable.

• Some minor reshuffling of static final fields between JMXWSConnectorServer and JMXWSDefinitions.

• The getMBeanInfo operation is now mapped as a Custom Action. Previously the MBeanInfo was a property of the MBean resource, alongside all the attributes of the MBean. This mix of data and metadata was strange.

• JMXAuthenticator.authenticate method. The passed Object parameter used to be a String[2]. It is now a String[3]. The third element contains a client certificate in case HTTPS Mutual authentication is enabled (null otherwise).

• XML Element NotificationLost has been removed from the schema. WS-Management already offers such definition by the mean of DroppedEvents XML element. NotificationLost was a redundant redefinition.

• The schema has been updated, mostly to reflect the changes above, and some minor refactoring.

Implementation changes since Early Access 2:

• Leverage of the Metro stack components (as detailed in my previous blog entry) in order to increase interoperability and efficiency of the Connector. We are currencly observing a *3 increase in performance.

• Rely on the latest JAX-WS 2.1.3 release.

• Introduction of an adaptor API that allows you to change the way MBeans are exposed in the WS-Management world (javadoc is bundled in the Early Access 3 auto extractible jar under docs directory).

The document that covers interoperability with Microsoft WinRM tooling (winrm command line, VBScript API and Event Viewer support for WS-Management events) has been updated in order to reflect the Public Review changes.

Feedbacks on the specification can be sent to jsr262-spec-comments@sun.com. Feedbacks on the connector implementation can be sent to users@ws-jmx-connector.dev.java.net.

Thanks.

Jean-Francois Denise

Monday February 11, 2008

Up the Metro stack to JMX Technology: A Wise Man's Journey

What a long trip... We started by driving on the HTTP highway, slid on SOAP, turned left at the XML crossroads, passed over the WS-Addressing bridge, visited WS-Enumeration, WS-Eventing and WS-Transfer, got a bit lost in WS-Management country, discovered that JMX was not so far off, and finally took a rest (but not a RESTful one) in the Web Services Connector... Did we do all that on our own? No, we did it with a set of friends, the Metros, or the high-performance, extensible, easy-to-use web service stack.

OK, I guess you get it. This little journey lists the set of standards that have been pressed into service to expose JMX MBeans as WS-Management resources. Luckily we didn't have to implement them all. The JAX-WS (a core Metro component) and WiseMan (a WS-Management Java implementation) projects have been of great help.

Now Sun has announced that the WiseMan project is to become part of the Glassfish community. So, it seems to be an appropriate time to look at how the Web Services Connector for JMX Agent RI makes use of WiseMan and Metro.

In this article, I will describe how we changed WiseMan to use part of the Metro stack, what the benefits were, and finally how these technologies are leveraged in the Web Services Connector for JMX Technology

Moving WiseMan to the Stack

In 2007, some significant engineering tasks were undertaken in the WiseMan project to bring the benefits of Metro to WiseMan. This section outlines the steps that were needed to make this move a reality.

The details of this technology switch are of interest to anyone who wants to make their existing applications benefit from Metro's features.

From Servlet to JAX-WS 2.0

WiseMan, was, in its pre-1.0 release, closely coupled with Servlet. At the time, Servlet container was the only way to deploy a WiseMan-based application. We decided to add a JAX-WS Endpoint to the WiseMan project and suddenly we found we could be much more agile. Deployments like the Java SE and Java EE platforms became possible. What a change! For example, being able to base our test suite on the lightweight HTTP server bundled with JAX-WS made our lives much easier...

The JAX-WS Endpoint we developed is a Provider<SOAPMessage>. Simply annotating with @WebService was not possible. WS-Addressing makes intensive use of SOAP headers to convey part of the protocol information. To access to such headers, we need full access to the SOAP Message. After some redesigning of the existing code we extracted a WSManAgent Class that is accessible from a JAX-WS Endpoint or a Servlet.

We took this opportunity to switch to the JAXB release contained in Metro.

So we were happy and impatient to move forward. We already knew that the JAX-WS team was working on adding WS-Addressing support and were waiting for it eagerly.

From JAX-WS 2.0 to JAX-WS 2.1

JAX-WS 2.1 comes with support for WS-Addressing. It's a very smart integration. By just using annotations, your Endpoint becomes addressable thanks to WS-A.

The @Addressing annotation that you can use to tag your Endpoint implies that the WS-Addressing 1.0 release is to be used. However, WS-Management requires the use of another release of the WS-A standard, which is a great pity. We discovered that we were not able to leverage this support. We were afraid this was the end of the road, but the JAX-WS engineering team came back to us showing us the Metro specific API. We were not the first customer with a dependency on a Membership Submission and they defined another “non standard but supported” Annotation to express our requirement : com.sun.xml.ws.developer.MemberSubmissionAddressing

With this annotation added, we removed all the code related to WS-A request-response headers.

So we were happy...but not for long. Our great friends Yves and Sandra of the JMX QA team started to develop some simple stress tests and we started to discover horrible things...

From SOAPMessage to Message

We discovered that 80% of the time spent handling a request was spent in the DOM/SAAJ layer. That's 80% of some big numbers, too...

You remember? Our Endpoint is a Provider<SOAPMessage>. This means that when called, a SOAPMessage object is injected and a SOAPMessage object is expected as the returned value. So we were either stuck, or we had to re-implement the SOAP layer to rely on the STAX API, or we were left with our disapointing figures.

Could the Metro API solve this issue? Yes, the Metro API solved this issue. Once again we were not the first customer, etc., etc., etc. As a solution, an efficient implementation of SAAJ, that is very well integrated with JAXB, is exposed as the Message API (located in the com.sun.xml.ws.api.message package). In order to benefit from its power, you must completely forget all the development you have done on top of SAAJ and rewrite everything on top of this new API. Which is exactly what we did. OK, we re-designed it to make both approaches live together nicely, but we also designed a new API for WiseMan, that was completely isolated from SAAJ. And things started to work much faster (3 to 4 times quicker). The numbers were becoming acceptable, for this kind of XML based processing.

Current WiseMan Architecture

The diagram above shows the different products that are used to build the WiseMan stack on the client and the server. You can see that the WiseMan client stack doesn't rely on Metro, yet. A work-in-progress aims to define and implement such a move, which will greatly improve performance and ease of use.

Improvements for the WiseMan Project

This move helped the project to:

• Improve deployability

• Improve alignment and reuse

• Improve interoperability

• Improve performance

Deployability, interoperability and performance are the key words of the Web Services Connector for JMX Technology. Based on these new capabilities, WiseMan is becoming the WS-Man implementation of choice for the Web Services Connector for JMX.

When using this connector you are not directly in contact with the stack but you indirectly benefit from the properties you are interested in.

Web Services Connector for JMX Continues Leveraging Metro

This section looks at how the the Web Services Connector for JMX Technology continues leveraging Metro.

Fast Infoset

- "Increase performance!"

- "Increase performance? Hmm..."

Simple, use Fast Infoset! It comes with the stack, it's efficient, it doesn't break interoperability and it's easy to use. We did it and we experienced another big improvement. To enable Fast Infoset in your client API, access the BindingProvider and provide a Fast Infoset key to the request context. For example:

BindingProvider provider = (BindingProvider) port;

Map<String, Object> requestContext = provider.getRequestContext();

requestContext.put("com.sun.xml.ws.client.ContentNegotiation", "pessimistic");

Dispatch<Message> on the client side.

On the client side of the connector, we use the JAX-WS Dispatch class. A Dispatch instance can be seen as an Endpoint proxy that deals with the complete SOAP Message (as opposed to classical JAX-WS proxies that expose a service level API). Since JAX-WS 2.1, Dispatch also handles WS-Addressing. Using Metro specific API, we are creating an efficient Dispatch that is compliant with the WS-Addressing Membership Submission release.

The following code extract shows how to create a Dispatch<Message> for Membership Submission:

Dispatch<Message> port = service.createDispatch(JMXWSConfiguration.JMX_WS_CONNECTOR_PORT, Message.class, Service.Mode.MESSAGE, new MemberSubmissionAddressingFeature(true, true));

Endpoint on the client side

WS-Management defines a way for a WS-Management Agent to deliver its Events in a PUSH mode. This delivery mode implies that the event sink (the client) is itself an Endpoint. We deploy a JAX-WS Endpoint (Provider<Message>) inside our connector client to receive pushed events.

WS-A EndPointReference API

The JMX Web Services Connector RI exposes a non-standard API to customize the mapping of MBeans to WS-Management resources. In this API, we allow customers to hook their customization as low as the WS-Management protocol itself. We needed a WS-Addressing Endpoint reference to expose.

The JAX-WS class javax.xml.ws.EndpointReference offers a standard 1.0 representation that we are internally translating into a Membership Submission via the Metro API (using the class com.sun.xml.ws.developer.MemberSubmissionEndpointReference).

Security

JAX-WS offers support for HTTP Basic Authentication and HTTPS. We are leveraging both.

HTTP Basic Auth

You provide the credentials to a Dispatcher via the request context. The following code extract shows how to provide your credentials.

BindingProvider provider = (BindingProvider) port;

Map<String, Object> requestContext = provider.getRequestContext(); requestContext.put(BindingProvider.USERNAME_PROPERTY, “user); requestContext.put(BindingProvider.PASSWORD_PROPERTY, “password”);

Server Side Security

The way you secure the server side depends on the container in which the Connector is deployed. For more details about how to use the HTTP server bundled in Metro, check the HTTP server API documentation.

Current Web Services Connector Architecture

Interoperability

Based on this architecture, we performed interoperability testing with WinRM (Microsoft's WS-Management based set of tools and API).

This document sums up the scenarios we are testing.

We didn't experience any problems related to interoperability.

Conclusion

Choosing Metro was definitely the right choice. On nearly all levels it has offered the features we were hoping for. When the boundaries of the standards were reached, Metro-specific API entered the scene and offered a solution. Metro really is ahead of the field in terms of improving performance.

So who is the wise man? The ones who defined such long stacks? No, I don't think so....

But Glassfish could be. Its management and monitoring features are based on JMX technology.

So?

So, by deploying the Web Services Connector for JMX (for which the Public Review specification and Early Access 3 RI will be available mid-February) in a Glassfish Server, all the existing MBeans (such as those defined by AMX, by JSR 77, and by you) are automatically exposed as WS-Management resources.

You can imagine the long journey we went through. Would you want to do this on your own, to expose your tiniest resources?

Be wise and use JMX, let us fill in the gaps!

Jean-François Denise

PS: My sincere thanks to Jitendra and Rama from the JAX-WS team for their help. Special thanks to Stuart Clements, my personnal editor and grammar checker.

Friday December 07, 2007

In this blog entry I want to provide you with a tip in order to help you answer to the fact that there is or not a memory leak in your application.

In some cases, where you are instantiating a lot of objects in a very short period of time, you can observe some huge memory consumption making the application to run out of memory. When it happens, we are (too?) quickly identifying this problem as a Memory leak and forget that the application can have reached some badly set JVM memory threshold causing OutOfMemory exception.

In order to distinguish between theses cases, I am using JConsole in a very very simple and quick way.

Steps to follow to isolate the case

1) Keep the JVM activity below the OutOfMemoryException to occur. For example, if your server crashes after 10000 received requests, tune your test case to deal with only 5000 requests. Doing so you will not run out of memory.

2) Start the application you want to monitor.

3) Start JConsole and attach it to your application. If your are using NetBeans, click on the "Run Main Project with Monitoring and Management" toolbar button.

Your project is compiled, run and JConsole automaticaly attached to it.

4) Click on JConsole Memory Tab and call Perform GC button located on the top right corner. The memory consumed during the application startup is released.

5) Start the activity (start a client application that will connect to the server, call a JMX MBean that initiates some activity, ...). Thanks to JConsole you will follow the memory allocation. Some big numbers can be reached there. In the example below, my server is simply serving simple Web Services Request. The memory consumed reaches 200 Megs...

6) Once the activity is stopped, you will notice that the amount of consumed memory is still very high. Call Perform GC multiple time to make sure that the GC is actually freeing the consumed memory.

7) Now you have 2 cases, or the memory consumed number is similar to step 4 and you are not experiencing a memory leak, or it stays high and you found a memory leak. In this example, the memory consumption is high but is not linked to any memory leak. You will notice that the Old Generation Memory pool (last vertical bar on top of Heap label) is cleared. There is no accumulation. In case of memory leak, you would have seen that this pool contains some objects. Mandy Chung wrote a blog entry that describes such case.

You can now take the right action. Or you allocate more memory for your application (use -Xmx JVM option for example), or you start investigate your problem (eg: by using NetBeans Profiler).

Jean-François

Monday December 03, 2007

NetBeans 6.0 is out. You should go experience the new Editor, really amazing... What does it add compared to the previous one? Well... not a lot of things... It simply anticipates your coding, checks your coding and drives you to write better code...no more than that... Kudo to the NetBeans team!

JMX module is part of the game and still downloadable from the stable update center. I am pasting a screenshot of the new update center UI that you now find in NetBeans 6.0.

In this blog entry I am focusing on JDK 6.0 JConsole related new features. You can access to the list of features, latest tutorials, documentation and known issues from this web site : http://management.netbeans.org/jmx.

JConsole to Monitor J2SE Project

If you are used to the JMX module, you will notice that the way to connect JConsole to your J2SE Project has not changed.

Just set your J2SE Project as the “main project”and click on this toolbar button:

Your project will be compiled, run and JConsole will attach to it. The way to setup JConsole and the Monitoring & Management features of your project has changed. The next chapter shows you how to achieve such configuration.

Better integration in J2SE project

Right click on your Project to access the project properties. You will notice a “Monitoring and Management” section. This is your configuration entry point.

From there you can:

• Enable / Disable JConsole auto attachment.

• Change JConsole polling period.

• Use (or not) the project classpath to make JConsole findout your MBean custom handled types. Unselecting this option will show you that you introduced some custom types that imply deployment of these classes on the JMX client application. Having custom types handled by your MBean interfaces is not something that we encourage and this is something that you can simply avoid by using MXBean

• Lookup the Project Classpath to load JConsole custom plugins. A new JConsole API has been defined in order to allow you to develop custom JConsole tabs and have them added to the list of tabs. The next chapter cover this feature. This article describes JDK 6.0 JConsole plugins new feature.

• Explicitly add paths to your JConsole plugins jar files or classes.

• Enable or disable Remote RMI access. By default, remoting is not enabled.

  • You can provide the RMI port the server will listen to. Doing so, you will start a non secure remote access.

  • Or you can provide a path to a management.properties file in which you can configure security.

New Support to load JConsole custom tabs

To do so, simply add JConsole plugins that you downloaded or developed, to the Monitoring and Management Properties Plugins Path list.

In this example, I am adding the JTop plugin that is bundled with JDK 6.0 (<java home>/demo/management/JTop/JTop.jar). This plugin demonstrates the use of the java.lang.management API to obtain the CPU consumption for each thread.

Add the plugin to the Project properties and run your application

In the “Monitoring and Management” Properties :

1. click on Add Jar/Folder button then provide a path to JTop jar file. The jar file is added to the list.

2. Close the project properties.

3. Run your project with by clicking on the JConsole toolbar button.

JConsole is started, JTop tab being loaded and displayed.

Some interesting JConsole plugins

• Top threads

• BeanShell

• Script Shell. Scripting JConsole plugin located in the demo directory.

• An interesting use of Script Shell with JRuby

JConsole to Monitor any Java project

If you want to connect JConsole to another kind of Java project (e.g.: Web Application), you can start JConsole and connect it to the application server running your application. To do so click on the following toolbar button to launch JConsole management console:

You can tune the JConsole default Target (default URL to connect to), polling period, plugins path, classpath, arguments or JVM arguments from the NetBeans Options. To do so:

1. Select Tools | Options from the main menu

2. Select Miscellaneous tab

3. Select JConsole tab

The following window is displayed:

Hope to have helped you discover a useful new feature of the JMX tooling.

Regards.

Jean-François Denise

(jean-francois dot denise at sun dot com)

Wednesday September 05, 2007

In a previous post, Securing JMX Web Services Connector, I explained how to enable HTTPS when using the JSR 262 RI. Basic Authentication and HTTPS are the basic building blocks on which you can rely to secure remote WS access to your JMX instrumentation. These technologies are very commonly used and are simple to put in place when you need point-to-point security. For more complex architectures, however, where messages contexts contain multiple nodes, HTTPS is not the best solution.

WS-Security

WS-Security, a Web Services standard defined by OASIS, aims to secure communication at the SOAP message level (as opposed to HTTPS, that secures messages at the transport level) in a unified, flexible and extensible way. WS-Security supports partial message signing and encryption, in case you need to encrypt the SOAP message body element and keep the headers non-encrypted; for example, when encrypting credit card numbers or any XML subtrees.

In addition to confidentiality and authentication (also offered by HTTPS + Basic Authentication), WS-Security offers message integrity, thanks to XML signatures. With XML signatures, you can protect your communication against message-tampering.

Interoperability at the security level

Due to the huge number of alternatives you can use when trying to secure your communications, you will generally run into interoperability problems. WS-Security has been designed to offer interoperability while still allowing you to plug in various technologies. The WSIT project on java.net (JavaOne 2007 Technical Session TS-4865 offers a good introduction to WSIT) offers an interoperable WS-* stack that, along with a full set of other WS standards, integrates a Java technology implementation of WS-Security. WSIT is the solution I have adopted to plug WS-Security into the Connector.

Plugging WS-Security into the JMX Web Services Connector

Now that we have briefly introduced WS-Security and WSIT, let's go back to what really interests us, namely plugging WS-Security into the Web Services Connector. The XWSS project, that is part of WSIT, offers everything we need to enable WS-Security in our context. How is this achieved? Simply because the JMX Connector is a JAX-WS Web Service (compliant with WS-Management but still a classical JAX-WS Web Service) and XWSS works well with JAX-WS.

So, I started by reading a XWSS article on how to add WS-Security to your Web Services on top of the Java SE 6 platform. I then adapted the XWSS sample application to create a simple JMX client and server. I interacted with Kumar Jayanti from the XWSS team to get our two technologies working well together. We encountered some issues with WSIT Milestone 6 (mainly related to the fact that WS-Policy was still linked to a Java EE platform type of deployment, and the JMX Connector was being deployed on the Java SE platform), but these have been fixed in the latest WSIT builds. The next WSIT FCS Milestone should contain all the necessary fixes. The usage of WS-Policy makes for a transparent usage of WS-Security. If you package your WS-Security configuration files properly, you can enable WS-Security on the client side and on the server side without touching a single line of code (see the next section for full details of how to make this work).
To enable WS-Security you simply need to package your application with the WSIT binaries and the WS-Security configuration file.

Building and Running the JMX WS Connector Sample Secured with WS-Security

1. Dowload the XWSS sample application.
2. Unzip it. A directory named src is created.
3. Download the sample JMX Agent Main.java file.
4. Copy Main.java into the src/simplejdk6ws/ directory. The ConnectorServer is started on port 8080. In case of conflict, update the Main.java file (be sure to update the client code accordingly, see the next section for the client side).
5. Download the sample JMX Client SimpleWSClient.java file.
6. Copy SimpleWSClient.java into the src/simplejdk6ws/ directory. The Connector connects to port 8080. In case of conflict, update the SimpleWSClient.java file (it should follow the value you previously provided when starting the server). This client application is a very simple one, it connects to the server and asks for the Default Management domain (MBeanServerConnection.getDefaultDomain()).
7. The server side security configuration file is src/META-INF/server_security_config.xml. Any message received and response sent is Authenticated, Signed and Encrypted according to the configuration defined in this configuration file. You don't need to update this file.
8. The c lient side security configuration file is src/META-INF/client_security_config.xml. Any message sent and response received is Authenticated (user Ron), Signed and Encrypted according to the configuration defined in this configuration file. Again, you don't need to update this file.
9. Download WSIT (which contains JAX-WS).
10. Install WSIT with the following command:

    java -jar jax-ws-latest-wsit-installer_nightly.jar

    The jax-ws-latest-wsit directory is created.
11. Create a directory named endorsed. Move jax-ws-latest-wsit/lib/webservices-api.jar into this directory.
12. If you have not already done so ;-), download JMX Web Services Connector Early Access 2.
13. Call the following command:

    java -jar jsr262-ri.jar

    The jsr262-ri directory is created.
14. To build your sample, go to the src directory and type the following command:
    

    javac -cp :../endorsed/webservices-api.jar:../jax-ws-latest-wsit/lib/webservices-rt.jar simplejdk6ws/*.java

15. Download the Java mail JAR file .
16. Start your server from the src directory. The command should be something similar to the following:

    java -cp .:../mail-1.4.jar:../jsr262-ri/lib/jmxws.jar:../jsr262-ri/lib/wiseman-core.jar:../jax-ws-latest-wsit/lib/webservices-rt.jar -Djava.endorsed.dirs=../endorsed/ simplejdk6ws.Main

    You should see the following message:

    JSR 262 ConnectorServer is ready to serve on http://localhost:8080/jmxws

17. Start your client from the src directory. The command should be something similar to the following:
    

    java -cp .:../mail-1.4.jar:../jsr262-ri/lib/jmxws.jar:../jsr262-ri/lib/wiseman-core.jar:../jax-ws-latest-wsit/lib/webservices-rt.jar -Djava.endorsed.dirs=../endorsed/ simplejdk6ws.SimpleWSClient

    You should observe a very verbose output containing the secured received and sent SOAP messages. You will notice that the user Ron never appears in the messages, because the user name is encrypted. At the end, the Default Management domain will be displayed.
18. You are done. Not a line of code was written, and all that was needed was some configuration and packaging.

Bridging with Authorization

When a request is received, the JMX WS Connector Server looks for the existence of Principals in order to create a javax.security.auth.Subject that will be used to check the Java platform's permissions. XWSS, after having dealt with Security, creates a Subject that contains all the authenticated principals (X500 certificate, Trusted user, etc.). XWSS makes this Subject accessible thanks to the call SubjectAccessor.getRequesterSubject(context). This call is XWSS-implementation-dependent and other WS-Security implementations are likely to offer their own way. This is why we have added a hook to plug a Subject extractor into the ConnectorServer to retrieve the Subject and make it accessible to the ConnectorServer.

The following XWSSSecuritySubjectExtractor class is an example of such an XWSS-aware SubjectExtractor:

        class XWSSSecuritySubjectExtractor extends JMXWSSubjectExtractor {
            protected Subject getExtraSubject(WebServiceContext context) {
                try {
                    return SubjectAccessor.getRequesterSubject(context);
                } catch (XWSSecurityException ex) {
                    ex.printStackTrace();
                }
                return null;
            }
        }
                        

To pass the extractor instance to the ConnectorServer, use the environment map :

        Map env = new HashMap();
        
        // Provide the extended SubjectExtractor to inject WS-Security Subject
        env.put(JMXWSConfiguration.JMX_WS_SUBJECT_EXTRACTOR, new XWSSSecuritySubjectExtractor());
        
        JMXConnectorServer server =
                JMXConnectorServerFactory.
                newJMXConnectorServer(new JMXServiceURL("service:jmx:ws:" +
                "//localhost:8080/jmxws"), env,
                ManagementFactory.getPlatformMBeanServer());
        server.start();
                                                                    

This is the updated Main.java file that contains the Subject Extractor.

WARNING: You need the updated JMX WS Connector RI jmxws.jar and wiseman-core.jar file to use the SubjectExtractor API. You can download these jars here.
Call java -jar jsr262-patch-ws-security.jar to extract the jmxws.jar and wiseman-core.jar files.
When compiling the agent, add both JAR files to your classpath.

So, if one day you reach the limits of the capabilities of HTTPS + Basic Authentication, you should think about using WS-Security. I hope that I have demonstrated to you that enabling it is straightforward. But you should be aware that enabling WS-Security will significantly impact the performance of the WS Connector. The time needed to check security added to the time needed to create and parse a secured SOAP Message (here is an example of a AUTH+PRIV+SIGN getDefaultDomain response SOAP message) makes for much slower client/server interaction.

Enjoy and have fun.

Jean-Francois

Thursday August 16, 2007

Securing the ConnectorServer

  JMXConnectorServer server =
                JMXConnectorServerFactory.
                newJMXConnectorServer(new JMXServiceURL("service:jmx:ws:" +
                "//localhost:8080/jmxws"), null,
                ManagementFactory.getPlatformMBeanServer());

  server.start();

  JMXConnectorServer server =
                JMXConnectorServerFactory.
                newJMXConnectorServer(new JMXServiceURL("service:jmx:ws-secure:" +
                "//localhost:8080/jmxws"), null,
                ManagementFactory.getPlatformMBeanServer());

  server.start();

        // Create the SSLContext
         SSLContext ctx = TestSSLContext.getInstance("SSLv3");
        // Create an env map
        Map env = new HashMap(1);
        // Provide the SSLContext
        env.put("jmx.remote.ws.sslcontext", ctx);
        // Create the ConnectorServer providing the env map
        JMXConnectorServer server =
                JMXConnectorServerFactory.
                newJMXConnectorServer(new JMXServiceURL("service:jmx:ws-secure:" +
                "//localhost:8080/jmxws"), env,
                ManagementFactory.getPlatformMBeanServer());
        
        server.start();

 JMXServiceURL url = new JMXServiceURL("service:jmx:ws://localhost:8080/jmxws");
 JMXConnector connector = JMXConnectorFactory.connect(url, null);
 //Get the MBeanServerConnection
 MBeanServerConnection mbsc = connector.getMBeanServerConnection();

 JMXServiceURL url = new JMXServiceURL("service:jmx:ws-secure://localhost:8080/jmxws");
 JMXConnector connector = JMXConnectorFactory.connect(url, null);
 //Get the MBeanServerConnection
 MBeanServerConnection mbsc = connector.getMBeanServerConnection();

Tuesday June 05, 2007

Extending JMXWSManServiceEndpoint JAX-WS endpoint

Extended class

import com.sun.jmx.remote.ws.wsman.JMXWSManAdaptor;
import com.sun.jmx.remote.ws.wsman.JMXWSManServiceEndpoint;
import javax.xml.ws.WebServiceContext;
import javax.annotation.Resource;
import javax.management.*;

@javax.xml.ws.WebServiceProvider
@javax.xml.ws.ServiceMode(value=javax.xml.ws.Service.Mode.MESSAGE)
public class ExtendedEndpoint extends JMXWSManServiceEndpoint {
   @Resource
   private WebServiceContext context;

   public ExtendedEndpoint() throws Exception {
       super(new JMXWSManAdaptor(null, (MBeanServer)MBeanServerFactory.findMBeanServer(null).get(0)));
        setURL(new java.net.URL("http://localhost:8080/samplewebapp/jmxws"));
   }

   protected WebServiceContext getWebServiceContext() {
        return context;
   }
}

Packaging the connector

<?xml version="1.0" encoding="UTF-8"?>
< endpoints
        xmlns="http://java.sun.com/xml/ns/jax-ws/ri/runtime"
        version="2.0">
        < endpoint
                name="JMXWSManEndpoint"
                implementation="ExtendedEndpoint"
                service="{http://jsr262.dev.java.net/jmxconnector/wsdl}JMXConnectorService"
                port="{http://jsr262.dev.java.net/jmxconnector/wsdl}JMXConnectorPort"
                url-pattern="/jmxws"
                binding="http://www.w3.org/2003/05/soap/bindings/HTTP/">
        </endpoint>
</endpoints>

Conclusion

Saturday May 05, 2007

• HP Open View Operations / JSR 262 (done by Denis Rachal from HP)
• Microsoft WinRM Scripting for JSR 262
• Microsoft Vista Event Viewer to subscribe to MBeans

Monday March 19, 2007

Premier Blog en Français pour annoncer notre présentation aux Tech Days à Paris ce Mercredi 21 Mars. Eamonn et moi même allons présenter la session TD1103: In-depth Session: Java SE Beyond Basics: JMX, Performance and Concurrency.
Bien sur NetBeans 5.5 et JConsole sont encore de la partie...
L'agenda des Tech Days est en ligne.
Dépêchez vous, ça à déjà commencé... et vous pouvez y rencontrer James Gosling !!!
A Mercredi.
Jean-François Posted by jeanfrancoisdenise ( Mar 19 2007, 12:46:31 PM GMT+02:00 ) Permalink

Monday February 05, 2007

All,
an alpha release of the JMX netbeans module is now available from the NetBeans Development Update Center (UC).

The support for JMX instrumentation has been enhanced with a new wizard for MXBeans, support for the Java SE plaform 6 JConsole Plugins and a lot of other improvements throughout.

How do you install and use it?

• Download NetBeans 6.0 from NetBeans download
• Download the module from the NetBeans Update Center : Tools/Update Center
• Select and install the Management/JMX module.

Jean-François

The main enhancements

MBean wizard

• New category JMX, which used to be named Management.
• Added support for MXBean.
• One wizard per type of MBean (MBean, MXBean, From an Existing Java Class, StandardMBean).
  Previous release wizard dealt with all kinds of MBeans. This added unnecessary complexity.
• Removal of Dynamic MBean support. If you want to add metadata to your management interface, use the StandardMBean wizard.
• Removal of JUnit test case generation. Instead, use NetBeans' native JUnit support to generate unit tests for your MBeans.

Contextual actions to update MBean classes

Agent Wizard

• This wizard has been removed. It used to place excess constraints on MBean registration.
• MBean registration in the Platform MBeanServer can now be generated in any Java source.
  Thanks to a new Contextual Editor action (JMX/Generate MBean Registration...) you generate MBean registration code where you want to.


• Position your mouse pointer where you want registration code to be generated
  
  
• Right click, select JMX/Generate MBean Registration...
  
• The MBean registration window pops up. Select (or type in) the Class of MBean you want to register
  
• Click OK, and the code to register your MBean in the Platform MBeanServer is generated
  
Simpler, no?

Management and Monitoring Support for Java SE projects

• Better integration of M&M configuration in a new "Monitoring and Management" Project Properties Node
• JConsole project auto-connect has been extended with the ability to provide JConsole plugins.

JConsole Options

Current issues and limitations

• Some generated code is not properly indented.
• Some generated comments are replicated.
• Manager wizard. Generation of the code to establish a secure connection is not working. Don't select this option.
• Tutorial and JavaHelp have not yet been updated.
• For people building the module from the sources (CVS module "management"), functional tests have not yet been updated.

Monday November 13, 2006

Thursday May 11, 2006

OK, I am sure that you think "Another one that wants to surf on the AJAX wave. JMX and AJAX, I know how to do that, it is easy, just write a Servlet and ...". No I am not speaking about that here. I am speaking of a REAL integration of JMX and AJAX. How is it possible? Come and see Eamonn and myself Technology Today and Tomorrow Technical Session.

I have the feeling that you are frustrated, you want some more... Ok, I can't resist (but just a little bit, come and see us, you will have it all :-)).

Web-Services Connector for JMX, does it sound to you? We are defining it in the JSR 262. It allows JMX client and non JMX client to access to JMX agent. It is based on JAX-WS 2.0 (that is Final since the 9th of May). Client and server part of the Connector are exchanging XML documents over HTTP. Client can be written in Java or in any other language such as JavaScript.

Do you better understand of what I am speaking about? Not fully? Come and see the talk, our demo should clarify the subject and give you new ideas for your future JMX development.

Oups... I nearly forgot to say that our NetBeans favorite IDE has a major role to play in the demo. You can't miss it!

Hope to see you there.

Jean-Francois Posted by jeanfrancoisdenise ( May 11 2006, 10:30:53 PM GMT+02:00 ) Permalink

JMX NetBeans lab at JavaOne 2006

This year at javaone I will lead the lab 5120 "Application Monitoring and Management with NetBeans™ 5.0".

During this lab you will learn how to instrument your own Java application using the JMX NetBeans tooling. Once the application instrumented, you will use JConsole to monitor the application.

You will also learn how to write a JMX client application and remotely connect to the JMX agent to access MBean information.

The lab is a 60 minutes one. All has been setup for you to focus on the exercices. The lab team, led by Sang Shin, has done a wonderful job to make your lab a delightful experience.

Hope to meet you there on Friday (05/19/2006 10:45 AM - 11:45 AM) at the Moscone Center Hall E 130/131

Jean-Francois Posted by jeanfrancoisdenise ( May 11 2006, 09:27:49 PM GMT+02:00 ) Permalink

Friday March 10, 2006

Luis-Miguel Alventosa, a member of JMX team, will be presenting Monitoring and Management in Java SE 5.0 at Java Expo '06 in Madrid.

During his talk he will use the JMX NetBeans module to demonstrate the Java SE 5.0 Monitoring and Management features.

You will see JConsole launched from NetBeans, connecting to a Java Swing application to manage its AWT event queue.

If you want more info on this AWT to JMX bridge, you can read the blog entry I wrote in december 2005.

You still have time to register for the event. Don't forget the NetBeans Software Day on Thursday 23rd.

Jean-Francois

Posted by jeanfrancoisdenise ( Mar 10 2006, 04:17:27 PM GMT+02:00 ) Permalink

search

navigation

links

Planet JMX

A. Sundararajan
Alan Bateman
Daniel Fuchs
Éamonn McManus
Joël Féraud
Mandy Chung
Luis-Miguel Alventosa

rss feeds