Arun Gupta, Miles to go ...

TOTD #63: jmx4r gem - How to manage/monitor your Rails/Merb applications on JRuby/GlassFish ?

TOTD #61 and TOTD #62 shows how to use JMX APIs to locally/remotely manage/monitor your Rails/Merb applications. This Tip Of The Day extends TOTD #62 and shows how remote management can be done using jmx4r gem.

This gem provides a clean mapping between JMX APIs and Ruby. It allows to write pure Ruby code to manage/monitor any Rails application. As a result it removes all dependency on the Java code used in TOTD #62.

Lets first install the gem!

~ >gem install jmx4r JRuby limited openssl loaded. gem install jruby-openssl for full support. http://wiki.jruby.org/wiki/JRuby_Builtin_OpenSSL Successfully installed jmx4r-0.0.6 1 gem installed Installing ri documentation for jmx4r-0.0.6... Installing RDoc documentation for jmx4r-0.0.6...

Here is the equivalent Ruby code to flush the class cache (as in TOTD #62):

require 'rubygems' require 'jmx4r' JMX::MBean.establish_connection :host => "129.145.133.163", :port => 8686 beans = JMX::MBean.find_all_by_name "org.jruby:*" beans.each { |bean|   service = bean.object_name["service"]   case service   when "ClassCache"     @cc_mbean = bean   when "Config"     @c_mbean = bean   end } # clear the cache if it's full if @cc_mbean.full   printf "JIT Max: %d, JIT Threshold: %d, Class load count: %d\n", @c_mbean.jit_max,     @c_mbean.jit_threshold, @cc_mbean.class_load_count     @cc_mbean.flush else   puts "Class Cache is not full"   printf "Loaded: %d, Reused: %d, Live: %d\n", @cc_mbean.class_load_count,     @cc_mbean.class_reuse_count, @cc_mbean.live_class_count end

As with all Ruby code, really clean and simple!

The key parts of the code are highlighted in bold. "JMX::MBean.establish_connection" establishes a connection with a JMX agent running on a remote machine identified by the IP address and port number specified. "JMX::MBean.find_all_by_name" queries the agent for all the MBeans in "org.jruby" domain. And then the code is self explanatory.

Just dump this code in a file and run it as:

jruby main.rb

to see the output similar to:

Class Cache is not full Loaded: 76, Reused: 0, Live: 76

Make sure Rails/Merb application is running after setting JAVA_OPTS as described in TOTD #62. For this blog, the JMX agent/Rails application ran on a Mac and the JMX client on a Solaris box.

The jmxr examples provides some more ways to use the gem.

There is also jmx gem that provides similar functionality. It even allows to create MBeans and provides a simple server where they can be registered. Tom's blog provide more details on usage and there is even a sample included.

Subsequent blogs in this series will discuss:

• How to remotely manage your Rails/Merb applications using JMX API ?
• How to publish your own metrics that can be managed using JMX API ?
• How to use jmx gem to manage/monitor ?
• How to use VisualVM to get more information about the underlying VM ?
• How to use NewRelic/FiveRuns to manage/monitor an application ?

Please leave suggestions on other TOTD (Tip Of The Day) that you'd like to see. A complete archive of all tips is available here.

Technorati: totd glassfish jruby rubyonrails merb jmx jmx4r manage monitor

Posted by Arun Gupta in web2.0  |  Comments[4]

| | | | | |
|

TOTD #62: How to remotely manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX API ?

TOTD #61 showed how to manage/monitor a Rails/Merb application running using JRuby/GlassFish using standard Java tools. Both the application and the management tools are running on the same machine. But that's never the case in a production environment. Instead remote management of an application is required when running in production mode. Fortunately, similar set of tools can be used for remote management as well.

However the underlying Java Virtual Machine needs to be configured to enable remote monitoring. This is achieved by setting JAVA_OPTS environment variables as shown below:

Set up the following envinronment variable before running the Rails/Merb application:

export JAVA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8686 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"

The first property enables the JMX remote agent (aka MBeans server) and local monitoring. The second property configures the port number for remote management using JMX/RMI connections. The third and fourth properties, for simplicity only, disable access control and SSL for remote monitoring. It is highly recommended to use them in a production environment. Using Password Authentication provide more details about configuring passwords for access control. Using SSL provide more details about configuring SSL.

Now for a Rails/Merb application running on GlassFish all the MBeans are also available on the JMX agent running on port 8686. This agent can be accessed using the standard URL as shown below:

service:jmx:rmi:///jndi/rmi://192.168.124.1:8686/jmxrmi

"192.168.124.1" is the host's IP address on which the Rails/Merb application is running and "8686" is the port configured in JAVA_OPTS. More details about the URL are available here.

First, lets use jconsole on a different machine to monitor this application. Start jconsole by typing the command in a shell as:

jconsole

In the "New Connection" window, select "Remote Process" and enter the URL defined above as shown below:



And then the management data snapshot is captured as shown below:



And here is a snapshot from a remote Solaris machine:



Notice the URL, as opposed to "pid", is shown on the jconsole window. This indicates that remote connection is being used instead of local process id. TOTD #61 further explains how jconsole can be used to manage/monitor your Rails/Merb applications.

As mentioned in TOTD #61, JRuby runtime exposes 4 different MBeans - ClassCache, Config, JITCompiler and ParserStats - all in "org.jruby" domain. Lets write a client application that query these MBeans to monitor the state of JRuby runtime and perform some management actions based upon the returned results.

Here is the Ruby client code using Java Management Extensions (JMX) APIs. 

include Java import java.io.IOException import java.util.HashSet import java.util.Set import javax.management.JMRuntimeException import javax.management.MBeanServerConnection import javax.management.MalformedObjectNameException import javax.management.ObjectName import javax.management.JMX import javax.management.remote.JMXConnector import javax.management.remote.JMXConnectorFactory import javax.management.remote.JMXServiceURL import org.jruby.management.ClassCacheMBean import org.jruby.management.ConfigMBean url = JMXServiceURL.new "service:jmx:rmi:///jndi/rmi://192.168.124.1:8686/jmxrmi" mbsc = JMXConnectorFactory.connect(url, nil).get_mbean_server_connection names = HashSet.new(mbsc.query_names(ObjectName.new("org.jruby:*"), nil)) names.each { |name|      service = name.get_key_property "service"      if service == "ClassCache"            @cc_mbean = JMX.new_mbean_proxy(mbsc, name, Java::org.jruby.management.ClassCacheMBean.java_class)      elsif service == "Config"            @c_mbean = JMX.newMBeanProxy(mbsc, name, Java::org.jruby.management.ConfigMBean.java_class)      end } # clear the cache if it's full if @cc_mbean.is_full     puts @c_mbean.get_jit_max     puts @c_mbean.get_jit_threshold     puts @cc_mbean.get_class_load_count     @cc_mbean.flush else     puts "Class Cache is not full"     printf "Loaded: %d, Reused: %d, Live: %d\n", @cc_mbean.get_class_load_count,       @cc_mbean.get_class_reuse_count, @cc_mbean.get_live_class_count end

Here are the main tasks achieved by this code:

• Do include/import for Java classes
• Connect with the JMX Agent where JRuby MBeans are registered and obtain a MBean Server Connection (explained here)
• Query for all the MBeans in "org.jruby" domain
• Create proxies for ClassCache and Config MBeans
• Flush the ClassCache if its full and print some debugging information

These MBeans are part of JRuby runtime so provide relevant management/montoring information corresponding to that. But as you can see it's fairly trivial to write a client to query MBeans and achieve dynamism using standard Ruby code.

Each JRuby-exposed MBean has multiple attributes that can be utilized to monitor your application and take appropriate actions. For example, an application could detect low memory and stop accepting new connections.

And you can continue to use GlassFish for running your Rails and Merb applications (now Sinatra as well)!

Subsequent blogs in this series will discuss:

• How to remotely manage your Rails/Merb applications using JMX API ?
• How to publish your own metrics that can be managed using JMX API ?
• How to use jmx gem to manage/monitor ?
• How to use VisualVM to get more information about the underlying VM ?
• How to use NewRelic/FiveRuns to manage/monitor an application ?

Please leave suggestions on other TOTD (Tip Of The Day) that you'd like to see. A complete archive of all tips is available here.

Technorati: totd glassfish jruby rubyonrails merb jmx manage monitor

Posted by Arun Gupta in web2.0  |  Comments[3]

| | | | | |
|

TOTD #61: How to locally manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX ?

GlassFish Gem can easily run both Rails and Merb application. Using JRuby, as opposed to MRI, allows you to use standard Java monitoring framework and tools. Java Management Extensions (JMX) is a standard management and monitoring solution for Java platform. This Tip Of The Day shows how to apply JMX techniques to monitor Rails/Merb applications.

The blog uses a trivial Rails/Merb application to demonstrate monitoring but the techniques can be applied to any Rails or Merb application. FYI TOTD #52 shows to create a simple Merb application and TOTD #28 shows how to create a simple Rails scaffold.

This blog will focus on local monitoring, i.e. monitor a Rails/Merb application running on the same machine as monitoring tool. A subsequent blog will demonstrate remote monitoring, i.e. monitor a Rails/Merb application from a different machine.

There is no extra configuration required to enable monitoring if your Rails/Merb application is running on GlassFish gem. You just need to launch an additional tool to monitor the already running application.

jconsole is a CLI bundled with JDK and launches a graphical tool to monitor and manage Java applications on a local or remote machine. jconsole can be used to monitor local application (those running on the same systems as jconsole) and remote applications (systems other than running jconsole).

Invoke "jconsole" in a shell as:

jconsole

It shows the following window:



The window shows different Java process that can be managed/monitored by jconsole. "org.jruby.Main -S glassfish" is the process corresponding to our Rails/Merb application running using GlassFish Gem. Click on that process (6179 in this case) and click "Connect".

The jconsole interface explains the different tabs and meaning of information displayed on each tab. Here is a snapshot of different tabs captured for a trivial Rails application.



It shows heap memory usage, threads, classes loaded, and CPU usage over a period of time. The "Memory" tab tracks different GC pools and shows their dynamic growth. Clicking on "Perform GC" on top-right corner performs a garbage collection (more on this later).



The "Threads" tab shows the highest number of threads since JVM started and current number of live daemon and non-daemon threads. Clicking on a particular thread shows it's current state and stack trace if it's waiting.



MBeans tab shows different managed beans registered with the platform MBean server, i.e. MBean server available in the JDK. As you can see there are several standard MBeans displayed on the left-hand side from java.lang and java.util.logging domains.



JRuby runtime also exposes 4 different MBeans - ClassCache, Config, JITCompiler and ParserStats - all in "org.jruby" domain. Clicking on each MBean shows detailed information collected by that MBean as shown below:



Clicking on "Attributes" node shows the current snapshot of all the attributes exposed by the MBean as shown below:



Clicking on "Refresh" button will read the fresh values from the MBean.

Some MBeans expose functions as well. Such MBeans have "Operations" node and clicking on them show the functions exposed as shown below:



These functions can be invoked by clicking on the button in right-hand panel.

These snapshots are captured without even a single invocation of the application. Accessing "http://localhost:3000/" in a browser updates heap usage screenshot as shown below:



And permgen is increased because new classes are JITed by JRuby as shown below:




After a Garbage Collection is performed (by clicking on "Perform GC" button in "Memory" tab), the updated screenshot looks like:



As the snapshot indicates the "Heap Memory Usage" is back to normal (which indicates no memory leak) and there is a spike in "CPU Usage" because CPU cycles are used to perform GC.

Monitoring a Merb application is no different at all. The "Overview" tab for a trivial Merb app looks like:



The wide range of JMX-compatible tools is another advantage of using JRuby over MRI :)

Some other alternatives are using a combination of jps and jstat.

The JMX Home Page provides all details about JMX framework.  Subsequent blogs in this series will discuss:

• How to remotely manage your Rails/Merb applications using JMX API ?
• How to publish your own metrics that can be managed using JMX API ?
• How to use VisualVM to get more information about the underlying VM ?
• How to use NewRelic/FiveRuns to manage/monitor an application ?

How do you monitor your Rails or Merb applications ?

Please leave suggestions on other TOTD (Tip Of The Day) that you'd like to see. A complete archive of all tips is available here.

Technorati: totd glassfish jruby rubyonrails merb jmx manage monitor

Posted by Arun Gupta in web2.0  |  Comments[5]

| | | | | |
|