Binu John's Weblog
Monitoring GlassFish Using BTrace
In one of my previous entries, GlassFish Tuning - HTTP Thread Pool, I discussed how to use the GlassFish monitoring framework to capture various server runtime statistics. This method required enabling the GlassFish monitoring framework.
There is an easier way to collect some of the same stats - using BTrace, the dynamic tracing utility in Java. Since BTrace can be attached dynamically, it can be used to monitor any JVM (needs JDK 6, I use 1.6.0_11), even a production instance where monitoring is not enabled. BTrace places restrictions on what code can be executed via the scripts (this is for your own good). Refer to the BTrace user guide for a complete list of restrictions. Due to these restrictions, not all the stats that can be displayed using the monitoring framework can be collected using BTrace. So, don't throw away the entire monitoring framework in GlassFish just yet.
Over the next few blogs, I'll provide some of the scripts that I use for my performance testing.
Let me start with the monitoring of the number of busy threads in the HTTP worker thread pool. Since the script instruments a specific class, this script works only for V2. The modification required for V3 is provided after the script listing.
| package com.sun.btrace.samples; import static com.sun.btrace.BTraceUtils.$; import static com.sun.btrace.BTraceUtils.print; import static com.sun.btrace.BTraceUtils.println; import com.sun.btrace.BTraceUtils; import com.sun.btrace.annotations.BTrace; import com.sun.btrace.annotations.Kind; import com.sun.btrace.annotations.Location; import com.sun.btrace.annotations.OnEvent; import com.sun.btrace.annotations.OnMethod; import com.sun.btrace.annotations.OnTimer; import java.util.concurrent.atomic.AtomicInteger; import static com.sun.btrace.BTraceUtils.newAtomicInteger; import static com.sun.btrace.BTraceUtils.str; import static com.sun.btrace.BTraceUtils.strcat; import static com.sun.btrace.BTraceUtils.addAndGet; import static com.sun.btrace.BTraceUtils.decrementAndGet; import static com.sun.btrace.BTraceUtils.get; /** * BTrace script to print the number of busy threads in the http worker thread pool. The output is written every 3 seconds. Change the value of OnTime annotation to change the frequency of printing the stats. * * @author Binu John */ @BTrace public class GFV2ThreadBusyStats { private static AtomicInteger ai = newAtomicInteger(0); @OnEvent() public static void onEvent() { println(strcat("Busy Threads = ", str(get(ai)))); } @OnMethod(clazz = "com.sun.enterprise.web.connector.grizzly.TaskBase", method = "run") public static void onGet() { addAndGet(ai, 1); } @OnMethod(clazz = "com.sun.enterprise.web.connector.grizzly.TaskBase", method = "run", location = @Location(Kind.RETURN)) public static void onReturn() { decrementAndGet(ai); } @OnTimer(3000) public static void dumpStats() { println(strcat("Busy Threads = ", str(get(ai)))); } } |
The script can be attached to a running application server instance as follows -
BTRACE_HOME/bin/btrace -cp GF_HOME/lib/appserv-rt.jar <pid> GFV2ThreadBusyStats.java
where BTRACE_HOME is the location of the Btrace installation, GF_HOME is the location of the GlassFish installation and pid is the process id of the server instance.
Example:
# /home/binu/Utils/btrace/bin/btrace -cp /space/binu/glassfishv2.1_b60b/glassfish/lib/appserv-rt.jar 10997 /home/binu/Utils/btrace/samples/GFV2ThreadBusyStats.java
Busy Threads = 20
The stats are printed out every 3 seconds. You can change the interval by modifying the value of the OnTimer annotation.
As mentioned earlier, the script needs to be modified for V3 - modify the clazz parameter (two instances) to com.sun.grizzly.http.ProcessorTask and the method parameter to doProcess.
Refer back to the earlier entry on how to tune the app server based on this data.
Posted at 08:20AM Apr 10, 2009 by binu in GlassFish | Comments[3]
what about using groovy for the scripts? - eduard/o
Posted by Eduardo Pelegri-Llopart on April 10, 2009 at 08:40 AM PDT #
BTrace places several restrictions on what can be executed in the script (BTrace uses the term program rather than script). Here is a description of a BTrace program (taken from the user guide) -
A BTrace program is a plain Java class that has one or more public static void methods that are annotated with BTrace annotations. The annotations are used to specify traced program "locations" (also known as "probe points").
So the BTrace scripts need to be written in Java. Additonally, only the public static methods of com.sun.btrace.BTraceUtils class may be called from a BTrace program. It does not even allow methods from JDK classes to be executed.
Posted by Binu John on April 10, 2009 at 01:58 PM PDT #
I guess I need to understand more about what is the associated run-time needed to execute groovy. I thought it would just compile to bytecodes and require no extra runtime. - eduard/o
Posted by Eduardo Pelegri-Llopart on April 10, 2009 at 02:39 PM PDT #