Weblog

All | Archive | General | Java
20070428 Saturday April 28, 2007

Multicasting with NIO

A long standing issue for many developers is that the java.nio.channels package has lacked support for Internet Protocol (IP) multicasting. In the NIO.2 early review draft specification you will see that we have addressed this issue by adding multicast support to DatagramChannel. Here's a small example that opens a DatagramChannel, binds the channel's socket to a local port, sets the network interface for multicast datagrams sent via the channel, and then joins a multicast group on the same interface: 

NetworkInterface interf = NetworkInterface.getByName("eth0");
InetAddress group = InetAddress.getByName("225.0.0.100");

DatagramChannel dc = DatagramChannel.open(ProtocolFamily.INET)
    .setOption(SocketOption.SO_REUSEADDR, true)
    .bind(new InetSocketAddress(5000))
    .setOption(SocketOption.IP_MULTICAST_IF, interf);

MembershipKey key = dc.join(group, interf);

Once the channel has joined the group then it reads or receives multicast datagrams in the same manner that it reads or receives unicast datagrams.

The most significant thing in this example is the static factory method to open the channel specifies a protocol family. In this example, the channel is to an IPv4 socket. In the existing APIs, the protocol family is transparent and all sockets created by the java.net or java.nio.channels packages are either all IPv4 or all IPv6. For IP multicasting it is important that the protocol family corresponds to the address type of the multicast groups that the socket joins; otherwise it is highly operating system specific if the socket can join the group, configure options, or receive multicast datagrams. Legacy java.net.MulticastSocket has suffered greatly from problems in this area. Another interesting thing to point out is that the channel's socket is bound and socket options are configured directly. The awkward and counterintuitive socket adaptor isn't required so it isn't necessary to mix java.net socket APIs when configuring the channel's socket.

In the example, the MembershipKey that is returned by the join method is a token to represent membership of the group. It defines methods to query information about the membership and defines the "drop" method to drop membership of the group.

Developers tracking multicast standards will know that the RFCs in this area have been updated in recent years to add source filtering and this is now supported by almost all modern operating systems. In the NIO.2 draft specification we have included basic support for source filtering. The following code fragment shows a channel joining a multicast group to only receive multicast datagrams sent by a specific IP source address (otherwise known as "include-mode" filtering): 

MembershipKey key = dc.join(group, interf, source);

"Exclusive-mode" filtering is where a group is joined to receive all multicast datagrams except those from specific IP source address: 

MembershipKey key = dc.join(group, interf).block(source1).block(source2);

So that's a brief introduction to the multicast support that we propose to add to the java.nio.channels package. There's a lot more detail in the draft specification for those interested in this topic. ( Apr 28 2007, 01:46:02 PM PDT ) Permalink Comments [6]

20070427 Friday April 27, 2007

Accessing file permissions

One of the many issues addressed by NIO.2 is that the Java SE platform doesn't have a complete set of APIs to access file permissions and other security related file attributes. It's not rare to encounter code that has to resort to using Runtime.exec to execute a command such as chmod(1).

In the NIO.2 early review draft specification you will see that we address the issue of file attributes by organizing related attributes into groups. A FileAttributeView is then defined to provide a read-only or updatable view of the attributes in the group. The specification defines a number of attribute views, of which only BasicFileAttributeView is required to be supported by a file system implementation (BasicFileAttributeView provides access to a small set of attributes such as file size and last modified time - essentially the basic attributes that are common to many file systems).

Aside from BasicFileAttributeView, a file system implementation can support other file attribute views. The specification defines PosixFileAttributeView for access to file attribtues commonly found on platforms that implement the POSIX family of standards. Here's a small example that uses an instance of this attribute view to print the owners and permissions of a file called "foo". It then changes the permissions to deny others access to the file: 

PosixFileAttributeView view = PathReference.from("foo")
    .newFileAttributeView(PosixFileAttributeView.class);

// bulk read
PosixFileAttributes attrs = view.readAttributes();

int perms = attrs.getPermissions();

// prints "rw-r--r-- alice bandits"
System.out.format("%s\t%s\t%s%n", 
    PosixFilePermission.toString(perms),
    attrs.getOwner(),
    attrs.getGroup());    

// deny others
perms &= ~OTHERS_READ & ~OTHERS_WRITE & ~OTHERS_EXECUTE;
view.updatePermissions(perms);

The only code that might need explanation here is the code that obtains an instance of PosixFileAttributeView. FileAttributeViews are selected by type-token and this code selects an instance of PosixFileAttributeView that is bound to the file "foo".

The comment hints that foo is owned by the "bandits" group and we can fix this by changing the group: 

UserPrincipal cops = view.lookupPrincipalByGroupName("cops");
view.updateOwners(null, cops);

The other part to file permissions is that sometimes you need to create a file with initial permissions (umask applies of course). Here's an example that creates a new file called "securefile" with initial permissions, opening the file for random access: 

PathReference file = PathReference.from("securefile");
PosixFileAttributeView view = ...

Attribute<Integer> attr = view.newPermissionsAttribute()
    .setValue(OWNER_READ | OWNER_WRITE | GROUP_READ | GROUP_WRITE);

SeekableByteChannel sbc = file.newSeekableByteChannel(CREATE_NEW | READWRITE, attr);

So that's a brief tour of PosixFileAttributeView. The other file attribute view in the specification that provides access to security related attributes is AclFileAttributView. This provides access to Access Control Lists (ACLs) based on the NFSv4 ACL model. That will be interesting to those on file systems that support ACLs. When I get time I'll write up a few words on how this attribute view is used. ( Apr 27 2007, 07:52:04 AM PDT ) Permalink Comments [2]

20070412 Thursday April 12, 2007

NIO.2: The Return From The Lost Planet

We (the JSR-203 expert group) recently submitted the Early Draft Review (EDR) specification for JSR-203. Otherwise known as NIO.2, this JSR is tasked with developing the second phase of the New I/O APIs for the Java SE platform. Specifically, it is tasked with developing a new file system interface, asynchronous I/O APIs, and to complete the socket-channel API carried over from the first NIO JSR (JSR-51). The plan is to put JSR-203 forward as a candidate component JSR to include in Java SE 7.

The draft specification that we have submitted can be downloaded from: http://jcp.org/aboutJava/communityprocess/edr/jsr203/index.html. We've setup a mailing list to discuss this JSR and anyone can subscribe by going to: http://groups.google.com/group/jsr203-interest/subscribe. Comments can also be sent to: jsr-203-comments@jcp.org. The most important goal with releasing this draft is to get feedback that we are addressing the right problems. In the file system area at least, this JSR is about solving some basic problems and adding support for functionality that is sadly missing from the platform today. A second goal is to poke holes in the API. As I'm sure all developers agree, API design benefits greatly from feedback and it can often take several iterations to get an API right.

For those interested in the topics in this JSR I have uploaded a few slides to get you started. In coming weeks I'll follow up on specific topics covered in this JSR. ( Apr 12 2007, 04:06:26 PM PDT ) Permalink Comments [3]

20060526 Friday May 26, 2006

HeapDumpOnOutOfMemoryError option in 5.0u7 and 1.4.2_12.

5.0 Update 7 was released this week. Among the changes is the backport of the HeapDumpOnOutOfMemoryError option from Mustang. This VM option tells the HotSpot VM to generate a heap dump when OutOfMemoryError is thrown because the java heap or the permanent generation is full. A heap dump is useful in production systems where you need to diagnose an unexpected failure. Lots of developers sent mail asking to make this option available in the shipping releases. For 1.4.2 it will be available shortly in 1.4.2_12.

So how do you analyze these dumps? The legacy Heap Analysis Tool (HAT) is one option but it's old, provides only limited queries, and it can't import heap dumps generated by 64-bit VMs. A better choice is jhat which fixes many issues with HAT, provides new queries, and most important, it provides a query interface called Object Query Language to write your own queries. Sundar has some great blog entries to introduce OQL and get you started:

The jhat utility is included in Mustang (which you can download from the binaries snapshot site). It will happily munch on dumps produced by 5.0u7 and 1.4.2_12.

Another tool to analyze the heap dump is the YourKit Java Profiler. Anton Katilin and his colleagues in YourKit recently released version 5.5 of the product and this includes an "Import HPROF Snapshot" option to import the heap dump and convert it to the format used by the profiler.

Two other useful things to know are: (i) the heap dumps are platform independent and so you don't need to analyze the dump on the same system that produced it, and (ii) running with -XX:+HeapDumpOnOutOfMemoryError does not impact performance - it is simply a flag to indicate that a heap dump should be generated when the first thread throws OutOfMemoryError.

[Update 08/04/06: In the original entry I mentioned a potential issue with the low-pause collector. That turns out not to be a concern.] ( May 26 2006, 04:26:23 AM PDT ) Permalink Comments [18]

20060104 Wednesday January 04, 2006

Another piece of the tool puzzle. One of the updates in Mustang b65 was the addition of the getAgentProperties method to the Attach API. This method gives tools access to a set of properties maintained in the Java virtual machine on behalf of agents. This may sound obscure but it gets interesting when you know that the JMX agent will create an agent property when it starts a local JMX connector server. A local JMX connector server is started when you start an application with -Dcom.sun.management.jmxremote or you start the management agent in a running application using the Attach API.

MemViewer.java is a simple "one page" example to demonstrate how a tool might use this method. MemViewer takes one argument to identify the target application and for this you use the process-id (or pid). MemViewer attempts to attach to the specified application and looks for the agent property named "com.sun.management.jmxremote.localConnectorAddress". If this property is set then it is the address of the local JMX connector server. If the property is not set then it means a local JMX agent is not running so MemViewer starts the management agent by loading a java agent named "management-agent.jar" into the target application. Once started, it re-obtains the agent properties to get the value of the agent property. Once MemViewer has the address of the connector server then it connects and prints information about each of the memory pools. Here is an example where MemViewer is used to print information about the memory used by a Java Web Start application. 

C:\> jps -l
2304 sun.tools.jps.Jps
2560 com.sun.javaws.Main

C:\> java -classpath .;%JDK_HOME%\lib\tools.jar MemViewer 2560
                Pool         Used    Committed          Max
Perm Gen [shared-ro]      5279360      8388608      8388608
          Code Cache      4901696      4980736     33554432
          Eden Space       313968      1638400      4194304
Perm Gen [shared-rw]      7021848     12582912     12582912
         Tenured Gen     14466928     23662592     61997056
            Perm Gen     10631352     12582912     67108864
      Survivor Space       196608       196608       458752

If you examine MemViewer then you will see that the code is relatively simple. VirtualMachine.attach is used to attach to the target Java virtual machine. Once attached the getAgentProperties method is used to read the agent properties. If the management agent is not running then it constructs the name of the management agent and then uses the loadAgent method to load it into the target. The Attach API is a simple tool API for bootstraping agents into a running application and ships in the JDK in tools.jar. This means that tools.jar must be on the classpath when compiling or running. The JAR file "management-agent.jar" is the java agent with the Agent-Class attribute in the main manifest to specify the class name of the agent. The agent property com.sun.management.jmxremote.localConnectorAddress is set by the agent when the local JMX connector has started. This property will be documented along with the other properties on the Monitoring and Management with JMX page.

Finally, in the example output you will see that I used jps utility to get a list of the Java virtual machines that I was running. An alternative approach would have been to use the VirtualMachine.list method and prompt the user to select from a list. ( Jan 04 2006, 08:14:04 AM PST ) Permalink Comments [3]

20051213 Tuesday December 13, 2005

Late binding agents and fun times for the tool maker! A few months ago, I highlighted an update to jconsole that allows it connect to applications that weren't started with special command line options. This ability to connect to a running application isn't unique to jconsole. When the NetBeans Profiler first appeared (as JFluid) it included a similar feature that allowed it to attach, apply instrumentation, and profile an application without needing to restart it with the profiler agent. At the time JFluid ran on a customized release of J2SE 1.4.x. The underlying implementation was very different, but the general idea is the same. There are many improvements in Mustang that mean that this ability to attach, start monitoring agents or instrumentation-based agents, is available to anyone developing tools. Today, I thought I might write up a few words on this topic.

First, some background. When I use the term "tool agent" I'm talking about the piece of the tool that runs in-process in the same VM as the application. Typically, tools are architected so that there is a "front-end" that the user interacts with, and a "back-end" that runs in the VM with the application. We see this general architecture with many of the profiling tools today. You'll also see essentially the same thing with the debugger architecture, or with JMX tools that connect to the JMX agent running in the target application. Agents doing exact and sample based profiling, space profiling, coverage analysis and so on are often developed as native agents (in C/C++) and make use of the JVM Tool Interface. Agents doing fine-grain monitoring, execution time profiling, journaling, and other tasks that make use of bytecode instrumentation are developed in the Java Language and make use of the instrumentation support provided by java.lang.instrument.

So how does a tool load its agent into a running application? In Mustang we have included the Attach API. This is a very simple API to bootstrap agents into a running application. The API is a Sun-specific API and is included in the JDK for use by tools (it can of course attach to applications that are deployed with the JRE). The VirtualMachine class is used to attach to an application. Once attached, the loadAgent or loadAgentLibrary methods can be used to load and start an agent. There is also a list method which can be used to enumerate the user's Java virtual machines (useful if you want to prompt a user to select the application to be monitored or profiled).

So what can an agent do once it is loaded in a running application? In the case of native agents using the JVM Tool Interface then it depends on the capabilities that the VM is able to provide in the live phase. Many of the capabilities needed by a fully featured debugger for example, are only available in the onload phase because they require special initialization or code generation that is only done at startup. Profilers starting in a running application will typically require the capabilities that will allow it instrument classes that are already loaded, instrument new classes as they are loaded, and perhaps tag objects and classes. In Mustang, these capabilities are always available in the HotSpot Server VM. Most are available in the HotSpot Client VM too except that the ability to instrument classes that are loaded from the shared archive (a short term implementation detail that will get resovled in time). For Java Lanaguage agents doing instrumentation then the story is the same - they can add ClassFileTransformers to instrument classes that are subsequently loaded, or retransform and instrument already loaded classes. Another aspect to instrumentation is that agents will often need to instrument classes so that they invoke methods on supporting classes provided by the tool. When an agent is loaded into a running application these supporting classes may not be available but Mustang has methods to make the supporting classes available for the bootstrap class loader or the system class loader. Many Java Language agents (like the JMX agent) won't be concerned with byetcode instrumentation - instead they will (for example) make use of java.lang.management package so that the tool can obtain telemetry information and manage the VM.

So now that we know what the agent can do you might wonder about security. This is very important and we've taken the position that there shouldn't be any suprises. That means that a tool shouldn't be able to interact with an application that won't normally be allowed by the security on the operation system. For UNIX environments this means that the tool must have the same effective uid/gid as the target application. On Windows, the implementation can only be used to attach to applications that you have privilege to open and interact with. So if Alice is running a memory profiler then she will only be able to attach to her own applications. She won't be able to attach her profiler to Bob's application that is churning out the winning numbers for next week's lottery.

So that's a brief overview of the late-binding agent support in Mustang. As you can see, all the major pieces to allow tools attach, observe, and profile a running application are in place. Fun times for the tool maker. ( Dec 13 2005, 02:50:31 AM PST ) Permalink Comments [1]

20051128 Monday November 28, 2005

OutOfMemoryError looks a bit better! OutOfMemoryError has always been a confusing error. For a long time the HotSpot Virtual Machine threw this error without a detail message or stack trace so typically the thread throwing the error would terminate with this: 

  Exception in thread "main" java.lang.OutOfMemoryError

That was confusing. Is my java heap full or does it mean something else? Those familiar with the heap layout that the HotSpot VM uses will know that the "something else" might mean the "permanent generation". This is place where reflective data such as class and method objects are allocated. It is also the place where interned strings are stored. If you've got an application that loads a huge number of classes or interns millions of strings then it's possible that the OutOfMemoryError is because the permanent generation is full rather than the java heap.

In 5.0 the error is less confusing as there is a detail message. This means you will see something like this: 

  Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
or: 
  Exception in thread "main" java.lang.OutOfMemoryError: PermGen full

In Mustang, there has been further changes to the way that OutOfMemoryError is reported. One obvious one is that the HotSpot VM will attempt to include a stack trace. This means you should see something like this: 

  Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at ConsumeHeap$BigObject.(ConsumeHeap.java:22)
        at ConsumeHeap.main(ConsumeHeap.java:47)

In this example we see the stack trace where the allocation failed. If the OutOfMemoryError is because the perm gen it full then you might see String.intern or ClassLoader.defineClass near the top of the stack.

Is a stack trace useful? In some limited cases it can be. For example, suppose you've got a thread looping and in the loop it is allocating objects and putting them into a collection. In that case the stack trace might direct you to a good starting place. More generally, the stack trace is not likely to be useful. If you've got a busy application and the heap is nearly full, then OutOfMemoryError will be likely be thrown in some random place by some random mutator.

One useful improvement is the -XX:+HeapDumpOnOutOfMemoryError option which tells the HotSpot VM to generate a heap dump when an allocation from the java heap or the permanent generation cannot be satisfied. There isn't any overhead to running with this option so it should be useful for production systems where OutOfMemoryError takes weeks (or longer) to surface. The heap dump is in HPROF binary format so it can be analyzed using any tools that can import this format. If you read Sundar's blog then you'll know that Mustang includes a useful tool called jhat which can be used to do rudimentary analysis of the dump. jhat supports Object Query Language so you can easily create your own queries and mine the heap dump.

The next improvement is visible when the system is almost out of swap space and an allocation from the native heap fails in the VM. In that case the VM aborts and you get a one-line error such as the following: 

  Exception java.lang.OutOfMemoryError: requested 16 bytes for CHeapObj-new. Out of swap space?

This message has been known to confuse a lot of developers. At first glance it might it might look like that the java heap is full. Of course if you increase the size of the heap with the -mx option it might make the problem worse as the larger java heap means there is less native memory available.

In Mustang this condition will trigger the VM to invoke the fatal error handling mechanism. This means you should get a fatal error log as you would get with a normal (abnormal?) crash. The fatal error log is named hs_err_<pid>.log and contains useful information about the thread, process, and system at the time of the crash. In the case of native heap exhaustion then the heap memory and memory map information in the log can be useful. The exact format is version and platform specific and you will get more information in the J2SE Troubleshooting Guide.

Hopefully, developers will find these improvements useful. It should make OutOfMemoryError a little less confusing and it means the error is no longer a one-line wonder. ( Nov 28 2005, 06:05:37 AM PST ) Permalink Comments [16]

20051103 Thursday November 03, 2005

To poll or epoll: that is the question: One of the updates in build 59 of Mustang (JavaTM SE 6) is that the New I/O Selector implementation will use the epoll event notification facility when running on the Linux 2.6 kernel. The epoll event mechanism is much more scalable than the traditional poll when there are thousands of file descriptors in the interest set. The work done by poll depends on the size of the interest set whereas with epoll (like Solaris /dev/poll) the registration of interest is separated from the retrieval of the events. A lot has been written on the topic. The C10K problem has been documenting I/O frameworks and strategies for several years. One relatively recent paper on Comparing and Evaluating epoll, select, and poll Event Mechanisms makes it clear the workloads where epoll performs a lot better than poll.

This isn't the first NIO Selector implementation to use epoll. The Blackdown folks added epoll support in their 1.4.x release. On Solaris, the /dev/poll based Selector has been default on Solaris 8 (and newer) since the original implementation of New I/O in J2SETM 1.4.

So if you are running on a Linux 2.6 system with an application that handles lots of simultaneous connections you might want to give b59 a test-run. The weekly builds have been appearing like clockwork on the binary snapshot release site so b59 should be available tomorrow (November 4). Will you see a difference? It depends on the workload. If you've registered lots of SelectableChannels with a Selector and you notice a lot of time spent in the kernel due to poll then you should see a difference. If you are doing test runs and you want to do a direct comparison with poll then you can set the java.nio.channels.spi.SelectorProvider system property to sun.nio.ch.PollSelectorProvider. This will select the poll-based Selector that will continue to be the default on 2.4 kernels. There is an epoll patch for 2.4 kernels but at this time anyway, the NIO implementation doesn't attempt to detect this.

[Update 08/04/2006: The epoll SelectorProvider will be included in 5.0 update 9. To enable it requires setting the system property java.nio.channels.spi.SelectorProvider to the value sun.nio.ch.EPollSelectorProvider. Sorry, it won't be the default in 5.0 but we must be very careful when changing anything in an update release.] ( Nov 03 2005, 04:19:35 AM PST ) Permalink Comments [10]