Watching A Directory for Changes — File Change Notification in NIO.2

Here is a sneak peek of the NIO.2 file coverage I am writing for the Java Tutorial. This will be available in the web push of the tutorial that coincides with the upcoming J1 Preview Release. (NIO.2 is part of the JDK 7 release.)

-- Sharon Zakhour

---

Have you ever found yourself editing a file, using an IDE or another editor, and a dialog pops up to inform you that one of the open files has changed on the file system and needs to be reloaded? Or perhaps, like the NetBeans IDE, it just quietly updates the file without making a fuss. The following sample dialog shows how this looks when using the free editor, jEdit:

jEdit Dialog — Modified File is Detected

To implement this functionality, called file change notification, a program needs to be able to detect what is happening to the relevant directory on the file system. One way to do this is to poll the file system looking for changes, but this approach is inefficient — it does not scale to applications that may have hundreds of open files or directories to monitor.

The java.nio.file package provides a file change notification API, called the Watch Service API. This API allows you to register a directory (or directories) with the watch service — when registering you tell the service which types of events you are interested in: file creation, file deletion, or file modification. When the service detects an event of interest, it is forwarded to the registered process. The registered process has a thread (or a pool of threads) dedicated to watching for any events it has registered for. When one comes in, it is handled as needed.

This section covers:

• WatchService Overview
• Try it Out
• Creating a Watch Service and Registering for Events
• Processing Events
• Getting the Filename
• When Not to Use this API

WatchService Overview

The WatchService API is fairly low-level, allowing you to customize it — you can use it as-is, or you may choose to create a high-level API on top of this mechanism suited to your particular needs.

Here are the basic steps required to implement a watch service:

• Create a WatchService "watcher" for the file system.
• For each directory that you want monitored, register it with the watcher. When registering a directory, you specify the type of events for which you want notification. You receive a WatchKey instance for each directory that you register.
• Implement an infinite loop to wait for incoming events.
• When an event occurs, the key is signaled and placed into the watcher's queue.
• The key is retrieved from the watcher's queue. You can obtain the file name from the key.
• Retrieve each pending event for the key (there may be multiple events) and process as needed.
• Reset the key and resume waiting for events.
• The watch service exits when either the thread exits or when each directory being watched becomes inaccessible.

WatchKeys are thread safe and can be used with the java.nio.concurrent package. You can dedicate a thread pool to this effort.

Try it Out

Because this is a more advanced API, we encourage you to try it out before proceeding. Save the WatchDir example to your computer and compile it. Create a test directory — this is passed to the example. WatchDir uses a single thread to process all events, so it blocks while waiting for events — either run it in a separate window or in the background, like this:

java WatchDir test &

Play with creating, deleting and editing files in the test directory. When any of these events occurs, a message is printed to the console. When you have finished, delete the test directory and WatchDir exits. Or, if you prefer, you can manually kill the process.

You can also watch an entire file tree by specifying the -r option. When you specify -r, WatchDir walks the file tree, registering each directory with the watch service.

Creating a Watch Service and Registering for Events

The first step is to create a new WatchService, using the newWatchService method in the FileSystem class:

WatchService watcher = FileSystems.getDefault().newWatchService();

Next, register one or more objects with the watch service. Any object that implements the Watchable interface can be registered. The Path class implements the Watchable interface, so each directory to be monitored is registered as a Path object.

As with any Watchable, the Path class implements two register methods. This section uses the two-argument version, register(WatchService, WatchEvent.Kind<?>...). (The three-argument version takes a WatchEvent.Modifier which is not currently implemented.)

When registering an object with the watch service, you specify the types of events you want to monitor. The supported StandardWatchEventKind event types are:

• ENTRY_CREATE — a directory entry is created.
• ENTRY_DELETE — a directory entry is deleted.
• ENTRY_MODIFY — a directory entry is modified;
• OVERFLOW — indicates that events may have been lost or discarded. You do not have to register for this event to receive it.

The following code snippet shows how to register a Path instance for all three event types:

import static java.nio.file.StandardWatchEventKind.*;

Path dir = ...;
try {
    WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
} catch (IOException x) {
    System.err.println(x);
}

Processing Events

The order of events in an event processing loop are:

1. Get a watch key. There are three methods provided:
   • poll — returns a queued key, if any is available. Returns immediately, with a null value, if none is present.
   • poll(long, TimeUnit) — returns a queued key, if one is available. If one is not immediately available it waits up until the specified time. The TimeUnit argument determines whether the specified time is nanoseconds, milliseconds, or some other unit of time.
   • take — returns a queued key. If none is available, this method waits.
2. Process the pending events for the key. You fetch the List of WatchEvents from the pollEvents method.
3. Retrieve the type of event using the kind method. No matter what events the key has registered for, it is possible to receive an OVERFLOW event. You may choose to handle the overflow or ignore it, but you should test for it.
4. Retrieve the file name associated with the event. The file name is the context of the event, so the context method is used to retrieve it.
5. After the events for the key have been processed, you need to put it back into a ready state by invoking reset. If this method returns false, the key is no longer valid and the loop can exit. This step is very important — if you fail to invoke reset, this key will not receive any further events.

A watch key has a state — at any given time, its state might be:

• Ready indicates that the key is ready to accept events. When first created, a key is in the ready state.
• Signaled indicates that one or more events are queued. Once the key has been signaled, it is no longer in the ready state until the reset method is invoked.
• Invalid — indicates that the key is no longer active. This happens when one of the following events occurs:
  • The process explicitly cancels the key using the cancel method.
  • The directory becomes inaccessible.
  • The watch service is closed.

Here is an example of an event processing loop. It is lifted from the Email example which watches a directory, waiting for new files to appear. When a new file becomes avaiable, it is examined to see if it is a text/plain file using the probeContentType method. The intention is that plain text files will be emailed to an alias, but that implementation detail is left to the reader.

The methods specific to the watch service API have been bolded:

for (;;) {

    //wait for key to be signaled
    WatchKey key;
    try {
        key = watcher.take();
    } catch (InterruptedException x) {
        return;
    }

    for (WatchEvent<?> event: key.pollEvents()) {
        WatchEvent.Kind<?> kind = event.kind();

        //This key is registered only for ENTRY_CREATE events,
        //but an OVERFLOW event can occur regardless if events are
        //lost or discarded.
        if (kind == OVERFLOW) {
            continue;
        }

        //The filename is the context of the event.
        WatchEvent<Path> ev = (WatchEvent<Path>)event;
        Path filename = ev.context();

        //Verify that the new file is a text file.
        try {
            //Resolve the filename against the directory.
            //If the filename is "test" and the directory is "foo",
            //the resolved name is "test/foo".
            Path child = dir.resolve(filename);
            if (!Files.probeContentType(child).equals("text/plain")) {
                System.err.format("New file '%s' is not a plain text file.%n", filename);
                continue;
            }
        } catch (IOException x) {
            System.err.println(x);
            continue;
        }

        //Email the file to the specified email alias.
        System.out.format("Emailing file %s%n", filename);
        //Details left to reader....
    }

    //Reset the key -- this step is critical if you want to receive
    //further watch events. If the key is no longer valid, the directory
    //is inaccessible so exit the loop.
    boolean valid = key.reset();
    if (!valid) {
        break;
    }
}

Getting the Filename

The file name is retrieved from the event context. The Email example retrieves the file name with this code:

WatchEvent<Path> ev = (WatchEvent<Path>)event;
Path filename = ev.context();

When you compile the Email example, it generates the following error:

Note: Email.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

This is a result of the line of code that casts the WatchEvent<T> to a WatchEvent<Path>. The WatchDir example avoids this error by creating a utility cast method that suppresses the unchecked warning:

@SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
    return (WatchEvent<Path>)event;
}

If you are unfamiliar with the @SuppressWarnings syntax, see the Annotations section.

When Not to Use this API

The Watch Service API is designed for applications that need to be notified about file change events. It is well suited for any application, like an editor or IDE, that potentially has many open files and needs to ensure that the files are in sync with the file system. It is also well suited for the app server that watches a directory, perhaps waiting for .jsp or .jar files to drop, in order to deploy them.

This API is not designed for indexing a hard drive. Most file system implementations have native support for file change notification — the Watch Service API takes advantage of this where available. But when a file system does not support this mechanism, the Watch Service will poll the file system, waiting for events.

Posted at 07:22PM May 13, 2009 by The Java Tutorial Team  |  Comments[12]