Weblog Chris Hegarty : Weblog

All
Biking
Java
Sun

Monday Feb 06, 2006

Diagnosing DeleteOnExit Issues.

Based on a peabody contribution by Matthias File.deleteOnExit was re-implemented in mustang b63 and now uses java level shutdown hooks. Previously there was a native implementation that registered filenames to be deleted at exit in a native linked list type structure and which was called during the vm exit to do the cleanup. Diagnosing issues where excessive use of deleteOnExit was causing OutOfMemoryErrors was very difficult as the list was held in native heap.

The re-implementation of deleteOnExit using java level shutdown hooks has several benefits:

A file name is only added once to the list of files to be deleted, so if deleteOnExit is invoked multiple times on the same File then it will not consume any additional memory. This issue was tracked by 4813777 in Sun bug database.
The new implementation stores the file names in a HashSet in the java heap. This allows for better diagnosability of further issues with deleteOnExit (and I will describe how to do this later).

There is still an issue with some code that creates a lot of temporary files and calls deleteOnExit on them to ensure that they will be removed during shutdown. This is currently a limitation of the way deleteOnExit is specified as you need to ensure that a potentially different file with the same abstract path is removed even if you specifically call delete on the current one. You need to be careful when using deleteOnExit with temporary files in this manner. So what do you do if you think you are running into this issue, read on...

As I mentioned earlier, in the past it was difficult to figure out if deleteOnExit was causing your OutOfMemoryError's because it allocates in the native heap and not the java heap, and inspecting the native heap is much more difficult. Here are 2 solutions if you are running with mustang b63 (or later) and think you are experiencing issues cause by deleteOnExit.

Examine the heap dump
You can use jmap to dump the java heap of a running process or add -XX:+HeapDumpOnOutOfMemoryError option to your startup arguments. You can see how to do this in Alans blog. Once you have your dump, run jhat on it. If you are not familiar with jhat then look at Sundar's blog. JHat will read the dump file and start a HTTP server. You will see something like this:
```
    Reading from heap.bin...
    Dump file created Wed Feb 01 13:57:42 GMT 2006<
    Snapshot read, resolving...
    Resolving 18956 objects...
    Chasing references, expect 3 dots...
    Eliminating duplicate references...
    Snapshot resolved.
    Started HTTP server on port 7000
    Server is ready.
```
Now connect to the HTTP server using a browser and follow the links, e.g.
select All classes including platform
search for class java.io.DeleteOnExitHook, select it
From its Static Data Members you will see files (L) : java.util.HashSet@0xXXXXXXXX (XX bytes), select it
In its Instance data members you will see map (L) : java.util.HashMap@0xXXXXXXXX (XX bytes), select it
Its Instance data members will look something like this:
```
  entrySet (L) : 
  keySet (L) : 
  loadFactor (F) : 0.75
  modCount (I) : 5000
  size (I) : 5000
  table (L) : [Ljava.util.HashMap$Entry;@0xbbc19ff8 (32776 bytes)
  threshold (I) : 6144
  values (L) : 
```
size is the number of files currently marked as deleteOnExit. In this example 5000. You can see the amount of heap space they are comsuming by looking at the number of bytes the table instance data member is occuping. In this example 32776 bytes.

You can also see the files that are currently marked as deleteOnExit by clicking on the table instance data member, which is a HashMap$Entry array. Each HashMap$Entry has a key which is a java.lang.String whose value is the abstract path name.

Write yourself a tool

This is a little diagnostic tool that allows you to poll the length of the deleteOnExit list. It uses reflection to access the same data members as in the above solution, but automates it so you don't need to know about the internal data structures.

The new Attach API lets you load an agent into a running vm. The following sample code contains a simple agent that uses reflection to find out the number of files marked deleteOnExit and sends this to the client over a Socket. This allows you to connect to an already running java process (started as usual without any special arguments) and obtain a count of the files marked deleteOnExit. You can use jps to retrieve the process id of the java process that you wish to attach to and then pass it as a command line argument to CountDeleteOnExitFiles


    weetabix : jps -l
    6829 DeleteOnExitTest
    6835 sun.tools.jps.Jps
    
    weetabix : java -cp /j2se/jdk6.0/lib/tools.jar:. CountDeleteOnExitFiles 6829
    Number of files registered to be deleted on exit is 32
    Number of files registered to be deleted on exit is 37
    Number of files registered to be deleted on exit is 42
    ....

Client that attachs to the target vm and loads the Agent:
CountDeleteOnExitFiles.java


import java.net.*;
import java.io.*;
import com.sun.tools.attach.*;

public class CountDeleteOnExitFiles
{
    public static void main(String[] args) {
	if (args.length != 1 || args[0].equals("-h")) {
	    System.err.println("Prints the number of files registered to be deleted on exit in process <pid> every 5 seconds");
	    System.err.println("Usage: \n  java CountDeleteOnExitFiles <pid>");
	    return;
	}

	try {     
	    ServerSocket ss = new ServerSocket(0);
	    int port = ss.getLocalPort();
	    
	    // attach to target VM and load Agent
            VirtualMachine vm = VirtualMachine.attach(args[0]);
            
            File jarfile = new File("DeleteOnExitAgent.jar");
            if (!jarfile.exists()) {
                System.err.println("agent not found");
                return;
            }
            String agent = jarfile.getCanonicalPath(); 

            vm.loadAgent(agent, Integer.toString(port));
            vm.detach();
            
            Socket socket = ss.accept();
	    ss.close();

	    DataInputStream in = new DataInputStream(socket.getInputStream());
	    for (;;) {
		int size = in.readInt();
		System.out.println("Number of files registered to be deleted on exit is " + size);
	    }
	} catch (Exception e) {
	    e.printStackTrace();
	    System.exit(0);
	}
    }
}

Agent that is loaded into the target vm and sends the count of files marked deleteOnExit to the client (agent uses reflection to access Sun's internal deleteOnExit implementation and is therefore Sun jdk specific). This Agent needs to be deployed as a jar file in accordance with the "Starting Agents After VM Startup" section of java.lang.instrument. Specifically it needs to be compiled and put in a jar file called DeleteOnExitAgent.jar with the attribute "Agent-Class: DeleteOnExitAgent" in its manifest.

  
weetabix : javac DeleteOnExitAgent.java
weetabix : more manifest.mf
Agent-Class: DeleteOnExitAgent
weetabix : jar -cvfm DeleteOnExitAgent.jar manifest.mf DeleteOnExitAgent*.class
added manifest
adding: DeleteOnExitAgent$1.class(in = 486) (out= 338)(deflated 30%)
adding: DeleteOnExitAgent.class(in = 2357) (out= 1381)(deflated 41%)
weetabix : cp DeleteOnExitAgent.jar /j2se/jdk6.0/jre/lib/ext

DeleteOnExitAgent.java


import java.util.*;
import java.io.*;
import java.net.*;
import java.lang.reflect.*;

public class DeleteOnExitAgent
{
    static int DEFAULT_INTERVAL = 5000;
    Socket socket = null;

    // invoked by attach mechanism
    public static void agentmain(String agentArgs) {
	//port number to send data to
	final int port;

	if (agentArgs != null && agentArgs.equals("")) {
            try {
                port = Integer.parseInt(agentArgs);
            } catch (NumberFormatException nfe)
                return;
        }
        else
            return;

	Thread agentThread = new Thread( new Runnable() {
		public void run() {
 		    startAgent(port);
		}});
	agentThread.setDaemon(true);
	agentThread.start();
    }

    static void startAgent(int port) {
	Socket socket = null;
	DataOutputStream dataOut = null;
	try {
	    socket = new Socket("localhost", port);
	    dataOut = new DataOutputStream(socket.getOutputStream());

	    Class DeleteOnExitHookClass = Class.forName("java.io.DeleteOnExitHook");
	    Field filesField = DeleteOnExitHookClass.getDeclaredField("files");
	    filesField.setAccessible(true);
	    HashSet hashSet = (HashSet)filesField.get(null);

	    Class hashSetClass = Class.forName("java.util.HashSet");
	    Field mapField = hashSetClass.getDeclaredField("map");
	    mapField.setAccessible(true);
	    HashMap hashMap = (HashMap)mapField.get(hashSet);
    
	    Class hashMapClass = Class.forName("java.util.HashMap");
	    Field sizeField = hashMapClass.getDeclaredField("size");
	    sizeField.setAccessible(true);

	    for (;;) {
	        int size = sizeField.getInt(hashMap);
	        dataOut.writeInt(size);
		Thread.sleep(DEFAULT_INTERVAL);
	    }
	} catch (SocketException se) {
	    try { if (socket != null ) { dataOut.close(); socket.close(); } }
	    catch (IOException e) { e.printStackTrace(); }
	} catch (Exception e) {
	   e.printStackTrace(); 
	}
    }
}

Posted at 04:04PM Feb 06, 2006 by chegar in Java | Comments[239] | Permalink

Comments:

[Trackback] Java SE 6 final release is now available! This presents an overview of the monitoring, management, and diagnosability features in Java SE 6 and also serves as a starting point for you to find the relevant blogs/articles/documentation.

Posted by Mandy Chung's Blog on December 11, 2006 at 11:40 AM GMT+00:00 #

Posted by Mandy Chung's Blog on December 11, 2006 at 11:58 AM GMT+00:00 #

Posted by Mandy Chung's Blog on December 12, 2006 at 08:22 PM GMT+00:00 #

thanks goood

Posted by kabin on July 06, 2007 at 09:42 PM GMT+00:00 #

thanks good text

Posted by kapı on July 06, 2007 at 09:56 PM GMT+00:00 #

Posted by kale kapı on September 09, 2007 at 07:09 PM GMT+00:00 #

Thanks good

Posted by ev on September 26, 2007 at 01:01 AM GMT+00:00 #

Thanks.

Posted by msn on November 03, 2007 at 09:10 AM GMT+00:00 #

This presents an overview of the monitoring, management.

Posted by makyaj on November 03, 2007 at 09:11 AM GMT+00:00 #

thanks. good :)

Posted by games on November 03, 2007 at 09:12 AM GMT+00:00 #

thank you

Posted by oyunlar on November 16, 2007 at 06:48 PM GMT+00:00 #

thanks

Posted by sandalye on November 16, 2007 at 06:49 PM GMT+00:00 #

Thanks.

Posted by kız oyunları on December 02, 2007 at 11:07 AM GMT+00:00 #

Thank You.

Posted by oyun on December 02, 2007 at 11:07 AM GMT+00:00 #

thank you good.

Posted by çanakkale on December 02, 2007 at 11:09 AM GMT+00:00 #

thank you

Posted by çanakkale on December 02, 2007 at 11:09 AM GMT+00:00 #

thanks.

Posted by bozcaada on December 02, 2007 at 11:10 AM GMT+00:00 #

thanks...

Posted by bozcaada on December 02, 2007 at 11:11 AM GMT+00:00 #

hii..thanks very nice. I Like this website

Posted by oyun on December 04, 2007 at 01:13 PM GMT+00:00 #

thanks..

Posted by forum on December 04, 2007 at 01:15 PM GMT+00:00 #

thanks..very good

Posted by en guzel oyun on December 04, 2007 at 01:16 PM GMT+00:00 #

thanks

Posted by aşk on December 20, 2007 at 07:54 PM GMT+00:00 #

thank you

Posted by oyunlar on December 20, 2007 at 07:54 PM GMT+00:00 #

sağolasinn

Posted by oyunlar on December 20, 2007 at 07:56 PM GMT+00:00 #

sağolll

Posted by kız oyunları on December 20, 2007 at 07:56 PM GMT+00:00 #

çok güzel

Posted by aşk on December 20, 2007 at 07:56 PM GMT+00:00 #

Spam

Posted by Скачать антивирус on January 02, 2008 at 04:39 PM GMT+00:00 #

thankssss

Posted by sohbet on February 14, 2008 at 03:28 PM GMT+00:00 #

thank you

Posted by sohbet on February 14, 2008 at 03:28 PM GMT+00:00 #

thankssss

Posted by sohbet on February 14, 2008 at 03:28 PM GMT+00:00 #

thansss

Posted by evden eve nakliyat on February 14, 2008 at 03:29 PM GMT+00:00 #

thanksss

Posted by evden eve nakliyat on February 14, 2008 at 03:29 PM GMT+00:00 #

thanksss

Posted by evden eve nakliyat on February 14, 2008 at 03:29 PM GMT+00:00 #

thanks so much

Posted by bodrum hotels on March 19, 2008 at 03:10 AM GMT+00:00 #

thank you

Posted by kadin on March 19, 2008 at 03:11 AM GMT+00:00 #

thank you very much

Posted by dershane on March 19, 2008 at 03:11 AM GMT+00:00 #

Great article

Posted by Cooking Games on March 20, 2008 at 02:52 PM GMT+00:00 #

Nice

Posted by oyunlar1 on March 20, 2008 at 02:53 PM GMT+00:00 #

Turkish building magazina, bid news fair news, decoration news, sector news.

Posted by insaat on March 26, 2008 at 07:20 AM GMT+00:00 #

very thanks

Posted by Oyun on May 12, 2008 at 10:30 PM GMT+00:00 #

teşekkürler

Posted by chat on May 19, 2008 at 08:39 PM GMT+00:00 #

sağola

Posted by muhabbet on May 19, 2008 at 08:39 PM GMT+00:00 #

o ne mrk ya, tenks

Posted by chat on May 28, 2008 at 09:11 AM GMT+00:00 #

very thanks you

Posted by Otel on May 28, 2008 at 06:46 PM GMT+00:00 #

thankss

Posted by mirc on June 07, 2008 at 03:22 PM GMT+00:00 #

thanks my brother

Posted by mirc on June 07, 2008 at 03:24 PM GMT+00:00 #

thanks very good

Posted by chat on June 07, 2008 at 03:24 PM GMT+00:00 #

thanks

Posted by chat on June 19, 2008 at 11:37 PM GMT+00:00 #

thanx

Posted by bloger on June 23, 2008 at 02:54 AM GMT+00:00 #

Speacial Thx !

Camfrog Video Chat Forums http://www.undeadcf.info

Posted by Camfrog on June 30, 2008 at 04:11 PM GMT+00:00 #

Congrats and thanks.

Posted by Seo Danışmanı on July 03, 2008 at 12:58 PM GMT+00:00 #

thanks you

Posted by Okey on July 09, 2008 at 12:16 AM GMT+00:00 #

thank you very nice article

Posted by oyun parki on July 15, 2008 at 12:34 PM GMT+00:00 #

thank you very nice

Posted by hikayeler on July 15, 2008 at 12:35 PM GMT+00:00 #

very good idea

Posted by rixoyun on July 15, 2008 at 12:35 PM GMT+00:00 #

thanks

Posted by miami on July 15, 2008 at 12:36 PM GMT+00:00 #

very nice articles

Posted by article on July 15, 2008 at 12:36 PM GMT+00:00 #

thank you

Posted by oyunlar on July 15, 2008 at 12:37 PM GMT+00:00 #

world wide web thanks

Posted by world search on July 15, 2008 at 12:38 PM GMT+00:00 #

local search thank you

Posted by local search on July 15, 2008 at 12:39 PM GMT+00:00 #

Very nice article! Thanks for this!
doumo_arigatou

Posted by 中古ビデオ on July 22, 2008 at 11:13 AM GMT+00:00 #

Thank you

Posted by Chat on July 26, 2008 at 01:35 AM GMT+00:00 #

özel güvenlik

Posted by özel güvenlik on July 26, 2008 at 03:10 AM GMT+00:00 #

Thank you

Posted by 二宮沙樹 on July 26, 2008 at 05:44 AM GMT+00:00 #

özel güvenlik görevlisi thank you

Posted by güvenlik görevlisi on July 27, 2008 at 09:39 PM GMT+00:00 #

thank youu

Posted by güvenlik görevlisi on July 27, 2008 at 09:49 PM GMT+00:00 #

thank youu

Posted by özel güvenlik on July 27, 2008 at 09:50 PM GMT+00:00 #

thanksss

Posted by evden eve nakliyat on August 01, 2008 at 12:08 PM GMT+00:00 #

very nice articles

Posted by prefabrik on August 03, 2008 at 05:52 PM GMT+00:00 #

thank you

Posted by hemşire on August 04, 2008 at 07:51 PM GMT+00:00 #

thank you

Posted by hemşire sitesi on August 04, 2008 at 07:52 PM GMT+00:00 #

thank you

Posted by amasya on August 21, 2008 at 01:34 PM GMT+00:00 #

evden eve nakliyat , nakliyat

Posted by Evden eve nakliyat on August 22, 2008 at 02:41 PM GMT+00:00 #

HMM.. very nice articles THANKS

Posted by iskele on August 31, 2008 at 09:57 AM GMT+00:00 #

thanks you

Posted by key ödemeleri on September 07, 2008 at 10:18 PM GMT+00:00 #

thanx you!!

Posted by Resim Yükle on September 09, 2008 at 07:29 PM GMT+00:00 #

Thanks so much.

Posted by oyun on September 14, 2008 at 02:19 AM GMT+00:00 #

Thanks you

Posted by manavgat on September 25, 2008 at 12:54 AM GMT+00:00 #

great code thanks

Posted by Oyunlar on September 25, 2008 at 03:51 AM GMT+00:00 #

great code thanks

Posted by muhabbetgulu on October 13, 2008 at 02:34 PM GMT+00:00 #

great code thanks

Posted by muhabbetgulu on October 13, 2008 at 02:34 PM GMT+00:00 #

great code thanks

Posted by chat on October 13, 2008 at 02:35 PM GMT+00:00 #

Thanx yOu .

Posted by kelebek on October 16, 2008 at 05:19 PM GMT+00:00 #

Thanx yOu .

Posted by kelebek on October 16, 2008 at 05:20 PM GMT+00:00 #

Thanks so much.

Posted by oyun on October 21, 2008 at 08:44 PM GMT+00:00 #

Thanks very good !.

Posted by cinsel urunler on October 22, 2008 at 03:41 PM GMT+00:00 #

thanks

Posted by chat on October 27, 2008 at 10:23 PM GMT+00:00 #

Mon	Tue	Wed	Thu	Fri	Sat	Sun
« June 2009
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

Today

Weblog Chris Hegarty : Weblog

Monday Feb 06, 2006

Diagnosing DeleteOnExit Issues.

About This Weblog

Archives

Feeds

Search

Bookmarks

Other Blog Links