Using DTrace with Java & Web 2.0 (Code Camp, Walldorf)

Date : 17.October.2007
Base URL : http://de.sun.com/sunnews/events/2007/20071017/

Yesterday, we organized our second Code Camp on using DTrace, the Swiss army knife for the Solaris platform. We had 8 participants, who were very enterprising and motivating. We're really glad that we are reaching out to our customers. 

 The event started off at 9:00 AM when we spent the first hour installing the software and loading the Solaris Developer Express Build 69 Vmware images. The images were pre-configured with Apache, MySQL, PhP, JDK 1.5 & 1.6.  Services were written to enable the MySQL and Apache servers to start automatically during bootup. This would save our participants from going through the menacing task of configuring the environment, which is definitely not the focus. I tested this several times myself. Stefan and Thomas went through the finalization phases when they gave me valuable feedbacks on improving the usablility and ease of working with the Vmware image.

The Vmware image is a menacing software. Solaris platform by itself needs 768 MB of physical memory and the native performance is appreciable. The nightmare comes on using this platform in a Vmware image. One of our participants had a notebook with 512 MB. The prescription was to use a notebook with 768MB. Though our participant had initial problems on using this setup, the system stabilized on leaving the setup to settle for the optimum cofniguration after the host platform has set the needed pages, caches, etc. Its quite impressive that we saw the use of Vmware images to deliver our content, at the same time retain the original machine setup of our participants. The use of Vmware images comes on its non-destructive methodology. Despite our virtualization solutions like Zones, LDoms, xVMs, etc., Vmware does have a use in delivering such content when the focus is on the materials than on the tools.

Well, we started our Code Camp with Thomas delivering the lecture on DTrace, its architecture, its strengths and weakness. After a brief introduction, we started getting our feet wet. We started off with exercises and the reception was excellent. Our participants started asking us questions on the DTrace concepts such as threads, stack, aggregation, buffers, etc. Though we use DTrace at our daily work, its really interesting to see its use among the spectrum of ISVs whose needs are quite broad. The guys who designed and implemented DTrace did indeed think ahead and see such a broad use. Good work! The queries made us think deeper about the concepts we never thought of before.

One of our ISVs had a particular need to determine the stack size of a multi-threaded application and from that information, restrict the stack size. I haven't had the need for such an approach and this was a thing which I had to learn. I approached the problem from the aggregation buffer. DTrace offers several tunables and options to fine tune buffers with parameters like switchrate, buffersize, etc. I guesses restricting the aggregation buffer size would bring about the same effect, but one had to try this out and I put this as an action item. Nevertheless, the participants were so fascinated by DTrace and with the hands-on experience that they gained from our Code Camp, I bet they are going to use it more often in analyzing bottlenecks, crashes, performance problems and issues like these that occur on a production system. DTrace, as I said before, is a swiss army knife that lets you do such analysis on live systems. The script he found out was very simple and could be found here.

From the discussion at the above URL, it seems that the stackdepth variable is not the key to the problem we are looking at. An one-liner could get the stacksize:

dtrace -n 'on-cpu { @[execname] = max(curthread->t_procp->p_stksize); }'

For multithreaded programs, this would fail for apparent reasons. We are not using any king of thread locality here. So, the solution we are looking for is something like this:

#!/usr/sbin/dtrace -s
this uintptr_t stkinfoptr;
this uintptr_t stkptr;

sched:::on-cpu, profile:::profile-997{
this->stkinfoptr = 0;
this->stkptr = 0;
}

sched:::on-cpu, profile:::profile-997
/execname != "sched"/{
this->stkinfoptr = curthread->t_lwp->lwp_ustack;
this->stkptr = (uintptr_t)0;
}

sched:::on-cpu, profile:::profile-997
/this->stkinfoptr != 0 && curpsinfo->pr_dmodel == PR_MODEL_ILP32/{
this->stkinfo32 = *(stack32_t *)copyin(this->stkinfoptr,
sizeof (stack32_t));
this->stktop = (uintptr_t)this->stkinfo32.ss_sp +this->stkinfo32.ss_size;
this->stkptr = (uintptr_t)uregs[R_SP];
}

sched:::on-cpu, profile:::profile-997
/this->stkinfoptr != 0 && curpsinfo->pr_dmodel == PR_MODEL_LP64/{
this->stkinfo = *(stack_t *)copyin(this->stkinfoptr,
sizeof (stack_t));
this->stktop = (uintptr_t)this->stkinfo.ss_sp +this->stkinfo.ss_size;
this->stkptr = (uintptr_t)uregs[R_SP];
}

sched:::on-cpu, profile:::profile-997
/this->stkptr != 0/{
@a[execname] = quantize(this->stktop - this->stkptr);
}

dtrace:::ERROR{
@b[execname] = count();
}

dtrace:::END{
printa(@a);
printf("\nErrors: \n" ) ;
printa(" %@d %s\n", @b);
}

Now, this approach should work. I tried executing this script on my x64 (Opteron) workstation and saw impressive results. This script works indeed and I've learnt something new. Its really impressive to see that our technology has appealed to our ISVs and they are taking the bulk of work from our shoulders and are using it in their daily work. Thats exactly what we are striving for. Way to go!

Well, I also remember something which my boss, Charles Addison told me once, "Young man, you are dead when you stop learning or your stop learning when you're dead". Now I see the point, Charles.

Summary of the  Code Camp:

 * The DVD worked for all the participants. No issues were found either in the content or in the Vmware image. The success rate was 100%
* The new DVD cover design is quite attractive. This was designed by "Milk & Honey", a small advertising firm in Vaterstetten, Munich.
* No glitches were observed in the exercises.
* The Walkthrough document was modified from version 1.4 on the DVD to version 1.5, which was handed out to our ISVs. The newer version shall be uploaded later and shall be the standard document for further Code Camps on DTrace.
* The Code Camps are meant for Sun partners and ISVs. If you wish to become one, contact Silke Pumberger or any one of use - Mithun Sridharan (thats me), Stefan Schneider, Thomas Bastian.
* Learn more about Sun Partner Advantage Program ( SPAP) here.

Unfortunately, I missed taking pictures of our participants. Else, I would have put up the pictures of the Code Camp here. Should make a mental not of it next time.

So, let me take this excellent opportunity to thank all our participants and my Sun colleagues, who were a part of our efforts in organizing such an excellent event. Kudos!!
  

Posted at 09:26AM Oct 19, 2007 by mithun in Sun  |  Comments[0]