Bill Sommerfeld's Weblog

Wednesday June 16, 2004

dtrace for reverse engineering. Since Bryan was so kind to list me as a "Friend of DTrace"..

Lately I've been prototyping some extensions to our IPsec implementation. While shaking them out, I've twice had the experience of the system not even getting to the new code when I was expecting it to..

In the pre-dtrace era, finding the point where we went astray was often a frustrating exercise in single-stepping, psychic breakpoint insertion, or iterated debug-by-printf.

With dtrace, it's a matter of running a truss to get the relevant syscall arguments and then doing something along the lines of:

#! /usr/sbin/dtrace -Fs

syscall::so_socket:entry
/arg0 == 29/
{
   self->traceme = 1;
}

fbt:::
/self->traceme/
{}

syscall::so_socket:return
/self->traceme/
{
  self->traceme = 0;
}

Drop the above in a file, chmod +x, run it, watch dtrace insert 45000 probes into the live system in a second or two..

.. and I quickly discovered that the reason I wasn't getting to the spdsock open routine was due to overly conservative device policy, which turned out to be easy to fix. Same deal for a new socket option -- whoops, added it to the wrong switch().. (2004-06-16 14:46:22.0) Permalink