Bill Sommerfeld's Weblog

Monday June 21, 2004

Emissions control computers vs the Bill of Rights? Last week, I brought my car in for its mandatory annual state inspection. While there, I learned that Massachusetts has just replaced its current dynamometer-based emissions test with a newer one which involves connecting the inspection station's computer up to your (post-1996) car's standardized OBD port.

Today, while reading Eugene Volokh's multiple blog posts on the Supreme Court decision in the Hiibel case, I started wondering about the applicability of the Fourth and Fifth Amendments to Massachusetts' new scheme for vehicle emissions tests. Is a mandatory telemetry system in a vehicle, combined with mandatory disclosure of some underspecified selection of the telemetry, an "unreasonable" search or a form of compelled self-incrimination?

From a database privacy angle, what values are actually recorded? Is maximum speed one of them? Who gets to do data-mining on the accumulated telemetry, or see individually identifiable records?

The state's FAQ isn't particularly informative.. (2004-06-21 15:59:48.0) Permalink

Wednesday June 16, 2004

Can't escape work... Last night, the NSPO had a private performance (we do a few of these a year to subsidize our operating expenses). During the pre-concert milling about, the guy sitting to my right (also a software engineer type..) asks me "so.. how do structure returns work in C?" which led to a short conversation about just how badly things can go wrong if you write code for an RTOS which returns structures a few times bigger than the typical RTOS thread stack size...

Concert went pretty well, too.. (2004-06-16 20:13:07.0) Permalink

dtrace for reverse engineering. Since Bryan was so kind to list me as a "Friend of DTrace"..

Lately I've been prototyping some extensions to our IPsec implementation. While shaking them out, I've twice had the experience of the system not even getting to the new code when I was expecting it to..

In the pre-dtrace era, finding the point where we went astray was often a frustrating exercise in single-stepping, psychic breakpoint insertion, or iterated debug-by-printf.

With dtrace, it's a matter of running a truss to get the relevant syscall arguments and then doing something along the lines of:

#! /usr/sbin/dtrace -Fs

syscall::so_socket:entry
/arg0 == 29/
{
   self->traceme = 1;
}

fbt:::
/self->traceme/
{}

syscall::so_socket:return
/self->traceme/
{
  self->traceme = 0;
}

Drop the above in a file, chmod +x, run it, watch dtrace insert 45000 probes into the live system in a second or two..

.. and I quickly discovered that the reason I wasn't getting to the spdsock open routine was due to overly conservative device policy, which turned out to be easy to fix. Same deal for a new socket option -- whoops, added it to the wrong switch().. (2004-06-16 14:46:22.0) Permalink

Saturday June 12, 2004

Knowing your tools.. An article featured on Slashdot today, Why Learning Assembly Language Is Still a Good Idea completely misses out one crucial reason why a reading knowledge of assembly and knowlege of a machine's low level architecture will help a systems programmer get their job done: Crash analysis. Over two decades of coding, it's been particularly obvious to me that programmers who don't know how to extract information from a core file, crash dump, or stack trace on average will take a whole lot longer to root-cause and fix bugs. And to do that, you need good tools and a modest ability to read assembler.

It's something like forensic analysis - the dump is the "scene of a crime" often containing many clues as to the cause of the fault.. (If you don't mind an analogy from Hollywood, it brings to mind a scene from a recent rerun of Crossing Jordan - a junior ME misses a bunch of more subtle clues at a murder scene which Jordan spots after a careful once-over; these clues give them a running start at the investigation).

On most unix systems, you generally get the register contents at the time of the fault and from that know which instruction was executing at the time. Source level debuggers can convert that into a source line, but that loses information -- when there's ambiguity, it's often better to work backwards from the instruction rather than try to guess which part of a complex expression the processor tripped over. And if you didn't build -g, a source-level debugger will throw up its hands and point in the vague direction of a function at fault.

So what do you do when the program crashes in production? Go back, rebuild with -g, and try to reproduce it? If it's a once-a-month, hard to reproduce race condition, you may end up waiting a while before it happens again.

Even if you can't conclusively prove anything from a crash, a careful analysis will often give you a big head start towards reproducing the problem, and knowing where to lay traps to catch a precursor to the crash.

Among the folks developing solaris, MDB is the debugger of choice for this sort of analysis. It's an evolving, extensible debugger, with a published API you can use to add your own extensions. GNU's GDB's also good for low-level debugging.

If you have to "lie in wait" for the bug to happen again, dtrace can't be beat. (And, well, it's also great for any number of other observability tasks.. but that's for another day..).

How to learn this stuff? Nothing beats a little experimentation. Sit down with an architecture manual, and a C compiler. Read enough of the front matter of the architecture manual to understand the register layout. Write some simple C code, compile it -S, and look at the assembly output. Learn how your compiler tends to generate code.. Learn how C-level identifiers appear in the low-level object symbol tables. Use your debugger to disassemble a "live" program, and compare with the -S output. Single step through the program an instruction at a time and watch the control flow and how the contents of registers and memory change. If you can't figure out what an instruction does, look it up. (2004-06-12 10:56:54.0) Permalink

Friday June 11, 2004

What's up with Planet Sun's date conversion? To cite a specific example, a post from Keith Bierman which appears to me dated "2004-06-11 00:33:50.0" is listed on Planet Sun at "June 12, 2004 07:33 AM", 31 hours later, and (as of the time of this writing) still in the future, even in New Zealand... I first thought it was time zones, but that doesn't explain a delta of more than 24 hours.. (2004-06-11 12:21:18.0) Permalink

Wednesday June 09, 2004

Funny that this is one of the default categories... as music is my most significant non-computer-related activity. I've played the trombone for many many years and I've somehow managed to keep a space for it in my life since college. These days I mostly do orchestral playing - both pit orchestras for shows/operas/operettas with a number of Boston-area groups, as well as symphonic playing here and there.

Groups I've played with recently include:

• The North Shore Philharmonic in Beverly, MA
• The Gilbert & Sullivan troupes of both MIT and Harvard
• The Brahms Society Orchestra

And, subject to free time, I wouldn't mind other chances to play..