Speaker To Machines

Erik O'Shaughnessy - erik.oshaughnessy AT Sun.COM


20040729 Thursday July 29, 2004

DTrace Instruction Tracing Continues..

Wow, this has me bugged. Using this script ( called itrc.d ):

pid$target:a.out::
{
        printf("%d %016llx",vtimestamp,uregs[R_PC]);
}

# dtrace -s itrc.d -c hello3 > rtrc.hello3

a custom SPARC disassembler, and a really awful perl script, I produced:

Delta NS Address          Instruction   (disassembled)
       0 0000000100000740 bc102000      or      %g0, 0x0, %fp
   12960 0000000100000744 e05ba87f      ldx     [%sp+0x87f], %l0
    7120 0000000100000748 1b000000      sethi   0x0, %o5
    5040 000000010000074c 9a136001      or      %o5, 0x1, %o5
    4960 0000000100000750 9b2b7020      sllx    %o5, 0x1020, %o5
    4640 0000000100000754 13000403      sethi   0x100c00, %o1
    3920 0000000100000758 9212400d      or      %o1, %o5, %o1
    4240 000000010000075c e0226208      stw     %l0, [%o1+ 0x208]
    4240 0000000100000760 a203a887      add     %sp, 0x887, %l1
 ...

I think I'm almost done.

-ejo

(2004-07-29 14:28:54.0) Permalink

DTrace Instruction Tracing

Adam Leventhal pointed out that the DTrace pid provider can instrument user-land instructions, so my interest was piqued enough to give it a whirl last night. The script I ended up with is just stupidly short, I feel like I'm cheating.

pid$target:a.out::
{
        printf("pc %p: %x",uregs[R_PC],*(uint )copyin(uregs[R_PC],4));
}

Using a new DTrace feature, I start the trace and get some output:

# dtrace -c ./hello2 -s itrc.d 
CPU     ID                    FUNCTION:NAME
  0  38695                     _start:entry pc 1000006a0: 91d02038
  0  38696                         _start:0 pc 1000006a0: 91d02038
  0  38697                         _start:4 pc 1000006a4: 91d02038
  0  38698                         _start:8 pc 1000006a8: 91d02038
...

That doesn't look right. The PC's look good, but those aren't the instructions at those addresses. Or are they? I was a bit foggy last night when I started messing with this, so I assumed that I wasn't using copyin() right. Somehow I was getting the address of the DTrace buffer that copyin returns or something. In the cold light of morning things look different. It occurs to me that 91d02038 looks familiar. Yes I know, it's numbers. The crawling itch in my lizard hacker brain was trying to tell me something:

91d02038 <=> ta      %icc, 0x38

What? Your lizard hacker brain doesn't tell you stuff like that? Lucky you. So I was using copyin correctly. I am fetching the instruction at the PC, just not the original instruction. As everybody knows, 0x38 is the DTRACE pid provider trap, so this instruction is the DTrace instrumentation that makes it all work.

I'm feeling moderately clever for having figured that out on only two cups of coffee. :)

-ejo

(2004-07-29 08:17:52.0) Permalink Comments [1]

20040728 Wednesday July 28, 2004

Adventures in Instruction Tracing: Part Duex!

Yesterday I began talking about cobbling together a user-land instruction trace tool using a variety of UltraSPARC and Solaris features. Today I'm going to cop out alittle and present the comments for the user trap handler I wrote which is the heart of the tool. For the truly bored, I've provided a hello.c and associated (post-processeed) trace.

-ejo

#if 0

illtrap_hndlr - Btrace3 ILLTRAP utrap handler

erik.oshaughnessy@sun.com


Btrace3 instruments the code of a program by substituting control
transfer instructions (CTIs) with specially prepared ILLTRAP
instructions.  When the code executes, the ILLTRAP instructions cause
the utrap handler to execute.  This allows us to observe the actual
path of execution by noting which ILLTRAP instructions are executed.

It probably is worth mentioning that the ILLTRAP instruction is a
sneaky way of hiding 22 bit constant data in your code.  According to
_The SPARC Architecture Manual_ Version 9, David Weaver and Tom
Germond, page 168:

"The ILLTRAP instruction causes a illegal_instruction exception. The
 const22 value is ignored by the hardware; specifically, this field is
 NOT reserved by the architecure for any future use."

I have taken advantage of this by crafting the ILLTRAP instructions so
that they serve as indices into the array of saved original
instructions. Used in a flat array, a const22 value gives us 2^22-1 or
roughly 4 million points where code can be instrumented.  For the
current generation of software this number of probe points appears to
suffice, but it is not difficult to conceive of future applications
where the number of probe points exceeds this flat space.

The first implementation of Btrace played a dangerous shell game,
storing the original instruction back into the executing stream of
instructions and re-inserting the ILLTRAP when handling the next
probed CTI.  Despite the odd corner case, this technique worked
surprisingly well in a single- threaded application.  It falls apart
when attempting to trace multi- threaded applications since the
complexity of managing multiple stores into the instruction stream and
guranteeing correctness of the instrumented program proved simply
overwhelming.

Btrace3 sidesteps the self-modifying code aspects of Btrace1 and 2 by
emulating the CTI instruction instead of storing the probed
instruction back into the instruction stream.  This is possible since
the utrap handler can choose where the trapping context begins
executing after the utrap handler returns.  Typical utrap handlers
either skip the trapping instruction or restart the trapping
instruction after modifying the state of the trapping context.  A CTI
instruction can be emulated by calculating the target address if the
branch is taken and then determining if the CTI is taken.  After
evaluating any conditions predicated by the type of instruction being
emulated, it is a simple matter to return with arbitrary PC and nPC
addresses.

Since the instruction stream remains instrumented during the lifetime
of of the process, the synchronization complexity of the tracing code
is greatly reduced.  The only remaining synchronization point is trace
buffer management, which is a relatively well understood problem.

The first Btrace implementation also uncovered the importance of
preserving the state of the trapping context before proceeding with
tracing activities.  The two macros UTRAP_PROLOG and UTRAP_EPILOG
preserve and restore the following registers: %ccr, %g1 - %g7.
Additionally, UTRAP_PROLOG writes ASI_PNF to %asi as required by the
__sparc_utrap_install(2) man page before calls are made to ABI
conforming functions.  Values are preserved on the stack, making the
utrap handler reasonably re-entrant.  The stack also has room for
sundry other values; %tick, %fsr, %l6 and %l7 (PC and nPC) from
trapping context, and the ILLTRAP instruction at [%l6].  The macro
UTRAP_EPILOG restores the state of %ccr, %g1 - %g7 which have may have
been corrupted by the utrap handler activites.

The following is a pseudo pseudocode description of how illtrap_hndlr works:

illtrap_hndlr{

   sampled_tick_register = gethrtime()

   save trapping context

   original instruction = ProbeTable[ trapping ILLTRAP instruction ]

   tid = thr_self();     /* re-implemented to avoid linking libthread */

   cpuid = getcpuid()    /* new in Solaris 9 */

   switch( original instruction )
   
     case CALL:
        emulate CALL instruction

     case BPr:
        emulate BPr instruction	
   	
     case FBfcc:
        emulate FBfcc instruction	
   
     case FBPfcc:
        emulate FBPfcc instruction	
   
     case Bicc:
        emulate Bicc instruction	

     case BPcc:
        emulate BPcc instruction	

     case JMPL:
        emulate JMPL instruction	
     
     default:   /* getting to default is an error condition */
        
        unregister illtrap_hndlr
        restore trapping context
        restart trapping instruction, this time without the handler to
          catch it when it falls.  the program should die.
     
   /* end of switch */

   pick PC and nPC values based on contents of FLG register which
   indicates if the branch was taken and if the delay slot instruction was
   annulled.

   if( !TraceOn )
     restore trapping context
     start executing at PC and nPC computed during instruction emulation
   fi
   
   lock(TrcLock,tid)
   			
   if( cTrcSlot > TRCMAX )
     write(btrc->fd,TraceBuffer,TRCBUFSZ)
     cTrcSlot = 0;
   fi

   store btfrec_t built in emulate_* to TraceBuffer[cTrcSlot]

   cTrcSlot++

   unlock(TrcLock)

   restore trapping conext

   start executing at PC and nPC computed during instruction emulation

}

Of course there's alittle bit of hand waving, but that should give the
reader a basic idea of what's being done and in what order. 

#endif
(2004-07-28 13:12:23.0) Permalink

20040727 Tuesday July 27, 2004

Anything You Can Do, D Can Do Better!

Not sour grapes honest! Jon Haslam saw my fsflush post and couldn't just bask in the glory that was old school hacking. He had to throw the new hotness at the problem. I, for one, welcome our new DTrace overlords!

-ejo

(2004-07-27 15:20:05.0) Permalink Comments [1]

Adventures in Instruction Tracing

One thing that performance people like to do is actually know what instructions are executed by a CPU. Crazy, I know. You can disassemble the program to know what specific machine instructions compose the program, but you still don't know what instructions specifically are executed at run-time due to branching in the code induced by the input to the program.

So way back when, clever programmers found ways to instrument their code ( or more often, somebody else's code ) in order to determine what instructions a CPU was executing. There are lots of techniques for doing this, but I'd like to talk about the one I implemented and was disclosed in a short paper that was published in the May 2004 issue of Research Disclosure Journal. I'm happy to have been published, but the two page limit did sort of cramp my style.

Branch Tracing

Some computer architectures provide a spiffy feature, often referred to as "trap on branch". It is possible to build a pretty efficient instruction tracing tool using trap on branch since it interrupts the code under observation only at the end of a basic block, rather than on every instruction executed. On average a SPARCv9 basic block consists of 5 instrutions, so using trap on branch to trace would trap 80% less than trapping on every instruction. Wait for it. Unfortunately, SPARCv9 does not define a trap on branch. Oh if only.

Work With What You've Got

Despite the lack of branch on trap in SPARCv9, there are some other architectural goodies defined which the intrepid hacker could exploit. The first thing I looked at was the Instruction Breakpoint Register( Ok, technically not defined by SPARCv9, it's a USIII implementation defined extension ), which caused a breakpoint trap to fire when an instruction matched a user definable mask. Originally intended as a means of hot-patching broken instructions in the field, this feature never really worked as designed and was disabled in USIII+.

Another cool feature that doesn't get alot of love is the user trap handler facility, described in __sparc_utrap_install(2) and in the SPARC architecture manual (verson 9). It allows a user to define and install a custom trap handler for a specific (but diverse) set of exceptions. Now that is pretty cool!

You Got Your Debugger In My Instruction Tracer!

Skipping ahead alittle bit, I had the idea of combining an old debugger technique for breakpointing with SPARC user trap handlers to create a poor-man's trap on branch. Many debuggers implement breakpointing by inserting an illegal instruction at the specified address in the program and catching the illegal instruction trap generated when the client program executes it. The debugger then patches up the replaced instruction or emulates it, whichever seemed easier to do at the time. An instruction tracing tool could patch up the victim target application, replacing all it's control transfer instructions (CTIs) with ILLTRAP instructions ( SPARC instructions whose specific job is cause an illegal instruction trap ). Then when the target runs, every time it tries to branch it causes a trap which we can intercept with a user trap handler. The handler could write a record to a trace file and then fix up the broken branch and restart the target. Simplicity itself!

It's Never That Easy

Where to start. First, user trap handling for ILLTRAP was broken as detailed in bug 4650095. Second, there's the small matter of introducing ILLTRAPs into a program. Third, what to do with the CTIs that you've replaced with ILLTRAPs and all the trace output you will generate?

I discovered that an LD_AUDIT shared object would be an excellent way to introduce the trace instrumentation into a target application. The runtime linker audit interfaces are described adequately in rtld_audit(3ext) and in excellent detail in the Linker and Libraries Guide. The audit interfaces give the determined hacker a great deal of freedom to mess with an executable before control is passed to it from the runtime linker. Another advantage of this technique is the instrumenting code and the target code all share the same address space, so it's very easy to find and replace all the CTIs in the target code with ILLTRAPs.

A Commercial Plug And Then A Pause For Breath

As an engineer for Sun, I have found these three books to be invaluable:

  1. The SPARC Architecture Manual, version 9
  2. Solaris Interals
  3. Linker And Libraries Guide

The Linker and Library Guide in particular is an amazing piece of work, especially when you learn that it's written by the same guys who write the linker code. Those guys rock! All that's missing is a blog from Siezo!

Tomorrow, I'll post some condensed notes on the design of a user-land instruction tracing tool built on user traps called Btrace3, and why it ultimately was a flop.

-ejo

(2004-07-27 15:10:15.0) Permalink

20040723 Friday July 23, 2004

fsflush: what is it doing?

Yesterday I was reading performance related bug reports, trolling for work, when I stumbled upon 5024143. I don't know alot about fsflush, so I did some reading and found that on10's fsflush is keeping some internal statistics concerning it's operation that it isn't sharing ( a kstat would be a good place to start ). So being a curious sort, I whipped up a libkvm(3LIB) consumer to expose fsflush's stats to userland. fsfstat is born! Caveat: on10 required.

Some Example Output

Here is sample output from fsfstat run on a 12-way 4800 with 28G of real memory, mostly idle ( Burninator! ).

trogdor:fsfstat -> sudo ./fsfstat 1
#3594320   SCANNED   PCT EXAMINED   LOCKED MODIFIED COALESCE RELEASES   TIME(ns)
15:08:44    119810   3.3      235        0        0        0        0     340320
15:08:45    119810   6.7      235        0        0        0        0     324880
15:08:46    119810  10.0      235        0        0        0        0     324800
15:08:47    119810  13.3      235        0        0        0        0     318000
15:08:48    119810  16.7      634        2        0        0        0     501840
15:08:49    119810  20.0    13802      773        6        7        0    4269440
15:08:50    119810  23.3      235        0        0        0        0     335520
15:08:51    119810  26.7      235        0        0        0        0     320800
15:08:52    119810  30.0      235        0        0        0        0     317840
15:08:53    119810  33.3      235        0        0        0        0     317360
15:08:54    119810  36.7      235        0        0        0        0     320640
15:08:55    119810  40.0      235        0        0        0        0     323680
15:08:56    119810  43.3      235        0        0        0        0     320400
15:08:57    119810  46.7      235        0        0        0        0     315440
15:08:58    119810  50.0      235        0        0        0        0     318800
15:08:59    119810  53.3      235        0        0        0        0     319440
15:09:00    119810  56.7     6367     3124        0        0        0    7615280
15:09:01    119810  60.0   110739    54692        0        6        0  118389680
15:09:03    119810  63.3    33125    14333      374       10        0   31190320
15:09:04    239620  70.0      470        0        0        0        0     663520
15:09:05    119810  73.3      235        0        0        0        0     324240
15:09:06    119810  76.7      235        0        0        0        0     320080
15:09:07    119810  80.0      235        0        0        0        0     317280
15:09:08    119810  83.3      235        0        0        0        0     319680
15:09:09    119810  86.7      235        0        0        0        0     318640
#3594320   SCANNED   PCT EXAMINED   LOCKED MODIFIED COALESCE RELEASES   TIME(ns)
15:09:10    119810  90.0     6572      773       18        0        0    3049280
15:09:11    119810  93.3   113238      617        4        0        0   20368080
15:09:12    119810  96.7   103493        0        0       21        0   17344240
15:09:13    119810 100.0      235        0        0        0        0     346960
15:09:14    119810   3.3      235        0        0        0        0     321040
15:09:15    119810   6.7      235        0        0        0        0     324240
15:09:16    119810  10.0      235        0        0        0        0     322960
15:09:17    119810  13.3      235        0        0        0        0     318400
15:09:18    119810  16.7     2139       96        0        0        0    1095680
15:09:19    119810  20.0    12249      679        6        0        0    3521120
15:09:20    119810  23.3      235        0        0        0        0     334400
15:09:21    119810  26.7      235        0        0        0        0     326960
15:09:22    119810  30.0      235        0        0        0        0     325840
15:09:23    119810  33.3      235        0        0        0        0     320240
15:09:24    119810  36.7      235        0        0        0        0     316720
15:09:25    119810  40.0      235        0        0        0        0     317040
15:09:26    119810  43.3      235        0        0        0        0     316640
15:09:27    119810  46.7      235        0        0        0        0     317120
15:09:28    119810  50.0      235        0        0        0        0     317040
15:09:29    119810  53.3      235        0        0        0        0     317600
15:09:30    119810  56.7    13010     6448        0        0        0   15070240
15:09:31    119810  60.0   115114    56225      109        0        0  121045120
15:09:32    119810  63.3    21994     9476      265        0        0   20595040
15:09:33    119810  66.7      235        0        0        0        0     339840
15:09:34    119810  70.0      235        0        0        0        0     319760

The first column is a timestamp, the second column is number of 8k pages scanned. The fourth column is number of pages examined, which can be a smaller number due to larger pages being composed of sets of smaller pages. Locked, modified, coalesce and release I'm still foggy on ( I need to read and understand fsflush.c some more ). The final column is the time in nanoseconds that fsflush ran over the previous interval. I've saved the third column for last because I thought I was being rather clever. The third column tracks how far thru memory fsflush has scanned since fsfstat has been observing it. In this way, we can tell when fsflush has made a complete sweep thru physical memory and is starting over again.

Some Interesting Observations

Using the locality information gained by the percentage progress thru memory, we can see a pattern emerging in the sample data. Starting at the 20%, 56% and 90% range, we see three second bursts of activity that recurr. I don't have a definite explanation for this activity, and I wouldn't mind hearing thoughts about what's going on here.

Anyway, today is a double-two-fer :) I'm just happy with how this hack turned out.

-ejo

(2004-07-23 13:56:44.0) Permalink Comments [5]

Friday Gadget: Double Whammy

I've got two new gadgets that I'd like to talk about today :)

Gadget 1: Dell Inspiron 8600

My previous laptop was really showing it's age: .5Ghz CPU, .128GB of memory and .080TB of disk. And no built-in networking (modems do not count). So of course I needed a new sexy laptop. The Alienware Sentia was my first love, but after mooning over it for months I realized there were some problems with it that would affect my future feelings for it. First, the keyboard. The control and the function keys were transposed. If you are a heavy emacs user (like me), this is a deal killer. Ok that was the only problem with the Sentia. Otherwise it was perfect. Excpet the new Dothan (Pentium M with 2M cache) hadn't quite made it into the product yet. Other than that, perfect.

So next I flirted with the ABS Mayhem. The Mayhem had so much potential: a great name, a good keyboard layout, a Dothan CPU, 802.11a/b/g, and a DVD-RW. And the gaming rags gave it such high marks! But, the resolution of it's 15 inch LCD was a disappointing 1280x800. My old and busted laptop sported a 1400x1050 resolution, it didn't seem right to settle for something less when I'm going spend that much cash.

So I looked at Dell, and before long I found the laptop I wanted ( even if it didn't have a glowing alien head on it or a terror inducing product name ). The Dell Inspiron 8600 had everything I wanted in one package:

  • 1.8Ghz Dothan ( 2M caches! Power frugal! )
  • 802.11a/b/g
  • CD-RW / DVD-RW
  • 1680x1050 Resolution
  • Control key in the right spot!
  • USB 2.0, Firewire
  • Capable of holding 2GB of RAM
  • Radeon 9600 Mobility Pro w/128M ( a real 3D card! )
  • Integrated 10/100 Ethernet

Ordering was breeze, and it was one week from order to delivery. Which isn't too bad considering it was manufactured in Malaysia! It always strikes me as strange when I order something from Dell, whose corporate headquarters can't be more than five miles from my house as the crow flies, and they ship it to me from anywhere but Roundrock.

The Inspiron 8600 is great! The screen is bright and completely free of dead or stuck pixels. The keyboard touch is shallow like most notebook keyboards, but crisp enough that I could stand to type on it for long durations. The thing boots up fast. Not just whatever OS ( XP or Linux currently ), but the BIOS doesn't spend alot of time messing around. The wireless is currently only working on Windows, but I've been able to connect to my home G and B networks without any trouble. In fact, the wireless is the only hardware feature I don't have working under Linux.

About the only downside that you could possible attribute to the 8600 is it's size. While being about the same weight and depth as my previous laptop, the 8600 is probably an inch or more wider. This makes finding bags that fit it alittle challenging.

I specifically bought this laptop with the smallest harddrive and least amount of memory configurable, since I can easily upgrade those two things over time. ( And Dell gouges it's customers when it comes to RAM ).

I love this machine. I can't wait to see how cool the next laptop I buy will be in four or five years if this machine is an indicator!

Gadget 2: Samsung LTP1795W 17" HDTV Monitor

After the great home office remodeling adventure, I decided it would be nice to have a flat panel TV in there. I could move my Playstation 2 up there and free up the downstairs TV, and perhaps even do double duty as a computer monitor. I wanted something with: 16:9 aspect ratio, component inputs, computer connectivity ( 15-pin or DVI ), and less than $1000. After much searching and review reading, I found the Samsung LTP1795W. This 17" wonder has two component inputs, 15-pin monitor input, TV tuner, 16:9 aspect ratio, builtin speakers, headphone output, and under $1000. Of course it has alot more, but those are the reasons I bought it.

The day it arrived, I thought I'd been ripped off. The box didn't feel nearly heavy enough to have a monitor in it. Going back and looking at the specs, I shouldn't have been surprised. The monitor without stand weighs something less than 17 pounds.

After carefully unpacking it, I was amazed at it's simplicity and graceful design. It is a beautiful little unit. And that was before I turned it on and plugged in my Playstation. The connections on the back of the unit are all very well labeled and oriented up and down, so it would be possible to mount this screen very close to a wall. The controls are on the top of the unit, and the remote control is very functional.

I've spent a couple hours using it to play SOCOM II ( I don't think I've gone a post witout mentioning SOCOM II ), and I couldn't be more pleased. I was worried that moving from a 36" Toshiba CN36X81 to a 17" screen might be an issue, but that hasn't been the case. About the only thing the Samsung is missing is the simulated 5.1 provided by the Toshiba ( it has a big ole subwoofer inside it's cabinet ).

I would have loved the 37 inch big brother of this unit, but the price scaled with the screen size. Maybe later that will change.

There you have it, two gadgets.

-ejo

(2004-07-23 12:28:11.0) Permalink

20040721 Wednesday July 21, 2004

Games I Like

Yesterday's entry got better response than I expected ( 3 people sent me mail! ). I hope you all find my hack useful. :)

Today I wanted to write about games, in particular the game that I'm hopelessly addicted to: SOCOM II. Wanted to write about it, but as you can see, I gave up because it devolved into "Oh Man! It's so great! You can shoot dudes online and then talk smack to them about it!". There, I'm done. Probably the aspect of the game that keeps me coming back is that I never grew-up enough to not want to play hide-and-go-seek.

So, I thought I'd just run down a list of games that stand out in my mind:

  • X-COM ( UFO Defense, Terror From The Deep, Apocolypse )
  • Total Annilihation
  • WipeOut for the Playstation
  • Half-Life
  • Unreal Tournament 2001
  • MechWarrior 3 / 4
  • Original War ( check Fry's for this $8 RTS jem ).
  • Command & Conquer Red Alert 2
  • Grand Theft Auto 3 & Grand Theft Auto: Vice City
  • SOCOM & SOCOM 2
  • Command & Conquer: Generals

As you might have noticed, most of the games on this list are either console games or "old school", with the exception of C&C: Generals. It wasn't until recently that I had a machine capable of running that game ( a new Dell Inspiron 8600, but I was saving that for Friday's entry ). My buying habits have also changed WRT games. I remember buying WipeOut for the original Playstation days before I owned a Playstation to play it on ( or maybe it was WarHawk, that was a long time ago ). Now I shop the bargain bins for year old games which have a chance of running on the hardware I have now.

-ejo

(2004-07-21 13:42:20.0) Permalink Comments [2]

20040720 Tuesday July 20, 2004

LD_PRELOAD, or how I added timestamps to the output of anything.

A pet peeve of mine forever has been the fact that the *stat family of tools don't timestamp their output. Oh sure, iostat has been granted a timestamp but vmstat, mpstat and the rest were left out in the cold. Yes, you could write a wrapper script which time stamps the beginning and the end of a ream of vmstat output but you still have to work to figure out when an event occurred in the middle of a 11,111,111 line capture of performance data.

Given my need to explore the odd nooks and crannies for what is possible, I thought it might be fun to write an LD_PRELOAD shared object which interjected a timestamp at the beginning of a line of output. All I'd need to do is provide a new printf and I'd be done right? Wrong!

After some messing about, it turned out that write(2) and _write() were the culprits that needed to be overridden to accomplish the mission. It also turned out that I needed to do some line buffering since I wanted the timestamp to be nice and tidy in column 1 of the output. For those of you with access to the internal network, you can find the fruit of my labor here. Those of you who don't can contact me and if my manager approves I could make it more widely available.

An example using venerable vmstat:

schwa:stamp-1.1 -> vmstat 1
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 s6   in   sy   cs us sy id
 0 0 0 1729512 1154760 0 116 1  0  0  0  0  0  0  1  0  634 1594  717  1  1 97
 0 0 0 1648560 1030352 0 21  0  0  0  0  0  0  0  0  0  633 1323  466  0  0 100
 0 0 0 1648560 1030352 0 12  0  0  0  0  0  0  0  0  0  585 1157  425  0  0 100
 0 0 0 1648560 1030352 0 15  0  0  0  0  0  0  0 52  0  756 1143  492  0 21 79
 0 0 0 1648560 1030352 0 15  0  0  0  0  0  0  0  0  0  722 1658  509  0  0 100
^C
schwa:stamp-1.1 -> setenv LD_PRELOAD stamp.so.1
schwa:stamp-1.1 -> vmstat 1
14:21:28  kthr      memory            page            disk          faults      cpu
14:21:28  r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 s6   in   sy   cs us sy id
14:21:28  0 0 0 1729504 1154744 0 116 1  0  0  0  0  0  0  1  0  634 1594  717  1  1 97
14:21:29  0 0 0 1648480 1030272 0 23  0  0  0  0  0  0  0  0  0  591 1211  415  0  0 99
14:21:30  0 0 0 1648480 1030272 0 15  0  0  0  0  0  0  0  0  0  577 1179  413  0  1 99
14:21:31  0 0 0 1648480 1030272 0 12  0  0  0  0  0  0  0  0  0  582 1240  438  0  0 100
14:21:32  0 0 0 1648480 1030272 0 15  0  0  0  0  0  0  0  0  0  584 1150  404  0  0 100
14:21:33  0 0 0 1648480 1030272 0 15  0  0  0  0  0  0  0  0  0  644 1543  492  1  0 99
14:21:34  0 0 0 1648480 1030272 0 15  0  0  0  0  0  0  0  0  0  677 1353  477  0  0 100
14:21:35  0 0 0 1648480 1030272 0 18  0  0  0  0  0  0  0 27  0  843 1644  552  0  1 99
14:21:36  0 0 0 1647280 1030032 0 325 0  0  0  0  0  0  0  0  0  777 2056  687  8  0 92
14:21:37  0 0 0 1648480 1030272 0 16  0  0  0  0  0  0  0  0  0  705 1422  526  2  0 98
14:21:38  0 0 0 1648480 1030272 0 40  0  0  0  0  0  0  0  0  0  772 2260  723  3  0 97
^C

Admittedly, not horribly clever but it scratched an itch.

-ejo

(2004-07-20 12:22:43.0) Permalink Comments [1]

20040719 Monday July 19, 2004

Laminate Floors, Mitre Saws and Caulk

Over the Fourth of July shutdown, my wife badgered me into cleaning out the bedroom that serves as my office/machine room at home and replacing the icky carpet with Pergo. This room has lain fallow for many years and I'd grown used to the game trails that had to be navigated in order to get to the various important parts of the room, but my wife thought it needed attention. She has been watching the myriad of shows on TLC which show the miraclous transformation of a room within the confines of a one hour TV show. "It will be fun and easy", she said! "Besides, you don't want to just sit around and play SOCOM 2 for a week straight do you"?

The Purge

First order of business was to get rid of the years of cruft and dead machines that had accumulated. Turns out there 5 computers in there I hadn't turned on in a very long time: A Macintosh Powerbook 140 (still working), a NeXT Turbo Monochrome Cube ( the first comptuer I truly lusted after ), a Pentium Compaq icky thing that ran my wife's website, a SPARC ELC that reminded me of my time at school, and a dot.bomb cast-off PC that I never got around to refurbing. Austin has a super-cool Goodwill that accepts computer hardware, so that's where all the old hardware went. So that was one pick-up truck load of stuff gone out of the room.

Preparation P

After moving everything I was going to keep into the guest bedroom, I started the hard work. After ripping out the carpet, pad, baseboards and ceiling fan I went to Home Depot and dropped the customary wad of cash on paint and a new ceiling fan. Never miss an opportunity to paint is the mantra of a home owner, and I never paint without my Wagner Paint Mate. It is simple to use, precise and very easy to clean. Home Depot was having a sale on Behr paint, so I went with that. Three coats later, I wasn't particularly pleased with Behr but at least the painting was done. Blue semi-gloss on top of an off-white semi-gloss should have only taken two coats IMHO.

Pergo's Law

This room is the third room I've installed Pergo in, so I was feeling pretty confident. The first room was at my father-in-law's house under his experienced instruction, and the second room was my wife's office. Both times there was a error in the calculation of how many boxes of Pergo would be required to do the job. This time I was determined to get it right, but it was not to be. My careful calculations neglected to take into account some unavoidable wastage, and so I was off by the traditional one box. This leads to my formulation of Pergo's Law: However many boxes of Pergo you think the job will take, add one. The floor took a day to install up to the last course, and I finished it the next day after another trip to Lowes for the last box.

Finishing It Up

With the walls painted, a new spiffy ceiling fan and the floor in the room has a whole new vibe. I'm pretty happy with it and can't wait to put it all back together. Unfortunately, there is the small matter of the baseboards. I broke the second wall of baseboards when taking them off, so I was going to replace them instead of putting the old ones back up. Another trip to Home Depot nets pre-primed baseboards and a gallon of Valspar white paint. The weather was relatively nice, so I set up saw horses on the back patio and painted the baseboards in the shade. This being the last day of vacation, I decided to hold off until the next weekend to install the baseboards. Fast forward two weeks and the job still isn't done. I'm worried about the carpentry skills required to do the job passably and spend time reading about installing baseboards and draw out an extensive plan of all the cuts required. So this weekend I screw up my courage and get to work.

Two Things I Learned While Installing Baseboards

I lost the plan I had so carefully constructed, but I retained the overall idea of what need to happen. I set up the mitre saw, constructed a jig to stand the work off of the fence so I could cut all the way thru baseboards, and brought in a saw horse to help hold the 8' boards while I was working on the ends. And went to work. Fast forward 5 hours and I'm looking at a room with baseboards which appear to have been installed by a four-year old. My wife looked in and told me that if she'd hired me to do this work, she'd tell me to do it over again. I told her to get out. The first thing I learned was: I am a poor carpenter. I have the patience, but I'm not very good at measuring things consistently with a tape measure.

  "I've cut this board three times and it's still too short".
    - Every amateur carpenter since the dawn of woodworking.

The second thing I learned was: Caulk covers a multitude of sins. I started caulking the tops of the baseboards and filling gaps in corner joints and scarf joints and the appearance improved immensely. This time the wife said she'd probably let the job go, but wouldn't hire me again. She's a tough boss. After filling nail holes and painting, I began the slow arduous task of filling that room with crap again. I wish I'd taken before and after pictures.

-ejo (2004-07-19 09:10:13.0) Permalink Comments [3]

20040716 Friday July 16, 2004

Friday Gadget: XM Radio & the Delphi SKYFi Receiver

Being a geek, I love gadgets. I also love music. So XM Radio is a geek dream come true. For a small monthly fee and a modest investment in a XM reciver, you can listen to commercial free CD quality music almost anywhere. I've been an XM customer for about a year and half, and I'm completely spoiled.

The Gadget

I have a Delphi SKYFi XM receiver which is the embodiment of flexibility. I can listen to XM radio in the car, at home, and at work. All using the same receiver and a series of kits available from Delphi. As long as you can arrange for the antenna to see one of the two satellites ( Rock and Roll appropriately enough ), you can get XM. I personally have a home kit, a car kit, and a boombox. The home kit is.. at work. The car kit is installed in my truck and the boombox is used by my wife at home. Delphi has a new compact quasi-home stereo kit that my wife is drooling over. It reminds me of mid-80's Bang & Olefson designed CD players, which I guess is cool :)

The Pros

The Delphi SKYFi is just a nifty gadget.

  • Compact.
  • Sturdy construction.
  • Reasonable ergonomics.
  • Flexible beyond expectations.
  • Did I mention over 100 channels of commercial free music?

The Cons

As neat as the SKYFi is, it's also a first generation product. So it has a few nits that need picking:

  • The display is bright and clear, but has a limited viewing angle (important in the car). It also appears to be polarized, so polarized sunglasses cause the display to appear blank.
  • Song titles are often truncated due to a 20-odd character limitation, presumably a deficiency in the receiver and not the data transmitted by XM.
  • The antenna for the home kit has a tuned cable, so you lose signal strength and quality if you build an extention cable for it.
  • The big scrolly knob on the front of the unit makes things happen in a counter-intuitive way (for me). I turn the knob one way expecting things to scroll up or down on the screen and the opposite happens. This is bad in the car. Presets go a long way towards ameliorating that problem, so it doesn't often annoy me.

XM radio has totally spoiled me. The range and variety of programming is truely astounding, as well as the responsiveness of the XM staff to email and phone calls.

XM Radio and the Delphi SKYFi rate 10/10 in my book.

-ejo

(2004-07-16 07:45:51.0) Permalink

20040715 Thursday July 15, 2004

First Post!!! I'm so pumped!

I'm blogging like all the other cool kids!

I'm Erik, I live and work in Austin Texas. I am a serious UNIX nerd, and I geek out on observability issues. I worked for IBM for 8 or so years; first as a co-op student, then a contractor and finally a regular employee. I worked with AIX on multiple platforms during my time there ( PS/2, RS/6000 and the RT briefly ). I joined Sun in August of 1999 .

Things I Claim to Know Something About

  • Solaris Kernel and Virtual Memory
  • SPARCv9 User Trap Handling
  • Application Linking and Loading (ld.so)
  • ELF File Format
  • UNIX C Development Environment
  • emacs and emacs lisp
  • SPARC Assembly
  • X11/Xlib/Xt and Motif Application Development
  • TCP/IP Application Development
  • Pthread and Thread Application Development
  • Kstat Observability Development
  • Audit Library Development
  • MDB Debugger Command (dcmd) Development
  • Solaris Device Driver and Loadable Module Development
  • Kernel and User-land Application Debugging
  • General UNIX Hackery

When I'm not pounding on a keyboard, I like:

  • Competitive Pistol Shooting
  • Playing SOCOM 2 online, my nickname is "Jöhnny".
  • Slowly re-modeling my house.
  • Assembling PCs from random parts.
  • Napping with my dogs ( Sam and Sid ).

Wow this is harder than I thought, writing all this stuff. Maybe I'm not cut out for blogging!

-ejo

(2004-07-15 14:28:27.0) Permalink Comments [2]


archives
links
referers