Let the Sunshine In: Software support engineer in Tokyo

How many times have you sent e-mail prematurely on Thunderbird? It's got "Ctrl+Return" shortcut key assigned for "Send Now". It will pop up confirmation dialogue so people like me don't send empty mail. Good... But..., in the dialogue, the default focus is on "Send" button. So, careless people like me will hit 'Return' or 'Space' and send an empty mail anyway.

I tend to use Control key more than an average guy so "Ctrl+Return Return" sequence happens often. So, I decided to change Thunderbird behaviour. I thought I might need to edit Thunderbird binary file. But a little googling found this plugin.

Keyconfig extension - MozillaZine Knowledge Base

The Keyconfig extension (available here) allows you to configure keyboard shortcuts. It works with Firefox, Thunderbird, and (probably) Mozilla Suite/SeaMonkey.

Here's how I downloaded this plugin. I did this because somehow my Firefox couldn't handle this URL well.

curl http://mozilla.dorando.at/keyconfig.xpi > /tmp/keyconfig.xpi

This plugin will add GUI interface for key shortcut config. I just removed the "Ctrl+Return" shortcut. This added one line to prefs.js file under Profiles folder.

user_pref("keyconfig.main.key_send", "!][][");

Now, I'm happy I'm empty e-mail free :)

I've been using this for 2 months for day-to-day work, both on IE and Firefox. This works pretty reliably.

iMacros for Firefox - Script your Firefox Web Browser

Whatever you do with Firefox, iMacros can automate it.
iMacros for Firefox is 100% free for private and commercial use.

This is a sample which toggles Sun Java System Identity Manager 7.1's trace flag. iMacros has recording capability so all but 2 lines were automatically generated.

VERSION BUILD=6210326 RECORDER=FX
TAB T=1
URL GOTO=http://idmserver:8080/idm/login.jsp?nextPage=debug%2Fsession.jsp
' These 2 lines were added by hand becase iMacros didn't record these.
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:mainform ATTR=NAME:accountId CONTENT=configurator
TAG POS=1 TYPE=INPUT:PASSWORD FORM=NAME:mainform ATTR=NAME:password CONTENT=configurator
TAG POS=1 TYPE=INPUT:BUTTON FORM=NAME:mainform ATTR=VALUE:Login
TAG POS=1 TYPE=INPUT:BUTTON FORM=ACTION:http://idmserver:8080/idm/debug/session.jsp ATTR=NAME:Show_Trace&&VALUE:ShowTrace
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=NAME:formname ATTR=NAME:isLogging CONTENT=YES
TAG POS=1 TYPE=INPUT:BUTTON FORM=NAME:formname ATTR=NAME:Save&&VALUE:Save

I had been using Adobe Reader 8 to read and print out PDF files. I usually print 2 pages per sheet and I want to use all the paper real estate to make it readable. I want NO white space on paper. Usually, "Fit to Printable Area" option works OK. But for PDF files I downloaded today(actually, one of our product document), this didn't work. About 50% of the papar was blank.

So, I googled and learned that free Adobe Reader *might be* crippled in print out capability.
Print a pdf at larger than intended size? - Scot's Newsletter Forums

Without buying the full Adobe Professional version, I don't know any "hack" methods around that.

I googled further and found free PDF viewer which gives users free hand on print out scaling.
PDF Viewer - Software To Open A PDF File - PDF Editing Software

The FREE PDF-XChange Viewer offers all the feature's you would expect such as support of all PDF file formats from Version 1.0 all the way to the latest 1.7 (Acrobat 8)

Last but most important, you may accuse me of printing out PDF to paper in the first place. Yeah, I know. There is  AmazonKindle. But I doubt this is worth the money outside of US. There are other similar devices from SONY and one from UK company and another from German company.

I recently downloaded bunch of StarOffice presentations. I view these on 2nd display and I want them to autoplay. These are XML files. So, I thought I could write XSLT but I'm not good at it. Instead, I thought I could use xml diff tool to apply the change to one file to all other files in batch.



Here are the xml diff/patch tools I tried. The bottom line is: I couldn't make none of these achieve what I want.

1. xmldiff/xmlrev
   

   a python tool that figures out the differences between two similar XML files
   

   It turned out that xmlrev hasn't implemented 'INSERT' yet.
2. Microsoft XmlDiff
   patch tool checks checksum of original file. So, the patch can be applied to only 1 file.
3. diffxml - XML Diff and Patch Utilities
   

   This project aims to provide Open Source XML diff and patch utilities which operate on the hierarchical structure of XML documents.

   This worked for some files but didn't work for files which have a little different structure.
4. Perl: xupdate - search.cpan.org
   

   xupdate - Process XUpdate commands over an XML document

   This page explains well. But it didn't work on OpenDocument xml.
5. another Python tool: 4suite.org
   

   command-line tool for performing XUpdates on XML documents

   I used the same xupdate output from #1.

So, what will I do next? I'm considering these.

1. OpenOffice API or macro
2. Disregard the xml aspect of content.xml file and process it by perl
3. Use OpenOffice XML tool from here and write XSL
   Xml - OpenOffice.org Wiki
   

These are the Wiki pages of the OpenOffice.org XML Project

It is possible on Solaris with the help from dtrace(with a caveat). On Linux/Windows, I don't think this is possible with off-the-shelf kernel. That's why I wrote perl in my previous entry. It can't associate short-lived socket to the owning process. So, I'm still not sure if it was worth my time.

Here's a man page from DTraceToolkit. Notice the caveat, i.e.) this only captures new connections.

$ man -M ~root/DTraceToolkit-0.99/Man tcpsnoop_snv
[...]
DESCRIPTION
     This analyses TCP network packets and prints the responsible
     PID  and  UID,  plus standard details such as IP address and
     port. This captures traffic of newly created TCP connections
     that were established while this program was running. It can
     help identify which processes is causing TCP traffic.

Here's how I show network traffic and owner process on Windows using the perl in my previous entry. This can't show sockets already closed and dis-owned. Also, I sorted by # of bytes rather than by # of packets.

$ perl /c/d/bin/active-port.pl 5 200 | sort -t $'\t' -k 3nr \
> | while read line; do
>   echo $line
>   pid=$(netstat -ano | awk '$2 == "'$(echo $line|cut -d' ' -f3)'" {print substr($0,72)}')
>   [ "$pid" != "" ] && tasklist /v /fi "PID eq $pid" | tail -1 | cut -c1-18,67-76,93-120,150-
> done
50 packets 220.16.82.26:2187 75700 bytes.
wmplayer.exe        10,828 K MY_W2K3\Administrator      0:00:20 Windows Media Player   
                                                 
81 packets other, e.g.) arp 68460 bytes.
41 packets 220.16.82.26:2255 29658 bytes.
firefox.exe        161,272 K MY_W2K3\Administrator      0:34:03 Let the Sunshine In - Mozilla Firefox   
                                
16 packets 220.16.82.26:2256 8475 bytes.
6 packets 220.16.82.26:137 660 bytes.
System                 264 K NT AUTHORITY\SYSTEM        0:36:29 N/A           
                                                          
4 packets 220.16.82.26:2144 364 bytes.
thunderbird.exe    111,220 K MY_W2K3\Administrator      0:16:12 Inbox : MyName@MyMail.COM - Thunderbird    
                    
2 packets 220.16.82.26:61466 298 bytes.

Here's how I show network traffic and owning process on Linux. I only show top 4. One restriction on Linux is that you can't use 'any' interface. Please specify real interface like 'eth0'.

$ sudo perl active-port.pl 1 1000 | head -4 \
> | while read line; do
>   echo $line;
>   echo $line|awk '$3 !~ "^o" {system("lsof -i@" _ $3)}' | tail -1
>   echo
> done
901 packets 129.158.21.152:32772 68016 bytes.
synergyc 3789 kinoue    4u  IPv4   9356       TCP myserver.jp.sun.com:32772->dhcp-jp-20-88.Jpn.Sun.COM:24800 (ESTABLISHED)

77 packets 129.158.21.152:32783 82222 bytes.
ftp     4109 kinoue    4u  IPv4  12469       TCP myserver.jp.sun.com:32783->labserv.Jpn.Sun.COM:53456 (ESTABLISHED)

8 packets other, e.g.) arp 480 bytes.

8 packets other, e.g.)UDP broadcast 1027 bytes.

Ever wanted to know which process is using the network bandwidth most? I googled and learned that, on Linux,  iftop seems to be able to tell which port is the busiest. But 'iftop' doesn't seems to be compilable on cygwin. So, I googled more and found several similar Windows GUI apps. But they are all shareware and they are overkill for my purpose anyway. So, I wrote a perl to parse tcpdump output to aggregate by local ports.

On Windows, it requires 'windump'.
WinDump: tcpdump for Windows

WinDump is the Windows version of tcpdump, the command line network analyzer for UNIX. WinDump is fully compatible with tcpdump

The output will look like this.

$ perl /c/e/tmp/active-port.pl 5 1000
962 packets     other, e.g.) arp                57850   bytes.
10 packets      220.14.81.26:1275               4526    bytes.
10 packets      220.14.81.26:1068               906     bytes.
10 packets      220.14.81.26:1256               570     bytes.
4 packets       220.14.81.26:1273               228     bytes.
2 packets       220.14.81.26:21         128     bytes.
1 packets       220.14.81.26:1026               636     bytes.
1 packets       other, e.g.)UDP broadcast               590     bytes.

Here's perl script. Borrowed code from here and here.

#!/usr/bin/perl -w
#
# Parse tcpdump and say which ports are more active
# Usage example: perl active-port.pl '\\Device\\NPF_{55899BD0-EC52-4066-ABA7-53849ACAA19A}' 100 | sort -t $'\t' -k 3nr
use strict;

if ($#ARGV != 1) {
    die "$0: Need exactly two arguments. Interface name/# and packet count to capture.\n";
}
my $tcpdump = "tcpdump";
my $index=0, my @local_ipaddrs, my $address;

if (defined $ENV{OS} && $ENV{OS} =~ "Windows_NT") {
    use Sys::Hostname;
    $tcpdump = "windump";
    my ($name,$aliases,$addrtype,$length,@addrs) = gethostbyname hostname;
    foreach $address (@addrs) {
        my ($a,$b,$c,$d) = unpack('C4',$address);
        $local_ipaddrs[$index++] = "$a.$b.$c.$d";
    }
} else {
    require IO::Interface::Simple;
    my @interfaces = IO::Interface::Simple->interfaces;
  loop: for my $if (@interfaces) {
      next loop if ( !defined $if->address );
      $local_ipaddrs[$index++] = $if->address;
  }
}

open(TCPDUMP, "$tcpdump -i $ARGV[0] -c $ARGV[1] -nntqep 2>/dev/null |");
eof(TCPDUMP) && die "$0: Could not run $tcpdump: $?\n";
my %activity, my %bytes;
while (<TCPDUMP>) {
    my ($link_level_length, $rest) = ($_ =~ /^[0-9a-f:]{17}\s>\s[0-9a-f:]{17},\s\S+,\slength\s(\d+):\s(.*)/);
    if (my ($addrport1,$addrport2) = ($rest =~ /(\d+\.\d+\.\d+\.\d+\.\d+)\s>\s(\d+\.\d+\.\d+\.\d+\.\d+):/)) {
        my $local;
      local_ipaddrses: foreach $address (@local_ipaddrs) {
          if ($addrport1 =~ /^$address\./) {
              $addrport1 =~ s/(.*)\./$1:/;
              $activity{$addrport1}++; $bytes{$addrport1}+=$link_level_length;
              $local = "true";
              last local_ipaddrses;
          }
      }
      local_ipaddrses: foreach $address (@local_ipaddrs) {
          if ($addrport2 =~ /^$address\./) {
              $addrport2 =~ s/(.*)\./$1:/;
              $activity{$addrport2}++; $bytes{$addrport2}+=$link_level_length;
              $local = "true";
              last local_ipaddrses;
          }
      }
        if (!defined $local) {
            $activity{"other, e.g.)UDP broadcast"}++; $bytes{"other, e.g.)UDP broadcast"}+=$link_level_length;
        }
    }
    else {
        $activity{"other, e.g.) arp"}++; $bytes{"other, e.g.) arp"}+=$link_level_length;
    }
}

foreach my $addr_n_port ( sort { $activity{$b} <=> $activity{$a} }  keys %activity) {
    print "$activity{$addr_n_port} packets\t$addr_n_port\t\t$bytes{$addr_n_port}\tbytes.\n";
}
exit 0;

There seems to be 4 profilers that's known to work with Java CAPS 5.

1. NetBeans Profiler
   
2. JProfiler
3. OptimizeIt
4. Wily
   

I have only seen NetBeans Profiler profiling Java CAPS 5.1.3. It takes a little while to setup but its interface is intuitive and it's versatile.

Yesterday, I found a 'lighter' profiler(sort of) while browsing blogs.sun.com so I tried it on my Java CAPS 5.1.3.

An application profiler using an embedded AspectJ based Java agent - Adventures in Java

Some applications of this facility that immediately come to mind are method tracing, method logging, heap tracing (dynamically tracing the amount of heap space used by an application as it runs), application health monitoring (trends of method invocations resulting in uncaught exceptions vs. “normal” execution), etc.. At development time, this framework can provide lightweight profiling capabilities.

I edited domain.xml file to change java options and restarted. And..., it worked! I could measure time spent in methods in particular classes.

Once Subhajit posts the source code, you can write your own 'lightweight' profiler.

Maybe I reinvented the wheel. Here's quick&dirty perl to convert 'truss -df' output. Unfortunately, this doesn't work if only "-d" is given. Here's how this works:

$ truss -df cat
Base time stamp:  1238745134.0279  [ Fri Apr  3 16:52:14 JST 2009 ]
16881:   0.0000 execve("/usr/bin/cat", 0xFFBFFDC4, 0xFFBFFDCC)  argc = 1
16881:   0.0051 resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12
16881:   0.0055 resolvepath("/usr/bin/cat", "/usr/bin/cat", 1023) = 12
16881:   0.0058 stat("/usr/bin/cat", 0xFFBFFBA0)                = 0
[...]

$ truss -d -f cat 2>&1 | ~kinoue/trusstimestamp.pl
Base time stamp:  1238745453.9931  [ Fri Apr  3 16:57:33 JST 2009 ]
16895:   16:57:33.9931  execve("/usr/bin/cat", 0xFFBFFDC4, 0xFFBFFDCC)  argc = 1
16895:   16:57:33.9996  resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12
16895:   16:57:34.0018  resolvepath("/usr/bin/cat", "/usr/bin/cat", 1023) = 12
16895:   16:57:34.0024  stat("/usr/bin/cat", 0xFFBFFBA0)                = 0
[...]

I hope this isn't necessary with truss on OpenSolaris. Linux's strace can do this. So, I think this has been on enhancement list. Here's perl.

#!/usr/bin/env perl
line: while (<>) {
    if ($. == 1) {
        @Fld = split(' ', $_, -1);
        $base = $Fld[3];
        print $_;
        next line;
    }
    if (m/^([0-9\/]*:[\t ]*)([0-9.]*)([\t ].*)$/o) {
        unless ($2) {
            print $_;
            next line;
        }
        my $seconds = $base + $2;
        ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($seconds);
        $subsec = substr($seconds, index($seconds, '.'));
        printf ("%s%02d:%02d:%02d%s %s\n",$1,$hour,$min,$sec,$subsec,$3);
        next line;
    }
}