Let the Sunshine In: Software support engineer in Tokyo

In my last entry, I uploaded a Google Map data which has way too many 'point's. So, I think it made browsers stop for upto 10 seconds to load all of them. I apologize for that.

So, now I want to 'prune' some points from that JavaScript part in the html generated by SportTracks FTPPlugin. While I'm googling, I stumbled upon a open source Java program called 'Prune'.

Activity Workshop - Prune

is an application for viewing, editing and converting coordinate data from GPS systems. Basically it's a tool to let you play with your GPS data after you get home from your trip.

Now, I want to load point data to Prune. I could have extracted those latitude/longtitude data from directly off MySQL to a file. But.., call me lazy and unloyal, I just used 'sed' on the html. Well, maybe Prune will have JDBC interface some day.

$ sed -n 's/^.*pts.*new GLatLng(\([0-9.]*\), \([0-9.]*\)).*$/\1,\2/p' /tmp/half-marathon.html > /tmp/points.data

Here's a screenshot. I can manually pick points to delete. Or let Prune do some 'intelligent' pruning.

Among the ERPs our Java CAPS customers use are SAP and J.D.Edwards(JDE). So, I know for years that both ERPs have so called 'Z objects'.  e.g.) Z table, Z events. Then, I recently learned that there's 'Z-segments' in HL7 standard as well, which our Java CAPS product is also good at handling.

So, today, I googled to know who started to use letter 'Z', whether SAP or JDE borrowed the idea from another...
I came to the understandings below what 'Z' means in each.

1. SAP R/3
   'Z' is a namespace for user-developed, as opposed to system(SAP R/3) provided,  stuffs.
   
   Yes, it is just 1 byte "Z". It is not necessary to follow it by a delimeter like '.' or '/', etc.
   Although in real world, I saw SAP developers using '_' as a delimeter to name things,
   e.g.) Z_BAPI_ACCOUNT_ADD .
   There's even a company who decided to name themselves Z*.
   
   
2. J.D. Edwards
   Unlike SAP, 'Z'* in JDE is system provided. And unlike SAP, 'Z' appears toward the end of object names, like 'F4311Z1'. "Z table" is kind of temporary area where data is loaded before they are batched into production table,
   ala Oracle Application's open interface table.
   Discussions abount Z-* on it.toolbox.com.
   
   
3. HL7
   What Are Z Segments?
   

   it is the “catch all” for data that does not fit into the HL7 Standard message definitions

While I couldn't find the answer as to who started to use letter 'Z', I think this started in mainframe era where there was no liberty of creating directories and when files are only sorted on alphabetical order. I found a trace here.

Customizing the IMS Environment

These customization jobs copy the load modules that contain your exit routines from your load library to the CYN.SCYNAUT3 library, and rename them by replacing the first letter "D" with the letter "Z".

As if HP-UX was not enough, I've been busy with mainframe customer issue. So, I had to look at several COBOL codes found by Google.

Since when don't COBOL source lines need to be prefixed with line numbers? COBOL sources I find internally here are all prefixed so I wrote a small script to take diff. It ignores case differences, white space, comment lines.

$ cat coboldiff.sh
#!/bin/bash
file1=$(echo $1 | sed 's|/|_|g')
file2=$(echo $2 | sed 's|/|_|g')
cut -c7-72 $1 | grep -v '^\*' > /tmp/$file1
cut -c7-72 $2 | grep -v '^\*' > /tmp/$file2
diff -Bicbw /tmp/$file1 /tmp/$file2

Many of our customers run our software on HP-UX. I had a call from one such customer recently and I had to reproduce the problem on in-house HP-UX.

I chose to use a new HP-UX machine on remote location. After a  few days, I noticed bash behaves a little differently. When I type in long command line, it scrolls the line to the left making head of the line invisible. This is not desirable because it makes copy&paste impossible. So, I started investigating.

First thing I found is that this feature is called 'horizontal-scroll-mode'.

$ man bash 2>/dev/null | ggrep -A 2 scroll
     horizontal-scroll-mode (Off)
          When set to On, makes readline use a  single  line  for
          display,  scrolling  the input horizontally on a single
          screen line when it  becomes  longer  than  the  screen
          width rather than wrapping to a new line.

So, I flipped the setting but it made no difference. It was set to 'off' to begin with, anyway.

$ bind -v | grep scroll
set horizontal-scroll-mode off
$ bind 'set horizontal-scroll-mode on'
$ bind -v | grep scroll
set horizontal-scroll-mode on
$ bind 'set horizontal-scroll-mode off'
$ bind -v | grep scroll
set horizontal-scroll-mode off
$
<a pretty long command line. This is a pretty long command line. This is a pretty long command line.

So, I googled and found that this is related to termcap/terminfo setting. On HP-UX, this capability seems to be called 'cuu1'.

$ man terminfo 2>/dev/null | grep cuu1
 cursor_up                 cuu1    up   Upline (cursor up)

It is defined in terminfo and for current session. It works, too.

$ echo $TERM
xterm
$ untic | grep -w cuu1
        cnorm=\E[?25h, cuf1=\E[C, cuu1=\E[A, cvvis=\E[?25h,

$ tput cuu1 | more
^[[A

In desperation, I logged on older HP-UX machine and checked. It worked! I compared 'untic' output but there's no difference.

So, I started to suspect bash bug and checked version on both machines.

1. older HP-UX
   

   $ bash --version
   GNU bash, version 3.2.0(2)-release (hppa2.0w-hp-hpux11.11)
   Copyright (C) 2005 Free Software Foundation, Inc.

2. newer HP-UX
   

   $ bash --version
   GNU bash, version 3.00.16(1)-release (hppa2.0w-hp-hpux11.23)
   Copyright (C) 2004 Free Software Foundation, Inc.

   
   

Yeah, this was it. Newer HP-UX had older version bash. Probably, bash 3.0 had HP-UX specific bug. I can now copy&paste happily, as I did in this entry. :)

本屋でこの特集を見つけ立ち読みしました。この雑誌はLinuxの記事が多く、個人ユーザの読者も多いと思っていたんですが、この特集に関してはずいぶんと企業のWindowsユーザ向けに書いてあるなと思いました。
Software Design 2008年12月号｜gihyo.jp … 技術評論社

特集
SD流 仮想化技術フルコース【マイグレーション編】

マイグレーションということでp2vの説明もあります。私がここに書いたVMware社のツール以外の商用製品も紹介されていました。
PlateSpin Products - PowerConvert

move and rebalance workloads in any direction between physical and virtual hosts – physical-to-virtual, virtual-to-physical, virtual-to-virtual, physical-to-physical, in and out of imaging formats

私が特集に期待していたのは、個人ユーザでもやると思われる、Linux の P2V でした。以下のようなフリーのバックアップツールが良く使われるのではと思います。
Mondo Rescue - GPL disaster recovery solution

Mondo Rescue is a GPL disaster recovery solution. It supports Linux (i386, x86_64, ia64) and FreeBSD (i386).

ググると以下のような個人ユーザがp2vした例はたくさん見つかります。
P2V using Mondo Rescue - madness.at

with Mondo Rescue, a Debian Etch box with two IDE disks (/dev/hda, /dev/hdc), Linux software raid (/dev/md0) and VMWare the process eventually goes like this:

Solaris x86 のp2vはどうでしょう。私が以前やってみたのは何回かに分けてこのブログに書きました。さすがに私だけではないだろうと思い、ググったところ見つかりました。企業ユーザの例のようです。
P2V - Solaris x86 and Linux workloads | Furoris News

we have so far virtualised (P2V'd) a number of Linux/Unix servers; Scientific Linux, Red Hat; Ubuntu and Sun Solaris 10

I found an jstack() line in one of DTraceToolkit script.

DTraceToolkit at OpenSolaris.org

DTraceToolkit is a collection of useful documented scripts developed by the OpenSolaris DTrace community

$ ggrep -B 4 -A 1 jstack DTraceToolkit-0.99/Proc/crash.d
proc:::signal-send
/self->crash && execname == "java"/
{
        printf("\nJava Stack Backtrace,");
        jstack();
}

Here's from man page.

$ man -M DTraceToolkit-0.99/Man crash.d | ggrep -A 1 -B 1 SIG
Reformatting page.  Please Wait... done
     crash.d monitors for applications that crash. When  a  crash
     via a SIGSEGV or SIGBUS is detected, a report of the process
     state is printed out.

So, I guess this script is supposed to dump java stacktrace when JVM crashes. Unfortunately, I couldn't confirm it really does. Maybe, I should look our bugdb harder how I can cause JVM to SIGSEGV.

Meanwhile, I experimented how I can do it on Windows. I think there are several ways. I'll explain here which I think is the safest way.

1. Download userdump from here and install.
   
2. Download SendSignal and install.
   SendSignal
   

   it dumps a list of threads and all their held locks to stdout.



3. Start "Process Dumper" from control panel. Add "java.exe" as an application to monitor.
   
   
   
   
4. Add custom rule to java.exe.
   
   1. Notice that I chose "Divide by Zero". SIGSEGV, SIGBUS equivalent on Windows is "Access Violation". For now, I choose "Divide by Zero"(SIGFPE equivalent) because causing "Access Violation" is harder.
      
   2. Notice that I set "Switch the Dumper" option and specified SendSignal.
      
      



5. Open NetBeans. Write "Divide by Zero" in try/catch block. Run.
   
   
   
   
6. Thread dump appears in "Output" window.
   

I love working on command line so history file is my knowledge base. I can't afford to lose it!

I knew that there's a new shell called Hotwire by a RedHat guy which saves history to embedded SQLite DB on every command execution.

Hotwire - A free object-oriented hypershell

Hotwire is an object-oriented hypershell.

But I want to do it in bash and haven't had no idea for several months. Then, recently, there's an interesting topic in internal mailing list discussing whether bash can show timestamp on every command invocation. There, I learned about 'trap'.

Call me a paranoia. These are the lines I added in .bash_profile. My bash session saves history in every 32nd command invocation.

historySaveViaTrap ()
{
    [ $((HISTCMD % 32)) -eq 0 ] && history -a
}
trap historySaveViaTrap DEBUG

This, however, didn't work on Cygwin. I googled a bit but couldn't find an answer.

I think this is the current status. Corrections and comments are welcome.

OS	Tool	space required	Boot device reconfiguration	Network device reconfiguration
Windows	VMware Converter	not much more than actual space used	automatic	automatic
Linux*	ext2/ext3 aware image backup tool	may require extra temporary space while restoring	boot with GRUB failsafe archive and reconfigure	manual
Solaris*	block by block HDD image backup tool	requires extra temporary space while restoring	GRUB. Doesn't require install media.	manual
others* (DOS,NetWare,etc)	file system aware or block by block image backup tool	depends on file system used	probably requires install media, additional driver disk.	may need to install driver

 

 

 

 

 

 

*) I omitted VMware Converter's Cold cloning (using a BootCD). It's $$ option and I haven't seen it. According to this FAQ,  the BootCD is Windows 2003 based WinPE (or BartPE?) and sounds quite flexible. How can it read Linux disk, then? I guess it has  ext2/ext3 driver installed and also capable of block-by-block backup.