Let the Sunshine In: Software support engineer in Tokyo

I wanted to check whether our business critical web server is up in every 6 hours.

First I thought I run ping every 6 hours via 'cron'. But I want to ping more often once server is detected down and until it comes back up.

So, I came up with this self registering 'at' script. This begins pinging the server in incremental backoff intervals once it's detected down. Starts at 1 minute, and then 2 minutes. 3,4,5... If you replace "+1" wit "*2", it will do exponential backoff. 1,2,4,8,16...

$ cat ~/misc/myServerPing.at
# THISFILE should be full path or relative from $HOME
# Run this in bash by ". {this file}"
THISFILE=misc/myServerPing.at
INTERVAL=1

curl --silent --connect-timeout 8 http://ourserver.sun.com | grep "Our critical page" > /dev/null

if [ $? -ne 0 ]; then
  date | mailx -s "Failed ping to ourserver" my.mail.address@sun.com
  sed "s/^\(INTERVAL=\)[1-9]*$/\\1$(($INTERVAL+1))/" $THISFILE | at now + $INTERVAL minutes > /dev/null 2>&1
else
  at now + 360 minutes < $THISFILE > /dev/null 2>&1
fi

先日UQ WiMAXの以下の端末が届いたので公式にはサポートされないWindows 2003でドライバーがインストールでき通信できるか試してみました。

UQ WiMAX | WiMAX USB TYPE UD01NA

# 製 品 名    UD01NA
# 製 造 元    NECアクセステクニカ株式会社

今のところ問題なく動いています。ドライバーをインストールために以下の変更をしましたが本当に必要だったかどうか定かではありません。

$ diff $SYSTEMDRIVE/DRV/UD01NA/drxvi314.inf{.orig,}
10c10
< signature="$CHICAGO$"
---
> signature="$Windows NT$"

I ran into this problem while running this "Stand-alone Eclipse RCP".

SAP Community Network Forums: Unable to open hprof file on Debian ...

on Solaris (and presumably Linux), this is because of an Eclipse bug -- the org.eclipse.core.internal.filesystem.local.LocalFileNatives.internalGetFileInfo() method calls the system lstat() function, which is limited to 2GB files

I couldn't find a patch for this bug but I guess fix is somewhere since this problem has been known for more than a year.
Luckily, for this particular app, 2 work arounds are available which is explained in the SAP forum above.
I tried 1st w/a of giving java heap dump file name on command line but I'm not sure it worked. I'm now testing 2nd w/a of hacking snapshotHistory.ser and it seems to work.
Maybe I should have used jhat or NetBeans to begin with.

Here's info of Eclipse native library called via JNI. The output only has 'lstat' but not 'lstat64'. I don't know if this proves Eclipse bug.

$ pldd $(pgrep MemoryAnalyzer) | grep eclipse | while read line; do nm $line | ggrep -E '\|lstat|internalGetFileInfo\>' && echo $line ;done
[72]    |      4732|     296|FUNC |GLOB |0    |9      |Java_org_eclipse_core_internal_filesystem_local_LocalFileNatives_internalGetFileInfo
[87]    |         0|       0|FUNC |GLOB |0    |UNDEF  |lstat
/mat/configuration/org.eclipse.osgi/bundles/69/1/.cp/os/solaris/sparc/liblocalfile_1_0_0.so

Windows上で作成されたzipをUN*X上でどうしても展開できなかったことはないでしょうか？ざっとググってみましたがUbuntuでの対応が早かったようです。JavaとSolarisでの対応を確認してみました。

Javaでの対応は以下のブログに説明されています。jarコマンドでの対応はまだ確認できてませんが以下ブログ中のソースをJDK7でコンパイル、実行すれば動作確認できます。

Non-UTF-8 encoding in ZIP file : Xueming Shen's Blog

zip is a stripped-down version of the Jar tool with a "-encoding" option to support non-UTF8 encoding for entry name and comment

/usr/bin/unzip での対応はOpenSolarisに入っています。私はOpenSolarisから/usr/bin/unzipファイルをSolaris10にコピーし動作確認してみました。

Bug ID: 6719511 unzip should support non ascii file name

For example, convertiong Japanese sjis, % unzip -O sjis .zip

以下に登録されているテスト用zipファイルでテストしたところ、両方ともSJIS日本語ファイル名のファイルを展開できましたが違う結果となりました。Javaでは抽出されたファイル名のエンコーディングがLANG環境変数の設定に一致しました。一方unzipではシフトJIS固定のようでした。特殊文字が含まれている場合、/usr/bin/unzipを使う方が良いかもしれません。

Unzip 6.0 is missing option -O - Info-ZIP Discussion Forum

Option -O that allows you to set an encoding for filenames is missing in the latest release.
To test I made a small zip file in Windows XP that has filenames encoded in shift-jis and tried to open it in Linux in UTF8 environment.

Windows has an interesting way of starting debugger on process inception.

greggm's WebLog : Inside 'Image File Execution Options' debugging

There are times that you need to debug the startup code for an application, but something else is launching the application.

To do this on Solaris/Linux is another interesting matter. Can dtrace be used? Or do we still use the classic way of replacing binary file with shell scripts? I'll leave it to experts for now.

Back to Windows.. This feature can be used in various good & bad way(e.g. virus) other than debugging. Today, I used this feature to pass an option to java.exe.

There are many applications that spawns "java.exe" and there often is no way to pass java flags. So, I think it's not only me who ever wished injecting java option dynamically, like -XMX. I'll probably write another blog after polishing this script and elaborate how I did for java.exe.

For today, I'll show a sample with "cmd.exe". With "/T:fc" option, cmd.exe shows up on red on white. I know this is a very lame example. But I think this is much safer to try than java.exe example. So..., here goes. Save this as *.vbs file and go ahead with registry editor. To test, run "cmd /k dir" from Windows menu, with system() call in perl, no matter how.

'ToDo: There should be 2 run mode.
'  1. Run from command line with .exe name and injection string passed in as argument.
'     Create registry key(*.exe) and create "Debugger" value under it.
'  2. Run via IFEO. This part is complete.
'
Option Explicit

Dim oShell, cmdLine, i
Set oShell = WScript.CreateObject("WScript.Shell")

'cmdLine = "cmd.exe /K " & WScript.Arguments( 1 ) & " "  'For those programs interacting with stdin/stdout
cmdLine = WScript.Arguments( 1 ) & " "
cmdLine = cmdLine & WScript.Arguments( 0 ) & " "
For i = 2 to WScript.Arguments.Length-1
  cmdLine = cmdLine & WScript.Arguments(i) & " "
Next

oShell.RegDelete "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\cmd.exe\Debugger"
oShell.Run cmdLine,,true   'setting bWaitOnReturn flag to "false" is dangerous!
oShell.RegWrite "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\cmd.exe\Debugger", _
                "cscript //NoLogo " & WScript.ScriptFullName & " /T:fc"
WScript.Quit

I wanted to run Wireshark on Solaris10 where I don't have root privelege. Wireshark itself is on blastwave already.

Blastwave.org - An OpenSolaris Community Site

description Wireshark (was Ethereal) is a free network protocol analyzer
vendor url http://www.wireshark.org/

Good! But it has 35 dependencies. How can I extract those dynamic link libraries(*.so files) under my home directory efficiently?

Here, a sh script which I wrote a year ago helps. It doesn't even use pkg-get or pkgutil. But..., my script is for downloading packages only. "*.so.*" files in the packages needs to be extracted somewhere.

I googled and I found a python code fragment to extract files from package and create bunch of symbolic links. So..., I thought if my sh script and this python code are combined, The whole process of downloading dependent packages and extracting to one's virtual root(~/csw) can be fully automated.

utils.py - release/pyutils - Code Search

for line in pkgmap:
 # Matching example:
 #1 s none lib/libglib-2.0.so=libglib-2.0.so.0.200.3
 try:
 (link, target) = re.compile('. . .*? (.*)=(.*)').search(line).groups()
 link = link.replace("/opt/csw/", "") # This is for blastwave packages, since they don't use relative paths
  os.symlink(target, link)

This time, I didn't use this python. Instead I used bash one-liners which roughly looks like this:

for i in *.pkg.gz
do
  gzcat $i | pkgtrans /dev/stdin . all
done
mkdir lib
mv CSW*/root/opt/csw/X11/lib/lib* CSW*/root/opt/csw/lib/lib* lib
cd lib
for i in *
do
  ln -s $i $(echo $i | gnused 's/\.[0-9]\+\.[0-9]\+$//')
done

BTW, this search result got me interested. Why the heck does Mono have to deal with Solaris package? I thought Mono is .net framework for Linux. So, I checked project's home page and learned I was wrong!

Mono:Solaris - Mono

Mono supports Solaris on SPARC, x86 and x86-64 architectures.

Packages for Mono on Solaris/SPARC are available from our Download page.

Mono is also available as part of Nexenta (http://www.nexenta.com), the Debian-based OpenSolaris (http://www.opensolaris.org) distribution.

Is it only in Japan? Is it because many of us office workers are not allowed to install tools like IrfanView or Gimp? Don't we all prefer to get image as *.png, *.jpg or in HTML mail?

Anyway, if you get image in newer MSOffice document format , such as *.xlsx *.docx *.pptx (as opposed to xls,doc,ppt), you may be lucky. These are just zip files, like OpenOffice files are.

This bash function worked to view all images in Excel file without cluttering my home directory. Tested on Solaris10 and Windows(Cygwin+X). This requires ImageMagick's display command.

imginxlsx () 
{ 
    unzip -l $1 | cut -c29- | grep -i --color '\.png$' | while read image; do
        unzip -p $1 $image | display -geometry +0+0 png:-;
    done
}

I don't think there are lot of applications that writes out gethrtime() return value in trace or log files. Oracle is one of them. So, I wrote a perl to convert those values in my previous blog. There are very visible caveats.

1. It doesn't run on other platforms.
2. Run it on the same machine you took the trace.
3. Run it before you reboot. Because gethrtime() counter starts on boot time.
4. You want to run it right after taking Oracle sql trace. Because:
1. gethrtime() counter rolls over to 0 after 497 days.
2. OS timestamp will drift for several reasons. e.g.) ntp kicking in.

Let's see how the perl works on one of my SPARC server.

1. oracle sql trace output fragment today

   *** SESSION ID:(491.26184) 2009-07-23 23:25:45.437
   =====================
   PARSING IN CURSOR #1 len=109 dep=0 uid=65 oct=3 lid=65 tim=3113736335727 hv=2518377154 ad='b9ccd050'
   SELECT timestamp, id, version, changeType FROM objchange WHERE timestamp > :1  AND type=:2 ORDER BY timestamp
   END OF STMT
   *** 2009-07-23 23:25:55.456
   WAIT #0: nam='SQL*Net message from client' ela= 8 driver id=1297371904 #bytes=1 p3=0 obj#=-1 tim=3113746119816
   

   Run it throught the perl.

   $ perl ~kinoue/oracle.Trace.Tim.pl /tmp/orcl_s000_11620.trc.2
   *** SESSION ID:(491.26184) 2009-07-23 23:25:45.437
   =====================
   PARSING IN CURSOR #1 len=109 dep=0 uid=65 oct=3 lid=65 tim=2009-07-23 23:25:45.43776  hv=2518377154 ad='b9ccd050'
   SELECT timestamp, id, version, changeType FROM objchange WHERE timestamp > :1  AND type=:2 ORDER BY timestamp
   END OF STMT
   *** 2009-07-23 23:25:55.456
   WAIT #0: nam='SQL*Net message from client' ela= 8 driver id=1297371904 #bytes=1 p3=0 obj#=-1 tim=2009-07-23 23:25:55.45667
   

2. oracle sql trace output fragment from 11 days ago

   *** SESSION ID:(481.17143) 2009-07-12 04:22:39.236
   =====================
   PARSING IN CURSOR #6 len=144 dep=0 uid=65 oct=3 lid=65 tim=2118622857650 hv=41747736 ad='b9cb2e10'
   SELECT task.id, task.type, name, '', '', summary, '', '', xmlSize  FROM task WHERE task.type='TaskInstance'
   and task.attr1='READY' order by name
   

   Run it throught the perl.

   *** SESSION ID:(481.17143) 2009-07-12 04:22:39.236
   =====================
   PARSING IN CURSOR #6 len=144 dep=0 uid=65 oct=3 lid=65 tim=2009-07-12 04:22:29.23621  hv=41747736 ad='b9cb2e10'
   SELECT task.id, task.type, name, '', '', summary, '', '', xmlSize  FROM task WHERE task.type='TaskInstance'
   and task.attr1='READY' order by name
   

3. oracle sql trace output fragment from 21 days ago

   *** SESSION ID:(492.30132) 2009-07-02 14:26:52.437
   =====================
   PARSING IN CURSOR #1 len=109 dep=0 uid=65 oct=3 lid=65 tim=1310268562358 hv=2518377154 ad='b9ccd050'
   SELECT timestamp, id, version, changeType FROM objchange WHERE timestamp > :1  AND type=:2 ORDER BY timestamp
   

   Run it throught the perl.

   *** SESSION ID:(492.30132) 2009-07-02 14:26:52.437
   =====================
   PARSING IN CURSOR #1 len=109 dep=0 uid=65 oct=3 lid=65 tim=2009-07-02 14:26:34.43783  hv=2518377154 ad='b9ccd050'
   SELECT timestamp, id, version, changeType FROM objchange WHERE timestamp > :1  AND type=:2 ORDER BY timestamp
   

My machine is not set up for ntp client. Yet, OS timestamp seems to constantly drift for certain seconds. Probably, the drift is caused by machine's internal hardware clock.