Kazuhiko's weblog: Updates on my interests

Few days ago, I found a bug and looking into it which happens only on PCK locale. It happens only for specific Japanese character. That character is, I used, '本' at that time. Other Japanese strings can be shown correctly but it does show English strings instead of Japanese if strings has that specific character.

The root cause was that the program uses '{' ascii character for searching in C function.
The Japanese character '本' is encoded as below.

#root@mamushi: echo 本 | od -t x1 -C
0000000  96  7b  0a
          本  **  \n

On the other hand, '{' is encoded as below.

#root@mamushi: echo '{' | od -t x1 -C
0000000  7b  0a
           {  \n
           0000002

These are encoded with same 0x7b byte value. The program handles strings as byte one by one and compare each byte with '{'. In the other Japanese encoding, EUC and UTF-8, ASCII characters can be handled with no care. There is no duplicated values with ASCII in Japanese multibyte character. So bug was not found in these two locales. PCK encoding interprets second byte based on first byte value and this second byte(even if it's ascii value) shows multibyte chararacter with first byte. Then program handles strings differently from what it intended.

In general, lot's of programs for command line does not do formatting or such strings operation, so program works fine with raw byte sequences instead of wide character strings. But if it handles characters one by one, strings are needed to convert wide character for that function.

Now there is discussion about Project Colorad in ha-clusters under OpenSolaris. It targets OpenSolaris release like OpenSolaris 2008.05 which based on IPS packaging system.

It's difficult to have cluster machine environment personally, so this project plans to update cluster to make it work on light weight environment. It looks to update the cluster to work without physical dedicated interconnecct,and update the quorum disk functionality to work without multi-hosted disks. There is plan to update fencing functionality, which prevent failed cluster node access to the shared disk. This fencing functionality seems to be optional.

Please see Project Colorad page for detailed information.

I've experienced some cases in my work that translator uses unicode character set(using UTF-8) in their translation and use 'oe' in one character as U+0153 code, but there is not corresponding character in 8859-1, Lattin-1 character set. Though, this, one character 'oe' is better for native speakers.

Of course, there are huge number of characters which 8859-1 does not have, but this is one of the good example that we need to take care at translation. If we use Unicode message file to show messages on other smaller charset locales, there would be conversion error on such locale at runtime.

I converted it to 'o' and 'e' with translator's advice in source side at that time.

PxFS introduction is described in SUN CLUSTER OASIS blog(Jul 11, 2008) . Because cluster core is now released as open source, there are links to actual code location with explanation. I think it's very helpful to understand what happens on cluster global file system.

I hope there will be more such articles on blog to be familiar to what's happened on software.

Evaluation about next OpenSolaris web site platform is described in these pages.(evaluation of XWiki, evaluation of JSPWiki). Currently XWiki and JSPWiki are candidates for current replacement. The evaluation also describes requirements around localization framework in next OpenSolaris web platform.

I worked recently to publish localization OHAC pages on OpenSolaris. Original English pages are picked up and send to translation cycle. Original pages are just six, but it's easy to be larger numbers for translation pages by number of translation languages. At that time, languages are nine. So final publication becomes 6x9=54 for just 6 original pages. Publication of these pages took much time actually.

One of the requirements mentions about API to take and publish contents. And there seems feature of XMLRPC to enable this kind of publication on wiki. This will be great help for this kind of translation.

There are other requirements and answers also for localization. Please see it and join in discussion if you are interested. OpenSolaris web page localization must be improved greatly with next platform.