Tuesday February 12, 2008

*Easier* localization in JavaFX Script

In the last blog entry, I explained the proposal for an easy way to localize string literals in JavaFX Script language. Now the implementation for it is in the OpenJFX Compiler's repository, let's take a look at this feature in action. As an example, I localized the JavaFXBalls demo into Japanese (it's in the repository as well):

You see the text in the button at the bottom left corner reading "ストップ", which means "Stop". In this button's object literal creation, the source code looks like:

text: bind if (not test._is_running) then ##"Start" else ##"Stop"

And it's JavaFX properties file entry is simply:

"Stop" = "ストップ"

Now, we are planning to extend this string literal localization proposal into a generic string localization runtime library function. Suppose that there is a JavaFX runtime library class, e.g., javafx.util.StringLocalizer, and it has the following APIs:

package javafx.util;

public class StringLocalizer {

    public attribute key: String;
    public attribute locale: java.util.Locale;
    public attribute packageName: String;
    public attribute propertiesName: String;
    public attribute defaultString: String;

    // this is lazily bound to the above attributes.
    public function localizedString(): String;

    public static function associate(packageName: String, 
                                     scriptFileName: String,
                                     properties: String): Void;
}

With these utility APIs, you could do something like:

// Object creation
var localizer = StringLocalizer{ key: "Hello, World!" };

// This prints localized text for "Hello, World!" for the default locale
System.out.println(localizer.localizedString()); 

// This prints localized text for "Duke" for the default locale
localizer.key = "Duke";
System.out.println(localizer.localizedString()); 

// This prints localized text for "Duke" for the French locale
localizer.locale = Locale.FRENCH;
System.out.println(localizer.localizedString()); 

// This prints localized text for "Duke", from 
// the FX properties file "foo/bar/MyBundle_fr.fxproperties
localizer.packageName = "foo.bar";
localizer.propertiesName = "MyBundle";
System.out.println(localizer.localizedString()); 

Also, with the "associate()" static function, you could associate a JavaFX Script source file (or a javafx package) to a JavaFX properties file so that the string literal localization consults the associated JavaFX properties file for searching the localized text. (By default, the JavaFX properties file with the same name/location as the source file is searched). This static function would provide developers with the granularity of how the JavaFX properties files are packaged. You could provide one single JavaFX properties for your entire application, or one properties file for each source script file, or in between.

This is still a rough idea and needs to be refined. But I'd hope this would contribute to an *easier* localization in JavaFX.

Thursday January 10, 2008

Easy localization in JavaFX Script

I've been recently working on the internationalization of the JavaFX Script, which is a scripting language based on Java. Although all the Java internationalization features are definitely available from JavaFX Script program as it runs on top of a Java runtime, it would not be very scripting language like, and could be error prone because of the complexity of mixing Java and JavaFX. One of the features that I would like to have in the JavaFX Script is an easy way to localize strings in the JavaFX source. In Java, it's like:

    import java.util.ResourceBundle;

    static final String GREETINGS_KEY = "HELLO_WORLD";

    String greetings;
    try {
        ResourceBundle rb = ResourceBundle.getBundle("foo.bar.resources.My_Resource");
        greetings = rb.getString(GREETINGS_KEY);
    } catch (MissingResourceException mre) {
        greetings = "Hello, World!";
    }

Instead, we are thinking of a new easier way to localize strings in the JavaFX Script. It would be pretty much like GNU's gettext() function where the key itself becomes the default translation. So the above code would be written in JavaFX Script like:

    var greetings = ##"Hello, World!";

Yep, just one line. How simple is that! Here "##" works like a unary operator to the string literal, in which it looks for the proper localized string with the key of "Hello, World!". Some of you might notice that PropertyResourceBundle recognizes a white space ' ' as the delimiter, so "Hello, World!" cannot be a key in it. You are correct. So what we would like to provide is a new resource bundle format for the JavaFX script. It simply accepts the form of property as

    <JavaFX string literal> = <JavaFX string literal>
e.g.,
    "Hello, World!" = "こんにちは、世界！"

In addition as a bonus, this FX properties file's default encoding is "UTF-8" (yay!) and yet also accepts the CSS style encoding declaration (@charset "<IANA defined charset name>";)

There could possibly be an issue of ambiguous meaning for short words. For example, "File" could both mean "to file" and "file (as a noun)". To address this situation, you can optionally insert an explicit key which will be used for resource look up. This optional key can be specified within a pair of square brackets, e.g., ##[FILE_NOUN]"File".

I wrote a proposal and prototype implementation at Planet JFX wiki, so you might want to try it out. Comments are welcome as I'd like to know what you think!

Thursday June 28, 2007

Currency enhancements in JDK7

In the JDK, applications can use java.util.Currency class to deal with currencies. They can query what currency is used in a given country, or what localized symbol is used for that currency in a particular locale. To provide applications with those currency information, the JDK contains the currency data that is based on the ISO 4217 standard. From time to time, due to a variety of reasons, the ISO 4217 data are often updated by the maintenance agency. Keeping up with these updates in the JDK is critical for applications that deal with currencies, such as a banking application. Otherwise, those applications would result in a wrong financial transaction (cf. see my old entry regarding the Turkish pastry story here)

Now, one of the issues in the current JDK's currency support is that the currency data is embedded in "lib/rt.jar" as a class file. So every time the ISO 4217 maintenance agency releases a currency update, such as Slovenia switching to Euro currency, the rt.jar file needs to be updated. This makes it impossible for customers to replace just the currency data portion when needed, without upgrading the whole JDK. This issue is analogous to the TimeZone data being separated from the JDK class files, and they can be upgraded separately.

The purpose of this blog entry is to announce that this will no longer be an issue in the JDK7, as I have just checked-in the necessary changes for this issue (woohoo!), which will soon be public in the OpenJDK build. In the new structure, the currency data is separately placed in "<JAVA_HOME>/lib" directory as a binary file "currency.data", which can be replaced not in sync with the JDK update release schedule. One added feature to this is that you could even provide your own properties file to override the contents in the currency data file. If you create a properties file as "<JAVA_HOME>/lib/currency.properties", which contains "key=value" pairs where the "key" is a ISO 3166 country code, and the "value" consists of three-letter currency code, three-digit numeric code, and the minor unit (separeted by a comma), it would override the currency data for that country designated by the "key". For example if it has "JP=JPZ,999,2", this would override the Japanese currency data.

Also, along with this Currency enhancement, I added a couple of new APIs. First one is to acquire all the available currencies in the JDK as follows:

public static Set<Currency> getAvailableCurrencies()

Next one is to return a numeric ISO 4217 code for a Currency instance:

public int getNumericCode()

The last one is to return the display name for a Currency instance. Instead of getSymbol() returning "$" for the US Dollar ("USD"), this API returns "US Dollar" in the "US" locale.

public String getDisplayName()
public String getDisplayName(Locale displayLocale)

The initial release of this API contains localized currency names for 10 locales, i.e., English, French, German, Italian, Japanese, Korean, Simplified Chinese, Spanish, Swedish, and Traditional Chinese.

I also added an SPI in java.util.spi.CurrencyNameProvider for the languages other than the above, so that one could provide localized display names in any language:

public getDisplayName(String currencyCode,
                      Locale displayLocale)

These are the changes that I have checked in, and I hope this is useful for developers. I would have liked to provide a working demo, but it's not possible to do it with a yet released build :-)

Sunday April 22, 2007

Ubuntu 7.04 (Feisty Fawn)

I know that Java 6 is included in the recently released Ubuntu 7.04 (Feisty Fawn), and am curious how they modify Sun's JDK6 in terms of internationalization. So I upgraded my Ubuntu from Edgy Eft to Feisty Fawn, grabbed the sun-java-6 package, and ran my locale demo...

Voila! CJK fonts seem to be supported out-of-the-box! Kudos to Ubuntu developers! BTW, I slightly modified the demo to display a small icon in front of each installed locale, so that it can identify itself as JDK bundled one or CLDR one. I am preparing to include this demo in one of the machines at this year's JavaOne Pavilion.

Wednesday March 21, 2007

New article about the new internationalization features in JDK 6

John O'Conner, who used to work with me, wrote a really nice article about the new internationalization features in JDK 6. The article is just released here, which includes lots of examples demonstrating the new features.

Kudos to John!

Tuesday November 14, 2006

Locale Demo on your own machine

"Seeing is believing", so I made the locale demo webstartable.

First, please check that your machine has Java 6 installed, if not, you can install it from here. Then just hit the following button:

The demo started? At this moment, it only displays locales that are provided by the Java runtime only. If you go to the "Locale Names" tab, it shows the total number of locales as "150".

Now the fun part begins. Download the CLDR Adapter into the extension directory of your Java runtime. For example on Windows, the path to the extension directory is typically "C:\Program Files\Java\jre1.6.0\lib\ext". Then hit the above "Launch" button again. It would take a bit longer time to start, because this time the CLDR adapter retrieves the locale data from the Unicode Consortium's web site on the fly. Take a look at the "Locale Names" tab again and you will see the locale count jumps to "365"!

This demo also showcases an interesting aspect, i.e., "locale on demand". Locales can be installed into the runtime only when it is really needed. One of the features that we are planning to do in the next release, um, that means after Java 6 :-), is how we can deploy locale data to the clients, and this demonstrates an interesting point.

Monday October 30, 2006

More than 300 locales in Java!

I am going to give a presentation at the upcoming Internationalization and Unicode Conference held in Washington D.C. next month. The main topic of the presentation is the new i18n features in the upcoming Java 6, and now I am preparing a demo to showcase the potential of the Locale Sensitive Services SPI feature (I know. Kinda long name), which I wrote in the previous entry (wow, it was 8 months ago :-)

The demo consists of two components. One is just a simple Swing application which displays a list of available locales in the platform (precisely, a list returned from Locale.getAvailableLocales()), and displays formatted dates, times, numbers, currencies, or language/country/timezone names in a particular locale which is selected in the locale list.

The other component is invisible, but more interesting one. That is, an adapter which reads the CLDR's locale data in XML format, then it transforms the locale data and plugs them into the Java runtime through the Locale Sensitive Services SPI. Although Java currently do not support three letter language codes defined in ISO 639-2 so some locales may not be plugged in, but still we have more than 300 locales available in the Java text processing classes with this CLDR adapter!

Here is a preview of the demo. This demonstrates the date and time formatting in the Tamil (India) locale, which the Java runtime does not yet support out of the box:

Click to enlarge

Note that since Java does not support Tamil, it does not contain the suitable font either. However, the OS I used for running this demo was Windows XP, which contains fonts that are capable for displaying Tamil such as "Latha" font. And with the "font fallback" mechanism we introduced in Java 5.0, the demo worked like a charm with the formatted date and time, which are derived from the CLDR data.

Here is another screen shot of the demo, which displays date and time in Amharic (Ethiopia) locale.

Click to enlarge

For this demo, since Windows XP does not support Amharic, I used an open source TrueType font for Amharic called "Jiret", which is available here.

If you are interested, and planning to be at the next Unicode conference, please come to our presentation. I am sure I can show you more interesting bits!

Wednesday February 15, 2006

About the Locale Sensitive Services SPI in Mustang Beta

Now that the Mustang (a.k.a. Java SE 6) beta is out (yay!), I think it is about time to describe one of the new i18n features so that you can ride the Mustang beta with pleasure. The feature I want to mention here has the name "Locale Sensitive Services SPI". Sounds complicated? No, it's not! OK, let me introduce it to you.

With this Mustang Beta release, we support more than 100 locales in java.text and java.util packages. Although this covers lots of areas in this Globe, there are areas that are yet to be supported by the Java runtime, and some customers do want them supported. Believe it or not, supporting a locale and its data requires a lot of investigation, such as the most popular date format or translation of a country name in that language. It's getting more difficult for smaller countries/regions. Sometimes even political ramifications come in. To resolve such a difficulty, there is an interesting project going on, which is called "Common Locale Data Repository (CLDR)" at the Unicode Consortium and standardizes the commonly used locale data. However, we cannot include all of the defined data into the Java runtime as it would bloat the Java runtime size (We introduced a couple of new locales in Mustang from the CLDR, though). So, the idea we came up with was to open up the interface with which the developers can plug in their own locale data and related services. I guess that the question in your mind now is, how do I use them, right?

To provide locale data/services, you first need to decide which functionality you want to provide for your own locale. With Mustang, you can provide the following functionalities by your locale data implementation.

• java.text.BreakIterator
• java.text.Collator
• java.text.DateFormat
• java.text.DateFormatSymbols
• java.text.DecimalFormatSymbols
• java.text.NumberFormat
• java.util.Currency
• java.util.Locale
• java.util.TimeZone

Once you decide which functionality you want to provide with your locale, then you will need to implement the corresponding SPI (stands for the "Service Provider Interface"), which resides either in java.text.spi or java.util.spi packages. Let's say you want to provide a DateFormat object for a new locale, then you need to implement java.text.spi.DateFormatProvider. Since java.text.spi.DateFormatProvider is an abstract class, you will need to extend it and implement the following 4 methods.

• getAvailableLocales()
• getDateInstance()
• getDateTimeInstance()
• getTimeInstance()

You'll notice that "getAvailableLocales()" is actually derived from the parent class, i.e., LocaleServiceProvider, so all the SPI providers need to implement it so that the providers can "declare" which locales they claim "supported". And you will also notice that the other three methods are mirrored factory methods from the corresponding API class, i.e., java.text.DateFormat class. This means that the object which your implementation returns is passed down to the application as it is.

After implementing those required methods, then you need to package your implementations, so that you can deploy it with the Java runtime. Since the Locale Sensitive Services SPIs are based on the standard Java Extension Mechanism, you can package them as a JAR file (with a few tricks in its MANIFEST file, which can be found in here) and place it in the extension directory.

That's it folks. If an application requests a DateFormat object for your locale, your providers object is now created and used. Happy locale-adding!

Wednesday January 05, 2005

I only bought a pastry!

Happy new year! After a hiatus, I try to resume this blog. Let's see :)

Some of you may have heard that Turkish government took place a denomination of their currency. On January 1st, 2005, they have introduced the New Turkish Lira (YTL), where one New Turkish Lira is equivalent to 1,000,000 (old) Turkish Lira (TL). The transition from the old currency to the new one seemingly went well, but it looks like some are confused. After the new currency inauguration, a Turkish man bought a pastry at some pastry store. It turned out that the credit card of the man was charged 1,400,000 New Turkish Lira (about $10,000) for the pastry! Both the man and the pastry shop were not sure what had happened until the credit card bank gave the man a call confirming the transaction. (cf. Man mistakenly buys Turkey's most costly pie)

Although the Java runtime cannot convert the amount in the old currency to the new one, this New Turkish Lira currency support has already been incorporated in the latest J2SE 5.0 Update 1 release. Let's take a look at the following piece of code:

Locale turkish = new Locale("tr", "TR");
Currency turkishCurrency = Currency.getInstance(turkish);
System.out.println(turkishCurrency.getSymbol(turkish));

This code prints "YTL" on or after 1/1/2005, otherwise prints "TL". Please note that getSymbol() returns the ISO 4217 currency code if the display language is not Turkish, such as getSymbol(Locale.ENGLISH). In such a case, the above code prints "TRY" and "TRL" for the New Turkish Lira and the (old) Turkish Lira respectively.

Monday September 06, 2004

New Internationalization Features of the Java™ Platform

That's the title of the talk Craig and I are going to give at the 26th Unicode Conference, starting from tomorrow. I am still swamped with lots of demos, machine setup, etc, spending Labor Day long weekend with those, but I believe that we are in good shape now.

Tuesday August 17, 2004

This is why the i18n is so important...

During the demonstration of the Sequoia machine last week, the machine worked fine when the company tested votes using an English-language ballot. But when the testers switched to a Spanish-language ballot, the paper trail showed no votes cast for two propositions.

Wrong Time for an E-Vote Glitch

This happened at the demo for the state Senate staffers in California. It seems that the company has not designed/developed/tested languages other than English, and the i18n glitch was revealed at a very bad moment.

BTW, I believe in E-Voting, but I guess it is still in a premature state.

Wednesday August 04, 2004

Guess what

I've just seen James Gosling wearing a suit with a tie at our all hands meeting. That was because he was supoosed to sink in a Dunk Tank at the party after that. And he did :)

James spoke about this at his own blog with a cool picture!

Monday July 26, 2004

I'm back

From my vacation, i.e. Las Vegas. It was nice except the temperature there, which was over 110F. We went to see a magic show by David Copperfield, and my wife was lucky enough to be selected to appear on stage and got lost into the thin air. She seemed to be put a seal upon her lips about how she got lost, by being given an autograph from David.

I have avoided to open my inbox till this morning. I expected a couple of hundreds emails were there, but that was not quite right. It turned out to be a couple of thousands :( However, soon I figured out that that was not because of my vacation, but the new virus (a variant of MyDoom) which spread out this morning.

My main machine has Java Desktop System as its OS, so there was no fear that it got affected by that virus, but the problem was the team email address we exposed to the public to solicit comments/questions. Looks like the virus was trying to extract email addresses from major search engines, such as Google, and that's why Google was down this morning as a result of DoS attack. It was kind of irony that Google was down when they announced the share price range for their IPO.

Anyway, I am back on business!

Friday July 16, 2004

Unicode Conference

Looks like I have to fill in for John at the next Unicode Conference, with Craig at Oracle. I knew that the tutorial sessions are longer than the technical sessions, but I did not know that the longest slot was assigned to our Java i18n tutorial session. I did technical sessions a couple of times, which are 40 minutes long, but I am not sure how we end up filling 2 hour and 40 minutes.

Well, let's forget it for now. Because I am off to Las Vegas!

Tuesday July 13, 2004

In the middle of two vacations

I was back from last week's shutdown, and will be off again next week. Believe it or not, I am going to Las Vegas for the first time in 6 years being here in the Bay Area.

So, I expected this week was calm and quiet, and planned to do some prototyping for the release after Tiger. However, when I got back, things has changed and I ended up fixing a bug for the Tiger release candidate build. You never know what's going to happen...