Gnome use RFC 2396 - Uniform Resource Identifiers (URI) to reprensent filename, so that filename are represented independent of encoding. [Read More]

This document is written when i build gnome cvs using jhbuild, and i write it just for reference by myself, there maybe many information which are not correct. Welcome any suggestion and comments!

1. jhbuildrc file

First copy below contents as your ~/.jhbuildrc

# -*- mode: python -*-

# information about cvs server
cvsroots['gnome.org'] = ':pserver:anonymous@anoncvs.gnome.org:/cvs/gnome'
cvsroots['fontconfig.freedesktop.org'] = ':pserver:anoncvs@cvs.freedesktop.org:/cvs/fontconfig'

# modules to build (deps are also built)
moduleset = 'gnome-2.16'
modules = ['gtk+', 'meta-gnome-desktop', 'meta-gnome-devel-tools' ]

skip = [ 'hal', 'mozilla' ]

skip.append('evolution-exchange')

# the prefix to install things to (user must have write access)
prefix = '/export/home/gnome/install/gnome-cvs'

# where the sources are checked out to
checkoutroot = '/export/home/gnome/src/cvsroot'

# where to put downloaded tarballs.
tarballdir = '/export/home/gnome/src/tarballs'

# arguments to pass to autogen script
autogenargs='--enable-maintainer-mode --disable-static --disable-gtk-doc'

# set CFLAGS:
os.environ['CFLAGS'] = '-g'
os.environ['PYTHON'] = os.path.join(prefix, 'bin/python')
os.environ['PYTHONPATH'] = os.path.join(prefix, 'lib/python2.4/site-packages')

# On SMP systems you may use something like this to improve compilation time:
# be aware that not all modules compile correctly with make -j2
os.environ['MAKE'] = 'make'

os.environ['INSTALL'] = os.path.expanduser('~/bin/install-check')

module_autogenargs['libgcrypt'] = autogenargs + ' --disable-asm'
module_autogenargs['howl'] = autogenargs.replace('--disable-static', '')
module_autogenargs['fontconfig'] = autogenargs + ' --with-expat=/export/home/gnome/install/gnome-cvs/'
module_autogenargs['gtk+'] = autogenargs + ' --with-included-loaders=png'
module_autogenargs['ekiga'] = autogenargs + ' --with-pwlib-dir=/export/home/gnome/install/gnome-cvs/ --with-opal-dir=/export/home/gnome/install/gnome-cvs/ --with-sdl-prefix=/export/home/gnome/install/gnome-cvs/'

module_makeargs['libIDL']= makeargs + ' CPP="/usr/lib/cpp"'
module_makeargs['ORBit2']= makeargs + ' CPP="/usr/lib/cpp"'
module_makeargs['libbonobo']= makeargs + ' CPP="/usr/lib/cpp"'
module_makeargs['dbus']= makeargs + ' CFLAGS="-I/export/home/gnome/install/gnome-cvs/include" LDFLAGS="-lexpat -lintl"'
module_makeargs['cairo']= makeargs + ' LDFLAGS="-lc"'
module_makeargs['gtk+']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['libgnome']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['libgnomeprint']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['bug-buddy']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['eog']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['zenity']= makeargs + ' LDFLAGS="-lintl"'
module_makeargs['evolution-webcal']= makeargs + ' LDFLAGS="-lintl"'

addpath('XDG_DATA_DIRS', '/export/home/gnome/src/gnome-cvs/share')
addpath('XDG_CONFIG_DIRS', '/export/home/gnome/src/gnome-cvs/share')

2. Issues met during building

export LC_ALL=C first

1. Install cbe

    look at opensolaris.org

2. Install jhbuild

3. jhbuild bootstrap

• gettext
  Using the install script shipped with cbe, don't use install-check.
• python
  Using gnu tar instead of tar bundled with solaris

4. jhbuild build

4.1fontconfig:

           - need expat: http://nchc.dl.sourceforge.net/sourceforge/expat/expat-2.0.0.tar.gz
           - need freetype: www.freetype.org
           - module_autogenargs['fontconfig'] = autogenargs + ' --with-expat=/export/home/gnome/install/gnome-cvs/', exit jhbuild and rerun jhbuild build.

            ps: build freetype and expat under jhbuild enviroment, otherwise freetype will link
  expat.so.0 under /usr/sfw/lib instead of $prefix/lib

4.2 cairo

cairo will be compiled with the following surface backends:
  image:      yes (always builtin)
  Xlib:       yes
  Quartz:     no
  XCB:        no
  Win32:      no
  PostScript: yes
  PDF:        yes
  SVG:        no
  glitz:      no
  BeOS:       no
  DirectFB:   no

the following font backends:
  FreeType:   yes
  Win32:      no
  ATSUI:      no

the following features:
  PNG functions: yes

and the following debug options:
  gcov support:  no
  test surfaces: no

using CFLAGS:
-DXTHREADS -I/export/home/gnome/install/gnome-cvs/include -I/usr/local/include/libpng12 -I/export/home/gnome/install/gnome-cvs/include -I/export/home/gnome/install/gnome-cvs/include/freetype2

 - libpng: http://prdownloads.sourceforge.net/libpng/
  cp scripts/makefile.solaris makefile
  change prefix
  make && make install

 - cairo:
  module_makeargs['cairo']= makeargs + ' LDFLAGS="-lc"'

4.3 gtk+

 - failed to load "./stock_about_16.png": Couldn't recognize the image file format for file './stock_about_16.png'
  ===> --with-included-loaders=png

  It is true that lt-gdk-pixbuf-csource doesn't seem to have png/tiff/jpeg built
into it, so that's why I believe this seems to be the corrent line of solution.

I'm trying to impliment a similar solution... I'm pretty sure the SHLIB stuff is
hp-ux specific. So I went into gdk-pixbuf/Makefile and added the -L./.libs to
the LINK= line and did a 'gmake clean; gmake' and now I get:

g_module_open() failed for
/auto/src-01/common/usc/gnu/gtk+/gtk+-2.2.3/gdk-pixbuf/libpixbufloader-png.la:
ld.so.1:
/auto/src-01/common/usc/gnu/gtk+/gtk+-2.2.3/gdk-pixbuf/.libs/lt-gdk-pixbuf-query-loaders:
fatal: relocation error: file /usr/usc/png/1.2.5/lib/libpng.so.3: symbol
inflateReset: referenced symbol not found

Which I'm confused about. png 1.2.5 is the latest, and it was compiled with the
same compiler.

I've been playing with this for 3 weeks. I actually had to hard code the
'-L/usr/usc/png/1.2.5/lib' into the configure.in file because the configure
script couldn't find it with --with-libpng= even if I set LDFLAGS... This is the
last of *many* problems I've had trying to get gtk+2 to compile...

Phil

 -  Undefined                       first referenced
   symbol                             in file
  libintl_dgettext                    /export/home/gnome/src/cvsroot/gtk+/gdk/.libs/libgdk-x11-2.0.so
  ld: fatal: Symbol referencing errors. No output written to .libs/gtk-query-immodules-2.0

  ==> -lintl

4.4.0 libIDL

 Use /usr/lib/cpp as c preprocessor, otherwise orbit-idl-2 will cause many problems.

4.4 ORBit2

 - CFLAGS="-Werror-implicit-function-declaration -std=gnu89 $CFLAGS"
  -> comment out this line in configure.in

 - test/inhibit: unknow Foo_..._EPV, this is caused by lack definition.
  add foo.h into baa.h.
  see: http://mail.gnome.org/archives/orbit-list/2001-May/msg00074.html
  ***: if we use /usr/lib/cpp for libIDL, then no such problem.

4.5 bonobo

 need libIDL, ORBit2 use /usr/lib/cpp on solaris as CPP(c preprocessor)

module_makeargs['libIDL']= makeargs + ' CPP="/usr/lib/cpp"'
module_makeargs['ORBit2']= makeargs + ' CPP="/usr/lib/cpp"'
module_makeargs['libbonobo']= makeargs + ' CPP="/usr/lib/cpp"'

 cpp under solaris:
  /usr/lib/cpp
  /usr/sfw/bin/cpp
  /opt/SUNWspro/bin/cc -E
  

4.6 howl

 module_autogenargs['howl'] = autogenargs.replace('--disable-static', '')

4.7 dbus

 - can't find expat.h

  module_makeargs['dbus']= makeargs + ' CFLAGS="-I/export/home/gnome/install/gnome-cvs/include LDFLAGS="-lexpat -lintl"'

 - ImportError: ld.so.1: python2.4: fatal: relocation error: file /usr/lib/python2.4/lib-dynload/array.so: symbol _PyArg_NoKeywords: referenced symbol not found
    export PYTHON and PYTHONPATH

4.8 hal:  still need patch in order to run on solaris correctly

 can't find pci.ids
 Patch hal for solaris: http://lists.freedesktop.org/archives/hal/2006-February/004630.html

 1) Can't find definition for __FUNCTION__, use __func__

 cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/pciids login
 cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/pciids co -P pciids

 can't find 'endian.h', 'byteswap.h'

 skip this module and delete all dependency on this.

4.9 gnome-vfs 

 - __PRETTY_FUNCTION__  undefined:  As noted in the previous subsection, the g++ compiler defines two standard built-in preprocessor macros: __FILE__ and __LINE__. The g++ compiler also defines two non-C standard preprocessor macros called __FUNCTION__ and __PRETTY_FUNCTION__ that return the name of the current function. We can use any of these special pre-defined macros to build our own debug utilities.

   GNU CC predefines two string variables to be the name of the current
function.  The variable `__FUNCTION__' is the name of the function as
it appears in the source.  The variable `__PRETTY_FUNCTION__' is the
name of the function pretty printed in a language specific fashion.

   These names are always the same in a C function, but in a C++
function they may be different.  For example, this program:

     extern "C" {
     extern int printf (char *, ...);
     }
    
     class a {
      public:
       sub (int i)
         {
           printf ("__FUNCTION__ = %s\n", __FUNCTION__);
           printf ("__PRETTY_FUNCTION__ = %s\n", __PRETTY_FUNCTION__);
         }
     };
    
     int
     main (void)
     {
       a ax;
       ax.sub (0);
       return 0;
     }

gives this output:

     __FUNCTION__ = sub
     __PRETTY_FUNCTION__ = int  a::sub (int)

   These names are not macros: they are predefined string variables.
For example, `#ifdef __FUNCTION__' does not have any special meaning
inside a function, since the preprocessor does not do anything special
with the identifier `__FUNCTION__'.

 This symbol are gcc/g++ only, fortunately it is mostly used by debug.
 We can disable debug, so that we don't compile it.

Sulotion:
 rm $(topdir)/CVSVERSION

AC_ARG_ENABLE(http-debug,
[  --enable-http-debug  Enable HTTP method debugging],
set_http_debug="$enableval",[
if test -f $srcdir/CVSVERSION; then
        set_http_debug=yes
else
        set_http_debug=no
fi
])

#ifdef HAVE_UTMP_H
        /* This sets stdin, stdout, stderr to the socket */
        /*if (login && login_tty (fd) == -1) {
                g_printerr ("mount child process login_tty failed: %s\n", strerror (errno));
                return -1;
        }*/
#endif

4.10 startup-notification

 - Undefined symbol: __FUNCTION__
  add one patch: use __func__ for Sun CC compiler instead of __FUNCTION__

4.11 gnome-doc-utils

 - ../../intltool-merge: No such file or directory
  copy from other modules

 - doc/ can't build
  Run under command line manually.

4.12 libgcrypt

 - "mpih-add1.S", line 44 : Syntax error
  --disable-asm

4.13 gnome-terminal

 - No rules to make 'help/bg'
  ignore it: need evaluate on this problem.

4.14 libgtop

  -
"shm_limits.c", line 62: incomplete struct/union/enum shminfo: sinfo
"shm_limits.c", line 73: improper member use: shmmax
"shm_limits.c", line 74: improper member use: shmmni
"shm_limits.c", line 76: improper member use: shmmin
"shm_limits.c", line 77: improper member use: shmseg
 - using libgtop-2.8.3 and one patch from http://www.whacked.net/2005/07/25/libgtop-on-solaris-10/

4.15 libxklaver

 - no doxygen
  modify autogen.sh to don't --enable-doxygen
 - illegal option -Wall
  patch $(topdir)/libxklavier to remove them

4.16 liboil

 - exmaple/Makefile: cantain '-O2' which can't be recognized by Sun cc.

4.18 gnome-cups

 - need install cups
  --need openldap: need berkely DB install to /usr/ diretory
   --berkely DB: http://www.sleepycat.com/
    cd build_unix
    ../dist/configure default prefix
    make
    make clean

 - When build example: can't found symbol definiton for libintl_dgettext
      

4.19 nautilus

 - need libexif: http://kent.dl.sourceforge.net/sourceforge/dropline-gnome/libexif-0.5.12.tar.bz2

4.20 gnome-control-center

 - "/export/home/gnome/install/gnome-cvs/include/libxklavier/xklavier_config.h",  : _XklConfigItem
cc:/opt/SUNWspro/prod/bin/acomp

4.20: yelp

 - fno-rtti can't be recognized by compile
 - liconv

4.21 nautilus-cd-burner

 - Wall illegal option
 - -lc -lintl
 =
Undefined                       first referenced
 symbol                             in file
void __Crun::ex_rethrow_q()       .libs/dvd_plus_rw_utils.o
void*operator new[](unsigned)                       .libs/dvd_plus_rw_utils.o
void operator delete(void*)                       .libs/dvd_plus_rw_utils.o
ld: fatal: Symbol referencing errors. No output written to .libs/libnautilus-burn.so.3.0.0

  add -lCstd -lCrun at proper position

4.22 libmusicbrainz

 - ./examples/:  ld: fatal: library -lstdc++: not found
  -cpp_example_LDADD = $(top_builddir)/lib/libmusicbrainz.la -lstdc++ -lm
  +cpp_example_LDADD = $(top_builddir)/lib/libmusicbrainz.la -lm -lCstd -lCrun

4.23 totem

 -  cannot find include file: <linux/cdrom.h>
  use patch from laca's spec file. 
 - no kill func:
  include <signal.h>
 

4.24 gnome-media

 - Replace '__FUNCTION__' with __func__
 - CDROM_GET_CAPABILITY undefined: #define CDROM... 0x42432

4.27 zenity: -lintl

4.29 gnome-mag:

 - can't found lib Xdamage:
ld: fatal: library -lXdamage: not found
ld: fatal: library -lXfixes: not found
ld: fatal: File processing errors. No output written to .libs/magnifier

  -R/usr/X/sfw/lib -L/usr/X/sfw/lib -lXdamage -lXfixes

4.30 epiphy:

 - need xpcom, skip this

4.31 gnome-system-tools skipped

 undefined symbol: NETSTAT_PROTOCOL_FORMAT

4.32 gnome-volume-manager

 skip because of hal

4.33 sound-juicer: skip

 -Wall
 -lstdc++:

4.34 evolution-webcal: intl

4.35 evolution-exchange: skip

 Package requirements (libsoup-2.2 evolution-shell-2.6 libedataserverui-1.2 libexchange-storage-1.2 libbonobo-2.0 libxml-2.0 gconf-2.0)

4.36 ekiga

 - need SDL
 - need PWlib: http://sourceforge.net/projects/openh323  or cvs
   --with-pwlib-dir
 - need opal: cvs -z9 -d :pserver:anonymous@cvs.sourceforge.net:/cvsroot/openh323 co opal
   PWLIBDIR=/path/to/pwlib
   

 - modify configure file manually
  --with-pwlib-dir=/export/home/gnome/install/gnome-cvs/ --with-opal-dir=/export/home/gnome/install/gnome-cvs/ --with-sdl-prefix=/export/home/gnome/install/gnome-cvs/

4.37 dasher - skip

4.38 gnome-keyring-manager: -Wall -W-Wno-format-y2k

 skip

4.39 sabayon: --with-disto

     Xnest binary not found.

4.40 poper

 --freetype: add include file

 

6. Run jhbuild gnome

6.1: create /usr/bin/jhbuild-session:

 #!/sbin/sh
 exec jhbuild --file=/.jhbuildrc run dbus-launch gnome-session

6.1.1: /usr/share/xsessions

 copy gnome.session jhbuild-gnome.session

6.1.2: /usr/dt/config/Xsession.jds

6.1.3: exec /usr/dt/bin/Xsession:

 SDT_ALT_SESSION="/usr/dt/config/Xsession2.jhbuild"

6.1.4: /usr/dt/config/Xsession2.jds

6.1.5: /usr/dt/config/Xinitrc.jhbuild

 jhbuild-session

6.2: copy /etc/fonts into $prefix/etc

 then execute fc-cache to handle with font issues.

6.3: update-mime-database $prefix/share/mime

   update-desktop-database -v

6.4:   bonobo-slay

    bonobo-activation-sysconf --display-directories
    bonobo-activation-sysconf --config-file-path

    $prefix/etc/bonobo-activation/bonobo-activation-config.xml
   add below: <item>$prefix/lib/bonobo/servers</item>

 we need make bonobo-activation-server be able to find your .sever file, it can be done in three ways:
  1) copy the server file to /usr/lib/bonobo/servers
  2) export BONOBO_ACTIVATION_PATH
  3) activation-client --add-path $(pwd)

  gtk-update-icon

6.5: gconf schema file: where to get?

  $prefix/libexec/gconfd-2: how to run it?

6.6: copy im-iiim.so to $prefix/lib/gtk-2.0/2.4.0/immodules/

7.0 References

 - Jhbuild issues:  http://www.gnomebangalore.org/wiki/index.php/Jhbuild_issues
      http://live.gnome.org/JhbuildIssues_2fpango

 - Build gnome on Ubuntu: http://live.gnome.org/JhbuildOnUbuntu
 - sample jhbuildrc: http://ftp.gnome.org/pub/gnome/teams/releng/2.13.1/sample-tarball.jhbuildrc
 - Python issue: http://mail.gnome.org/archives/garnome-list/2005-July/msg00199.html

1. Oddly enough, Python's use of whitespace stopped feeling unnatural after about twenty minutes.


2. An important measure of effort in coding is the frequency with which you write something that doesn't actually match your mental representation of the problem, and have to backtrack on realizing that what you just typed won't actually tell the language to do what you're thinking.


3. it is compact--you can hold its entire feature set (and at least a concept index of its libraries) in your head. C is a famously compact language. Perl is notoriously not; one of the things the notion ``There's more than one way to do it!'' costs Perl is the possibility of compactness.


4. The gold Python rule is "explicit is better than implicit"

Gtk+ provide several widget to let user input characters, such as GtkEntry, gtk multiline widget, and then you can using some method such gtk_entry_get_text() got what user input. But question is: what is the encoding of returned text on different locale?

The answer is that the encoding is utf8 no matter on which locale user input it.

#include <gtk/gtk.h>
#include <stdio.h>

void
cb_activate (GtkEntry *entry, gpointer user_data)
{
    const gchar *text;

    text = gtk_entry_get_text (GTK_ENTRY(entry));

    if (text != NULL)
        printf ("The text returned from entry: %s\n", text);
    else
        printf ("The text returned from entry: NULL\n");
}

int
main (int argc, char **argv)
{
    GtkWidget *window;
    GtkWidget *entry;

    gtk_init (&argc, &argv);

    window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
    gtk_window_set_title (GTK_WINDOW (window), "Test the encoding of text input by user");
    gtk_window_resize (GTK_WINDOW (window), 300, 100);
    g_signal_connect (window, "delete-event", G_CALLBACK (gtk_main_quit), NULL);
    g_signal_connect (window, "destroy", G_CALLBACK (gtk_main_quit), NULL);

    entry = gtk_entry_new ();
    g_signal_connect (entry, "activate", G_CALLBACK (cb_activate), NULL);
    gtk_container_add (GTK_CONTAINER (window), entry);

    gtk_widget_show_all (window);
    
    gtk_main ();

    return 0;
}

po file are handled by gettext/dgettext/dcgettext.

1. The encoding of po file is specified in po file through 'charset' keyword, gnome messages files are utf8.
2. The output encoding of gettext is dependent to LC_CTYPE at default
3. You can use bind_textdomain_codeset() to specify what encoding of the output of gettext is.

char * gettext(const char *msgid);

char * dgettext(const char *domainname, const char * msgid);

char * dcgettext(const char *domainname, const char *msgid, int category);

1. What is message domain?
   1. A set of translatabel msgid messages.
   2. Use domain name to determine where the translation is looked up.
2. The message domain of gettext is specified by a preceding textdomain call.
3. Translation Operation
   1. gettext, dgettext use LC_MESSAGES to lookup. It is determined by a preceding call to the setlocale function. setlocale(LC_ALL, "") initializes the LC_MESSAGE locale based on the first nonempty value of the three enviroment variabled LC_ALL, LC_MESSAGES, LANG.
   2. Use LC_CTYPE to convert the translated message from the translator's codeset(specified in po file, mostly is UTF-8) to the current locale's codeset. You can use bint_textdomain_codeset to specify the output messages encoding.

1. Introduction

glib always use utf8 as internal encoding, such as all text you see on any widget are utf8, so if you need display one filename in gtklable or gtkbutton, you need convert filename to utf8 first.

glib provide one convenient function g_filename_display_name() to get the utf8 display name of one filename which will convert filename from one encoding to UTF-8.

And problem appears: How glib know the encoding of one filename? The answer is that glib don't know the encoding of filename, user or system administrator tell glib the encoding of filename. And glib do provide one way to let user tell him the filename encoding:  G_FILENAME_ENCODING and G_BROKEN_FILENAMES enviroments.

• G_FILENAME_ENCODING can be one encoding list, if G_FILENAME_ENCODING is set, the first one in encoding list is treated as glib filename encoding (g_filename_to_utf8 and g_filename_from_utf8use it). When display one filename, glib try this encoding list to convert filename to UTF-8.
• G_BROKEN_FILENAMES can be 'yes' or null, If G_FIELNAME_ENCODING is unset and G_BROKEN_FIELNAMES=yes, then the current locale encoding( setlocale ) is treated as glib filename encoding.
• If both enviroment variables are unset, then UTF-8 is the glib filename encoding and current locale encoding will be append to filename encoding list which will be used when generate display name.

g_get_filename_charsets() encapsulate above operations and return the glib encoding list.

2. Current Implementation of g_filename_display_name()

Try each encoding in encoding list which is returned by g_get_filename_charsets() and convert it to UTF-8 until the convert success. If still failure, then call make_valid_utf8() which will call g_utf8_validate() and return one utf8 eventually.

Notes:

1. 'Try' means that glib has no accurate way to get the encoding of filename cause it may be created on any locale and maybe in any encoding.
2. 'Convert success' means the converted result is not NULL, and thus maybe still incorrect, such as for one gb18030 encoded string, it may return non-NULL value when convert it from Big5 to UTF-8( treat the gb18030 string as big5 ).
3. The ability of g_utf8_validate() is limited, it can only determine whether the string is utf8 characters stream or not, but can't ensure that the string (especially filename) is utf8 encoded even it pass the validation.
   

3. Examples

The common situation is that G_FILENAME_ENCODING and G_BROKEN_FILENAMES are both unset or set to '@locale,UTF-8' and 'yes' seperately.

3.1 Set to '@locale,UTF-8' and 'yes'

The Glib filename encodiing list is '@locale,UTF-8' and the glib filename encoding is locale encoding. eg on zh_CN.GB18030 locale, the glib filename encoding list is 'GB18030,UTF-8' and the filename encoding is GB18030(used in g_filename_to_utf8). If the actually filename encoding on disk is:
    1) GB18030: All works fine, glib will try convert the string from gb18030 to utf8 first, and return correct result.
    2) UTF-8: glib will first try convert the utf-8 string from gb18030 to utf8, it will return NULL (In most cases it return null more accurately, but I am not sure whether it will always return NULL or not), then try convert it from utf8 to utf8 and got correct result. So glib can display utf-8 on-disk encoded filename correctly.
    3) Other encoding, such as Big5: glib still try convert this big5 encoded filename from gb18030 to utf8 first, and usually the conversion will return one result but it is not correct, normally will be garbled characters for end user. If the converstion failed, the try convert this big5 string from utf8 to utf8, commonly it will failed again. Then g_filename_display_filename will call make_valid_utf8 to return one utf8 filename.

3.2 Both unset

The glib filename encoding list is 'UTF-8,@locale' and assume the filename is utf8 by default in this situation. eg: on zh_CN.GB18030 locale, the filename encoding list is 'UTF-8,GB18030', and filename encoding is UTF-8. If the actually filename encoding on disk is:
    1) GB18030: First glib will call g_utf8_validate to validate that whether this gb18030 encoded string is valid utf8 or not. Commonly g_utf8_validate will fail, but I am not sure whether there exist one gb18030 string which is valid utf8 also or not. Then try convert this gb18030 string from gb18030 to utf8, and the result is correct, so glib can display it correctly.
    2) UTF-8: it will pass g_utf8_validate and duplicate it and return directly.
    3) Other encoding, such as big5: like gb18030, it will failed on g_utf8_validate and then glib try convert this big5 filename from gb18030 to utf8. commonly it will return one garbled string, if it return NULL, then call make_valid_utf8 to return one valid utf8 string.

4. Advantage and Disadvantage of display name

4.1 Advantage

Using display name, glib can display current locale encoded filenames and utf8 encoded filenames correctly, reduce garbled characters to users and make gnome more friendly to end users.

As most of users and developers use just one locale most time and use two locale at most(one popular legacy locale such as zh_CN.gb18030 and UTF-8 locale), the above strategy is very well for these situation.

4.2 Disadvantage

As glib can display multiple encoded filename correctly at the same time, gnome program (such as gtk+,libgnomeui) need handle these filename(not display name), if deal with it incorrectly, problem will appear, and in fact some of bugs are caused by this. So need programmer give more attention on this, especially handle filename in gconf(gconf always store utf8 string internal), please see below sample:

if (g_utf8_validate (item->filename, \-1, NULL)) {
      gconf_client_set_string (capplet->client, WP_FILE_KEY, 
                               item->filename, NULL);
} else {
      utf8_filename = g_locale_to_utf8 (item->filename, \-1, NULL, NULL, NULL);
      gconf_client_set_string (capplet->client, WP_FILE_KEY, 
                               utf8_filename, NULL);
      g_free (utf8_filename);
}

if (g_utf8_validate (item->fileinfo->name, \-1, NULL))
       item->name = g_strdup (item->fileinfo->name);
else 
       item->name = g_locale_to_utf8 (item->fileinfo->name, \-1, NULL, NULL, NULL);
item->options = gconf_client_get_string (capplet->client,
                                               WP_OPTIONS_KEY,
                                               NULL);

even if programmer handle these problem correctly, there still exist problem, because

• g_utf8_validate can't ensure the filename is utf8 encoded, but can only ensure it is valid utf8 characters stream.
• no accurate way to determine the encoding of filename

1. Introduction

Glib use utf8 as internal encoding and thus all gtk+/gnome application use utf8 to represent text, so all text you got from widgets are utf8. In order to use legacy encoding, you need do some converstion.

Before converstion between two encoding, you should know what is encoding it is first. Of course we can guess the encoding of text, but unfortunately there is not perfect way to determine the encoding of one segment of text throught program. So many applications provide one encoding list to user and let user make the decision.

Filename handling is especially hard, because there is no indication whatsoever what character encoding a filename is in (it might have been created when the user was using a different locale, so filename encoding is basically unreliable and broken).

Glib has no idea to get the filename encoding either, so it let user to config filename encoding through enviroment: G_FILENAME_ENCODING and G_BROKEN_FILENAME. By default, Glib assumes that filenames on disk are in UTF-8 encoding, and through these enviroment variables, user can instruct Glib to use that particular encoding for filenames raterh than UTF-8.

2. Get Encoding Using Glib Function

2.1 g_get_charset

g_get_charset will get the character set from the C runtime on the current locale, that is to say g_get_charset will get the current locale encoding if you call setlocale (LC_ALL, "") in your applications; if you call setlocale (LC_ALL, "zh_CN.GB18030") in apps, then the later encoding of C runtime will be GB18030, thus g_get_charset will got GB18030.

2.2 g_get_filename_charsets

g_get_filename_charsets determine the prefered character sets(encoding maybe more accurate) used for filenames. The firest character set from the character sets is treated as filename encoding by Glib, the subsequent character sets are used when trying to generate a displayable respresentation of a filename, see g_filename_display_name().

On Unix, the character sets are determined by consulting the environment variables G_FILENAME_ENCODING and G_BROKEN_FILENAMES. On Windows, the character set used in the GLib API is always UTF-8 and said environment variables have no effect.

G_FILENAME_ENCODING may be set to a comma-separated list of character set names. The special token "@locale" is taken to mean the character set for the current locale. If G_FILENAME_ENCODING is not set, but G_BROKEN_FILENAMES is, the character set of the current locale is taken as the filename encoding. If neither environment variable is set, UTF-8 is taken as the filename encoding, but the character set of the current locale is also put in the list of encodings.

• G_FILENAME_ENCODING set: filename encoding is the first encoding in list.
• G_BROKEN_FILENAMES set: filename encoding is locale encoding.
• Both unset:
      -  filename encoding is utf8,
      -  and put the current locale charset into encoding list.

Notes:

• Because glib will cache the filename charsets internal, so if you change the value of G_FILENAME_ENCODING or G_BROKEN_FILENAMES dynamically, it doesn't  effect. So don't use putenv() to change their values. 
• On Unix, regardless of the locale character set or G_FILENAME_ENCODING value, the actual file names present on a system might be in any random encoding or just gibberish.

3. Conversion between C Runtime encoding and UTF-8

3.1 g_locale_to_utf8

The string parameter of  g_locale_to_utf8 is a string in the encoding of the current locale  of applications(C Runtime locale). On  Windows this means the system codepage.

Converts a string which is in the encoding used for strings by the C runtime (usually the same as that used by the operating system, cause most of applications use setlocale (LC_ALL, "") to set it locale) in the current locale into a UTF-8 string.

eg: If the current locale is gb18030, while you set your applications using setlocale (LC_ALL, "zh_TW.BIG5), then the C runtime encoding is BIG5, while the OS encoding is gb18030.

If the current C runtime encoding is UTF-8, then duplicate simply.

3.2 g_locale_from_utf8

Converts a string from UTF-8 to the encoding used for strings by the C runtime (usually the same as that used by the operating system) in the current locale.

4. Conversion between Glib filename encoding and UTF-8

4.1 g_filename_to_utf8

Converts a string which is in the encoding used by GLib for filenames into a UTF-8 string. Filename encoding is the first encoding of list returned by g_get_filename_charset().

4.2 g_filename_from_utf8

Converts a string from UTF-8 to the encoding used for filenames. Filename encoding is the first encoding of list returned by g_get_filename_charset().

4.3 g_filename_from_uri

Converts an escaped ASCII-encoded URI to a local filename in the encoding used for filenames.

4.4 g_filename_to_uri

Converts an absolute filename to an escaped ASCII-encoded URI.

5. Display Name

5.1 g_filename_display_name

Converts a filename into a valid UTF-8 string. The conversion is not necessarily reversible, so you should keep the original around and use the return value of this function only for display purposes. Unlike g_filename_to_utf8(), the result is guaranteed to be non-NULL even if the filename actually isn't in the GLib file name encoding(always return one name for display purpose).

If you know the whole pathname of the file you should use g_filename_display_basename(), since that allows location-based translation of filenames.

Parameters:

1.  filename: a pathname hopefully in the GLib file name encoding
2. Returns : a newly allocated string containing a rendition of the filename in valid UTF-8

5.2 g_filename_display_basename

Returns the display basename for the particular filename, guaranteed to be valid UTF-8. The display name might not be identical to the filename, for instance there might be problems converting it to UTF-8, and some files can be translated in the display

You must pass the whole absolute pathname to this functions so that translation of well known locations can be done.

This function is preferred over g_filename_display_name() if you know the whole path, as it allows translation.

Parameters:

• filename: an absolute pathname in the GLib file name encoding(hopefully)
• Returns : a newly allocated string containing a rendition of the basename of the filename in valid UTF-8

Notes:

• If glib encoding is UTF-8( returned by g_get_filename_charsets ) while filename parameters is not valid utf8 (g_utf8_validate), then the sequence encoding in glib encoding list will be used.
• If filename is not in hopeful encoding, then return one valid utf8 display name by make_valid_utf8(filename) in the form: "valid_utf8_string(invalid encoding)"

6. Lower Level Function

glib use native iconv routines or libiconv if has no native iconv implementation to do encoding converstion

g_iconv_open (to_codeset, from_codeset) will try codeset alias, so that provide more powerful conversion. (will learn g_charset_get_aliases() later)

7. Example

Please got simple example from http://blogs.sun.com/roller/resources/yydzero/main.c

8. References

• http://scentric.net/tutorial/sec-treeview-col-utf8-pango.html
• http://developer.gnome.org/doc/API/2.0/glib/glib-Character-Set-Conversion.html