Kyle Yuan's Weblog

Wednesday June 08, 2005

Goodbye, Sun

Tomorrow, the 10th of June, is my last working day at Sun. After 3.5 years working here, I've decided to leave for another opportunity.

I've really enjoyed working with so many talented people, I've learnt a lot of stuff that can't learn in anywhere else.

This is my last post at blog.sun.com.

Farewell, my friends
Good luck, Sun

Thursday December 09, 2004

DTrace & Mozilla

I gave a brief talk about how to use dtrace to debug mozilla on Wed. Here are the examples I used for the talk.

Examples

Ex1. How many files opend by mozilla and what they are?

dtrace -n syscall::open:entry'/execname=="mozilla-bin"/{trace(copyinstr(arg0))}'

Another form:

#!/usr/sbin/dtrace -s

syscall::open:entry
/execname=="mozilla-bin"/
{
  trace(copyinstr(arg0));
}

Highlights:

• syscall provider - lets you trace every system call entry and return
• predicate - / ... /
• copyinstr - copy user process data into the address space of kernel

Ex2. The .mozill bug - which convertor is mozilla using?

#!/usr/sbin/dtrace -s

BEGIN
{
  self->start = 0;
}

/* nsLocalFile::GetPath */
pid$1::__1cLnsLocalFileHGetPath6MrnJnsAString__I_:entry
{
  self->start = 1;
}

pid$1::__1cLnsLocalFileHGetPath6MrnJnsAString__I_:return
{
  self->start = 0;
}

/* http://aggregate.eng/ws/on10_nightly/source/usr/src/lib/libc/port/gen/iconv.c */
pid$1::_icv_iconv:entry
/self->start/
{
  trace(probemod);
}

Highlights:

• pid provider - allows you to trace any instruction in a process
• D language variable type
  • associative array
  • thread-local variable - self
  • clause-local variable - this
  • built-in variables - execname, probemod, probefunc, pid, etc.

Ex3. Constructor & Destructor

#!/usr/sbin/dtrace -s

BEGIN
{
  self->div = 0;
}

pid$1:libgklayout:__1cQnsHTMLDivElement2t*6M_v_:entry
{
  self->div++;
  printf("Div++: %d", self->div);
}

pid$1:libgklayout:__1cQnsHTMLDivElement2T*6M_v_:entry
{
  self->div--;
  printf("Div--: %d", self->div);
}

Highlights:

• some utilities: nm - symbol lookup; c++filt - C++ symbol demangler
• no optimized build

Ex4. What's leaked?

#!/usr/sbin/dtrace -s

BEGIN
{
 self->div = 0;
}

pid$1:libgklayout:__1cQnsHTMLDivElement2t*6M_v_:entry
{
 @t[arg0] = count();
 self->div++;
 printf("Div++: %d", self->div);
}

pid$1:libgklayout:__1cQnsHTMLDivElement2T*6M_v_:entry
{
 @t[arg0] = count();
 self->div--;
 printf("Div--: %d", self->div);
}

Highlights:

• arg0 - arg9: for entry probe - first ten arguments, int64 (in C++ program, the last argument is always "this".); for return prob - arg1 - return value;
• aggregating functions
  • count - the number of times called.
  • sum - the total value of specified expressions.
  • quantize - see Ex 6.

Ex5. More aggregations 1 - How many system calls did mozilla make?

dtrace -n syscall:::entry'/execname=="mozilla-bin"/{@t[probefunc] = count()}'

Ex6. More aggregations 2 - How many bytes write to disk for every "write" system call?

dtrace -n syscall::write:entry'/execname=="mozilla-bin"/{@t[execname] = quantize(arg2);}'

Ex7. Timing & aggregations - How many time do js* functions consume?

#!/usr/sbin/dtrace -s

pid$1:libmozjs:js*:entry
{
  time[probefunc] = vtimestamp;
}

pid$1:libmozjs:js*:return
/time[probefunc]/
{
  @t[probefunc] =  sum(vtimestamp - time[probefunc]);
  time[probefunc] = 0;
}

Ex8. Which gtk functions have been called by gtk_window_new?

#!/usr/sbin/dtrace -Fs

pid$1::$2:entry
{
  self->trace = 1;
}

pid$1::$2:return
/self->trace/
{
  self->trace = 0;
}

pid$1:libgtk-x11-2.0::entry,
pid$1:libgtk-x11-2.0::return
/self->trace/
{
}

Ex9. Tracing arbitrary instruction

dtrace -n pid29967:libmozjs:JS_MaybeGC:

Highlights:

• pid provider can trace any instruction in any user function;
• useful to determine the code path;

Ex10. Watch the stack - Which files failed to open?

#!/usr/sbin/dtrace -s

syscall::open:entry
/pid == $1/
{
  self->path = copyinstr(arg0);
}

syscall::open:return
/self->path != NULL && arg1 == -1/
{
  printf("open for %s failed!", self->path);
  ustack(30);
}

Highlights:

• ustack() - records a user stack trace (stack() is for kernel stack);

Ex11. $target - Tracing from the beginning

#!/usr/sbin/dtrace -s

pid$target:libc::entry
{
  @[probefunc] = count();
}

Highlights:

• pid provider can only be used on processes that are already running;

Resources

• Home page: http://dtrace.eng
• Mail alias: dtrace-interest
• Solaris Dynamic Tracing Guide: http://docs.sun.com/app/docs/doc/817-6223?q=dtrace
• Examples: /usr/demo/dtrace

Tuesday November 02, 2004

Mozilla performance decline on new kernel (Linux 2.6.5)

We noticed that mozilla's startup and page load performance slightly declined after we upgraded the Linux kernel from 2.4 to 2.6. Below are the result got from our internal tinderbox. The value is an average of 5 runs on the exactly same hardware.

             2.6     2.4   change
----------------------------------
Startup      1662    1463  +13.6%
Page Load   432.2   410.8   +5.2% 

Thursday September 16, 2004

Mozilla/Gecko Keyboard Navigation Proposal

Finally, we got it published at here. It took me a whole week for writing, discussing, gathering feedback etc. Now it is in the position to get public feedback. Cheers!

This week I'm working on another document that analyzes the advantage and disadvantage of Firefox adoption. This wasn't an interesting job, though it may affect our browser strategy.

In the meantime, I have to work on moving our accessibility site to a new location: http://www.mozilla.org/access/unix/. It's not only about moving file, but also about updating the content, refining the directory structure and file name, recoding the HTML to fit mozilla.org Documentation Style Guide. The coming 2-3 days probably will spend on this :(

I hate writing document.

Tuesday September 14, 2004

Get rid of the duplicate entries in your address book

After developed the Firefox extension Show java console, I came up with an idea to develop another extesion for mozilla address book that can remove the duplicate entries (in terms of email address), because there are many duplicate email addresses in my address book, like foo@sun.com, foo@Sun.Com, Foo@sun.com etc. It's very annoying especially when you use the auto-complete feature - the drop-down list is filled with the same guy's name which you don't want to send email to. Manually editing the abook.mab is not an option because that file is in a computer-friendly format.

The extension itself is very straightforward - it goes through every entry in gAbView (card = gAbView.getCardFromRow(i)), then get email address from each card (email = card.primaryEmail.toLowerCase()) and lower case it, compare to existing addresses to see whether it's a dup, if so, select this entry so that user can remove all of them later by just pressing Del key. It's also a good idea to lower case all email address at the mean time, but card.editCardToDatabase(gAbView.URI) must be used for committing the change to database.

BTW, the problem of mozilla not treating foo@sun.com and foo@Sun.Com as the same entry is here. Changing 4th argument from PR_FLASE to PR_TRUE would fix this problem. Any regressions? God knows.

I prepared two packages for the extension and source code, but I can't upload them to this site, sigh...

Monday September 13, 2004

Apache

This is the first time I build and configure apache web server from scratch. Thanks to this document, I was able to setup apache with mod_perl and mod_ssl together.

To make .pl files work as cgi scripts, I have to add the following lines into httpd.conf (not sure whether all of them are necessary).

PerlModule Apache::Registry
<Directory "/path/to/cgi-bin">
    SetHandler perl-script
    PerlHandler Apache::Registry
    PerlSendHeader On
    Options +ExecCGI
</Directory>

Authentication, Authorization, and Access Control

Here you can find a very useful document about how to setup authentication, authorization and access control stuff for apache. You can add those directives either in the <Directory> section in httpd.conf or a .htaccess file placed in the directory you want to control. But in order for Apache to pay attention to .htaccess, the directories in question need to be within the scope of a AllowOverride directive that includes the AuthConfig (for voluntary controls) or Limit (for involuntary controls) keywords. For example,

<Directory /secure/dir>
    AllowOverride AuthConfig Limit
</Directory>

Saturday September 11, 2004

Wiki software

There are two web pages that list many wiki software for setting up a wiki server (or just application):

• http://www.informationblast.com/Wiki_software.html
• http://www.nationmaster.com/encyclopedia/Wiki-software

I tried UseModWiki which is a perl-based cgi program. It's pretty simple (only 1 perl file) and very easy-to-setup (only need to change the database directory).

Thanks for Brion's reminder, the original article of wiki software is from Wikipedia. September 11, 2004 07:54 PM PDT Permalink

Thursday September 09, 2004

Connecting to home

I am using a wireless router for my home internet connection, that says, all my home computers will be behind a firewall. To connect my home computers from office, Ineed to:

• know the ip address of my router (it was allocated by ISP dynamicly);
• enable router's "port forwarding" function to forward the accessing of special port (say 20, 21 for FTP) to a particular computer.

Firstly, create a perl script in a http server to determine the remote ip address:

#!/usr/bin/perl -w

print "Content-type: text/plain\n\n";
$query_string = $ENV{'REMOTE_ADDR'};
print "$query_string";

Then, use another perl script poking that server to get the real internet address:

#!/usr/bin/perl

use IO::Socket;

$server = 'site';
$port = 80;
$script = '/path/to/cgi/myip.cgi';

$socket = IO::Socket::INET->new(
 Proto => 'tcp',
 PeerAddr => $server,
 PeerPort => $port,
 Timeout => 10,
 );
unless($socket) {
 die("Could not connect to $server:$port");
}
$socket->autoflush(1);
print $socket ("GET $script HTTP/1.0\nHost: $server\nUser-Agent: YY\n\n");

$ip = "";
$off = 1;
while ($line = <$socket>) {
 $off = 0 if ($line =~ /^\s*$/)?
 unless ($off) {
     $ip = $line if (length($line) > 0)?
 }
}

[snip]

Finally, I use blat to send the ip to my office's mailbox:

blat ip.addr -to xxx -cc xxx -s "ip" -u xxx -pw xxx

Now I'm able to do FTP, VNC, etc. from office to my home!

Wednesday September 08, 2004

RSS On BBS

Yesterday, I created a RSS feed in our BBS. The template was got form nytimes. It is pretty simple and straight forward (compare to the others using rdf).

Below is the file structure:

<?xml version="1.0" encoding="iso-8859-1" ?>
 <rss version="2.0">
 <channel>
     <title>...</title>
      <link>...</link>
      <description>...</description>
      <copyright>...</copyright>
      <language>...</language>
      <lastBuildDate>...</lastBuildDate>
      <item>
          <title>...</title>
          <link>...</link>
          <description>...</description>
          <author>...</author>
          <pubDate>...</pubDate>
          <guid isPermaLink="false">...</guid>
      </item>
      <item>
          ...
      </item>
  </channel>
</rss>

I hacked into the post.cgi, updated the rss file when a new post coming by

1. copy the header,
2. fill in the new item,
3. remove the oldest item if the total items exceeded 10.

Both lastBuildDate and pubDate need to be a RFC 822 compliant date format, like Tue, 07 Sep 2004 00:00:00 EDT. I spended some time to study how to do it in Perl, and came up with the following code:

use Time::gmtime;

$gm = gmtime(time);
$weekday = (qw(Sun Mon Tue Wed Thu Fri Sat Sun))[$gm->wday()];
$month = (qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec))[$gm->mon()];
$date = sprintf("%s, %02d %s %d %02d:%02d:%02d GMT", $weekday, $gm->mday(),
                $month, $gm->year() + 1900,
                $gm->hour(), $gm->min(), $gm->sec());

The guid field value need to be globally unique. So I combine the site URL and the current time together to make it a really GUID.

I downloaded a RssReader to verify my work. It works very well. I also found a useful site that can verify whether your feed is valid.

RSS 2.0 spec

Wednesday September 01, 2004

Open file:// in java applet

One customer reported 4 months ago that she can't do this in java applet (bug 5005607):

URL url = new URL("file:///path/to/file");
applet.getAppletContext().showDocument(url,"_blank");

After some investigation, I found it is controlled by nsScriptSecurityManager::CheckLoadURIWithPrincipal. If the source scheme is http://, the other protocols it can access are controled by this table. The entry for file:// is "PrefControlled", and the pref entry is "security.checkloaduri" which is originally set to true in all.js. That means opening file:// url from http:// is generally disallowed.

This bug does not appear in mozilla 1.2.1 because there was no this kind of security checking at that time. Our problem is since the java applet is signed, it is supposed to do anything it wants. But unfortunately, mozilla does not know what "signed applet" means yet. So...

Tuesday August 31, 2004

Dependency

I have struggled with a library dependency issue for days. It is the WebBrowser component in JDIC.

As the xpcom library has a lot of backward compatibility issue, my app has to use xpcomglue library, and it also depends on libgtkembedmoz.so, libgtkembedmoz.so depends on libxpcom.so (my app doesn't). But if I link my app without -lxpcom, ld will complain:

ld: warning: libxpcom.so, needed by ../../../../dist/bin/libgtkembedmoz.so, not found (try using -rpath or -rpath-link)

I wonder why the dependency of libxpcom.so was not solved during linking libgtkembedmoz.so? Another thing confused me is that this only happens on Linux, Solaris' ld works fine with the same source code/makefile. How does Linux ld work? Is there any tools on Linux which can find out the exact dependencies, i.e. what symbol belongs to what library?

Friday August 27, 2004

My first project @mozdev

A few days ago, I posted my first project to mozdev - Show Java Console. It is a Firefox extension which adds a menu under Tool menu to show the Java Console.

The project itself is trivial - not more than 20 lines of code. But it lets me get familiar with many things:

• how to create an extension for Firefox/Mozilla (what files should be included, what files should be put to where, how to write the makefile, installation script, etc.);
• the procedure of how to register a new project in mozdev;
• how to setup the webpages of your project;
• how to submit your extension to the mozilla update site.

OK, now click this link to get showjavaconsole installed on your Firefox.

Thursday August 26, 2004

GCC 3.3.3

Yesterday, a build error reported against mozilla/modules/libpref/src/nsPrefBranch.cpp, it was

nsPrefBranch.cpp:88: error: brace-enclosed initializer used to
initialize `const char*'

It used to work fine for months with gcc 3.2.2, but apparently gcc 3.3.3 changed the rule. (I wonder why the Ireland team changed their kernel & compiler without notifying us.)

It's not difficult to find the gcc-3.3.3 source tarball from one of their mirror sites. The build procedure is pretty straight forward:

1. extract the source into srcdir;
2. make a new dir to hold the obj files objdir;
3. run ../srcdir/configure --prefix=$HOME/local from objdir;
4. make bootstrap;
5. make install;

Everything will be built up and installed in your $HOME/local/bin, setting CC=$HOME/local/bin/gcc, CXX=$HOME/local/bin/c++, you will be able to build mozilla using the new compiler.

BTW, the error was:

static const char* gSystemPrefKeys[] = {
    {"network.proxy.http"},
    {"network.proxy.http_port"},
    ...

Removing the {} around the strings fixed it. I don't understand why all other compilers (VC, gcc 3.2.2, Forte) are happy with that but gcc 3.3.3 doesn't.