Back from my summer hols in sunny Nice and armed with a nice VMWare image of Solaris 10 GA wanted to take a look at memory consumption of JDS and Gnome in particular on Solaris.
Looked at the "Memory Reduction Project":http://live.gnome.org/MemoryReduction page which gave me plenty of ideas. A lot of tweaking had also been done internally by Sun for the build process back in the Gnome 2.0 days, but I wanted to look at things afresh so to speak.
Grabbed a script one of the RE engineers had put together to process the pmap -x output of all running processes under JDS to get a system snapshot of memory usage, both shared and private.
Shows JDS is using around 35M private and 45M shared memory. What was a surprise is the amount of Private taken up by the heap for various gnome daemons, apps and XServer, around 25M in all. And this was just after a clean boot - so no web browser running or gedit up and running, this was just the basic desktop.
Wanted to dig a little more so grabbed a "helloworld gtk+":http://www.gnomebangalore.org/?q=node/view/14
app and built and ran it. Doing a pmap -x on it I get heap usage of ~850K. It is also pulling in about 18 other libs and ends up consuming 1.5 M Private memory 
*lazyloading*
So does it need to pull in all these libs? Found "Rod Evan's blog":http://blogs.sun.com/rie really useful here when digging into all things to do with the linker. So wanted to explore what was going on and if I could reduce the large library load for the simple hello world app by tweaking the linker options.
Initially just built it with:
* cc helloworld.c -o helloworld `pkg-config -cflags -libs gtk+-2.0`
* Used $ldd -U -r helloworld to look for unused dependencies. Gives indication on how successful lazy loading might be. Saw a few so thought worth pursuing.
Went back the gnome team and Damien in RE kindly rebuilt the gnome stack with:
-z ignore and -z lazyload [-z ignore refs to unused dependencies and -z lazyload defer loading of referenced dependencies until required by running app]
So back to my hello world, but with my gnome stack built with lazyload.
* Rebuild hello world for lazy loading
* Confirm linked libraries will be lazyloaded, use $elfdump helloworld - have a [LAZY] entry preceding library entry to indicate lazy loading will be used
* Run in linker debug mode $LD_DEBUG=files helloworld
Can see what is being lazy loaded and when.
See gtk_init symbol being referenced from helloworld, see all the other 17 libs getting pulled in via lazyloading at this stage.
So bottom line is, lazyloading seems to make very little difference to memory consumption of the hello world gtk+ app Presumably what's happening is that the gnome init functions are pulling in all the additional libs on first load. So to prevent this happening some rearchitecting of the init functions would need to happen.
*DTrace aside:*
Saw "Brian Cantrill's blog":http://blogs.sun.com/roller/page/bmc?entry=using_dtrace_to_understand_gnome on using dtrace to check out gnome startup and was curious to see if having built the gnome stack with lazyload there would be any noticeable difference in amount of I/O [lots due to loading of shared libs]. Ran his script on vanilla Solaris 10 FCS with Gnome 2.6 and after with modified Gnome 2.6 built with ignore and layzload.
No noticeable time difference in both startup's [bit of variation in time it takes me to kill dtrace when it all comes up, but in the noise level].
* I/O reduction of about 10% using ignore and lazyload directives.
* About half of it seemed to be coming from Nautilus's loading of icons [i did not open nautilus directly in either case], rest of this reduction was spread over the various gnome app's daemons. Know there's work ongoing with icon cache to reduce this in Nautilus anyway.
Will rerun the tests when I get me hands on a Gnome 2.10 build for Solaris. Also vague memory of Jeff Waugh posting about lazyloading in Ubuntu where they did see improvements in memory usage and load time, but could be wrong.
*lazyloading*
So does it need to pull in all these libs? Found "Rod Evan's blog":http://blogs.sun.com/rie really useful here when digging into all things to do with the linker. So wanted to explore what was going on and if I could reduce the large library load for the simple hello world app by tweaking the linker options.
Initially just built it with:
* cc helloworld.c -o helloworld `pkg-config -cflags -libs gtk+-2.0`
* Used $ldd -U -r helloworld to look for unused dependencies. Gives indication on how successful lazy loading might be. Saw a few so thought worth pursuing.
Went back the gnome team and Damien in RE kindly rebuilt the gnome stack with:
-z ignore and -z lazyload [-z ignore refs to unused dependencies and -z lazyload defer loading of referenced dependencies until required by running app]
So back to my hello world, but with my gnome stack built with lazyload.
* Rebuild hello world for lazy loading
* Confirm linked libraries will be lazyloaded, use $elfdump helloworld - have a [LAZY] entry preceding library entry to indicate lazy loading will be used
* Run in linker debug mode $LD_DEBUG=files helloworld
Can see what is being lazy loaded and when.
See gtk_init symbol being referenced from helloworld, see all the other 17 libs getting pulled in via lazyloading at this stage.
So bottom line is, lazyloading seems to make very little difference to memory consumption of the hello world gtk+ app Presumably what's happening is that the gnome init functions are pulling in all the additional libs on first load. So to prevent this happening some rearchitecting of the init functions would need to happen.
*DTrace aside:*
Saw "Brian Cantrill's blog":http://blogs.sun.com/roller/page/bmc?entry=using_dtrace_to_understand_gnome on using dtrace to check out gnome startup and was curious to see if having built the gnome stack with lazyload there would be any noticeable difference in amount of I/O [lots due to loading of shared libs]. Ran his script on vanilla Solaris 10 FCS with Gnome 2.6 and after with modified Gnome 2.6 built with ignore and layzload.
No noticeable time difference in both startup's [bit of variation in time it takes me to kill dtrace when it all comes up, but in the noise level].
* I/O reduction of about 10% using ignore and lazyload directives.
* About half of it seemed to be coming from Nautilus's loading of icons [i did not open nautilus directly in either case], rest of this reduction was spread over the various gnome app's daemons. Know there's work ongoing with icon cache to reduce this in Nautilus anyway.
Will rerun the tests when I get me hands on a Gnome 2.10 build for Solaris. Also vague memory of Jeff Waugh posting about lazyloading in Ubuntu where they did see improvements in memory usage and load time, but could be wrong.
Note, lazy-loading can be compromised by dlsym(3c) lookups that are unsatisfied. See lazy loading fallback.
A dlsym(3c) can exhaust many pending lazy loads in an attempt to locate the symbol. Using dlopen(3c) with the new RTLD_FIRST flag, or dlsym(3c) with RTLD_PROBE can suppress this lazy load rescan.
Posted by Rod Evans on July 29, 2005 at 05:52 PM IST #