Paul Hinker's Weblog

More on Custom Libraries

I've been proposing the idea of custom libraries within the Sun Studio product group and the Sun Performance Library group for years. In a previous blog I wrote about some of the benefits this idea could bring to both developers and to people like me who need to make decisions concerning where to apply (ever scarcer) tuning resources. I've been using an in-house version of a tool that can create custom libraries from a variety of sources. I thought I would show a couple examples of its usage:

Custom Lib from a object file

One of the most straightforward examples is the case of an object file that has external references to Perflib

% cat main.f
      program driver
      implicit none
  
      integer, parameter    :: n = 100, incx = 1, incy = 1
      real(8), dimension(N) :: x, y, result
      real(8)               :: ddot, alpha = 1.0

      x = 2.0
      y = 1.2
 
      call daxpy(n, alpha, x, incx, y, incy)
      print *, "Sum(daxpy) = ", sum(y)
      end

Here's a very simple main that calls the double precision AXPY from the Sun Performance Library. Looking at the nm on the .o file we can see that there's a single undefined external reference to Perflib (of course there are additional references to the fortran run time).

% nm main.o | grep -i undef
[28]    |             0|           0|FUNC |GLOB |0    |UNDEF  |__f90_eslw
[19]    |             0|           0|FUNC |GLOB |0    |UNDEF  |__f90_init
[26]    |             0|           0|FUNC |GLOB |0    |UNDEF  |__f90_slw_ch
[27]    |             0|           0|FUNC |GLOB |0    |UNDEF  |__f90_slw_r8
[25]    |             0|           0|FUNC |GLOB |0    |UNDEF  |__f90_sslw
[24]    |             0|           0|FUNC |GLOB |0    |UNDEF  |daxpy_
[18]    |             0|           0|FUNC |GLOB |0    |UNDEF  |f90_init

The object file can now be used as a target to generate a custom library:

% createCustom main.o -lib /opt/SunStudioExpress/prod/lib/amd64/libsunperf.a
Created libCustom.a
Created libCustom.so

The tool creates a static and shared version of the library that contains only the routines either directly referenced by the .o file or routines that support those directly referenced routines. There are a number of benefits to this. One is that the resulting custom library is considerably smaller than the entire Performance Library :

% ls -l libCustom*
-rw-r--r--   1 hinker   staff      14812 Nov 20 13:15 libCustom.a
-rwxr-xr-x   1 hinker   staff      12288 Nov 20 13:15 libCustom.so
% ls -l /opt/SunStudioExpress/prod/lib/amd64/libsunperf.*
-rwxr-xr-x   1 root     sys      28289228 Oct 30 15:31 /opt/SunStudioExpress/prod/lib/libsunperf.a
-rwxr-xr-x   1 root     sys      20747884 Oct 30 15:31 /opt/SunStudioExpress/prod/lib/libsunperf.so.3

Of course, this is a simple example but it's not uncommon to see a custom library that's 10% the size of the full library.

Another nice feature is that a custom library can be generated using a variety of targets:

* Object files
* Dynamically linked executables
* Shared libraries
* Archive libraries

The custom shared library that's produced can be used as a drop-in replacement for the existing libsunperf.so without requiring the executable to be relinked.

We hope to rollout the tool with an upcoming release to see how it's received by our users.

Posted at 01:45PM Nov 20, 2008 by hinkthink in General  |