One of my responsibilities on the
Update Center 2.0
project is to perform
builds of Python and wxPython for all of our supported platforms.
One unique aspect of our environment is that we need these builds
to be relocatable as far as the filesystem is concerned in order to
support our
multi-install requirement. That is:
plop the build anywhere on your system and it must work.
A key to making relocatable software work is relative paths. All
references to files within the relocatable install image must be
relative (or at least start out relative and only made absolute
dynamically at runtime). This includes any dependencies that binaries
in the install image might have on dynamic libraries in
the install image.
You can always use LD_LIBRARY_PATH (or
DYLD_LIBRARY_PATH on
the Mac) to force the runtime linker to locate the right
dynamic libraries -- but that's cheating, and should only be used as
a hack of last resort. Better to construct the binaries correctly
so that they can locate their dependencies without the need to
alter the user's environment.
I'm pretty familiar with dynamic libraries on Solaris and Linux.
On those platforms you can embed an RPATH into a binary that is
searched by the runtime linker to locate libraries. Plus both Solaris
and Linux support the $ORIGIN token so you can make these paths
relative to the install location of the binary.
On Solaris you can see these settings by using dump -Lv:
$ dump -Lv wx/_core_.so
wx/_core_.so:
**** DYNAMIC SECTION INFORMATION ****
.dynamic:
[INDEX] Tag Value
[1] NEEDED libCrun.so.1
[2] NEEDED libwx_GTK2u_richtext-2.8.so.0.4.0
[3] NEEDED libwx_GTK2u_aui-2.8.so.0.4.0
[4] NEEDED libwx_GTK2u_xrc-2.8.so.0.4.0
[5] NEEDED libwx_GTK2u_qa-2.8.so.0.4.0
[6] NEEDED libwx_GTK2u_html-2.8.so.0.4.0
[7] NEEDED libwx_GTK2u_adv-2.8.so.0.4.0
[8] NEEDED libwx_GTK2u_core-2.8.so.0.4.0
[9] NEEDED libwx_baseu_xml-2.8.so.0.4.0
[10] NEEDED libwx_baseu_net-2.8.so.0.4.0
[11] NEEDED libwx_baseu-2.8.so.0.4.0
[12] INIT 0xc8ef4
[13] FINI 0xc9000
[14] RUNPATH $ORIGIN/../wxWidgets/lib
[15] RPATH $ORIGIN/../wxWidgets/lib
[16] HASH 0x94
Here the _core_.so binary has dependencies on a number
of wx libraries. RPATH is set to look for libraries relative to the
install location of _core_.so, and therefore the linker can
find these libraries without needing to set LD_LIBRARY_PATH. You
specify the value for RPATH at link time using the
-R option.
Linux is similar. In this case you use objdump -p to inspect
the binaries:
$ objdump -p wx/_core_.so
_core_.so: file format elf32-i386
. . .
Dynamic Section:
NEEDED libwx_gtk2u_richtext-2.8.so.0
NEEDED libwx_gtk2u_aui-2.8.so.0
NEEDED libwx_gtk2u_xrc-2.8.so.0
NEEDED libwx_gtk2u_qa-2.8.so.0
NEEDED libwx_gtk2u_html-2.8.so.0
NEEDED libwx_gtk2u_adv-2.8.so.0
NEEDED libwx_gtk2u_core-2.8.so.0
NEEDED libwx_baseu_xml-2.8.so.0
NEEDED libwx_baseu_net-2.8.so.0
NEEDED libwx_baseu-2.8.so.0
NEEDED libstdc++.so.6
NEEDED libm.so.6
NEEDED libgcc_s.so.1
NEEDED libpthread.so.0
NEEDED libc.so.6
RPATH $ORIGIN/../wxWidgets/lib
INIT 0x1d384
. . .
But what about the Mac? That's new territory for me. Here is what
I learned -- I welcome comments since I likely know just enough to
be dangerous.
On the Mac a dynamic library (dylib) has an "install name". The
install name is a path baked into the dynamic library that says
where to find the library at runtime. When you link against the dylib
this path is saved in your binary so that your binary can find
the dylib at runtime. Seems a bit backwards to me -- but that's how
it works.
You can see the install name of a dylib by using otool. For example:
$ otool -D libwx_macu-2.8.0.4.0.dylib
libwx_macu-2.8.0.4.0.dylib:
/Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib
You can also use otool to inspect binaries and list their dependencies:
$ otool -L wx/_core_.so
wx/_core_.so:
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0)
/Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib
(compatibility version 5.0.0, current version 5.0.0)
. . .
So wx/_core_.so depends on libwx_macu-2.8.0.dylib,
which is fine. But notice that evil absolute path. That won't work in our
"install anywhere" world. So how do we fix this? What's the Mac OS equivalent
of $ORIGIN and RPATH?
Well, there isn't any. But there is something we can do instead. After
the build is complete I use the install_name_tool utility to
fix up the dylib install names and dependencies in our binaries. Fortunately
the Mac also supports a magic token: @loader_path that can be used
in a fashion similar to $ORIGIN on Solaris/Linux.
So when building wxPython on the Mac I first do a complete build. This results
in the absolute paths being used as mentioned above. Then as part of
my "make install" step I fix up these paths to be relative using the
install_name_tool utility.
For example, this command changes the install name of
libwx_macu-2.8.0.4.0.dylib
to be relative to the location of the binary using it:
$ install_name_tool -id "@loader_path/../wxWidgets/lib/libwx_macu-2.8.0.4.0.dylib"
libwx_macu-2.8.0.4.0.dylib
And this changes the dependency in a binary to use a relative path
to locate the library (relative to the install location of the binary).
$ install_name_tool -change "/Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib"
"@loader_path/../wxWidgets/lib/libwx_macu-2.8.0.dylib" wx/_core_.so
This mechanism is still not as flexible as the Solaris/Linux approach, since
binaries can't specify a search path. It's also a bit more difficult to
automate since you must determine the current install name in order to replace
it with the new install name. But it does work.
Note that for your project you may be able to simplify this. For example,
if you build your dynamic libraries with the correct install names first,
then your binaries will pick up the correct install names at link time
and you shouldn't need to change the dependencies post-build.