Morgan Herrington's Blog Adventures in Porting and Tuning

I've learned to expect almost complete upward compatibility when moving to a new release of Solaris and to expect nearly identical behavior when moving between platforms running Solaris. Only occasionally will a platform differences show through -- for example, the different VM page sizes on x86 as compared to SPARC.

Occasionally, I'll run across a difference which is not so obviously forced by the platform. One example of this showed up when a simple interpose routine (used to gather some runtime statistics) would not compile on 32-bit x86 Solaris even though there were no problems compiling it for x64 or for SPARC:

#include <sys/types.h>
#include <sys/stat.h>
#include <dlfcn.h>

typedef int (*stat_functype)(const char *path, struct stat *buf);

int
stat(const char *path, struct stat *buf)
{
    static stat_functype stat_handle = 0;
    if (stat_handle == 0)
        stat_handle = (stat_functype)dlsym(RTLD_NEXT, "stat");

    int retvalue = stat_handle(path, buf);
    /* interposed instrumentation code elided for brevity */
    return retvalue;
}

The problem turned out to be that in the x86-specific header file, <sys/stat_impl.h>, stat is defined (not just declared) as a file-local "static". Something like:

static int
stat(const char *_path, struct stat *_buf)
{
    return (_xstat(2, _path, _buf));
}

When the application code tried to define the interpose routine, the compiler complained about seeing multiple definitions for the symbol.

I couldn't see a platform-specific reason for this implementation difference, but others explained to me that it was probably a historical difference. If I understand correctly, Solaris on x86 used to provide some level of binary compatibility with a legacy Unix (maybe SCO Unix or Interactive Unix). By using this implementation trick, Solaris-compiled binaries would have SVR4 semantics (by using _xstat()), but non-Solaris binaries would get their legacy semantics from stat(). Please take my version of this rationale with a grain of salt; however, it's the best explanation I've heard.

As of today, one of the kernel engineers submitted a request to remove this feature (ie. treat it like a bug). However, until/unless it is changed, how do you work around it?

The most common suggestion I received was to compile the 32-bit code large file aware. Since this only requires a tweak to the compiler options, it may be the easiest solution:

$ cc -D_FILE_OFFSET_BITS=64 interpose_stat.c

However, if compiling for large files is not acceptable (for example if you're also using /proc), another work-around would be to interpose on the _xstat function. Don't settle on this solution lightly though, because it is NOT guaranteed to be supported or portable. Even the header file warns:

  /*
   * NOTE: Application software should NOT program
   * to the _xstat interface.
   */