Okay, perhaps I should have said, welcome to OpenSolaris, a much more exciting topic! But if you intend to make changes to any of the headers defined by the various operating systems standards, or you simply want to better understand what the heck is going on in the headers, this entry will hopefully get you started. Just by their very nature, the headers have always been public and viewable, but in this context, many would probably agree that viewable and readable are not the same, at least when it comes to those headers defined by standards.

First things first

I have worked at Sun since 1987 and during much of that time I have been involved in one form or another with operating system standards, specifically X/Open and POSIX. I was the Sun X/Open Verification Working Group representative for a number of years and eventually represented Sun on the POSIX.1 working group. In the latter role, I was a technical editor for the POSIX.1a amendment which was eventually folded into what is commonly referred to as the Austin Group Specification, the merged POSIX and X/Open specifications. Though my role in standards and standards implementation is more peripheral these days, I have either led teams or been part of  teams implementing POSIX and X/Open standards in the Solaris OS. My focus has been in the areas of the C library, commands, and headers.

Why header hell?

It seems the most visible casualty of all these standards requirements are the headers. Each of the standards is very specific about which symbols are allowed within that particular standards name space. The Solaris headers continue to cause grief to both myself and anyone who has ever had the misfortune to touch a header defined by the C, C++, POSIX, or X/Open standards. Coupled with the fact that we continue to support a number of different versions of the standards in order to maintain compatibility across releases and the fact that each of the standards and versions are not always a direct subset or superset of each other, and you begin to see why things get so confusing. And this doesn't even begin to address the fact that there may be different interpretations of standards requirements. A prime example of this is the Sun versus GNU C interpretation of the value of __STDC__. In Solaris 10, we have tried to accommodate these differences in interpretations in an effort to co-exist more peacefully, thus hopefully creating a win win for everyone.

Determining which headers are defined by a standard

How does one know if a header is defined by a particular standard? Unless you have access to the standards specifications, on the Solaris implementation, a good indication of whether or not a header is defined by a standard is if the header either includes <sys/feature_tests.h> or uses any one of the following feature test macros:

__STDC__
__STDC_
_STRICT_STDC
_XOPEN_*
__XOPEN_*
__XPG*
_POSIX_*

If you see any of these in a header, tread lightly! Adding a new symbol in the wrong area of the header could result in standards name space pollution and an automatic high priority bug report.

Rules for name space

Adding a new symbol to a header defined by any of the aforementioned standards comes with the risk of polluting the standards name space if either the new symbols are not guarded properly (i.e. hidden from the X/Open, POSIX, or C name space) or do not use names otherwise acceptable in the standards name space. I won't try to explain all of the rules here, but there are some general rules that apply to the restricted name space:

• Except for headers defined by the C standard, any type definition ending in _t is allowed

• Any symbol beginning with _[A-Z] is allowed

• Any symbol beginning with __[A-Za-z] is allowed

Of course any symbol added by the implementation or application must not already be reserved for use by a particular standard.  For the Austin Group Specification referenced earlier and also referred to as UNIX03, or more accurately as IEEE Std 1003.1-2001 or the Single UNIX Specification Version 3, name space is described under the section on Compilation Environment under XSH. While the tables are specific to the latest revision of the specification, the text and tables are helpful in understanding if you are at least on the right path.

So back to Solaris...

You may not be convinced, but <sys/feature_tests.h> is your friend. There is a lot of information in this file including detailed explanations of many of the feature test macros, and in particular Sun implementation macros, used in the standards headers. Whether you intend to add a new interface in a header defined by a standard or simply want to understand why it is things are not behaving as expected with respect to visibility of symbols, the first place you should look is this header. The header was initially created as a means of consolidating a number of standards feature test macro definitions into a single place. In Solaris 10, we expanded the definitions by collapsing a number of commonly used feature test macros into additional Sun implementation feature test macros. For example any place that previously had:

#if defined(_XOPEN_SOURCE) || defined(_POSIX_C_SOURCE)

instead uses:

#if defined(_XOPEN_OR_POSIX) 

And some of you may recall the use of the following throughout headers defined by the C Standard:

#if (__STDC__ - 0 == 0)

or

#if (__STDC__ - 0 != 1)

On Solaris, we define __STDC__ to be 1 for a strictly conforming ANSI C application -- one that uses only the set of interfaces defined by the C standard. When __STDC__ is 0, we accept ANSI semantics, but in a much less restricted name space. Reference the Sun Studio Compilers manual page cc(1) for more information on the different compilation modes designated via the use of the -X[acts] option. We created a new feature test macro _STRICT_STDC so that the above guards could be replaced with something a bit more readable such as:

#if !defined(_STRICT_STDC)

Note that our _STRICT_STDC_ is equivalent to the GNU C __STRICT_ANSI__ build environment.

We collapsed some of the commonly used macro combinations for a number of reasons:

• The #ifdef lines were getting unreasonably long and increasingly difficult to parse

• A consensus among our OS developers that the second form was more readable

• The ability to combine a number of macros into a single macro, so that multiple versions of a specification can be specified with a single macro

Of course, the down side of this is that the headers can be a bit more confusing if <sys/feature_tests.h> is not consulted when reviewing or making changes to any header that relies on the definitions within that header. For example, a number of macros exists to indicate different versions of X/Open. These include _XPG3, _XPG4, _XPG5, etc. One should be aware that the use of #ifdef _XPG4 now implies support for XPG4, XPG5 and XPG6, not just XPG4. It does not imply support for XPG3.

Guidelines for adding new symbols

So let's assume you are adding a new interface that is related to a number of other interfaces already defined in a header specified by one of the standards. Or perhaps you are adding functionality to a previously existing interface that requires symbol extensions. In the first case, rather than create a new header, it seems best to group this new interface within the family of the previously existing interfaces.  In the second case, adding a new symbol with a reserved prefix (e.g LC_ in <locale.h>) may be allowed, but let's assume that this is a new symbol that will pollute the standards name space. In general, when adding a new symbol to a header defined by the standard, if that symbol is not included amongst the names reserved for the implementation, you will likely be guarding it with one of following:

For headers specified by both X/Open and POSIX, such as <unistd.h>:

#include <sys/feature_tests.h>

#if !defined(_XOPEN_OR_POSIX) || defined(__EXTENSIONS)

For headers specified by X/Open but not POSIX, such as <dlfcn.h>:

#if !defined(_XOPEN_SOURCE) || defined(__EXTENSIONS__)

For headers specified by POSIX but not X/Open:

#if !defined(_POSIX_C_SOURCE || defined(__EXTENSIONS__)

For headers specified by X/Open, POSIX, and the C Standard, such as <signal.h>:

#include <sys/feature_tests.h>

#if !defined(_XOPEN_OR_POSIX) && !defined(_STRICT_STDC) || \
        defined(__EXTENSIONS__)

It is quite likely these guards are already in place and adding the new interface is as simple as making sure it is placed in the correct area.

Worth noting is that in all of these examples, __EXTENSIONS__ is included in the guard macros. Whether we are guarding visibility within the standards name space or adding a new feature test macro for restricting visibility, we generally make any new symbols visible if -D__EXTENSIONS__ is defined. The exception would be if there is a conflict with another symbol visible in the default name space, but this is rare. The use of __EXTENSIONS__ essentially results in a non-restricted name space, though in combination with other feature test macros, one may get different definitions or declarations for those few cases where there are multiple definitions or declarations.

Note also the inclusion of <sys/feature_tests.h> in two of the examples. It is unlikely that you will need to add the inclusion of this header as it is probably already included either directly or in a top level header, but I've included it here just to help clarify the distinction between the various feature test macros. It is important to understand the difference between the standards specified feature test macros such as _XOPEN_SOURCE, _POSIX_C_SOURCE and Sun implementation specific macros such as _XOPEN_OR_POSIX or _XPG6. The latter are defined in <sys/feature_tests.h>.

Also note that from an application standpoint, the only valid feature test macros are those specified by the desired standard. In other words, the Sun implementation feature test macros such as _XPG3 or _XPG4 should never be specified at compilation time; their use is limited to within the headers. While the first example that follows may build without warnings or errors, it is likely to result in unexpected behavior.

c99 -D_XPG6 foo.c [ WRONG ]

c99 -D_XOPEN_SOURCE=600 foo.c [ CORRECT ]

See standards(5) for more detail on the standards compilation environments.

Testing and debugging headers

When making updates to the headers, there are few rules worth noting. In addition to doing full builds in the case of public header updates, testing should also include but not necessarily be limited to the following:

• Building of a test program with the Sun Studio C compiler using each of the -Xa (default), -Xc (strictly conforming), -Xt (transitional), and -Xs (K&R) compiler options. See cc(1) for more detail. Verify preprocessor output via cc -E.

• Building of a test program with each of -D_XOPEN_SOURCE, -D_POSIX_C_SOURCE, and -D__EXTENSIONS__ feature test macros specified, as well as combinations of all three. Verify preprocessor output via cc -E.

• If using the Sun Studio C compiler, testing in both C89/C90 mode and C99 mode via the use of the -xc99=none option. See cc(1) for more detail.

• Not necessarily needed for testing, but like cc -E, sometimes helpful in debugging, is the use of cc -H to see exactly what headers are getting pulled in and from where. This can be especially helpful if either the Sun Studio C++ compiler or the GNU C compiler is being used, as both of these have their own versions of some common headers.

• Building with the GNU C compiler. Note that it too supports the -E and -H options.

• If you are using GNU C, be aware that some Solaris headers may have been updated via the use the fixincludes utility, fixincl. For gcc 3.4.3, these are limited to <math.h>, <stddef.h>, <stdio.h>, and <sys/types.h>. If you touch any of these headers and try to build using gcc, you will likely need to rerun the fixinc utility.

• Build a test program using C++ (CC and g++).

• Ideally, any updates to the standards headers should also be tested using the applicable X/Open test suites assuming one has access to the test suites. The header tests are very good at identifying name space problems.