Say goodbye to /usr/ccs/bin. Actually, the path is still intact in Solaris Nevada. The contents however have been moved to /usr/bin, with symbolic links remaining in /usr/ccs/bin for compatibility.

With this update, it is no longer necessary to include /usr/ccs/bin in your path. The exception is if you call either of the sccs subcommands "admin" or "help" directly rather than via "sccs admin" or "sccs help". These two sccs subcommands, owned by the compiler team, were not moved to /usr/bin with their putback. However, everything else that teams owns, including make and sccs, moved to /usr/bin a number of builds back (build 42). This latest integration to build 68 completes the migration of the /usr/ccs/bin utilities, including many of the linker and build tools, to /usr/bin.

How will this affect you? It shouldn't unless you have an identically named utility included in your search path that was previously before /usr/ccs/bin but after /usr/bin. The ld command in /usr/ucb/bin is one possibility. Another is gnu yacc in /opt/csw/bin. But for most, the change will be transparent; one will simply have the option now of no longer needing to include /usr/ccs/bin in the PATH variable.

Technorati Tag: Opensolaris
Technorati Tag: Solaris

I have discovered that having a visible injury is a real attention getter, almost like walking a new puppy, minus the oohing and aahing. Maybe if I had one of those nifty hot pink casts, I would get the "Ooh, what a pretty cast!" comments. Then again, maybe not...

Instead, my heavily fortified hand and wrist generates the more common "What happened to your hand/wrist/arm?" question from both friends and strangers. Telling them that I broke the 5th metacarpal is usually a bit too clinical and just not good enough. Folks want to know how I broke it and most are expecting a pretty good story, especially those that know me well. There is an odd almost eager anticipation of a pretty good bicycle or motorcycle crash story. Not that I have a habit of crashing... Well, okay, I have had a couple of good bicycle get-offs in recent years, but nothing resulting in broken bones. A few have asked in jest -- or at least I hope in jest -- whether or not I decked my husband. For the record, my husband may have his moments, but for the most part he is a sweetheart. Decking him would be highly inappropriate, wrong, and just plain mean.

So how did I break my hand? Well, it was like this... I was on my Ducati enjoying a most excellent open track day at Laguna Seca Raceway. I was feeling great! No one could catch me! As I entered the Corkscrew, I threw the bike hard to the left and then made the quick transition to the right. What a rush! And then disaster! At that very moment, a wild turkey leapt out in front of me. In an effort to avoid the little gobbler, I made the unfortunate mistake of grabbing the front brake. Very bad idea. The bike stood up and off I went in a most impressive high side, my hand taking the first hit as my body returned to the ground. Ouch! Poor bike, poor hand, happy turkey.

Did that really happen? Well, not exactly. But the fact of the matter is that the true story is both boring and causes some folks to get a bit squeamish. And those that have heard both stories, overwhelmingly prefer the first. You decide, but remember, you have been warned. The slightly embellished real story...

So I thought I was safe in the confines of my own home. Without warning, a stackable desk chair suddenly appeared in front of me, grabbed my hand and then tossed me to the ground while my hand remained attached to the chair, firmly wedged between the backrest and the armrest until it literally snapped free. Yes, snapped. It was a freak accident and entirely unexciting except for a couple of kittens who were tittering in the corner while I did my best to maintain composure. Did I really catch a glimpse of an outstretched paw as I went down? And to think that I was worried about them tearing up the furniture. Who would have thought that the little monsters would also be a threat to my well-being.

So that is the embarrassing truth -- I was attacked by a killer chair with the help of a couple of rambunctious kittens. As for the kittens who were unnamed at the time of the accident, I have seriously considered naming them after two of my doctors, or perhaps Pin1 and Pin2 for the pins that now keep the broken bone intact as it heals. But then I might forever be forced to repeat this story when asked how I came about naming the kitties....

Okay, perhaps I should have said, welcome to OpenSolaris, a much more exciting topic! But if you intend to make changes to any of the headers defined by the various operating systems standards, or you simply want to better understand what the heck is going on in the headers, this entry will hopefully get you started. Just by their very nature, the headers have always been public and viewable, but in this context, many would probably agree that viewable and readable are not the same, at least when it comes to those headers defined by standards.

First things first

I have worked at Sun since 1987 and during much of that time I have been involved in one form or another with operating system standards, specifically X/Open and POSIX. I was the Sun X/Open Verification Working Group representative for a number of years and eventually represented Sun on the POSIX.1 working group. In the latter role, I was a technical editor for the POSIX.1a amendment which was eventually folded into what is commonly referred to as the Austin Group Specification, the merged POSIX and X/Open specifications. Though my role in standards and standards implementation is more peripheral these days, I have either led teams or been part of  teams implementing POSIX and X/Open standards in the Solaris OS. My focus has been in the areas of the C library, commands, and headers.

Why header hell?

It seems the most visible casualty of all these standards requirements are the headers. Each of the standards is very specific about which symbols are allowed within that particular standards name space. The Solaris headers continue to cause grief to both myself and anyone who has ever had the misfortune to touch a header defined by the C, C++, POSIX, or X/Open standards. Coupled with the fact that we continue to support a number of different versions of the standards in order to maintain compatibility across releases and the fact that each of the standards and versions are not always a direct subset or superset of each other, and you begin to see why things get so confusing. And this doesn't even begin to address the fact that there may be different interpretations of standards requirements. A prime example of this is the Sun versus GNU C interpretation of the value of __STDC__. In Solaris 10, we have tried to accommodate these differences in interpretations in an effort to co-exist more peacefully, thus hopefully creating a win win for everyone.

Determining which headers are defined by a standard

How does one know if a header is defined by a particular standard? Unless you have access to the standards specifications, on the Solaris implementation, a good indication of whether or not a header is defined by a standard is if the header either includes <sys/feature_tests.h> or uses any one of the following feature test macros:

__STDC__
__STDC_
_STRICT_STDC
_XOPEN_*
__XOPEN_*
__XPG*
_POSIX_*

If you see any of these in a header, tread lightly! Adding a new symbol in the wrong area of the header could result in standards name space pollution and an automatic high priority bug report.

Rules for name space

Adding a new symbol to a header defined by any of the aforementioned standards comes with the risk of polluting the standards name space if either the new symbols are not guarded properly (i.e. hidden from the X/Open, POSIX, or C name space) or do not use names otherwise acceptable in the standards name space. I won't try to explain all of the rules here, but there are some general rules that apply to the restricted name space:

• Except for headers defined by the C standard, any type definition ending in _t is allowed

• Any symbol beginning with _[A-Z] is allowed

• Any symbol beginning with __[A-Za-z] is allowed

Of course any symbol added by the implementation or application must not already be reserved for use by a particular standard.  For the Austin Group Specification referenced earlier and also referred to as UNIX03, or more accurately as IEEE Std 1003.1-2001 or the Single UNIX Specification Version 3, name space is described under the section on Compilation Environment under XSH. While the tables are specific to the latest revision of the specification, the text and tables are helpful in understanding if you are at least on the right path.

So back to Solaris...

You may not be convinced, but <sys/feature_tests.h> is your friend. There is a lot of information in this file including detailed explanations of many of the feature test macros, and in particular Sun implementation macros, used in the standards headers. Whether you intend to add a new interface in a header defined by a standard or simply want to understand why it is things are not behaving as expected with respect to visibility of symbols, the first place you should look is this header. The header was initially created as a means of consolidating a number of standards feature test macro definitions into a single place. In Solaris 10, we expanded the definitions by collapsing a number of commonly used feature test macros into additional Sun implementation feature test macros. For example any place that previously had:

#if defined(_XOPEN_SOURCE) || defined(_POSIX_C_SOURCE)

instead uses:

#if defined(_XOPEN_OR_POSIX) 

And some of you may recall the use of the following throughout headers defined by the C Standard:

#if (__STDC__ - 0 == 0)

or

#if (__STDC__ - 0 != 1)

On Solaris, we define __STDC__ to be 1 for a strictly conforming ANSI C application -- one that uses only the set of interfaces defined by the C standard. When __STDC__ is 0, we accept ANSI semantics, but in a much less restricted name space. Reference the Sun Studio Compilers manual page cc(1) for more information on the different compilation modes designated via the use of the -X[acts] option. We created a new feature test macro _STRICT_STDC so that the above guards could be replaced with something a bit more readable such as:

#if !defined(_STRICT_STDC)

Note that our _STRICT_STDC_ is equivalent to the GNU C __STRICT_ANSI__ build environment.

We collapsed some of the commonly used macro combinations for a number of reasons:

• The #ifdef lines were getting unreasonably long and increasingly difficult to parse

• A consensus among our OS developers that the second form was more readable

• The ability to combine a number of macros into a single macro, so that multiple versions of a specification can be specified with a single macro

Of course, the down side of this is that the headers can be a bit more confusing if <sys/feature_tests.h> is not consulted when reviewing or making changes to any header that relies on the definitions within that header. For example, a number of macros exists to indicate different versions of X/Open. These include _XPG3, _XPG4, _XPG5, etc. One should be aware that the use of #ifdef _XPG4 now implies support for XPG4, XPG5 and XPG6, not just XPG4. It does not imply support for XPG3.

Guidelines for adding new symbols

So let's assume you are adding a new interface that is related to a number of other interfaces already defined in a header specified by one of the standards. Or perhaps you are adding functionality to a previously existing interface that requires symbol extensions. In the first case, rather than create a new header, it seems best to group this new interface within the family of the previously existing interfaces.  In the second case, adding a new symbol with a reserved prefix (e.g LC_ in <locale.h>) may be allowed, but let's assume that this is a new symbol that will pollute the standards name space. In general, when adding a new symbol to a header defined by the standard, if that symbol is not included amongst the names reserved for the implementation, you will likely be guarding it with one of following:

For headers specified by both X/Open and POSIX, such as <unistd.h>:

#include <sys/feature_tests.h>

#if !defined(_XOPEN_OR_POSIX) || defined(__EXTENSIONS)

For headers specified by X/Open but not POSIX, such as <dlfcn.h>:

#if !defined(_XOPEN_SOURCE) || defined(__EXTENSIONS__)

For headers specified by POSIX but not X/Open:

#if !defined(_POSIX_C_SOURCE || defined(__EXTENSIONS__)

For headers specified by X/Open, POSIX, and the C Standard, such as <signal.h>:

#include <sys/feature_tests.h>

#if !defined(_XOPEN_OR_POSIX) && !defined(_STRICT_STDC) || \
        defined(__EXTENSIONS__)

It is quite likely these guards are already in place and adding the new interface is as simple as making sure it is placed in the correct area.

Worth noting is that in all of these examples, __EXTENSIONS__ is included in the guard macros. Whether we are guarding visibility within the standards name space or adding a new feature test macro for restricting visibility, we generally make any new symbols visible if -D__EXTENSIONS__ is defined. The exception would be if there is a conflict with another symbol visible in the default name space, but this is rare. The use of __EXTENSIONS__ essentially results in a non-restricted name space, though in combination with other feature test macros, one may get different definitions or declarations for those few cases where there are multiple definitions or declarations.

Note also the inclusion of <sys/feature_tests.h> in two of the examples. It is unlikely that you will need to add the inclusion of this header as it is probably already included either directly or in a top level header, but I've included it here just to help clarify the distinction between the various feature test macros. It is important to understand the difference between the standards specified feature test macros such as _XOPEN_SOURCE, _POSIX_C_SOURCE and Sun implementation specific macros such as _XOPEN_OR_POSIX or _XPG6. The latter are defined in <sys/feature_tests.h>.

Also note that from an application standpoint, the only valid feature test macros are those specified by the desired standard. In other words, the Sun implementation feature test macros such as _XPG3 or _XPG4 should never be specified at compilation time; their use is limited to within the headers. While the first example that follows may build without warnings or errors, it is likely to result in unexpected behavior.

c99 -D_XPG6 foo.c [ WRONG ]

c99 -D_XOPEN_SOURCE=600 foo.c [ CORRECT ]

See standards(5) for more detail on the standards compilation environments.

Testing and debugging headers

When making updates to the headers, there are few rules worth noting. In addition to doing full builds in the case of public header updates, testing should also include but not necessarily be limited to the following:

• Building of a test program with the Sun Studio C compiler using each of the -Xa (default), -Xc (strictly conforming), -Xt (transitional), and -Xs (K&R) compiler options. See cc(1) for more detail. Verify preprocessor output via cc -E.

• Building of a test program with each of -D_XOPEN_SOURCE, -D_POSIX_C_SOURCE, and -D__EXTENSIONS__ feature test macros specified, as well as combinations of all three. Verify preprocessor output via cc -E.

• If using the Sun Studio C compiler, testing in both C89/C90 mode and C99 mode via the use of the -xc99=none option. See cc(1) for more detail.

• Not necessarily needed for testing, but like cc -E, sometimes helpful in debugging, is the use of cc -H to see exactly what headers are getting pulled in and from where. This can be especially helpful if either the Sun Studio C++ compiler or the GNU C compiler is being used, as both of these have their own versions of some common headers.

• Building with the GNU C compiler. Note that it too supports the -E and -H options.

• If you are using GNU C, be aware that some Solaris headers may have been updated via the use the fixincludes utility, fixincl. For gcc 3.4.3, these are limited to <math.h>, <stddef.h>, <stdio.h>, and <sys/types.h>. If you touch any of these headers and try to build using gcc, you will likely need to rerun the fixinc utility.

• Build a test program using C++ (CC and g++).

• Ideally, any updates to the standards headers should also be tested using the applicable X/Open test suites assuming one has access to the test suites. The header tests are very good at identifying name space problems.