Casper Dik's Weblog

Tuesday June 14, 2005

User Credentials and all that

Peter Harvey's story reminds me of the unforeseen consequences of creating the ucred in Solaris 10. The ucred was motivated by two factors: the introduction of privileges and a way to propagate information about process credentials through the system in userland.

Before Solaris 10, we had several mechanisms, some internal, some public, all propagating a subset of that information.

in sys/door.h:

/*
 * Structure used to return info from door_cred
 */
typedef struct door_cred {
        uid_t   dc_euid;        /* Effective uid of client */
        gid_t   dc_egid;        /* Effective gid of client */
        uid_t   dc_ruid;        /* Real uid of client */
        gid_t   dc_rgid;        /* Real gid of client */
        pid_t   dc_pid;         /* pid of client */
        int     dc_resv[4];     /* Future use */
} door_cred_t;

in sys/tl.h:

#define TL_OPT_PEER_CRED 10
typedef struct tl_credopt {
        uid_t   tc_uid;         /* Effective user id */
        gid_t   tc_gid;         /* Effective group id */
        uid_t   tc_ruid;        /* Real user id */
        gid_t   tc_rgid;        /* Real group id */
        uid_t   tc_suid;        /* Saved user id (from exec) */
        gid_t   tc_sgid;        /* Saved group id (from exec) */
        uint_t  tc_ngroups;     /* number of supplementary groups */
} tl_credopt_t;

in rpc/svc.h:

/*
 * Obtaining local credentials.
 */
typedef struct __svc_local_cred_t {
        uid_t   euid;   /* effective uid */
        gid_t   egid;   /* effective gid */
        uid_t   ruid;   /* real uid */
        gid_t   rgid;   /* real gid */
        pid_t   pid;    /* caller's pid, or -1 if not available */
} svc_local_cred_t;

and in the project I missed this one in sys/stropts.h:

struct k_strrecvfd {    /* SVR4 expanded syscall interface structure */
        struct file *fp;
        uid_t uid;
        gid_t gid;
        char fill[8];
};

There was also the need to be able to enquire about other processes and perhaps network connections and packets; a getpeereid interface was requested.

Now, what information should such an interface return? Network interfaces often only allow you to shape requests as a blob of bytes. And that blob needs to have a predictable maximum size too. As you can see from the above examples, even declaring a number of filler elements is not sufficient; none of the above structures which include a filler have space for the full complement of 16 groups, let alone Pete's proposed 65536 maximum number of groups.

The most natural way of implementing a blob which such restrictions is using an opaque data structure with accessor functions (in <ucred.h>):

extern ucred_t *ucred_get(pid_t pid);

extern void ucred_free(ucred_t *);

extern uid_t ucred_geteuid(const ucred_t *);
extern uid_t ucred_getruid(const ucred_t *);
extern uid_t ucred_getsuid(const ucred_t *);
extern gid_t ucred_getegid(const ucred_t *);
extern gid_t ucred_getrgid(const ucred_t *);
extern gid_t ucred_getsgid(const ucred_t *);
extern int   ucred_getgroups(const ucred_t *, const gid_t **);

extern const priv_set_t *ucred_getprivset(const ucred_t *, priv_ptype_t);
extern uint_t ucred_getpflags(const ucred_t *, uint_t);

extern pid_t ucred_getpid(const ucred_t *); /* for door_cred compatibility */

extern size_t ucred_size(void);

extern int getpeerucred(int, ucred_t **);

extern zoneid_t ucred_getzoneid(const ucred_t *);

extern projid_t ucred_getprojid(const ucred_t *);

The ucred_t itself is defined in sys/ucred.h, a header which isn't installed on the system because programs are not supposed to use it; it is a private interface between the kernel and the library.
One function of note is perhaps ucred_size() which returns the maximum size of a credential on the system; it can be used to size credentials allocated on the stack or embedded in structures.
In many cases, the system will just allocate one for you and return the allocated one, but the interfaces have been structured so you can reuse ones returned earlier or ones you allocated yourself.

By now you may be asking yourself where you get creds; well, here are some examples in the OpenSolaris source code: nscd getting a door cred, rpcbind getting an rpc caller credential and the use of the TL option by RPC.

And your typical use of the function in an inetd started daemon:

#include <ucred.h>

int
main(int argc, char **argv)
{
	ucred_t *uc = NULL;

	if (getpeerucred(0, &uc) == 0) {
		/* we know something about the caller */
	}

        return (0);
}

And a slightly bigger example where we use XPG4 recvmsg to receive a UCRED control messages:

/*
 * Send a 1 byte UDP packet; print the response packet if one is
 * received.
 */

#include <sys/socket.h>
#include <sys/uio.h>
#include <sys/signal.h>
#include <netinet/in.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <netdb.h>
#include <stdlib.h>
#include <arpa/inet.h>

int
main(int argc, char **argv)
{
        struct sockaddr_storage stor;
        struct sockaddr_in *sin = (struct sockaddr_in *)&stor;
        struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&stor;
        ssize_t bytes;
        union {
                struct cmsghdr hdr;
                unsigned char buf[2048];
                double align;
        } cbuf;
        unsigned char buf[2048];
        struct msghdr msg;
        struct cmsghdr *cmsg;
        struct iovec iov;
        int one = 1;

        msg.msg_name = &stor;
        msg.msg_iov = &iov;
        msg.msg_iovlen = 1;

        iov.iov_base = buf;

        setsockopt(0, IPPROTO_IP, IP_RECVDSTADDR, &one, sizeof (one));
        setsockopt(0, IPPROTO_IPV6, IPV6_RECVPKTINFO, &one, sizeof (one));
        setsockopt(0, SOL_SOCKET, SO_RECVUCRED, &one, sizeof (one));
        alarm(30);

        while (1) {
                char abuf[256];
                msg.msg_control = &cbuf;
                msg.msg_controllen = sizeof (cbuf);
                msg.msg_namelen = sizeof (stor);
                iov.iov_len = sizeof (buf);

                bytes = recvmsg(0, &msg, 0);

                if (bytes >= 0) {

                        if (msg.msg_namelen != 0 &&
                            connect(0, (struct sockaddr *)&stor,
                            msg.msg_namelen) != 0)
                                exit(1);
                        printf("you connected from %s with the credential\n",
                                inet_ntop(stor.ss_family,
                                    stor.ss_family == AF_INET ?
                                        (void *)&sin->sin_addr :
                                        (void *)&sin6->sin6_addr,
                                        abuf, sizeof(abuf)));
                        for (cmsg = CMSG_FIRSTHDR(&msg); cmsg;
                            cmsg = CMSG_NXTHDR(&msg, cmsg)) {
                                if (cmsg->cmsg_level == SOL_SOCKET &&
                                    cmsg->cmsg_type == SCM_UCRED) {
                                        ucred_t *uc = (ucred_t *)
                                            CMSG_DATA(cmsg);

                                        /* We have a ucred here !! */
                                }

                        }
                        if (msg.msg_namelen != 0)
                                (void) connect(0, NULL, 0);
                } else {
                        exit(1);
                }
        }
}

But thinking back of Pete's problem, we see a problem when increasing max groups, even worse, this libnsl private datastructure is abused and multiple copies exist which need to be kept in sync (so parts of the system broke when I changed it in this one place). The bug is an illustration why cut & paste programming doesn't work and why even when you share a private defintion, you must use a proper header file. I filed the bug as soon as I did the quick fix for the Solaris Express respin, the bug is 4994017.