I've been away from the spe code for some time doing bug work, gatekeeping, and web design. I'm back to it and trying to get it ready to integrate into our development gate. I've decided to move the spe decision making into the kernel and as such I'll need to pass the policies from user space to the kernel.

Traditionally, you have to define 64 and 32 bit versions of your data structures (user land is typically 32 bit address space). Your application packs in the data and sends it to the kernel via syscall. And then you have to copy your data from user space to the kernel.

The STRUCT_* macros in the following code chunk show the steps you take:

1. You declare a local variable, u_sped in this case, which allows two views of the memory (see STRUCT_DECL(9F))
2. You determine whether the user land address space is 32 or 64 bit
3. You configure your local variable to use the current model
4. You copy the memory from user land (sped_in in this case) to the kernel address space
5. You get a local copy out of there.
6. You get a local copy out of there.
7. This one is different, it is a variable length "string"

void
nfs4_sped_svc(struct nfssped_args *sped_in)
{
        spe_policy_t    *sp = NULL;
        nfssped_op_t    opcode;
        char            *buf = NULL;
        size_t          len;
        size_t          tlen = 0;

        XDR             xdrs;

        model_t         model;

        STRUCT_DECL(nfssped_args, u_sped);  /* 1 */

        model = get_udatamodel();  /* 2 */

        /*
         * Initialize the data pointers.
         */
        STRUCT_INIT(u_sped, model); /* 3 */
        if (copyin(sped_in, STRUCT_BUF(u_sped), STRUCT_SIZE(u_sped))) {   /* 4 */
                set_errno(EFAULT);
                return;
        }

        opcode = STRUCT_FGET(u_sped, nsa_opcode);  /* 5 */
        len = STRUCT_FGET(u_sped, nsa_xdr_len);    /* 6 */

        if (len) {
                buf = kmem_zalloc(len, KM_SLEEP);

                if (copyinstr(STRUCT_FGETP(u_sped, nsa_xdr),         /* 7 */
                    buf, len, &tlen)) {
                        goto err_out;
                }
        }

One problem is that you need to know how big the buffer is to extract from it. If you have a fixed number of elements, i.e., 1, this is an easy problem. But I want to send an arbitrary number of elements. I could cycle through a linked list, sending one at a time, but that sounds gross.

But XDR was created to handle differences in network byte order, machine address ranges, variable number of data sets, etc.

So we take a pass in userland to encode the data into XDR, which also tells us how large the resulting data will be. And then we can unpack it in the kernel.

The largest issue I faced was that you typically start out with XDR notation and use rpcgen to produce headers and code. I already had the data structures, so I took my header and turned it into a .x file. And I had a hard time getting a union in there:

typedef union {
        uid_t           uid;
        gid_t           gid;
        int             i;
        char            *sz;
        spe_network_t   net;
} spe_data_t;

I would have twigged onto what was needed if I had started earlier - discriminated unions. So now I need to provide a way to tell the XDR code how to determine which field was to be used. Luckily, I knew that:

typedef enum {
        SPE_DATA_ADDR,
        SPE_DATA_GID,
        SPE_DATA_INT,
        SPE_DATA_NETNAME,
        SPE_DATA_NETWORK,
        SPE_DATA_STRING,
        SPE_DATA_UID
} spe_type_t;

I rearranged the .x file to get:

enum spe_type {
        SPE_DATA_ADDR,
        SPE_DATA_GID,
        SPE_DATA_INT,
        SPE_DATA_NETNAME,
        SPE_DATA_NETWORK,
        SPE_DATA_STRING,
        SPE_DATA_UID
};

union spe_data switch (spe_type data_type) {
        case SPE_DATA_UID:
                uid_t           uid;
        case SPE_DATA_GID:
                gid_t           gid;
        case SPE_DATA_INT:
                int             i;
        case SPE_DATA_NETNAME:
        case SPE_DATA_STRING:
                char            *sz;
        case SPE_DATA_ADDR:
        case SPE_DATA_NETWORK:
                struct spe_network   net;
};

But that produced the following for the header file:

struct spe_data {
        spe_type data_type;
        union {
                uid_t uid;
                gid_t gid;
                int i;
                char *sz;
                struct spe_network net;
        } spe_data_u;
};
typedef struct spe_data spe_data;

I wanted a nice clean separation, so I fought this for a while. But it is logical. In the end I went with:

typedef struct {
        spe_type_t      sd_type;
        union {
                uid_t           uid;
                gid_t           gid;
                int             i;
                char            *sz;
                spe_network_t   net;
        } sd_u;
} spe_data_t;

It fits my naming style and I also avoided doing the following:

#define uid sd_u.uid
#define gid sd_u.gid

Of course, with that, I should pick more unique field names. And what I hate about this method is that if you are sitting there with one window open to code and another on a debugger, you can't examine foo.uid. You have to dig through the headers to find that you really need to be looking at foo.sd_u.uid. I find it easier to avoid the macros.