At a previous employer, I had worked with someone who was incredulous that Sun's implementation rpcgen(1) did not eliminate tail recursion. After all, he had worked there and had that code change sitting in a workspace when he left.

So at the time, I manually fixed a XDR call that was blowing the stack.

Since then, I've occasionally had to relearn XDR and rpcgen. And I've always assumed that I had to manually do the tail recursion code. The other day I was in a hurry and I decide to generate output from both Linux and Solaris to see the differences. I thought that surely the Linux implementation would be optimal.

We can see there is a simple linked-list which would be NULL terminated:

#define SPED_MAX_GUUIDS         8
typedef struct spe_policy {
        uint_t                  sp_id;
        uint_t                  sp_stripe_count;
        uint_t                  sp_interlace;
        spe_interior_t          *sp_attr_expr;
        char                    *sp_name;
        uint64_t                sp_guuids[SPED_MAX_GUUIDS];
        struct spe_policy       *next;
} spe_policy_t;

I was looking at the diff and I was struck by how simple and elegant the few lines of Linux generated C code were:

bool_t
xdr_spe_policy (XDR *xdrs, spe_policy *objp)
{
	register int32_t *buf;

	int i;
	 if (!xdr_uint32_t (xdrs, &objp->sp_id))
		 return FALSE;
	 if (!xdr_count4 (xdrs, &objp->sp_stripe_count))
		 return FALSE;
	 if (!xdr_uint32_t (xdrs, &objp->sp_interlace))
		 return FALSE;
	 if (!xdr_pointer (xdrs, (char **)&objp->sp_attr_expr, sizeof (spe_interior), (xdrproc_t) xdr_spe_interior))
		 return FALSE;
	 if (!xdr_pointer (xdrs, (char **)&objp->sp_name, sizeof (char), (xdrproc_t) xdr_char))
		 return FALSE;
	 if (!xdr_vector (xdrs, (char *)objp->sp_guuids, SPED_MAX_GUUIDS,
		sizeof (uint64_t), (xdrproc_t) xdr_uint64_t))
		 return FALSE;
	 if (!xdr_pointer (xdrs, (char **)&objp->next, sizeof (spe_policy), (xdrproc_t) xdr_spe_policy))
		 return FALSE;
	return TRUE;
}

And I was looking at the Solaris generated output and was worried about the apparent complexity:

bool_t
xdr_spe_policy(XDR *xdrs, spe_policy_t *objp)
{
        rpc_inline_t *buf;

        spe_policy_t *tmp_spe_policy;
        bool_t more_data = TRUE;
        bool_t first_objp = TRUE;


        if (xdrs->x_op == XDR_DECODE) {
                while (more_data) {
                        if (!xdr_uint32_t(xdrs, &objp->sp_id))
                                return (FALSE);
...
                        if (!xdr_bool(xdrs, &more_data))
                                return (FALSE);

                        if (!more_data) {
                                objp->next = NULL;
                                break;
                        }

                        if (objp->next == NULL) {
                                objp->next = (spe_policy_t *)
                                        mem_alloc(sizeof (spe_policy_t));
                                if (objp->next == NULL)
                                        return (FALSE);
                                bzero(objp->next, sizeof (spe_policy_t));
                        }
                        objp = objp->next;
                }
        } else if (xdrs->x_op == XDR_ENCODE) {
                while (more_data) {
                        if (!xdr_uint32_t(xdrs, &objp->sp_id))
                                return (FALSE);
...
                        if (!xdr_vector(xdrs,
                            (char *)objp->sp_guuids,
                            SPED_MAX_GUUIDS,
                            sizeof (uint64_t),
                            (xdrproc_t)xdr_uint64_t))
                                return (FALSE);
                        objp = objp->next;
                        if (objp == NULL)
                                more_data = FALSE;
                        if (!xdr_bool(xdrs, &more_data))
                                return (FALSE);
                }
        } else {
                while (more_data) {
                        if (!xdr_uint32_t(xdrs, &objp->sp_id))
                                return (FALSE);
...
                        if (!xdr_vector(xdrs,
                            (char *)objp->sp_guuids,
                            SPED_MAX_GUUIDS,
                            sizeof (uint64_t),
                            (xdrproc_t)xdr_uint64_t))
                                return (FALSE);
                        tmp_spe_policy = objp;
                        objp = objp->next;
                        if (objp == NULL)
                                more_data = FALSE;
                        if (!first_objp)
                                mem_free(tmp_spe_policy_t,
                                    sizeof (spe_policy_t));
                        else
                                first_objp = FALSE;
                }

        }
        return (TRUE);
}

I didn't look at the actual code, I just assumed the worst. So I went searching for a version of rpcgen that would work for me. I also was kinda looking for the bug id of the code my friend had sitting in a private workspace.

The first hit in google on the keywords "rpcgen tail recursion" was Bug ID: 4023944 rpcgen generates linked list routines that blow stacks. Hmm, that is on opensolaris.org, which is good.

When I opened it, I was struck by several facts:

1. State     10-Fix Delivered (Fix available in build)
2. Fixed In     snv_14
3. Responsible Engineer     Bill Ricker
   Hey, I know this guy! I met him at Connectathon several times.
4. One of the problems leading to this fix was a blown stack in mountd.

I was a victim of low expectations. I assumed that the Solaris rpcgen(1) still did not handle tail recursion (it hadn't been a priority to putback when my friend had a working solution), so I didn't inspect the generated code to see that it did do so. I assumed that fewer lines of code meant elegant and more functional. Instead it meant a blown stack at some point.

So thanks Bill for getting that change made and protecting stacks everywhere!