common_dispatch() in the NFS server and ma Buddies Dtrace and MDB.

Over here Mike Eisler asked a couple of interesting questions with respect to the NFS server and dtrace. I thought i'd expand on Eric's answer. So, Eric's Dtrace script is using the FBT provider which is function boundary tracing. Not the old 'vtrace point' that Mike added back in S8 time (which *I* just ripped out :) ). We can figure out what function calls happen in the NFS Server by using Dtrace like this :-

# dtrace -m nfssrv       
dtrace: description 'nfssrv' matched 1392 probes
CPU     ID                    FUNCTION:NAME
0  36317                    nfs_svc:entry 
0  35475    nfs_srv_set_sc_versions:entry 
0  35476   nfs_srv_set_sc_versions:return 
0  35477          rfs4_server_start:entry 
0  35478         rfs4_server_start:return 
0  36318                   nfs_svc:return 
0  35489               rfs_dispatch:entry 
0  35485            common_dispatch:entry 

That's nice but what are the arguments to common_dispatch() ? -- there are probably other ways to obtain the information (ugh... look at code ? who has time for that) but i like mdb :-

# mdb -k
Loading modules: [ unix krtld genunix specfs dtrace ufs ip sctp usba uhci s1394 random fctl nca lofs nfs audiosup sppp crypto ptm ]
<MDB> common_dispatch::nm
Value      Size       Type  Bind  Other Shndx    Name
0xf763063d|0x00000774|FUNC |LOCL |0x0  |1       |common_dispatch

<MDB> common_dispatch::nm -f ctype
C Type                                            
void (*)(struct svc_req *, SVCXPRT *, rpcvers_t, rpcvers_t, char *, struct rpc_disptable *)

so now we know what the arguments are.. but what on earth is in a svc_req ?

<MDB> ::print -a -t struct svc_req
{

0 rpcprog_t rq_prog 
4 rpcvers_t rq_vers 
8 rpcproc_t rq_proc 
c struct opaque_auth rq_cred {

c enum_t oa_flavor 
10 caddr_t oa_base 
14 uint_t oa_length 

}
18 caddr_t rq_clntcred 
1c SVCXPRT *rq_xprt 

}

Hmmm what could we do with this info ? -- How about counting the number of NFS Program dispatches ?

# dtrace -n fbt:nfssrv:common_dispatch:entry'/args[0]->rq_prog == 100003/ {@nfsc["NFS Dispatch count"]=count()}'
dtrace: description 'fbt:nfssrv:common_dispatch:entry' matched 1 probe
^C
NFS Dispatch count                                             6584