Lisa Week's Weblog

pNFS Server Control Protocol Documentation
The NFSv4.1/pNFS server control protocol describes the communication between the MDS and DS components of the pNFS server. The following picture (taken from the NFSv4.1 specification) illustrates where the control protocol fits in:

    +-----------+
    |+-----------+                                 +-----------+
    ||+-----------+                                |           |
    |||           |        NFSv4.1 + pNFS          |           |
    +||  Clients  |<------------------------------>|   Server  |
     +|           |                                |           |
      +-----------+                                |           |
           |||                                     +-----------+
           |||                                           |
           |||                                           |
           ||| Storage        +-----------+              |
           ||| Protocol       |+-----------+             |
           ||+----------------||+-----------+  Control   |
           |+-----------------|||           |    Protocol|
           +------------------+||  Storage  |------------+
                               +|  Devices  |
                                +-----------+

The control protocol is not standardized as a part of the NFSv4.1 specification, therefore, it is left up to each implementer to specify the control protocol for their implementation.

The OpenSolaris implementation's control protocol documentation has been available on the NFSv4.1/pNFS OpenSolaris project for a while, but I have recently migrated the documentation to our wiki in order to allow for more collaboration.

I am currently working on the implementation of the control protocol along with some others in our team. Specifically, I am working on DS_REMOVE. This is the capability to remove a file from the namespace and from storage. The implementation will come in a couple of phases. The first phase of the DS_REMOVE is fairly simple. It just instruments mds_op_remove() function (in nfs41_srv.c) in order to send a DS_REMOVE message to the data servers contained in the layout. This will prompt the data servers to remove the objects identified by the arguments DS_REMOVE. The first phase will not address any failure to contact the data servers contained in the layout. It will also not address any failure of the MDS. Future phases will deal appropriate state invalidation upon remove, truncation of files (e.g. SETATTR of size 0) and failures of the MDS/DS.