In no particular order...
Here is a bad result
mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k mds #
And here is a good result:
# echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
{
dbe = 0xffffff02fbb21f08
dev_addr = {
na_r_netid = 0xffffff02f6d17b20 "tcp"
na_r_addr = 0xffffff02ec14c000 "10.1.233.119.217.62"
}
dev_flags = 0x3
dev_infop = 0xffffff02fbb22f78
dev_list_next = {
list_next = 0xffffff02fbb22fb8
list_prev = 0xffffff02fbb22fb8
}
}
Unfortunately, many problems have this symptom.
client # mount pnfs-3-14:/ /mnt Mar 19 11:29:21 pnfs-3-12 nfs: NOTICE: enabling pNFS on pnfs-3-14 client # cd /mnt client # cd client # snoop pnfs-3-14 pnfs-3-13 Using device bge0 (promiscuous mode) ^Cclient # cd
I typically open another window and do simple copies from /etc:
[pnfs-3-12 zippy]> cp /etc/motd . [pnfs-3-12 zippy]> cp /etc/motd kkk
It is only when I see valid traffic that I know I am set.
# snoop pnfs-3-12 pnfs-3-13 Using device bge0 (promiscuous mode) Mar 19 11:39:07 pnfs-3-12 nfs: NOTICE: enabling pNFS on pnfs-3-14 pnfs-3-12 -> pnfs-3-13 TCP D=55614 S=1014 Ack=93767047 Seq=74796989 Len=0 Win=49640 pnfs-3-12 -> pnfs-3-13 NFS C 4 (exchange_id ) EXCHANGE_ID Verf=11E050B247E141B2 COID=F6C0 PNFS_DS NONE
ds # zpool create -f dpool /dev/dsk/c1t0d0s7 ds # dservadm addmds 10.1.233.120.8.1 ds # dservadm addpool dpool ds # dservadm enable
If you don't, then you won't see traffic going from the client to the DS.
client # snoop pnfs-3-12 pnfs-3-13 Using device bge0 (promiscuous mode) Mar 19 11:19:26 pnfs-3-12 last message repeated 2 times
So go back and do:
ds # zfs set sharenfs=on dpool
Note you will also not see a record for the auth in the mds:
mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
ds # dservadm disable
And
# mdsadm -o add -t auth -a ip=10.1.233.119 adding: IP Addr - 10.1.233.119
ds # dservadm enable
mds # mdsadm -o add -t auth -a ip=10.1.233.119 adding: IP Addr - 10.1.233.119 mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
Whoops, forgot to disable the ds!
ds # dservadm disable
Still nothing!
mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
Okay, issues again:
mds # mdsadm -o add -t auth -a ip=10.1.233.119 adding: IP Addr - 10.1.233.119 nfssys:: File exists mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
And finally:
ds # dservadm enable
Which yields:
# echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
{
dbe = 0xffffff02f4493f08
dev_addr = {
na_r_netid = 0xffffff02f76f28e0 "tcp"
na_r_addr = 0xffffff02f51ec820 "10.1.233.119.254.189"
}
dev_flags = 0x3
dev_infop = 0xffffff02f4494f78
dev_list_next = {
list_next = 0xffffff02f4494fb8
list_prev = 0xffffff02f4494fb8
}
}
Hmm, I wasn't able to reproduce the double record thing... Oh yes I was!
ds # dservadm disable ds # dservadm enable
And
mds # echo '::walk Device_entry_cache | ::print struct rfs4_dbe data | ::print mds_device_t' | mdb -k
{
dbe = 0xffffff02f4493e18
dev_addr = {
na_r_netid = 0xffffff02f76f2fc0 "tcp"
na_r_addr = 0xffffff02f51ecb20 "10.1.233.119.212.217"
}
dev_flags = 0x3
dev_infop = 0xffffff02f4494f78
dev_list_next = {
list_next = 0xffffff02f4494fb8
list_prev = 0xffffff02f4494fb8
}
}
{
dbe = 0xffffff02f4493f08
dev_addr = {
na_r_netid = 0xffffff02f76f28e0 "tcp"
na_r_addr = 0xffffff02f51ec820 "10.1.233.119.254.189"
}
dev_flags = 0x3
dev_infop = 0xffffff02f4494f78
dev_list_next = {
list_next = 0
list_prev = 0
}
}
The point is with multiple auth records you may not get the results you want. I'd reboot them both at this point. :->
client # ./server -p /nippy -m /mnt/pnfs-3-14 pnfs-3-14 Start tests on path /mnt/pnfs-3-14/pnfs-3-12.test [y/n]? y sh ./runtests -a -t /mnt/pnfs-3-14/pnfs-3-12.test Starting BASIC tests: test directory /mnt/pnfs-3-14/pnfs-3-12.test (arg: -t) mkdir: Failed to make directory "/mnt/pnfs-3-14/pnfs-3-12.test"; Permission denied Can't make directory /mnt/pnfs-3-14/pnfs-3-12.test basic tests failed Tests failed, leaving /mnt/pnfs-3-14 mounted
So:
mds #share -@nippy /nippy rw "" mds # zfs set sharenfs=rw,anon=0 nippy