Kool Aid Served Daily

I just saw an interesting question on one of the internal Sun developer lists:

ssh -fN -L "3049:somehost.someext:2049" jonb@somehost.someext
mount -o port=3049 localhost:/f /f

This allows you to mount a filesystem from a remote machine over a ssh tunnel. The question was why doesn't automounting also work?

so I have a

/-              auto_direct     -nosuid,rw

in auto_master

/f -rw,port=3049 localhost:/f

in auto_direct.

And since this is over Solaris, by default it is NFSv4 and we shouldn't need anything but port 2049. The question was why didn't this work. A good resource on this is at: Tunneling NFS traffic via ssh, which is provided by Spencer Shepler. In the comments, it specifically mentions that all you need is port 2049 for NFSv4 traffic.

In my reply on the mailing list, I pointed out that this does not apply to the auotmounter. It has to call on the server's portmapper, which is port 111. Let's repeat the experiment and see if we can prove this to be what is happening.

[tdh@sandman ~]> ssh -fN -L "3049:ultralord:2049" ultralord
The authenticity of host 'ultralord (192.168.2.104)' can't be established.
RSA key fingerprint is a5:e6:e6:9c:e2:9c:2e:d9:6c:d2:91:c0:ed:41:8f:38.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ultralord,192.168.2.104' (RSA) to the list of known hosts.
Password:

And

# mount -o port=3049 localhost:/f/1 /f
# ls -la /f
total 4
drwxr-xr-x   2 root     root         512 Dec 30 15:08 .
drwxr-xr-x  24 root     root         512 Dec 30 15:08 ..
# tcsh
# ls -la /f
total 4
drwxr-xr-x   2 root     root         512 Dec 30 15:11 .
drwxr-xr-x  24 root     root         512 Dec 30 15:08 ..
-rw-r--r--   1 root     root           0 Dec 30 15:11 ultralord
# showmount -e localhost
showmount: localhost: RPC: Program not registered

I'll claim I don't need to setup the automounter. I'll do that later to confirm things. Okay, let's try some snoop. Krap! snoop only works on non-loopback interfaces. I found a dtrace script which is supposed to work: DTrace TCP Snoop, but I need to work on it a bit more. Anyway, let's look at how showmount works over a regular interface:

[tdh@sandman ~]> showmount -e ultralord
export list for ultralord:
/f/1 (everyone)
/f/2 (everyone)

And the snoop:

# snoop sandman ultralord
Using device /dev/hme (promiscuous mode)
     sandman -> ultralord PORTMAP C GETPORT prog=100005 (MOUNT) vers=1 proto=TCP
     ultralord -> sandman PORTMAP R GETPORT port=36795
     sandman -> ultralord MOUNT1 C Get export list
     ultralord -> sandman MOUNT1 R Get export list 2 entries

Which can be translated as:

sandman asks ultralord's portmapper: "Do you support MOUNT vers 1 over TCP?"
ultralord replies: "Yeah and you can get it on port 36795."
sandman then asks ultralord's mountd: "What exports do you have?"
ultralord then replies: "/f/1 and /f/2"

It makes sense that the automounter would also use the mount protocol. It basically needs to do a MOUNTPROC3_DUMP (see RFC 1813) in order to determine what the server has available. In effect you would have to punch a hole for both port 111 and the port ultralord advertises mountd on. And nothing specifies that the server has to use the same port everytime.

With NFSv4, you do not need an automounter to discover all of the exports on a server. But the implementation shipped in Solaris 10 does not support the automatic mounting of new filesystems as you cross boundaries. In part, it was an engineering decision to ship NFSv4 without this functionality. You can use automounters to get some of the equivalent functionality. You also have to look at typical usuage of shares in Solaris pre-zfs - small sets of shares per server and no child shares.

The problem is given this set of shares on a server:

/    ro
/f   rw=.eng.sun.com,ro=@172.16
/f/1 rw=.lab.sun.com,anon=0
/f/2 rw,root=.lab.sun.com

How do you dynamically determine where you can go? We haven't specified that this is a Solaris server, so let's assume that '/', '/f', '/f/1', and '/f/2' are all different filesystems. Via the NFSv4 spec (RFC 3530) you should be able to do the following:

# mount -o vers=4 sandman:/ /sandman
...
[tdh@sandman ~]> cd /sandman
[tdh@sandman ~]> ls -la
total 6
drwxr-xr-x   3 tdh      staff        512 Dec 30 16:18 .
drwxr-xr-x   5 tdh      staff        512 Dec 30 16:18 ..
drwxr-xr-x   2 tdh      staff        512 Dec 30 16:18 f
[tdh@sandman ~]> cd f
[tdh@sandman ~]> ls -la
total 10
drwxr-xr-x   5 tdh      staff        512 Dec 30 16:18 .
drwxr-xr-x  24 root     root         512 Dec 30 15:07 ..
drwxr-xr-x   2 tdh      staff        512 Dec 30 15:11 1
drwxr-xr-x   2 tdh      staff        512 Dec 30 15:11 2
drwxr-xr-x   3 tdh      staff        512 Dec 30 16:18 3
[tdh@sandman ~]> cd 1
[tdh@sandman ~]> ls -la
total 4
drwxr-xr-x   2 tdh      staff        512 Dec 30 15:11 .
drwxr-xr-x   5 tdh      staff        512 Dec 30 16:18 ..
-rw-r--r--   1 tdh      staff          0 Dec 30 15:11 ultralord
[tdh@sandman ~]> cd ../3
[tdh@sandman ~]> ls -la
total 6
drwxr-xr-x   3 tdh      staff        512 Dec 30 16:21 .
drwxr-xr-x   5 tdh      staff        512 Dec 30 16:18 ..
drwxr-xr-x   2 tdh      staff        512 Dec 30 16:18 g

Under the current Solaris implementations, you would need to manually make the mounts before each of the cd commands. Except for the last, in that case you are staying on the filesystem represented by '/f'.

The NFS development team is actively working on fixing this issue, which we call Mirror Mounts. Once that functionality is in place, you will be able to explore a remote host across both a firewall and a ssh tunnel via only one port: 2049.