If you have ever used an ifconfig command to plumb a network interface
like:
>ifconfig ce0 plumb
and wondered what the internal STREAMS plumbing looks like,
then read on. This blog entry is especially for those who are
interested in a simplified view of the network interface plumbing and
data flow in Solaris.
The plumbing, for those interested in looking at code, happens in
plumb_one_device(). First, ifconfig opens the network device:
Fig 1: ifconfing opening a stream to the network interface.
It then pushes IP on top of the network device. This forms the stream over which IP packets are sent and recieved:
Fig 2: IP pushed on top the network device.
Assuming that a resolver is needed, an ARP module is then pushed on this stream to form the IP-ARP stream. This allows IP to send queries direclty to ARP and process responses from it:
Fig 3: ARP pushed on top of IP.
If ARP is needed (as in this example), then another stream to the device is opened:
Fig 4: Opening another stream for ARP
And ARP pushed on top of it. This forms the ARP-device stream over which the actual ARP queries are sent out and responses processed from the wire:
Fig 5: ARP pushed on top of the network device.
To allow ifconfig to exit, these two streams are PLINKed underneath a multiplexor stream:
Fig 6: PLINK below the mux
Since PLINK is used, ifconfig can close the corresponding file descriptors and exit:
Fig 7: ifconfig exits
The above diagram describes the "lower half" of setup. To understand how a packet actually travels from the application out to the wire and vice versa, one needs to see the other half (i.e "upper half") of the setup that happens when a socket is opened by the application:
Fig 8: Final plumbing showing data path
On the outbound side (packet going from application to the wire), the packet travels upto IP where a routing lookup is done to determine the correct "network interface" (i.e "lower half") to send the packet out on. Accordingly the packet is placed in the correct output queue. On the inbound side (packet coming in from the wire), the packet is "fanned out" to the correct "upper connection" based on five-tuple (src IP addr, dest IP addr, src port, dst port, protocol) match. The bi-directional arrow highlights both the paths.
Technorati Tag: OpenSolaris
Technorati Tag: Solaris
Fig 1: ifconfing opening a stream to the network interface.
It then pushes IP on top of the network device. This forms the stream over which IP packets are sent and recieved:
Fig 2: IP pushed on top the network device.
Assuming that a resolver is needed, an ARP module is then pushed on this stream to form the IP-ARP stream. This allows IP to send queries direclty to ARP and process responses from it:
Fig 3: ARP pushed on top of IP.
If ARP is needed (as in this example), then another stream to the device is opened:
Fig 4: Opening another stream for ARP
And ARP pushed on top of it. This forms the ARP-device stream over which the actual ARP queries are sent out and responses processed from the wire:
Fig 5: ARP pushed on top of the network device.
To allow ifconfig to exit, these two streams are PLINKed underneath a multiplexor stream:
Fig 6: PLINK below the mux
Since PLINK is used, ifconfig can close the corresponding file descriptors and exit:
Fig 7: ifconfig exits
The above diagram describes the "lower half" of setup. To understand how a packet actually travels from the application out to the wire and vice versa, one needs to see the other half (i.e "upper half") of the setup that happens when a socket is opened by the application:
Fig 8: Final plumbing showing data path
On the outbound side (packet going from application to the wire), the packet travels upto IP where a routing lookup is done to determine the correct "network interface" (i.e "lower half") to send the packet out on. Accordingly the packet is placed in the correct output queue. On the inbound side (packet coming in from the wire), the packet is "fanned out" to the correct "upper connection" based on five-tuple (src IP addr, dest IP addr, src port, dst port, protocol) match. The bi-directional arrow highlights both the paths.
Technorati Tag: OpenSolaris
Technorati Tag: Solaris