Thursday February 23, 2006

Solaris Link Aggregations (2): Configuration

In my previous entry, I described the architecture of the Solaris Link Aggregations. Today, we'll take a quick look at how easily this feature can be used to create aggregations of NICs with higher bandwidth and availability.

Currently, only devices that plug into the GLDv3 (a.k.a. Nemo) framework can be aggregated. Out of the box, this currently includes bge (1 Gb/s Broadcom based), e1000g (1 Gb/s Intel based), and xge (10 Gb/s Neterion based). More drivers are being ported from DLPI or GLDv2 to GLDv3, and the Nemo Unification project currently underway and led by Cathy is going to provide a shim layer that will allow all DLPI-based drivers to plug into the GLDv3 framework.

Suppose your machine has four Gigabit Ethernet NICs, bge0-3, that you want to aggregate (our newest servers such as the Niagara-based Sun Fire T2000 and T1000, as well as our AMD Opteron-based Sun Fire X4100 and X4200 servers already come with four on-board gigabit ethernet ports, and it's also possible to add single, dual, or even quad gigabit-ethernet adapters to a system.) To aggregate these network interfaces, you simply run the following command:

# dladm create-aggr -d bge0 -d bge1 -d bge2 -d bge3 1

That's it! You now have a 4 Gb/s pipe to your machine (yes, it loves to scale, I'll show you in a future article). The previous command caused a new device "aggr1" to be created, which you can plumb and configure with ifconfig(1M) like any other device, for example:

# ifconfig aggr1 plumb
# ifconfig aggr1 inet 192.168.1.1 up

All the aggregation configuration information is persistant across reboot automatically, so you don't have to edit any other file than the usual /etc/hostname. entries.

The full set of options of the create-aggr subcommand are described in details in the dladm(1M) man page. Some of these options allow enabling LACP, changing the traffic distribution policy, setting an explicit MAC address (by default, the aggregation driver uses the address of one of the constituent ports), etc.

Note that the last argument of the create-aggr subcommand above corresponds to the key of the aggregation, which you can pick but must be unique on your machine (a future version of dladm will pick one for you.) That key is used as the PPA of the aggregation data-link that can configured using ifconfig(1M). In the example above, the specified key value was 1, so the data-link name is aggr1.

Another useful dladm(1M) you may also find useful for now is show-aggr, which allows you to display the status of an aggregation and its constituent ports, as well as traffic distribution statistics, etc.

Technorati Tag: Solaris
Technorati Tag: OpenSolaris

Posted by droux ( Feb 23 2006, 12:12:59 AM PST ) Permalink Comments [2]