I was recently at a customer site trying to determine why a Sun Fire v880 server would not consistently establish a link with the network equipment on the other end of a 1-Gig fiber.
I tried to configure ce.conf to force the link to 1-Gig instead of allowing it to auto-negotiate. It might be that I was too lame to understand the examples in the ce.conf man pages, or maybe the man pages need a little help there.
Anyway, I ended up modifying a custom init script which the customer had written to tweak the /dev/ce driver at boot time. They had the right idea, which was to force the link to a certain speed, but the problem was that they had multiple instances of this 1-Gig fiber card. Our customer had mistakenly thought that if they did the following commands, they would operate on *all* instances of the /dev/ce type:
This situation seems perfectly reasonable to me since the interface names are 'ce0' and 'ce1' and we wanted to modify all of the interfaces, so we operate on /dev/ce driver with ndd and "It should just work!" But things are not always so intuitive in UNIX. I think this is what might be called a "modal interface with no visual feedback" and it defies all forms of intuition. It's my opinion that when using ndd to operate on any driver and there exists more than one instance, ndd should emit a message to STDERR with a warning that multiple instances exist and it should also inform the user about which instance will receive the given changes. Of course, ndd should have an option to make it suppress said messages...
OK, where was I... Oh yeah, for a while, I also thought like the customer that using ndd on /dev/ce would operate on all of the ce* interfaces until it hit me like a brick that I needed to add a little tweak to their init script to make ndd operate on each instance like so:
They were so close!
It's easy to understand once you see the solution, but we needed to set the 'instance' parameter of the /dev/ce driver to "point to" a certain instance of the driver and then make the necessary changes. So, ndd will only operate on the default instance (instance 0) without explicitly setting the 'instance' parameter.
I hope that was interesting and helpful enough to keep you from falling into the same little trap.
I tried to configure ce.conf to force the link to 1-Gig instead of allowing it to auto-negotiate. It might be that I was too lame to understand the examples in the ce.conf man pages, or maybe the man pages need a little help there.
Anyway, I ended up modifying a custom init script which the customer had written to tweak the /dev/ce driver at boot time. They had the right idea, which was to force the link to a certain speed, but the problem was that they had multiple instances of this 1-Gig fiber card. Our customer had mistakenly thought that if they did the following commands, they would operate on *all* instances of the /dev/ce type:
ndd -set /dev/ce adv_autoneg_cap 0
ndd -set /dev/ce adv_1000fdx_cap 1
This situation seems perfectly reasonable to me since the interface names are 'ce0' and 'ce1' and we wanted to modify all of the interfaces, so we operate on /dev/ce driver with ndd and "It should just work!" But things are not always so intuitive in UNIX. I think this is what might be called a "modal interface with no visual feedback" and it defies all forms of intuition. It's my opinion that when using ndd to operate on any driver and there exists more than one instance, ndd should emit a message to STDERR with a warning that multiple instances exist and it should also inform the user about which instance will receive the given changes. Of course, ndd should have an option to make it suppress said messages...
OK, where was I... Oh yeah, for a while, I also thought like the customer that using ndd on /dev/ce would operate on all of the ce* interfaces until it hit me like a brick that I needed to add a little tweak to their init script to make ndd operate on each instance like so:
ndd -set /dev/ce instance 0 <---- added
ndd -set /dev/ce adv_autoneg_cap 0 <---- original
ndd -set /dev/ce adv_1000fdx_cap 1 <---- original
ndd -set /dev/ce instance 1 <---- added
ndd -set /dev/ce adv_autoneg_cap 0 <---- added
ndd -set /dev/ce adv_1000fdx_cap 1 <---- added
They were so close!
It's easy to understand once you see the solution, but we needed to set the 'instance' parameter of the /dev/ce driver to "point to" a certain instance of the driver and then make the necessary changes. So, ndd will only operate on the default instance (instance 0) without explicitly setting the 'instance' parameter.
I hope that was interesting and helpful enough to keep you from falling into the same little trap.
ce=`prtconf -D | egrep 'ethernet|network' | grep "driver name: ce" | wc -l` if [ $ce -gt 0 ] ; then interface=0 while [ $interface -lt $ce ] ; do ndd -set /dev/ce instance $interface ndd -set /dev/ce adv_1000fdx_cap 1 ndd -set /dev/ce adv_1000hdx_cap 0 ndd -set /dev/ce adv_100fdx_cap 1 ndd -set /dev/ce adv_100hdx_cap 0 ndd -set /dev/ce adv_10hdx_cap 0 ndd -set /dev/ce adv_10fdx_cap 0 ndd -set /dev/ce adv_autoneg_cap 1 interface=`expr $interface + 1` done fiPosted by Alex Madden on September 21, 2006 at 08:20 AM PDT #
Well, the customer is happy and the system works so I won't "fix" it at this point. But your suggestion has been tucked away into my "home directory" (a UNIX euphemism for "the bottomless abyss of every bit of data ever acquired") for later use:
############################################################################# # ndd script suggestion from Alex Madden at: # http://blogs.sun.com/blahblah/entry/ndd_script_tweak#comment-1158852055000 ############################################################################# ce=`prtconf -D | egrep 'ethernet|network' | grep "driver name: ce" | wc -l` if [ $ce -gt 0 ] ; then interface=0 while [ $interface -lt $ce ] ; do ndd -set /dev/ce instance $interface ndd -set /dev/ce adv_1000fdx_cap 1 ndd -set /dev/ce adv_1000hdx_cap 0 ndd -set /dev/ce adv_100fdx_cap 1 ndd -set /dev/ce adv_100hdx_cap 0 ndd -set /dev/ce adv_10hdx_cap 0 ndd -set /dev/ce adv_10fdx_cap 0 ndd -set /dev/ce adv_autoneg_cap 1 interface=`expr $interface + 1` done fiPosted by Dale Sears on September 21, 2006 at 03:08 PM PDT #