We knew customers would not want an unmanaged, "black box" for external I/O. The system administrator needs access to the status of the IO Box. They also need to know when the IO Box fails, and why it is failing (and how that failure is affecting the rest of the system). And being able to light "locator" indicators to find one of hundreds of IO Boxes located remotely from the host server was critical for service. Providing a managed IO Box, with fault and status information available using standard protocols, and LEDs controlled by the host, is essential to provide a highly available system.
On the other hand, we didn't want the IO Box to be another system that the customer had to manage. The IO Box does not have a service processor; it doesn't have its own MIB; it doesn't need software upgrades; it doesn't have an ethernet port.
The IO Box is just a bunch of I/O slots available to the host; it shouldn't matter if they are located in the same chassis as, or several meters (or several dozen meters) away from, the CPUs and memory.
The IO Box is fully managed, as a part of the host, reporting status and faults back to the host to which it is connected. The host SP includes IO Box status as part of the overall system status. The IO Box does not require any special cables between the host and the box, other than the PCI-Express cables. Separate cabling would introduce the risk that a customer could cross-wire a box to the wrong host. And we wanted to make the wiring as simple as possible.
This is accomplished by using extra signals in the PCI-Express cable to implement a management connection between the host service processor (SP) and the IO Box. The standard PCI-Express connector has two pins for SMBus: SMCLK (B5) and SMDAT (B6). Using those two pins, the host (SP) is able to talk to a "slave" microcontroller on the link card. The microcontroller then uses spare signals in the cable to communicate in a reliable fashion with the microcontroller on the link card in the IO Box to forward requests from the host SP. The link card in the IO Box then uses SMBus on those same two pins B5 and B6, this time as "master", to access devices in the IO Box, reading or writing any device that the host SP requests. In effect, the I2C devices in the IO Box become "local" to the host service processor, even though the IO Box itself is remote. The microcontrollers on the link cards act as proxies.
With this arrangement, the host SP can retrieve environmental information about the IO Box: temperatures, fan speeds, voltages, currents, switch positions, etc. In addition, the host SP can control the IO Box, turning off power to a power supply unit so it can be removed, lighting the "locator" indicator so the IO Box can be found in the datacenter, etc. And when the IO Box experiences an error, the host SP can gather error information, and factor it into other host-detected errors to diagnose the fault.
IO Box management is entirely optional -- the IO Box as a standalone unit can function with no host management. But management by the host SP provides an added dimension of availability and serviceability which is not found on low-end I/O expansion units.