SPARC Enterprise M-class Servers The Secrets of Olympus

As I mentioned in my blog XCP 1041 Now Available, Sun SPARC Enterprise M-class servers support an External I/O Expansion Unit, or IO Box. This week, IO Box started shipping to customers! IO Box is one of my pet projects.

The IO Box addresses a critical problem with previous generations of enterprise servers: The I/O-to-CPU ration was too low for some customer applications. On a Sun Fire 25K fully populated with I/O boards and CPU boards, there are 72 PCI slots, which is plenty for most customers. But with 72 CPU sockets filled with dual-core UltraSPARC IV CPUs, that yields a PCI-slot to CPU core ration of 1/2 -- one PCI slot for ever two CPU cores. The SPARC Enterprise M9000-64 has 128 PCI-Express slots, and while that may seem like a lot, with 64 dual-core SPARC64-VI CPU chips, that's still just 1 PCI-Express slot per core (and it will get worse when we pack more cores into each CPU chip). Some customers really care about I/O, and a higher I/O-to-CPU ratio is important.

The IO Box allows you to connect one PCI-Express slot in the host to an IO Boat, which has six additional, hot-plug-capable PCI-Express or PCI-X slots. Each IO Box can support up to two IO Boats (either PCI-Express or PCI-X), independently connected to the same host. The host-to-box link can either be copper (low cost, but short and bulky, so only really applicable if the server and the IO Box are in the same cabinet) or fibre optic (higher cost, but with 25m cable lengths you can locate the IO Boxes together in a separate cabinet from your servers).

There are other IO expansion units on the market, but the Sun version is really designed for the enterprise-class environment, with features like:

• Fully redundant, hot-swappable power supplies.
• Support for Sun's indicator standard, with LEDs for locating field replaceable units (FRUs), showing FRU power state, identifying FRUs that are ready to remove, and faults.
• Ability to monitor IO Box internal voltages, currents, temperatures, LEDs, power state and switch settings from the host's Service Processor.
• Host's ability to detect and diagnose IO Box faults seamlessly with other errors and faults in the host.

In sort, the IO Box is less a peripheral, and more like a part of the system; it just happens to be located several meters away. Disconnecting compute power and I/O only makes sense if you reconnect them virtually, so there's only one system to manage.

While IO Box is initially supported on the M-class servers, it's history predates the Sun/Fujitsu APL agreements, and will almost certainly be supported on other Sun products in the future. I've got some neat things I'd like to share about IO Box in future posts...