Marc Hamilton's Weblog: Marc Hamilton's Weblog

Like the Parthenon in Athens or countless other ancient monuments in Greece and around the world, Tokyo Tech's TSUBAME cluster derives much of its power from the simplicity of its design. The Parthenon consists of 46 fluted columns, 8 at the ends and 17 down each side. TSUBAME consists of 655 Sun Fire x64 servers, mounted 11 to a rack, in groups of two facing rows. The Parthenon's columns incline slightly inwards to correct an optical distortion which would make them appear further apart at the top than at ground level. TSUBAME's racks are not inclined, and in earthquake prone Japan have their own special engineering to ensure they stay that way.

Many supercomputers come in oversized cabinets that must be assembled using complicated techniques in clean-room environments. The logistics of building Asia's fastest supercomputer in Tokyo Tech's second floor computer room would have made it very difficult and time consuming to install a traditional supercomputer. Instead, TSUBAME uses standard 19 inch wide racks from NEC. The 46 rack unit high NEC racks, fully assembled, would be too tall to ship via standard air freight. In addition, special earthquake bracing required for the racks could only be installed when the racks where partially empty. Luckily, this posed no problems for the Sun Fire x64 servers. While several NEC racks were shipped to Sun's HPC Solutions Center in Hillsboro Oregon to ensure they would work with the Sun Fire x64 servers, the production racks were shipped directly to Tokyo Tech and installed empty, simplifying greatly the task of earthquake bracing. Below is show one set of empty racks being installed in March.

The empty racks were then cabled for power, InfiniBand, and Ethernet before any of the Sun Fire x64 servers arrived. From March 31 when the last of the 655 Sun Fire x64 compute servers arrived at Tokyo Tech, it took less than six weeks to install and test the system, and complete the record setting 38.18 TFlop Linpack run.

Check back tomorrow for the next installment of the TSUBAME story. But until then, one more picture, showing the same racks as above, this time with the Sun Fire x64 compute servers installed.