LaoTsao's Weblog (老曹的網路記)

Infiniband Switches

Recently, to replay some RFP, I have to investage offers from many different vendors.

I summarize my finding in this weblog:

1. Mellanox:
   • InfiniScale II MT43132 chip, 8 4x ports, 240 Gbps ,foundation for many IB switches
   • InfiniScale III MT47396 chip, 24 4x ports, 480 Gbps ,foundation for many IB switches
   • InfiniHost dual port 4x HCA/TCA, support 128MB, 256MB or 512GB memory, 8 Data VLs and VL15 mgmt
   • MTS2400, 24 4x port,1U, InfiniScale III based<
   • MTS14400, 144 4x port,10U, InfiniScale III based
   • MTS9600, 96 4xport, 7U, InfiniScale based
2. topspin:
   • TS-90 , 12 4x port plus one dual port FC-GW or 6 ported IP-GW, 1U
   • TS-120, 24 4x port , 1U
   • TS-360, 24 4x port,4U, plus upto 12 modules for FC-GW or IP-GW
   • TS-170, 96 4x port, 7U,
   • TS-270, 96 4xport , 6U
   • HCA, 125MB RAM,
3. Infinicon:
   • InfinIO9000, 288 4x port , 7U for 12 cards,14U for 24 cards
   • infinIO5000, 12 4x port ,1U
   • InfinIO7000, 60 4x ort, 1U
   • InfinIO3000, 32 4x port , 1U
   • InfiniServ HCA, 256 MB, dual port 4x
4. Voltaire:
   • ISR9288, 288 4x ports,14U, 3 blade drawer, 2 port GE, 4 port FC
   • ISR9024, 24 4x port, 1U
   • ISR6000, 1U, 3 slots, upto 3 6 4x IB, 2 2ge-GW, 2 4-FC ports

Most those products are using the silicon from Mellanox.

Using 24 port silicon to build an non-blocking switches, in 3 layers setup, one can get:

• 48 ports: 6 chips, 6 links between chips, 2 cores and 4 leafs
• 72 ports: 9 chips, 4 links between chips, 3 cores and 6 leafs
• 96 ports: 12 chips, 3 links between chips, 4 cores and 8 leafs
• 144 ports: 18 chips, 2 links between chips, 6 cores and 12 leafs
• 288 ports: 48 chips, 1 link between chips, 12 cores and 24 leafs

To form a HPC clusters, one can use the two ports of HCA to have a HA configuration of two core switches.

The connection from the leaf switches to the core switchs will determines the blocking factor e.g. for 1 24 port switch,

• 8 uplink and 16 downlink is consider 50% blocking.
• 12 uplink and 12 downlink is consider 0% blocking
• 4 uplink and 20 downlink is consider 80% bloacking
• 8 uplink and 12 downlink is 25% blocking

Posted at 04:28PM Sep 26, 2004 by hstsao in Infiniband  |