Ted H. Kim's Weblog

Tuesday April 14, 2009

iSER

As other folks have mentioned, iSCSI Extensions for RDMA (iSER) was recently integrated into build 112 of OpenSolaris. Congratulations to the iSER team! The general background on iSER is here with the Solaris architecture specifics here. But the general idea is iSCSI with InfiniBand RDMA to drive up performance.

Technorati Tags: InfiniBand

(2009-04-14 15:25:50.0) Permalink

IB QDR and Nehalem (updated)

In general, InfiniBand DDR HCAs were already limited by the IO bus under PCIe 1.X. With today's Nehalem (Xeon 5500 series) related announcements come a bunch of PCIe 2.0 servers (including x2270, x4170, x4270, x4275 servers plus x6270 and x6275 blades). Now x4 DDR should be able to operate at full speed with a PCIe 2.0 x8 slot and x4 QDR is now realistic (though probably still IO bus limited). The x6275 blade even has on-board QDR HCAs installed. For some great shots of the boards, look here.

So in addition to earlier HCA offerings, we now have this QDR HCA story:

Sun Option	Form Factor	Chip	IO Bus	IB Ports (per HCA)	Card Memory	Solaris Driver
X4237A	Card	ConnectX-IB	PCIe 2.0 x8	Two x4 QDR	Memfree	hermon
X4219A-Z	Express Module

Edit: I think I might finally understand this now! The X5500A SunBlade 6048 QDR NEM has switches, but no HCAs. This design is different from how the DDR NEM works, which has the HCAs in it too. Instead, the QDR NEM is meant to be used with on-board HCAs like with the x6275 blade.

But both IB NEMs for the SunBlade 6048 are meant to be used as part of an overall Sun Constellation system with the final stage of the fat-tree (or torus for the QDR one) switch network incorporated directly in the NEM itself.

And of course, if there is any contradiction, I am sure the official stuff takes precedence over anything I say.

Technorati Tags: InfiniBand

(2009-04-14 14:02:21.0) Permalink Comments [1]