Simon Bullen's Weblog : Weblog

Tuesday Feb 20, 2007

10Gb takes it's next step forward.

10Gb moves on yet again while IB stands still, but this time the movement has not come from a network vendor but from Sun.
This product is a brand new, internally designed multi-threaded 10Gb network card.

Why is it unique ?

Sun's 10Gb Ethernet Adapter extends CPU and OS parallelism to networking with its support for hardware-based flow classification and multiple DMAs. Using CPU thread affinity to bind a given flow to a specific CPU thread, it enables a one-to-one correlation of Rx and Tx packets across the same TCP connection. This can help avoid cross-calls and context switching to deliver greater performance while reducing the need for CPU resources to support I/O processing.

The Sun 10 Gigabit Ethernet Adapter utilizes Sun's own innovative MAC Controller to map the 10Gb XAUI interface onto the PCI Express form factor. It supports 10 GB/sec bandwidth using four transmit and four receive lanes, enabling higher density and a lower pin-count to reduce costs while improving performance. As a result, local area networks using the Adapter can immediately benefit from increased speed and efficiency.

* Dual 10 GE port x8 PCI Express 1.1 compliant, Fiber XFP MSA compliant Low Profile plug-in adapter

* IEEE 802.3ae 2002 compliant

* Uses Sun's own ASIC and Software for innovative Throughput Networking design

* Networking I/O virtualization is built in to support progression in the upper layer virtualization software

* Hardware-based flow classification for extending parallelism and virtualization to networking

More information from www.sun.com

http://blogs.sun.com/hendel/entry/russian_dolls - Ariel Hendel
http://blogs.sun.com/markusflierl/entry/crossbow_and_neptune - Markus Flierl

tags [10Gb][neptune]

Posted at 09:56AM Feb 20, 2007 by Simon Bullen in Networking | Comments[13]

Comments:

Sounds pretty exciting.... your links aren't working for me though. Can you talk about price/latency vs IB?

Posted by Donal McMullan on February 20, 2007 at 11:12 AM GMT #

Sorry, slight typo on the links, it is all fixed now. Pricing is from $498 USD, as for latency I will try to find out more. I am hoping to link this to a cut through switch such as the Force10 S2410 which has a 200-300ns latency hit on all packet sizes.

Posted by simon bullen on February 20, 2007 at 11:45 AM GMT #

Many await netperf figures (TCP_RR in addition to TCP_STREAM) with bated breath :)

Posted by rick jones on February 20, 2007 at 10:54 PM GMT #

Sounds fantastic, but I see no info on offload capabilities - any idea if this will offload TCP, or especially UDP, checksum processing?

Posted by Kemp Watson on February 21, 2007 at 03:40 PM GMT #

We have been doing TCP/UDP checksum for a decade and do not call that offload...Of course it is there. Offload lately means a futile attempt of having a $50 NIC chip do with 12Watts something better than the host processor can with 80W and 32GB of DRAM. We do not believe in that and a future entry will explain how to use threads instead.

Posted by simon bullen on February 22, 2007 at 08:41 AM GMT #

Why is it unique? Because it's one of the few 10gbe products that has no information published on how to interface to the hardware. But this is hardly surprising coming from Sun, where is the Cassini documentation? The schizo UltraSPARC III host bridge documentation and what would have to be a large list of errata are no where to be found either, despite being repeatedly requested over a period of many years.

Posted by Jonathan Gray on February 22, 2007 at 11:29 AM GMT #

>>We have been doing TCP/UDP checksum for a decade Really? I've read a number of recent papers indicating UDP checksum offload has only been available for a year or so. BTW, it appears this card doesn't support CX4 - is that so? I'd love to try this in my T2000, but I'm planning on copper. Great that it's on the try'n'buy program.

Posted by Kemp Watson on February 22, 2007 at 02:12 PM GMT #

Kemp, If you are wanting copper wait until the end of the year when you will be able to run 10Gb on cat6 and not CX4. That way you can keep the cost down and be able to run 100m.

Posted by simon bullen on February 22, 2007 at 03:47 PM GMT #

The price information is completely deceptive. The base card costs $995. Transceivers (required) start at $599 each. A single port solution is $1,594, not $498. A dual-port solution is $2,193.

I do not understand why Sun does this dishonest marketing. The price per Gb (based on the stated 16 Gb/sec throughput of dual ports) is $137/Gb, which is cheaper than the new Neptune-based Quad GigE NIC or any Sun GigE NIC.

Posted by Mark on February 22, 2007 at 07:56 PM GMT #

I don't know how long Sun have offered NICs with ChecKsum Offload, but the industry has had them going back to the early 1990s (if not the rather late 1980's) with FDDI cards from various major systems vendors of the time. It has only been in somewhat recent years that the "PC" space has seen them though.

Posted by rick jones on February 23, 2007 at 02:08 AM GMT #

Re CX4, what about customers wanting copper _now_? By the same logic, you could offer CX4 now for customers not wanting/requiring the cost of optical links. I can't wait a year, and other vendors have 10GbE CX4 for Sun already.</p

Posted by Kemp Watson on February 23, 2007 at 03:32 PM GMT #

Re UDP checksum offload, it appears that Sun has had a minimal implementation of 1's complement checksum offload for IP-sized packets only, since Solaris 2.6. 2's complement, checksum-over-fragment, large send offlead, and full protocool stack offload have not existed until project Crossbow, today.

Unfortunately, those are all features that become really important when using UDP over high bandwidth links, so saying 'of course it is there' sounds a lot like the response I often get from salespeople when I try to find out what a product _really_ does.

See Priyanka's blog for details. Also, see 2006 paper and presentation.

BTW, I'm not claiming that competitive products are better than this NIC, they may well not be, but in this category, details count.

Posted by Kemp Watson on February 23, 2007 at 03:33 PM GMT #

Re CX4, what are you going to connect to ? Very few network vendors supply CX4 ports on the switch front, a few supply CX4 modules on stack units the other option is to purchase CX4 XPF's which keeps the cost high.

Posted by simon bullen on February 26, 2007 at 12:13 PM GMT #