Thursday February 07, 2008 Ted H. Kim's Weblog
Musings of a Random Dude
Thursday February 07, 2008 Here are some recent blog entries featuring Sun hardware using InfiniBand in action. In each case, InfiniBand is being used with Sun hardware to challenge for world records.
More Ranger Facts and Figures gives a run down on the stats for the huge TACC installation which includes 62,976 CPU cores in the system. We won't know how it places on the Top500 until June.
A trio of stories about world record benchmarks using Sun gear connected by InfiniBand: ABAQUS v6.7, EXA PowerFLOW, Fluent.
Technorati Tags: InfiniBand
(2008-02-07 17:42:17.0) Permalink
Back when we came out with our first Solaris HCA driver, InfiniBand only had what is now called single data rate (SDR). Now the situation is different. Not only are double (DDR) and quad (QDR) rates spec'ed for IB, DDR is common out in the customer base. To address this, we have fixes that are going to be released in the next update of Solaris 10, which means they are also in OpenSolaris already. But to get it out faster, the fixes are also being pushed out in official Solaris patches.
So if you have a DDR capable HCA that runs our "tavor" driver, try out the patch (sparc: 126647-02, x86: 126648-02). Of course, remember to be sure your switch is configured for DDR too. Folks using Sun Connection Update Manager should have seen these come out already in the patch stream. BTW, if you are wondering about the "arbel" driver, these fixes are already in our recently released InfiniBand Updates 1.
I should also mention that this patch contains some other fixes which improve our uDAPL performance (used by Sun's MPI implementation in ClusterTools) as well as other fixes which might be of interest to our Solaris IB users. Check it out.
Technorati Tags: InfiniBand
(2008-02-07 17:13:11.0) Permalink
Friday January 25, 2008 Recently, some parts of the Sun download site were reorganized. Now, it is a bit easier to find some InfiniBand items, since they are now in the InfiniBand section under the Hardware Drivers category.
Currently, we have some Linux stuff there as well as these add-ons to the base InfiniBand support in Solaris:
Solaris InfiniBand Updates 1 - which I talk more about here
SCSI RDMA Protocol (SRP) Driver
Firmware Flash Update Tool - to update your HCA firmware when necessary
Of course, this is not the complete list of downloads with InfiniBand content. I don't totally understand the classification system, but I guess it depends on whether they are primarily InfiniBand or more generic than that.
So for example, there is HPC ClusterTools 7.1 which Sun's MPI product which also has InfiniBand support and can be downloaded. But it's in a different section. I think there are some others like this, but you get the idea.
Oh yeah, did I mention that all the downloads I mentioned above are all FREE.
Technorati Tags: InfiniBand
(2008-01-25 13:34:53.0) Permalink
Monday January 14, 2008 Not much has been posted yet, but I should point out that there is now an OpenSolaris project for iWARP.
Look here: RDMA Offload Framework (ROF)
Technorati Tags: iWARP
(2008-01-14 13:18:58.0) Permalink
Just wanted to point out some OpenSolaris projects for InfiniBand ...
IP on InfiniBand - Of course, IPonIB is already in Solaris, but this OpenSolaris project is about futures.
iSCSI Extensions for RDMA (iSER) - This project includes both initiator and target.
NFS over RDMA - Of course, this is both client and server.
OpenFabrics User Verbs - This project is a port of the OpenFabric's userland verb interface to Solaris. Once this is done, many apps which run on OFED could then be ported to run on Solaris.
Technorati Tags: InfiniBand
(2008-01-14 13:10:04.0) Permalink
Friday January 11, 2008 Like many other bloggers, I have to simply own up to the reality that blogging is bursty. Often it's a particular event rather than a lifestyle habit that motivates posting.
Well, this time I am motivated by the release of Solaris InfiniBand Updates 1. If you want cut to the chase, then get it here. But if you have no idea what this is about, then read on ...
This product is a FREE software download containing additional software for Solaris 10 InfiniBand. To run it, you need Solaris 10 8 / 07 (Update 4). From the name, you can probably guess that it will not be the only release in this series. You may wonder if this software is also going back into the other Solaris release trains. The answer is YES. It is our intention to eventually fold all this stuff back into the mainstream Solaris releases. So it is going to end up there. But for now, to satisfy the demands of our rapidly growing InfiniBand market in both HPC and the Enterprise, it's coming out NOW to minimize delay in getting to the market.
So what's in this release? Let me be the first to admit the name of the product is something only an accountant would like. The name really does not tell you what's in it. This product contains support for MemFree InfiniBand HCAs based on the Mellanox MT25204 and MT25208 chips (these PCI IDs: 15b3,5e8d 15b3,6274 15b3,6828). For the non-IB crowd, "MemFree" means that the HCA uses host memory instead of memory on the I/O adapter card for the HCA state (so it's LESS EXPENSIVE). So in particular the Sun 10-Port 4x DDR IB NEM for the SunBlade 8000/8000P (option X1289A-Z) is supported (see here, link to a drawing of the NEM). Of course, there are also numerous third party cards also using these Mellanox chips. Note that this software is only for the "MemFree" cards, other Sun software should be used for the cards (and Express Modules) with on-board memory.
In other Sun InfiniBand news, I should also point out that ClusterTools 7.1 (Sun's MPI based on OpenMPI 1.2.4) has been released. See this article.
Technorati Tags: InfiniBand
(2008-01-11 10:39:21.0) Permalink Comments [2]
Wednesday June 27, 2007 Others have mentioned it, but in case you missed it ...
Folks may want to check out the YouTube video of the unveiling of Sun's 3456 port InfiniBand switch.
If you are wondering what you do with such a big switch, you build an HPC setup something like this.
Technorati Tags: InfiniBand
(2007-06-27 15:09:20.0) Permalink
Friday April 06, 2007  | Previously I talked about Early Access versions of Sun ClusterTools 7. Now the production version has been released here. Just to repeat again, this version is based on OpenMPI and works on Solaris for both SPARC and x64. As well as InfiniBand, it also supports TCP, shared memory and has a Myrinet MX module too. |
Technorati Tags: InfiniBand
(2007-04-06 10:43:52.0) Permalink
Friday January 19, 2007 ClusterTools 7 Early Access (2)
For those following along, there is an update available for Sun's MPI product-- ClusterTools -- on InfiniBand. This update is a refresh of the ClusterTools 7 Early Access release. This OpenMPI based release includes a uDAPL BTL with support for both send/recv and RDMA to improve IB performance, plus other bug fixes and improvements. Plus like most of our software, it's free. So download away.
Technorati Tags: InfiniBand
(2007-01-19 14:03:18.0) Permalink
Wednesday November 15, 2006 A while ago, I described RNICPI. I would have liked to have said it's taking over the world. But alas, that is not to be. I really did want RNICPI to succeed. After all, I am the co-chair of the group that authored it. Now, that is not to say there won't be some RNICPI implementations. I wish them luck.
 | But as far as Sun is concerned, we have decided that the industry momentum is increasingly behind the Open Fabrics Alliance (OFA). Now OFA was originally called OpenIB because of it's InfiniBand roots. And most of its software is still heavily influenced by that. But it also has iWARP support under development. There are some significant milestones still to come before OFA iWARP support can be considered mature, but it's coming along. |
Now if you have been reading my blog, you know I am a Solaris developer. Folks familiar with OFA know it's based around Linux (and there is a Windows version too!). So there are various issues involved with leveraging OFA stuff for iWARP on Solaris and making it all work together. But that is what we are doing. And since it's SC06 time, I guess I shouldn't delay blogging about this any longer.
So there you have it. We are trying to take advantage of the growing interest in iWARP based on OFA code and interfaces. Many of the iWARP IHVs have OFA code internally or under development. We feel that OFA is going to become the de facto standard for iWARP software.
More on this story to come ...
Now I would be remiss if I didn't offer all the standard disclaimers. YMMV. I don't time travel. But who knows, I might still be right. :-)
Technorati Tags: iWARP
(2006-11-15 16:00:00.0) Permalink
Friday November 03, 2006 I offer this anecdotal data point on the availability of InfiniBand:
InfiniBand on Amazon
How about this refurbished switch and module-box item for around $1500! Standard Disclaimer: I don't endorse this particular item or the company offering the item. But I am trying to claim IB is creeping into the mainstream.
InfiniBand on eBay
Currently, this item
seems to be the best deal on eBay for HCAs. Wow, $50 new! Standard Disclaimer: I don't endorse this particular item or the person offering the item. But I do hope you get my point about falling prices and availability.
Technorati Tags: InfiniBand
(2006-11-03 12:28:25.0) Permalink
Friday September 22, 2006 So it goes it without saying that I should blog more. But just to remind people about what is going on, I thought I would highlight some recent Solaris InfiniBand news.
Folks might be wondering about our MPI story on InfiniBand. For a while now, Ohio State's MVAPICH has run on top of our Solaris uDAPL. More recently, Scali has announced support for their implementation of MPI on Solaris 10 on our Opteron machines. And an Open MPI version is in the works for the official Sun product.
Finally, I should mention that the Solaris InfiniBand SCSI RDMA Protocol (SRP) driver 1.0 is now available for free download. So download away and start using IB SRP storage. As you might imagine, other IB storage options are also under development, but nothing has been officially announced yet.
Technorati Tags: InfiniBand
(2006-09-22 13:00:00.0) Permalink
Monday June 19, 2006  | I took a few days off at the begining of June to go to ConsimWorld Expo 2006. This event is a game convention for the serious board wargaming. Not much else is going on there. But if you are a serious board wargamer, it's the best convention of its kind that I have been to. The facilities are good, the activities are well organzied, there are a lot of free door prizes (but I wasn't lucky enough to win one), there are lots of friendly players and wargame industry folks there, and of course the games are great fun! But Arizona this time of year is quite hot. |
This convention specializes in hosting so-called Monster Games which are huge board wargames. Given that emphasis, it's not surprising that the convention is mostly about the playing experience. Some other well respected wargame conventions like the WBC focus on competition and tournaments. My own experience at the convention (playing the monster game Streets of Stalingrad) is chronicled on the ConsimWorld discussion board starting here".
Technorati Tags: boardgames
(2006-06-19 11:30:00.0) Permalink
Wednesday April 19, 2006  | Sun just recently put PCI-Express HCAs on the pricelist. This product uses an 8x PCI-E interface and should help in cases where folks were bandwidth limited by PCI-X or PCI buses. This HCA has 256MB on-card and is also available in low-profile. Take a look here. |
Technorati Tags: InfiniBand
(2006-04-19 12:00:00.0) Permalink
Monday January 30, 2006 Just an update on InfiniBand certified vendors ...
Fans of Voltaire equipment will be delighted to see that the company's products are now listed on the Sun website as Solaris Ready here.
The overall list of certified InfiniBand vendors is here.
Technorati Tags: InfiniBand
(2006-01-30 10:00:00.0) Permalink
Calendar
| « November 2009 | ||||||
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 | |||||
| Today | ||||||
RSS Feeds
All
/Boardgames
/Books
/General
/InfiniBand
/IO
/iWARP