Code Complete
20040927 Monday September 27, 2004

Take a SWIG

Before I begin, it occurs to me that my posts might be just a bit too long. Maybe I should split stuff over a number of posts (or even days), but when I get on a roll, I can type forever (frequency of posting shows my average motivation level ;). Maybe I'll change the format, but probably not... I'm unpredictable like that.

SWIG, the Simple Wrapper Interface Generator, is a great way to wrap C/C++ libraries for use with scripting languages. It supports the favorites - Java, Perl, Tcl, Python, and PHP as well as some newer and perhaps more obscure languages such as Ruby, Guile, MZscheme, Ocaml, and Chicken (huh?). For most standard calling conventions, it is possible to point swig at the header files for a library, and it will spit out a C interface file to that library. I say most standard calling conventions here, because as I've found out over the last couple of weeks, some things are hard to do in a language agnostic fashion.

For instance, consider the case of the Solaris libnvpair(3LIB) interface. In my project, I want to be able to wrap a function from another library which returns a pointer to a nvlist. I don't want to wrap the entire libnvpair interface in all of it's glory just to be able to read a nested nvlist. I'm not interested in adding to or modifying the nvlist that is returned to my code, so I will turn the nvlist into a structure which is native to whatever scripting language I'm using. As I mentioned in my last article regarding extension and embedding, this boils down to Perl, and then maybe Python and Java (adoption for Perl for system admin tasks is high, less so for Python; Java is not widely used, but could be).

For Perl, it seems most natural to have the nvlist copied out into a hash reference, which the Perl code would then be able to walk through to make decisions about the returned data. The code to do this is quite interesting, and needs to be added on as inline code in the SWIG interface file. For instance, to create a simple hash reference, here is the Perl API code:

  HV * hash = (HV*)sv_2mortal((SV*)newHV());
  hv_store(hash, "mykey", 3, newSVpv("my value", 0));

This is similar to the following Perl code:

  my $href = {};
  $href->{mykey} = "my value";

Now, for the real application, C code will be required to traverse the nvlist and copy keys and values to an HV. Obviously, the above code is very Perl specific. To do the same thing in Python, I need to traverse the nvlist and create a Python dictionary object, for Java a java.util.HashMap or similar. Each will require a bit of VM magic code to make the translation. Luckily, SWIG is there to save the day, and makes it quite easy to #ifdef each language specific section, as it gives the interface file a once over with the C pre-processor before creating the wrapper. Then I just have to bind each wrapper with the corresponding language headers and libraries, and I get a nice loadable module.

My main concern, as C is really not a core competency in my department, is to keep the interface as clean and understandable as possible. I also want to have the flexibility to experiment and explore with other languages. SWIG gives me the best of both worlds.

Permalink

x86 Shuffle

I don't work from home that often, so I don't require a SPARC based workstation like I enjoy in my office, but I do enjoy a good game of Doom 3 now and then. To support this habit, I've decided to upgrade my current home system. I built my current system in early 2000, and it has remained future proof up until very recently. Specifically, releases of games such as Doom 3 and Halflife 2 have required more GL power than my old GeForce 2 32MB card could provide. My current rig:

  • ABit VP6 Dual PIII mainboard
  • 2 x PIII 1GHz Coppermine (C0 stepping)
  • 512MB Corsair CAS3 SDRAM
  • SB Audigy Platinum
  • Netgear MA311 802.11b
  • ATI Radeon 9800PRO 128MB
  • 2 x IBM Deskstar 60GXP 60GB/ATA100/7200RPM/2MB Cache
  • IBM Deskstar 60GXP 40GB/ATA100/7200RPM/2MB Cache
  • Pioneer DVD-106S (Slot Load)
  • LITE-ON LTR-48125W 48x12x48 CD/RW
  • Antec Sonata Case w/380W PS
  • NEC MultiSync FE1250+ 22"
  • Cambridge Soundworks 5.1 Audio
  • Dual boot Windows XP and Fedora Core 2

Bold items are the lucky components to be included in my new system. You might be thinking -- this seems like enough, why upgrade? Well, it all began when I upgraded my video card to the Radeon. The board layout for the VP6 puts CPU1 directly above the AGP slot, which I didn't believe to be that big an issue when I bought the board. When I bought the Radeon, I knew that it would produce a lot more heat, so I bought some nice new Thermaltake copper heatsinks for the P3s. Really, the point was to reduce the overall noise level of the system, which I've since learned is next to impossible with a dual CPU VP6 (larger heat sinks from Zalman were recommended here, but don't fit).

After I moved all of my hardware into the Antec case, added in the Radeon card, and strapped on my shiny new copper heatsink, almost everything came up, which is obviously not acceptable. Now, the VP6 board comes with a RAID controller on board, which allows me to run RAID 0, 1 or 0+1 with up to four drives. Alternatly, I can turn off the RAID function, and have two more IDE controllers to use, which allows for up to a total of 8 IDE devices. I had configured my system to run each drive as a channel master, with only the 40GB storage drive and DVD sharing a bus. Normally, when I start the system, it displays the normal hardware probe screen, and then switches to a drive detection screen for the RAID controller (Highpoint HPT370).

So, when I started up the system in a new case with the fantastic Radeon card inside, I was puzzled when the RAID detection screen didn't come up. Switching the card for my old GeForce caused the detection screen to reappear, but I wasn't about to take the Radeon back to the store. I posted a few times on ArsTechnica Forums, probing my initial suspicion that there was some sort of IRQ sharing problem between Mr. RAID controller and Mr. Radeon. Someone on the Ars forums postulated that the Radeon might not be properly releasing resources after initialization, leaving insufficient space in BIOS mapped RAM to allow the RAID controller to initialize. Of course, I am explaining this phonetically, as I really don't fully understand how ACPI initializes PCI cards.

I still haven't figured out how to fully resolve the issue, but I figured that it was partly due to the age difference between the Radeon and my mainboard. Without the HPT370 drivers in Windows XP, the devices attached to the RAID controller do not show up. Installing the drivers makes the devices appear -- Linux gets it right the first time with no extra drivers or configuration. So, dispite the fact that I have to do a bit of a jig after a fresh Windows XP install to see my drives, at least I now can access them. Problem solved enough.

Now that the all singing, all dancing Radeon is working under Linux, I made a special effort to find the highest performance, higest resolution, most fantastic and earth shattering screen saver. This came in the form of a translucent dancing torus (4D Hypertorus, for you KDE users). After staring at it for several minutes, I locked the screen and allowed the torus to dance into the night. An hour later, I came back to the system --- locked solid. I thought that it might be an issue with Linux, which has really never worked 100% with my dual cpu VP6 anyway (issues mostly with ACPI), so I rebooted, and no lockups while I worked for the rest of the evening. The next time I left the system on the screensaver, it locked up again. Hmmm. I switched screensavers, and now I haven't had a lockup for over a month.

The issue turned out to be heat. As I mentioned before, the VP6 puts CPU1 directly above the AGP slot. With a Radeon installed, this is similar to putting CPU1 in a toaster oven. Checking temps on the CPUs after running some 3D demos, I found that CPU0 was hovering around 35-40C, and CPU1 was up around 55C. Heat was rising off the Radeon, and warming the CPU heatsink. There's no opportunity for a larger heatsink, and I really don't want to mess around with liquid cooling for this old system. This was really the last straw for the dual CPU setup.

The new rig will use the bold items in the listing above, with the addition of the following:

  • MSI Neo2 Platinum nForce3 Ultra mainboard
  • AMD Athlon 64 3500+ (Socket 939, Newcastle core, 2.2GHz)
  • 1GB(512x2) OCZ DDR PC3200 (2-3-2-6 1T)

From what I gathered from a whole Sunday afternoon of research and pricing comparisons, the Socket 939 platform seems to be the most future proof for AMD64 desktop systems. Socket 939 is similar to the original AMD64 Socket 745, but provides a dual-channel memory controller on board instead of a single channel controller. It seems that the number of socket types is blooming at the moment in the AMD space. This is due to the on-board memory controller -- every time a new type of RAM appears, silicon must be changed in the CPU. As a result, AMD needs to make sure that the CPU cannot be installed in the wrong system, and their solution is to change the socket interface. From my observation, Sun has sidestepped this by using proprietary RAM, so nearly everything can be held constant to allow multiple models of CPUs to use the same RAM and mainboards.

Anyway, we will see once the new system comes together whether I will finally be free of the upgrade struggle for another four years. I like to upgrade my system, but don't like doing it more than once every couple of years or so, or less for major brain transplants like the above.

Permalink