Thursday November 19, 2009

Judging from a quick look at the survey results from this weekend's Sun HPC Consortium meeting in Portland, Oregon, Marc Parizeau's talk was a favorite with both customers and Sun employees.

Marc is Deputy Director of CLUMEQ and a professor at Université Laval in Québec City. His talk, Colossus: A cool HPC tower! [PDF, 10MB], describes with many photos how a 1960s era Van de Graaff generator facility was turned into an innovative, state of the art, supercomputing installation featuring Sun Constellation hardware. Very much worth a look.

A nicely-produced CLUMEQ / Constellation video that describes the creation of this computing facility is also available on YouTube.

Henry Tufo from NCAR and CU-Boulder spoke this weekend at the Sun HPC Consortium meeting here in Portland, OR. As part of his talk, More than a Big Machine: Why We Need Multiple Breakthroughs to Tackle Cloud Resolving Climate [PDF], he estimated the number of floating-point operations (FLOPs) needed to compute a climate model over a one-century time scale with a 1 km atmosphere model.

His answer was the highlight of the Consortium for me: A Century Experiment requires about a mole of FLOPs. :-)

Friday November 13, 2009

I'm at Logan Airport waiting for my flight to O'Hare and then to Portland, Oregon for Sun's HPC Consortium this weekend and SC09 next week.

Google is sponsoring free wifi access at Logan through January 15th, which is how I'm able to write this blog entry -- I would not usually pay the usual $10 fee since my flight is leaving in only an hour.

After clicking through the landing page to access the Internet, I was redirected to a Give Back site that lets me make a donation to either Engineers Without Borders USA, One Economy Corporation, or Climate Savers Computing. Even better, Google will match any donation I choose to make.

I wanted to make a donation, but I didn't. Why? Because making the donation requires I create a Google Checkout account. I have a Paypal account already and I'm trying to reduce my credit card exposure on the web whenever possible, so I opted not to sign up.

Thursday November 12, 2009

On November 3rd, I received an email congratulations about my upcoming 20th anniversary with Sun (for those keeping score at home, the 20 includes some credit for time at Thinking Machines prior to our arrival at Sun) and an invitation to select a commemorative gift of my choice. My immediate thought was that I should place the order immediately, given all the current craziness and future uncertainties. My recognition award arrived via FedEx yesterday. Parrot not included.

(Wondering what's in the box?)

Wednesday November 11, 2009

Once again, I am delighted by Apple's customer service.

After having many problems with my original Macbook Pro, which Apple eventually replaced, my system has been stable and problem-free for quite awhile. Until my screen started losing pixels about a month ago.

Every other vertical line on the display became light grey, making it nearly impossible to read the screen. The problem briefly appeared and then disappeared about a month ago, but it happened again last week and stayed broken for over 12 hours despite reboots, PRAM/NVRAM resets, and SMC resets. I made the problem go away eventually by scheduling a Genius appointment at my local Apple store --- the display spontaneously started working again within an hour of making the appointment. But of course! However, not trusting the machine and needing it for an upcoming business trip, I decided to keep my appointment at the Apple store.

Without being able to actually see the problem at the store, the Genius couldn't make an absolute diagnosis, but we both felt the MBP's display was probably flaky. This conclusion was partly influenced by the fact that when I ran the system in dual screen mode, the problem was only visible on the built-in LCD -- the external monitor did not show the problem. While there still might be a logic board(*) or other problem, I felt comfortable enough to request that the screen (actually, the clamshell assembly -- the top part of the laptop, including the cables that run from the clamshell to various locations on the motherboard) be replaced. Since the MBP was no longer covered by AppleCare, I was going to have to pay for this repair myself.

I learned Apple has two repair programs. I could either opt to have the machine shipped to an Apple repair depot and expect to receive the machine in 7-10 days, shipped directly to my house, or I could have the machine repaired at the Apple store and it would likely be ready the next day if the parts were available. The depot option has a fixed price -- about $300 regardless of what the problem is or what parts need to be replaced. The in-store option is generally more expensive since you pay for the required parts and for labor. In my case, the in-store option would cost about $600 or twice as much as the depot option. What to do? I needed to work on my presentation for an upcoming conference and would be leaving for that conference in seven days. The depot might ship my machine back earlier than 7-10 days, but I'd be taking a risk.

Because I was able to make arrangements to use another laptop, I decided to opt for the cheaper depot option and wait the 7-10 days. Imagine my surprise when I got a call the next afternoon informing me that my repair had been completed. Apple had opted to do the repair in their store and they honored the depot rate I had been quoted. How cool is that?

So far, I've not had a recurrence of the problem. As a side benefit, this new display is much more evenly illuminated than the old one so even in the unlikely event the problem turns out to be something else, my machine has a nice, new LCD display that to me is worth the $300 I've paid so far. Not that I expect the problem to recur, of course.

(*) If you have this problem with your machine, look carefully at the cursor. Does it seem to "float above" the bad display or is it also affected by the dropped vertical lines? Noticing this can help diagnose the problem, since an unaffected cursor means it is more likely that the problem is either at the logic board or earlier, while an affected cursor pushes the diagnosis more towards the screen/clamshell.

The New England OpenSolaris User Group is holding its first meeting at Boston University this evening, hosted by the BU Department of Electrical & Computer Engineering. It is open anyone interested in learning more about OpenSolaris -- both students and professionals are welcome. This first meeting features three talks: What's So Cool About OpenSolaris Anyway, OpenSolaris: Clusters and Clouds from your Laptop, and OpenSolaris as a Research and Teaching Tool.

The meeting runs from 6-9pm tonight (Wed, Nov 11th, 2009) at the BU Photonics Center Building. Follow this link for directions, full agenda details, etc. If you think you'll be coming, please RSVP so we have a rough headcount for food.

See you there -- I'm bringing the pizza!

Tuesday November 10, 2009

I recently received an email promotion from Patagonia, the upscale purveyor of adventure clothing and gear, which invited me to visit one of their stores in person (how quaint) and enter their 2009 Holiday Giveaway. Reading the first paragraph of the official rules, I began to wonder if the typical Patagonia customer has landed on their head once too often while adventuring and the company therefore feels it needs to cater specifically to this demographic. Here is what I read:

OFFICIAL RULES PATAGONIA HOLIDAY 2009 GIVEAWAY

BUYING WILL NOT HELP YOU WIN. NO PURCHASE NECESSARY. A PURCHASE WILL NOT INCREASE YOUR ODDS OF WINNING. Your chances of winning without making a purchase are the same as the chances of someone winning who purchases something.

Monday October 19, 2009

As discussed in several earlier posts (here and here), virtualization technology is destined to play an important role in HPC. If you have an opinion about that or are interested in learning more, consider attending or submitting a technical paper or position paper to the Workshop on High Performance Virtualization: Architecture, Systems, Software, and Analysis, which is being held in conjunction with HPCA-16, the 16th International Symposium on High Performance Computing Architecture, and also in conjunction with PPoPP 2010, the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Apparently, Bangalore is the place to be in early January 2010!

Time is short: Workshop submissions are due November 10, 2009.

Saturday October 17, 2009

Friday October 16, 2009

I gave a short talk about Fortress, a new parallel language, at the Sun HPC Workshop in Regensburg, Germany and thought I'd post the slides here with commentary. Since I'm certainly not a Fortress expert by any stretch of the imagination, my intent was to give the audience a feel for the language and its origins rather than attempt a deep dive in any particular area. Christine Flood from the SunLabs Programming Languages Research Group helped me with the slides. I also stole liberally from presentations and other materials created by other Fortress team members.

The unofficial Fortress tag line, inspired by Fortress's emphasis on programmer productivity. With Fortress, programmer/scientists express their algorithms in a mathematical notation that is much closer to their domain of expertise than the syntax of the typical programming language. We'll see numerous examples in the following slides.

At the highest level, there are two things to know about Fortress. First, that it started as a SunLabs research project, and, second, that the work is being done in the open under as the Project Fortress Community, whose website is here. Source code downloads, documentation, code samples, etc., are all available on the site.

Fortress was conceived as part of Sun's involvement in a DARPA program called High Productivity Computing Systems (HPCS,) which was designed to encourage the development of hardware and software approaches that would significantly increase the productivity of the application developers and users of High Performance Computing systems. Each of the three companies selected to continue past the introductory phase of the program proposed a language designed to meet these requirements. IBM chose essentially to extend Java for HPC, while both Cray and Sun proposed new object-oriented languages. Michèle Weiland at the University of Edinburgh has written a short technical report that offers a comparison of the three language approaches. It is available in PDF format here.

I've mentioned productivity, but not defined it. I recommend visiting Michael Van De Vanter's publications page for more insight. Michael was a member of the Sun HPCS team who focused with several colleagues on the issue of productivity in an HPCS context. His considerable publication list is here.

Because I don't believe Sun's HPCS proposal has ever been made public, I won't comment further on the specific scalability goals set for Fortress other than to say they were chosen to complement the proposed hardware approach. Because Sun was not selected to proceed to the final phase of the HPCS program, we have not built the proposed system. We have, however, continued the Fortress project and several other initiatives that we believe are of continuing value.

Growability was a philosophical decision made by Fortress designers and we'll talk about that later. For now, note that Fortress is implemented as a small core with an extensive and growing set of capabilities provided by libraries.

As mentioned earlier, Fortress is designed to accommodate the programmer/scientist by allowing algorithms to be expressed directly in familiar mathematical notation. It is also important to note that Fortress constructs are parallel by default, unlike many other languages which require an explicit declaration to create parallelism. Actually to be more precise, Fortress is "potentially parallel" by default. If parallelism can be found, it will be exploited.

Finally, some code. We will look at several versions of a factorial function over the next several slides to illustrate some features of Fortress. (For additional illustrative Fibonacci examples, go here.) The first version of the function is shown here beneath a concise, mathematical definition of factorial for reference.

The red underlines highlight two Fortressisms. First, the condition in the first conditional is written naturally as a single range rather than as the more conventional (and convoluted) two-clause condition. And, second, the actual recursion shows that juxtaposition can be used to imply multiplication as is common when writing mathematical statements.

This version defines a new operator, the "!" factorial operator, and then uses that operator in the recursive step. The code has also been run through the Fortress pretty printer that converts it from ASCII form to a more mathematically formatted representation. As you can see, the core logic of the code now closely mimics the mathematical definition of factorial.

This non-recursive version of the operator definition uses a loop to compute the factorial.

Since Fortress is parallel by default, all iterations of this loop could theoretically be executed in parallel, depending on the underlying platform. The "atomic" keyword ensures that the update of the variable result is performed atomically to ensure correct execution.

This slide shows an example of how Fortress code is written with a standard keyboard and what the code looks like after it is formatted with Fortify, the Fortress pretty printer. Several common mathematical operators are shown at the bottom of the slide along with their ASCII equivalents.

A few examples of Fortress operator precedence. Perhaps the most interesting point is the fact that white space matters to the Fortress parser. Since the spacing in the 2nd negative example implies a precedence different than the actual precedence, this statement would be rejected by Fortress on the theory that its execution would not compute the result intended by the programmer.

Don't go overboard with juxtaposition as a multiplication operator -- there is clearly still a role for parentheses in Fortress, especially when dealing with complex expressions. While these two statements are supposed to be equivalent, I should point out that the first statement actually has a typo and will be rejected by Fortress. Can you spot the error? It's the "3n" that's the problem because it isn't a valid Fortress number, illustrating one case in which a juxtaposition in everyday math isn't accepted by the language. Put a space between the "3" and the "n" to fix the problem.

Here is a larger example of Fortress code. On the left is the algorithmic description of the conjugate gradient (CG) component of the NAS parallel benchmarks, taken directly from the original 1994 technical report. On the right is the Fortress code. Or do I have that backwards? :-)

Several ways to express ranges in Fortress.

The first static array definition creates a one dimensional array of 1000 32-bit integers. The second definition creates a one dimensional array of length size, initialized to zero. I know it looks like a 2D array, but the 2nd instance of ZZ32 in the Array construct refers to the type of the index rather than specifying a 2nd array dimension.

The last array subexpression is interesting, since it is only partially specified. It extracts a 20x10 subarray from array b starting at its origin.

Tuple components can be evaluated in parallel, including arguments to functions. As with for loops, do clauses execute in parallel.

In Fortress, generators control how loops are run and generators generally run computations in any order, often in parallel. As an example, the sum reduction over X and Y is controlled by a generator that will cause the summation of the products to occur in parallel or at the least in a non-deterministic order if running on a single-processor machine.

In Fortress, when parallelism is generated the execution of that work is handled using a work stealing strategy similar to that used by Cilk. Essentially, when a compute resource finishes executing its tasks, it pulls work items from other processor's work queues, ensuring that compute resources stay busy by load balancing the available work across all processors.

Essentially, a restatement of an earlier point: In Fortress, generators play the role that iterators play in other languages. By relegating the details of how the index space is processed to the generator, it is natural to then also allow the generator to control how the enclosed processing steps are executed. A generator might execute computations serially or in parallel.

A generator could conceivably also control whether computations are done locally on a single system or distributed across a cluster, though the Fortress interpreter currently only executes within a single node. To me, the generator concept is one of the nicer aspects of Fortress.

Guy Steele, who is Fortress Principal Investigator along with Eric Allen, has been working in the programming languages area long enough to know the wisdom of these statements. Watch him live the reality of growing a language in his keynote at the 1998 ACM OOPSLA conference. Be amazed at the cleverness, but listen to the message as well.

The latest version of the Fortress interpreter (source and binary) is available here. If you would like to browse the source code online, do so here.

Some informational pointers. Christine also tells me that the team is working on an overview talk like this one. Except I expect it will be a lot better. :-) Though I only scratched the surface in a superficial way, I hope this brief overview has given you at least the flavor of what Project Fortress is about.

Friday September 04, 2009

Sun is holding a virtual HPC conference on September 17th featuring Andy Bechtolsheim as keynote speaker. Andy will be talking about the challenges around creating Exaflop systems by 2020, after which he will participate in a chat session with attendees. In fact, each of the conference speakers (see agenda) will chat with attendees after their presentations.

This is a free event. Register here.

Thursday August 27, 2009

Two weeks ago the Parallel Computing Laboratory at the University of California Berkeley ran an excellent three-day summer bootcamp on parallel computing. I was one of about 200 people who attended remotely while another large pile of people elected to attend in person on the UCB campus. This was an excellent opportunity to listen to some very well known and talented people in the HPC community. Video and presentation material is available on the web and I would recommend it to anyone interested in parallel computing or HPC. See below for details.

The bootcamp, which was called the 2009 Par Lab Boot Camp - Short Course on Parallel Programming covered a wide array of useful topics, including introductions to many of the current and emerging HPC parallel computing models (pthreads, OpenMP, MPI, UPC, CUDA, OpenCL, etc.), hands-on labs for in-person attendees, and some nice discussions on parallelism and how to find it with an emphasis on the motifs (patterns) of parallelism identified in The Landscape of Parallel Computing Research: A View From Berkeley. There was also a presentation on performance analysis tools and several application-level talks. It was an excellent event.

The bootcamp agenda is shown below. Session videos and PDF decks are available here.

Friday July 24, 2009

This microfinance stuff is pretty cool. By placing funds into my Kiva account I've been able to make small loans to a variety of people around the world and then continue to make additional loans as loan payments are deposited to my account.

I've been using Kiva now for just over two years and have made a total of 20 loans, ten of which have been completely repaid and ten of which are currently in active repayment. I've had no delinquencies and no defaults. While I can withdraw my funds once they are paid back to my account, I plan to continue making loans indefinitely since, 1) psychologically it feels as if I spent the money two years ago when I opened my account, and 2) I can see that these loans really make a difference for people.

My previous loans were to people in Ghana, Tanzania, Nicaragua, Peru, Lebanon (2), Pakistan, Azerbaijan, Cambodia, and Samoa. My current loans are for construction, clothing sales, crafts, transportation, hardware, tailoring, etc. to people in Lebanon, Nicaragua, Nigeria, Cambodia (2), Samoa, Mexico, Tajikistan, and Azerbaijan (2).

As part of the 40th anniversary commemoration of the Apollo 11 moon landing, the source code for both the Command Module and the Lunar Module have been released. In addition to being able to view the code, you can also run it on an emulator. Go here for details. Cool!!

Thursday July 16, 2009

Skitch is a free, Mac-based utility that I use all the time when writing blog entries because it makes preparing images for my blog so simple. Specifically, this what I do with Skitch:

I use Skitch for screen captures. Preview's "grab" function is pretty good, but Skitch makes this much easier. When I select "Capture --> Crosshair" the Skitch window helpfully disappears from the desktop, exposing everything underneath. No more fiddling with window positions to ensure no current Preview windows are in the way of what you are trying to capture.

With Skitch I can resize an image by dragging. Or crop it by dragging. I especially like that the expected size of the image (KB or MB) is always displayed, which is really useful for ensuring that I don't embed overly large images in my blog entries. The size will change depending on what output format is selected (e.g. jpg, gif, png, tif, svg, bmp, etc.)

I can also annotate images with Skitch by drawing or writing on them. I use this frequently in my blog. Some examples are here and here.

Skitch is also a web service, though I don't use that capability. And the Skitch application as many more functions than I've described here. It really is worth a look -- for me it has become an indispensable tool.