Watercooler Signing Off - Hire Me!

Rich Brueckner will not be joining Oracle, so this is the last Sun HPC Watercooler entry.

It's all good! I've had a great 10-year ride here-- besides writing this blog (1780 stories!) I had the privilege to manage ten Supercomputing shows and numerous other HPC events for Sun. My only regret is that I will no longer get to work with Sun's brightest talents and our wonderful customers any longer.

So, in closing, if any of you know of some outfit that needs a creative, somewhat twisted, HPC Marketing guy with a Red Hat, please keep in touch!

Cheers,

-Rich

email: richard.brueckner@gmail.com
Webhome: rich.brueckner.com

Topology-Aware Scheduling in Sun Grid Engine

Dan Templeton continues his deep-dive into Sun Grid Engine capabilities with a look at Topology-Aware Scheduling:

"With topology-aware scheduling, the user who submits the job can specify how that job should be laid out across a machine's CPUs. Users are allowed to specify three different flavors of distribution strategy: linear, striding, or explicit. In linear distribution, the execution daemon will place the job's threads/processes on consecutive cores if possible. If it can't fit the entire job on a single socket, it will span the job across sockets. The striding strategy tells the execution daemon to place the job on every nth core, e.g. every 4th core or every other core. The explicit strategy lets the user decide exactly which cores will be assigned to the job. Note that the core binding is a request, not a requirement. If for some reason the execution daemon can't fulfill the request, the job will still be executed; it just won't be bound." Full Story

Posted by Rich Brueckner [Cloud Computing] ( January 29, 2010 05:00 AM ) Permalink | Comments [0]

Sandia Tests HPC Virtualization

We've heard a lot about virtualization for HPC from folks like Josh Simons. Now Sandia National Labs is putting the idea to the test thanks to the computing power of their Red Storm supercomputer:

"Thus far, supercomputing hasn't made great use of virtualization. "The conventional wisdom in the HPC community is that virtualization has too high of an overhead to be useful," Pedretti said, in an e-mail interview. "We hoped to challenge this by showing reasonable overhead was possible even for tightly-coupled, communication-intense scientific applications running at a large scale in a virtual machine."

Virtualization could prove valuable, Pedretti contends, because researchers wishing to do large-scale simulations or number-crunches would no longer be bound to using only those applications that run on supercomputer operating systems. Full Story

Sun and QDR InfiniBand Power Africa's Largest Super

Voltaire today announced that the Centre for High Performance Computing (CHPC) in Cape Town, South Africa has selected a Voltaire 40 Gb/s QDR InfiniBand director switch as part of its new Sun Microsystems supercomputer. The Sun Constellation System, includes three Sun Blade 6048 Modular Systems, with 144 Sun Blade X6275 server modules using the Intel Xeon processor 5500 series, and a Sun SPARC Enterprise M9000 server with 64 SPARC64 VII quad-core processors. Full Story

New Whitepaper: Double Your Performance with Flash

Increasing your application performance is easier than you think. Find out how by reading this new white paper, Turbocharging HPC Application Performance with Flash Storage, which covers both the business benefits and technical implications.

You'll get details about the performance gap between CPU and I/O performance and learn why the use of Flash technology is a simple and cost-effective methodology to bridge the gap. In addition, the paper explains the performance boosts that can be gained with Flash-based storage solutions, using commercially-available applications benchmarked in before and after scenarios. Download Whitepaper

Sun HPC News January

The latest Sun HPC Newsletter is out. Don't miss an issue--Subscribe today!

Dispelling Green Grid Rumors

Despite rumors to the contrary, Dave Douglas blogs that the Green Grid remains a critical resource in the effort to make the world's datacenters sustainable:

"What I particularly appreciate is the focus on practical information and tools, in addition to the standards and advocacy work." Full Story

Andy Bechtolsheim to Keynote HPC Financial Markets, April 19, NYC

Andy Bechtolsheim will keynote the 2010 High Performance Computing Financial Markets show on April 19, 2010, in New York City. Mr. Bechtolsheim will address key trends in HPC, High Performance networks, and the performance impact of solid state memory on data bandwidth. Also speaking will be Stanley Young, CEO of NYSE Technologies and co-CIO of NYSE Euronext. Mr. Young will be joined by leading Wall Street professionals covering Low Latency, High Frequency Trading, and the need for speed.

Oracle will have a booth at the show, so we'll see you there! Full Story

Video: Developing a Curriculum for Parallelism Part 1

Recorded at SC09.

Abstract: Time is no longer a best performance programming tool, by just waiting 18 months for a doubling of performance. The discussion initiated by last year's panel led to the conclusion that parallelism must seamlessly be woven throughout the CS curriculum. This year provides a diabolic panel focused on details; the six panelists will spend 5 minutes each describing what needs to be done, what they have done, and what they plan to do. The second half will be another near free-for-all with the room and the panelists further discussing these hard to implement simply stated goals.

Panelists include:Tom Murphy-Contra Costa College, Robert Chesebourgh-Intel Academic Program, Dr. Jim Larus-Microsoft Research, Dr. John Gustafson-Intel Labs, Dr. Bill-NVIDIA, Dr. Simon Mclintosh Smith-Bristol College, Charles Peck-Earlham College.

This video has more chapters available here.

Want to see more? The Intel Academic Community has posted a complete set of videos from SC09 including BoF's, panel discussions, and more. Full Story

Webinar: Turbo-Charge HPC with Flash Storage

In this webinar, you will learn how to move data into and out of your cluster at the highest rate possible using the Sun Storage 7000 Unified Storage System coupled with Flash technology. Launch Webinar

Hadoop Integration with Sun Grid Engine 6.2u5

Dan Templeton takes a closer look at the new Apache Hadoop integration with Sun Grid Engine 6.2u5:

"The Hadoop integration consists of two halves. The first half is a parallel environment that can start up a jobtracker and tasktrackers on the nodes assigned by the scheduler. (HDFS is assumed to already be running on all the nodes in the cluster.) In the end, it's really no different than an MPI integration (especially MPICH2). The second half is called Herd. It's the part that talks to HDFS about blocks and racks, and it has two parts. The first part is the load sensor that reports on the block and rack data for each execution host (hosts that are running the SGE execution daemon). The second part is a Job Submission Verifier that translates requests for HDSF data paths into requests for racks and blocks." Full Story

Posted by Rich Brueckner [HPC Article of the Day] ( January 20, 2010 05:00 AM ) Permalink | Comments [0]

Virtualization for HPC: Heterogeneity

Josh Simons contends that virtualization has much to offer HPC customers in terms of heterogeneity:

Homogeneity can be a real problem for end-users whose applications only run on either other versions of the chosen cluster OS or, worse, on completely different operating systems. These users are generally not able to use these centralized facilities unless they can port their application to the appropriate OS or convinced their application provider to do so.

The situation with respect to heterogeneity for software providers, or ISVs -- independent software vendors, is quite different. These providers have been wrestling with expenses and other difficulties related to heterogeneity for years. For example, while ISVs typically develop their applications on a single platform (OS 0 above,) they must often port and support their application on several operating systems in order to address the needs of their customer base. Assuming the ISV decides correctly which operating systems should be supported to maximize revenue, it must still incur considerable expenses to continually qualify and re-qualify their application on each supported operating system version. And maintain a complex, multi-platform testing infrastructure and in-house expertise to support these efforts as well." Full Story

Posted by Rich Brueckner [HPC Article of the Day] ( January 19, 2010 05:00 AM ) Permalink | Comments [0]

Video: Preparing the World for Ubiquitous Parallelism

This SC09 Panel Session, "Preparing the World for Ubiquitous Parallelism" was moderated by Dr. Matthew Wolf (Georgia Institute of Technology).

Abstract:
Multicore platforms are transforming the nature of computation. Increasingly, FLOPs are free, but the people who know how to program these petascale, manycore platforms are not. This panel, composed of a diverse set of industry and academic representatives, is focused on presenting and discussing the abstractions, models, and (re-)training necessary to move parallel programming into a broad audience. The panelists, often vigorous competitors, will present the situation from their respective viewpoints, likely fueling a lively discussion of the entire room as has been seen recently in other venues.

Panelists:

Dr. Benedict Gaster (AMD)
Kevin Goldsmith (Adobe Systems Inc.)
Dr. Charlie Peck (Earlham College)
Dr. Steven Parker (NVIDIA)
Dr. Michael Wrinn (Intel Corporation)
Dr. Evan Smyth (DreamWorks)

This video has four more chapters:
Part 2
Part 3
Part 4
Part 5

Want to see more? The Intel Academic Community has posted a complete set of videos from SC09 including BoF's, panel discussions, and more. Full Story

Video: John Gustafson - It's Time to Teach Parallelism

Do universities need to totally rethink how they teach Computer Science for today's world of Parallelism? Sun Alum John Gustafson, now Director of Intel Research Labs, speaks his mind at the Teach Parallel studio at SC09.

Sun Grid Engine: Now Cloud-Aware

The Virtual Steve Blog describes why Sun Grid Engine is the world's first Cloud-aware distributed resource manager:

"Many auto companies use Grid Engine to coordinate the resources on the Grid/Cluster/Cloud they use for mechanical design and simulation. Users across the company submit jobs (e.g. a crash simulation) and Grid Engine queues them and dispatches them based on priority and policy. However, what happens when your submissions start to outpace the ability of your systems to keep up? In the traditional model, you'd have to buy new hardware and add it to your Grid/Cluster/Cloud. With the Grid Engine you can now configure rules that allow you to "cloud burst" these workloads out to another cloud." Full Story

Video: High-Performance Virtualization - A Benchmark Guide

In this recorded eSeminar you will learn about:

* Innovations in AMD Opteron™ processor technology
* Connections between processor innovation and virtualization
* Benchmarks that show Sun servers utilizing AMD Opteron processors deliver scalable performance

Are your virtualized systems yielding the scalability you set out to achieve? Watch the Video (requires registration)

TACC Ranger: Turning Environmental Info into Action

Patrick Reed of Penn State is using the Sun HPC-powered TACC Ranger to test an observational framework that will improve our understanding of long-term environmental change:

“Despite the long history of using observations of the natural world in our management frameworks, our approach to environmental monitoring is largely ad hoc,” Reed said. “We don’t have a very good understanding of how to design or adapt our long-term monitoring efforts. This is a fundamental problem.” Full Story

Defensive Programming at TACC

To ensure the quality of answers derived from computation, TACC senior research associate Robert McLay recommends users consciously design their code to limit errors and bugs, a process called defensive programming:

“Programming is hard,” said McLay. “It’s easy to have a simulation that doesn’t simulate real-world gravity at all. Because it’s so hard to get the simulation right, and you can’t check these things by hand, you have to do everything in your power to protect the integrity of your research.” Full Story

Monmouth College Shares Sun HPC Know-How

Monmouth College will host a collaborative two-day HPC workshop enitled “Developing a High Performance Computing Consortium for Multi-Disciplinary Applications on ACM Campuses.”

“HPC will allow us to model complex systems to solve some of the most important problems that face our world today,” said Fasano. “HPC may dramatically improve our ability to model and understand our economy, to develop viable energy sources and to engineer new drugs and treatments.” Full Story

A Closer Look at Sun Grid Engine 6.2 update 5

Dan Templeton blogs on the latest features in Sun Grid Engine:

"The Sun Grid Engine 6.2 update 5 release is now available. Don't let the unassuming version number fool you; there's quite a few interesting features packed into this release including Integration with Apache Hadoop, Topology-aware Scheduling, Slotwise Subordination, SGE Inspect, and Improved Power Management." Full Story

Make The Most Of Your SSD With ZFS

The Directory Services Blog shows you how to make the most of your SSD with ZFS:

"If you're anything like me, you're lucky if you have a single SSD in your workstation or server. I had a dilemma in the past, I couldn't quite find a way to make the most of my one flash drive: I had to chose between making it a ZIL or use it as L2ARC. I dreaded having to make a definitive choice for one or the other." Full Story

Futures of Data Storage

Our favorite storage pundit Henry Newman lays out his predictions for data storage in 2010:

"Flash SSD usage will increase, with multiple RAID vendorsa and multiple controller card vendors providing better support, which means higher bandwidth and more IOPS in 2010. 10 Gigabit Ethernet will become the standard connectivity for almost all systems. Higher-end home PCs from Dell, HP and others will support this technology. Home routers from multiple vendors will have this support, likely before the end of the year." Full Story

Posted by Rich Brueckner [HPC Article of the Day] ( January 09, 2010 05:00 AM ) Permalink | Comments [0]

Video: Business Ready HPC for MCAE

Numerical simulation of real-world problems has become standard practice in manufacturing — and Sun HPC solutions are leading the way. Download Solution Brief: Sun Business Ready HPC Solutions for MCAE

Podcast: Flash - A Disruptive Technology for the Enterprise

Flash memory technology has been a total game changer in the consumer space over the last twenty years. Now, enterprises are starting to reap the benefits of flash. Host Hal Stern, VP Global Systems Engineering, Sun Microsystems, interviews Lisa Noordergraaf, a Senior Performance Engineer in Sun's Performance Technology group, on flash as a disruptive technology and how Sun's innovation with flash in the storage hierarchy has vastly improved system performance for enterprises. Full Story

Video: Sun Storage 7000 Simulator

This video introduces the Analytics that are a part of the Sun Storage 7000 series. Try it for yourself; now you can install and manage a virtual Sun Storage 7000 Unified Storage System on your desktop. Download Simulator