Twitter test: Preview of HPC on Wall St.

We are trying something new here. Sun is planning for some major excitement at the High Performance on Wall Street show on September 22 in New York City. Now you can follow the show live as Sun folks and our partners twitter live from the show floor.

At the show, Sun's Ian Pearl will present on Low Latency Trading and Market Data:

Ian Pearl has been involved with trading and market data systems for over 20 years. He has worked for trading technology suppliers including Misys and CSK Micrognosis as well as for a number of banks including Allied Irish, and has been responsible for deployment projects for trading rooms in many others. He has founded and grown a financial software house focused on trading applications on the Reuters RMDS platform, and was the developer of the RTT toolkit which was the original Reuters' API for building real-time trading applications using Visual Basic and Excel. Ian was also one of the founding directors of EasyScreen, the trading system built for the LIFFE Connect market when London's LIFFE futures market moved from floor to screen-based trading. He has been working for Sun Microsystems for 3 years and currently has global responsibility for Low Latency Trading and Market Data systems in the Global Financial Systems Industry team. Conference Site

Kotura awarded $14M silicon photonics contract with Sun, DARPA

Kotura, a leading provider of silicon photonic components, today announced that Sun Microsystems has signed a five year $14M development contract with Kotura for DARPA’s Ultraperformance Nanophotonic Intrachip Communications (UNIC) program. Previously, Sun announced its participation in DARPA's $44M program to advance a virtual supercomputer using an on-chip network of low-cost optical interconnects. Full Story

Monitoring Your HPC Cluster

Linux has detailed story on how to monitor your linux cluster with Ganglia:

"Ganglia consists of several components designed specifically for the different aspects of monitoring, collecting, and displaying metrics from HPC systems in an efficient and scalable way. It was originally written by Matt Massie at the University of California, Berkeley (unsurprisingly, Ganglia was released under a BSD license), and is actively maintained by a small group of developers. Ganglia is used by commercial, educational, government, and non-profit organizations across the world to monitor some of the largest clusters currently in operation. A partial, but still impressive, list of organizations using Ganglia can be found on the Ganglia homepage. The current stable release is ganglia-3.1.0. This new release has a number of improvements over the previous 3.0.x series, including a new modular interface for adding metrics directly to gmond (with C and Python bindings), the addition of several new core metrics, and a number of display improvements. While the screenshots below were taken using version 3.0.5, the setting up a new Ganglia 3.1.0 installation is essentially the same." Full Story

And if any of my colleagues read this and care to comment how some of Sun's new xVM system management offerings compare, I'd love to write that up ;-)

Server Rack Noise: A Barrier To HPC Adoption

Server racks are very noisy due to all the fans that it takes to remove the heat from today's processors, memory, and storage devices. Is it hurting business?

"The high level of noise generated by server racks is one of the main obstacles in companies adopting high-performance computing (HPC) in offices, one firm has claimed. According to Colfax, businesses are sceptical about employing HPC technology due to the noise associated with server racks, however the organisation claims it has found a solution to the problem." Full Story

Solaris & Linux: Say Hello to Sun HPC ClusterTools 8.0

Kuldip Oberoi writes:

The growing popularity of High Performance Computing (HPC) environments, which are based on supercomputers and computer clusters, is being driven not only by (P)erformance, but also by the (P)roductivity in using these environments. In addition to the expansion in traditional scientific and engineering market segments, HPC is being adopted in Enterprise IT segments, such as Finance and Business Intelligence.

With its AMD x86, Intel x86, and Sun UltraSPARC-based servers, Sun continues to be a leader on the hardware side. However, as a systems company, we are also investing into software and now offer complete HPC software stacks on both Solaris and Linux OSs. For developers, the freely available Sun Studio compilers and tools offers up record-setting C, C++, and Fortran compilers, support for MPI and OpenMP development, performance analysis tools, and optimized math routines (e.g. LAPACK, BLAS, etc.). Central to this HPC stack is a high-performance, scalable, MPI (Message Passing Interface) runtime. MPI is the dominant communication paradigm for parallel applications written for clusters.

Today we are announcing the availability of Sun HPC ClusterTools 8.0, which now supports Linux distributions as well as Solaris and OpenSolaris OSs, and has the following highlights:

Based on the open source Open MPI project, version 1.3
For the first time, available and supported on Linux (RHEL 4&5, SLES 9&10) as well as Solaris 10 OS
Available with the next OpenSolaris OS release
Supported with Sun Studio compilers and tools and GNU/gcc toolchains on both Solaris and Linux OSs
MPI profiling support with Sun Studio Analyzer, plus support for VampirTrace and MPI PERUSE
Infiniband multi-rail support
Mellanox ConnectX Inifiniband support
DTrace provider support on Solaris
Enhanced performance and scalability, including processor affinity support
Support for InifiBand, GbE, 10GbE, and Myrinet interconnect

With available Sun support options, download a free, and unrestricted, copy of Sun HPC ClusterTools 8.0 today!

What Is the Metaverse and Should HPC Care?

Other than the geek factor, why should HPC people care about the Metaverse and alternate spaces like those depicted in the book, Snow Crash?

"If these technologies become as commonplace and important as we believe they will, people who choose not to participate may end up as left out of commercial and civic discourse as Web-ignorant people are today." Full Story

The Metaverse Matrix (Courtesy of the Acceleration Studies Foundation)

YouTube: Hadron Collider Rap

You know what? Bad rap music can be extremely compelling and informative:

"Science can be pretty weird, especially theoretical physics, but weirder still is watching someone rapping about the Large Hadron Collider. (That's the giant device in Switzerland that will recreate the Big Bang, among other things.) Time to recalibrate your strangeness meters - science writer Kate McAlpine and some friends filmed themselves busting various moves deep in the caverns of the LHC while Kate dropped mad verse about the collider."

In other CERN news, the Lustre file system is getting major traction as the way to do paralell I/O:

"In previous meetings, Lustre had appeared as an interesting file system. This week, judging by the number of times it was talked about, it is now the file system of choice across much of the HEP world. This was confirmed by the presentation of the results of carefully prepared tests performed by the HEPiX Storage Working Group for whom Lustre is now the recommendation for a cluster file system." Full Story

Sun: Open Standards to Usher New Era in Manufacturing

Sun announced today that it will join the MTConnect Technical Advisory Group (MTAG) to further define the open communication protocol standard it helped create for the manufacturing technology industry a year ago. MTConnect is an open manufacturing technology standard that uses proven, royalty free Internet communications technologies as its basis to allow manufacturing technology vendors and customers to safely and easily communicate.

"Sun Microsystems has a very long history of working with the industry and academia to create and promote open technology standards that drive genuine innovation," said Dave Edstrom, Chief Technologist of the Americas Software Practice for Sun Microsystems. "Open source and open standards are the keys to unlocking manufacturing innovation and efficiency around the world, particularly in growing emerging markets. I am thrilled Sun has been able to play a pivotal role in the development of such an important initiative as MTConnect." Full Story

What can nature teach us about really big clusters?

InsideHPC points us to this story in Linux Magazine on how large colonies of organisms work in nature and what this can teach us about managing extremely large compute clusters:

"Insect colonies are in the truest sense of the word parallel computers. Think about it. How do you control one million ants. They work together surprisingly well without MPI and Gigabit Ethernet. Entomologists will tell you that ants communicate using chemicals, but much of what they do is preprogrammed and governed by what I would assume are simple rules. An individual ant is useless on its own and expendable, but very good at doing its particular job when working with others." Full Story

Virtual Clustering at TACC with MyCluster

The TeraGrid is designed to enable scientists to leverage the most powerful supercomputers to accomplish their research goals, regardless of where they are located. But how can researchers to harness all the different systems on the Grid together, maximizing the efficiency and usefulness of the network? Enter MyCluster::

"MyCluster creates a virtual cluster, a group of linked computers that work together closely and can be managed remotely. Interacting with the user’s preferred job management system (with which most scientists are familiar), the tool sets up job proxies on whatever system can accomplish the computing tasks most quickly. When these jobs run, the provisioned computing resources appear on the user’s personal computer as a virtual cluster that dynamically shrinks or expands over time.

“All the user has to worry about is submitting his own jobs into his own personal cluster,” Walker said. “The user doesn’t have to worry about the infrastructure or deal with the heterogeneous usage policies at the different supercomputing sites. Full Story

Sun xVM White Paper is one compelling story

A new Sun xVM White Paper provides a fairly high-level look at the Sun xVM product family in white paper form. For those of us looking for a better way to do System Management, the paper is a great introduction to xVM Virtual Box, xVM Server, xVM Ops Center and xVM VDI. Download PDF

Two More Xeon-Based Galaxy Servers from Sun

The ITJungle takes a look at Sun's newest x86 servers for HPC:

"The Xeon versions of these rack-mounted machines, Sun hopes, will help boost its Galaxy server volumes, which have not grown as fast as Sun needs them to if it is to be a powerhouse in the X64 server space. "Some customers are simply not open to an AMD machine," explains Huynh--a lesson that Sun has learned the hard way after it endorsed Opterons early and came to the Intel party late. This is one of the reasons why the Galaxy server line has been running only at an estimated $500 million to $600 million annual run rate, when it should be a lot larger given the excellence of Sun's engineering on the Galaxy products. Sun also has a much smaller partner channel than either Hewlett-Packard and IBM, who sell the bulk of the X64 boxes these two vendors put in the field. Getting these partners to add a second or third vendor is a very tricky task--and one that Sun has not had a lot of success with to date, despite the benefits of the Galaxy line compared to its X64 server competition." Full Story

Dedicated Servers or Grid Computing?

Is the grid computing model finally getting traction in commercial markets in the form of Cloud Computing? The HostingNews takes a look:

"So why aren't all applications and websites simply converting over to grid computing? The answers are numerous and in some cases complex. There are companies that don't want to be bothered by the migration and switching costs of moving operations to grid computing. New systems and vendors mean additional overhead in switching billing, moving data, and working out start up bugs that smooth running websites simply don't want to tolerate for the marginal savings. There is also the learning curve as system administrators learn the skills needed to manage the grid computing systems versus the dedicated servers that they are used to and well-skilled in. Ultimately, there is simply the 'tried and true' system of dedicated servers that has yet to be outdone by grid computing. However, this is not necessarily the fault of grid computing - but simply a byproduct of the massive scalability of the system." Full Story

Grand Challenge: Millisecond-scale Molecular Dynamics Simulations

David E. Shaw, chief scientist of D.E. Shaw Research and a senior research fellow at the Center for Computational Biology and Bioinformatics, Columbia University, opened the SIAM Conference on Parallel Processing for Scientific Computing with a plenary talk on his group’s exciting new research: designing massively parallel machine architectures and algorithms for the grand challenge of millisecond-scale molecular dynamics simulations of biological macromolecules.

D.E. Shaw Research (DESRES) conducts research in computational biochemistry to elucidate structures and dynamic behaviors of proteins. Molecular dynamics simulation is an important tool for the modeling of protein-size systems (25,000–50,000 atoms in water); applications include the development of new drugs. The grand challenge is to simulate such systems as long trajectories, in the millisecond time scale, where biologically interesting phenomena occur. Among these phenomena are the folding of proteins, the binding of drugs to molecular targets, interactions between proteins, and the dynamics of conformational changes in macromolecules. To put this molecular dynamics challenge into perspective, a single processor can simulate about one nanosecond in a day, and a massively parallel code might be able to simulate about one hundred nanoseconds per day. Meeting this grand challenge will thus require close to a hundred-fold speedup, which in turn will require new massively parallel architectures and innovative algorithms. Full Story

Which Technologies are Transforming IT?

Research firm IDC asked hundreds of line-of-business execs and IT professionals to name the one technology that has the greatest impact on transformation of business process. The results may surprise you; In reality, it’s the merging of several leading technologies will shape the future of enterprise technology.

“I look at the addition of Web services, SOA, and virtualization – those three things together are really all about this next generation of IT," Gens says. It’s a world in which users will have relatively efficient, low cost access to business and IT resources. Full Story

Sun Updates Open Storage

Sun is continuing down the road of Open Storage:

"In 2007, for example, the company published the source code on a number of its NAS technologies, including the ZFS file system and parallel NFS. In April 2008, Sun formally announced its OpenSolaris Storage initiative, opening up Solaris code related to iSCSI device drivers, QLogic Fibre Channel HBA drivers, and Java implementations of the RPC and NFS protocols, so third parties could enhance its solutions. In addition, new developer tools and expanded professional services were designed to make it simpler for third parties to deliver OpenSolaris Storage solutions.

"To change its marketing approach successfully, Sun will have to make its plans clear to end users. “Potential customers will have to understand that Sun’s movement to open source is not just a concept; the technology will be incorporated into various Sun products,” said Forrester’s Balaouras. The vendor has tried to do that with recent announcements of the Sun Fire X4500 family of hybrid server/storage systems and its J4000 line of storage arrays." Full Story

IDF: Inside Nehalem

This Hot Hardware article combines several IDF presentations to give us the clearest picture yet on Intel's pending Nehalem processor:

"Intel claims that Nehalem represents the biggest platform architecture change to date. This might be true, but it is not a grounds-up, completely new architecture design. An Intel representative told us that Nehalem "shares a significant portion of the P6 gene pool"--it does not include many new instructions and has approximately the same sized pipeline as Penryn. Nehalem is built upon Penryn, but with significant architectural changes to improve performance and power efficiency. It includes more external ports and deeper buffers. Nehalem will be manufactured on a 45nm process and will be the basis for Intel's forthcoming platforms, including the desktop, server, and mobile spaces." Full Story

Altair Ports Hyperworks to Solaris x86

Altair Engineering, Inc., a leading global provider of technology and services that strengthen client innovation and decision-making, today announced the company has now ported solver solutions to Sun's Solaris(TM) 10 Operating System (OS) on x86 systems. The ported versions of RADIOSS, OptiStruct and MotionSolve are scheduled to be available by the end of this year.

"This agreement affirms our long-term relationship with Sun and our desire to work with them to give our customers powerful CAE solution choices," said Michael Humphrey, Altair vice president of partner programs. "We're also very pleased that Sun is a global Gold Sponsor for our HyperWorks Technology Conference (HTC) series this year, which will be held at locations around the world from July to November."

Altair has already modified the MPP and SMP versions of RADIOSS, as well as the OptiStruct and MotionSolve codes, to run on the Solaris 10 OS and provide compatibility with Solaris containers and future releases of the operating system. Altair has also received hardware and technical support from Sun. Full Story

Video: James Gosling on the Multi-core state of play

Multi-core CPUs promise to be the biggest revolution in programming since object orientation -- but it remains virtually unheard of to most developers. Thanks to the development and uptake of multi-core CPUs, developers must begin to consider truly programming in parallel:

"To take advantage of modern architectures, applications will increasingly need to be parallelised as much as possible. There is no point in creating an application with four threads because the latest chip has four cores, as in 12 months that chip will be obsolete and replaced by a chip with even more cores. It has never been a good maintenance idea to base client applications tightly on inconsistent and easily upgradable hardware -- and that idea remains true for multi-core." Full Story

AMD Hits Intel With Shanghai Surprise

The Register reports on AMD's chipset plans for the HPC market:

"AMD plans to dish up a server platform based on a new chipset in the first half of 2009 - meanwhile, its 45 nanometre Shanghai processor will be released in the fourth quarter of this year. The chip maker made the announcements on Friday, just before the Intel Developer Forum kicked off in San Francisco. The new chipset is aimed at servers with multiple sockets to plug in additional server chips. AMD claimed the platform would improve performance through new virtualisation resources and support for HyperTransport 3.0 bus technology." Full Story

YouTube: AMD Powers Lifelike Animation

With the technology demonstrated in this video, I'm thinking that visualization has taken a major step forward.

"The woman above is not real. I mean, she was real once, when real actress Emily O’Brien provided Image Metrics (you know their work from GTAIV) with 35 facial poses in front of a pair of digital cameras. From there, O’Brien was dismissed so the animators could go to work. Apparently "ninety per cent of the work is convincing people that the eyes are real." And the results—while not always perfect—are pretty extraordinary." Full Story

You can also check out Sun's advanced visualization solutions.

Bake-off: OpenSolaris 2008.05 vs. Linux

Linux Format takes OpenSolaris for a test drive, examining the similarities and differences between the OS and a typical Linux distro:

"If you want to sample the mighty ZFS filesystem, OpenSolaris is definitely the way to go." Full Story

Xeon Powers Newest Sun Fire Server for HPC

Sun Microsystems today extended its family of Intel Xeon-based servers including the fastest 1U server for HPC workloads and the most expandable 2U enterprise-class system based on Intel Xeon processors.

The Sun Fire X2250 and Sun Fire X4250 servers, powered by one or two dual- or quad-core Intel Xeon processors 5200 or 5400 series, run a variety of operating systems, including Solaris, Linux and Windows. To take advantage of special offers and promotions for the new servers, visit the Sun Try & Buy site.

The Sun Fire X2250 server is a perfect 1U server for HPC workloads such as MCAE, EDA, energy and financial services, giving customers an inexpensive compute engine for highly dense, power-sensitive environments that delivers lightning-fast performance and can easily scale for parallel processing. When used in conjunction with the freely available Lustre(TM) file system, customers running these applications can scale to tens of thousands of nodes, petabytes of data, and billions of files. Full Story or find out more on the Sun HPC Radio Podcast.

Podcast: HPC Radio #6: Sun Fire x2250 Xeon server

In episode #6 of the Radio HPC podcast, Tony Warner interviews David Maples from Allinea. After that, his guest Brian Huynh fills us in on our newest x86 server-- the Sun Fire x2250 based on the Intel Xeon architecture. Download MP3 or Subscribe on iTunes.

GPU Trumps CPU

As reported in Advance Imaging Magazine, the evolution of processing speed, driven by the gaming industry, has grown faster and faster, outstripping CPUs and FPGAs:

"Analyses of intense computing operations indicate a rise in performance by a factor of 2 to 10 when using a GPU in place of a CPU, freeing the CPU for other tasks simultaneously." Full Story