Storage Networking: The Basics

Our favorite storage pundit Henry Newman has written a nice primer on Storage Networking:

"The storage stack is complex because it really has not changed much over the last 20 years and yet there have been huge changes in our industry. The move from Fibre Channel and other connectivity options to 10GbE is designed to reduce complexity and break the model of having networking administration separated from storage administration both from a hardware perspective and from a personnel perspective. If you are a network administrator, I would strongly recommend learning something about storage, and if you are a SAN administrator, I would strongly recommend learning something about networking." Full Story

Posted by Rich Brueckner [HPC Storage] ( November 04, 2009 05:00 AM ) Permalink | Comments [0]

Video: Active Archive Solution for Life Sciences Research

Mark Leggott, University Librarian for the University of PEI, presents on the System Architecture for Sun's Active Archive Solution for Life Sciences Research.

RAID's Days May Be Numbered

Storage pundit Henry Newman writes that RAID's days may be numbered:

"The bottom line is this: Disk density has increased far more than performance and hard error rates haven't changed much, creating much greater RAID rebuild times and a much higher risk of data loss. In short, it's a scenario that will eventually require a solution, if not a whole new way of storing and protecting data." Full Story

Screencast: HPC Storage and the Lustre File System

In this screencast you will learn about an implementation of the Sun Lustre file system as a scalable storage cluster using Sun Fire servers, high speed/low latency InfiniBand interconnects, and additional networking and storage devices. In addition, we will explore the use of the Sun Lustre file system at a shared government/education research site, including configuration information and details on testing that was performed on-site to evaluate the performance of Sun’s Lustre Storage System.

To learn more, check out the new whitepaper entitled: "Lustre File System: High-Performance Storage Architecture and Scalable Cluster File System."

Sun Technologist Urges Flash Industry to Adopt NAND SSDs

At the recent Flash Memory Summit, Michael Cornwell, lead technologist for flash memory at Sun, urged the industry to adopt or emulate Sun's NAND flash module strategy for high-end servers:

"During a frank and sobering keynote address at the Flash Memory Summit here, he said NAND vendors are going down the wrong path by racing each other in process technology--at the expense of customer needs. Few sub-50-nm parts are suitable for the high-end computing or enterprise market, due to reliability and endurance issues, he said. So, OEMs like Sun are looking to devise flash-based systems using trailing-edge but more reliable NAND, but the trouble is that those type of parts are in short supply, he said." Full Story

Whitepaper: Accelerating Data-Intensive HPC Workflows

Sun provides a wide range of cost-effective options for HPC storage performance, helping customers improve throughput for HPC applications while reducing costs. This paper provides a guide to open storage solutions for data-intensive HPC environments and describes the trends and challenges in delivering the necessary I/O throughput for high-performance computing applications. Full Story

SSDs Test Limits of RAID Controllers

Henry Newman writes that the potent combination of SSDs and pNFS will test RAID controller design:

"RAID controllers designed for hard disks and IOPS aren't going to cut it in the new world of solid state drives (SSDs) and pNFS, which are capable of feeding so much more data into the channel that streaming bandwidth is going to become the problem for RAID vendors." Full Story

Whitepapers: Lustre Storage Architecture and Networking

Check out these white papers for information on Lustre architecture, networking, high availability, and comparison with other shared/cluster file systems:

Lustre File System: High-Performance Storage Architecture and Scalable Cluster File System
Scalability is one of Lustre's most attractive attributes, offering as it does, the ability for users to deploy a Lustre file system as a general-purpose, datacenter back-end file system at a variety of sites, from Internet service providers (ISPs) to large financial institutions. Enhancements to wide area support scheduled for imminent release in Lustre networking (LNET) and storage software, should make these market segments even more important.

Lustre Networking White Paper
This paper provides information about Lustre networking that can be used to plan cluster file system deployments for optimal performance and scalability. The paper includes information on Lustre message passing, Lustre Network Drivers,and routing in Lustre networks, and describes how these features can be used to improve cluster storage management. The final section of this paper describes new Lustre networking features that are currently under consideration or planned for future release.

Video: Andy B. on the SSD Revolution

The new Sun Video Blog brings us this keynote presentation by Andy Bechtolsheim on the Solid State Storage Revolution. Recorded at the Storage Developer Conference, September, 2008. Download mp4 video for iPod or PDF slides

Sun unveils entry-level SSD-hybrid storage array

Sun Microsystems recently unveiled a low-end addition to its lineup of Sun Storage 7000 Unified Storage arrays. The new Sun Storage 7310 array allows solid-state disk (SSD) drives and hard disk drives to be used in the same chassis and under the same management interface.

The Sun Storage 7310 array is also capable of clustering high-availability applications. And it uses Sun's Hybrid Storage Pool technology, which allows users to add up to 600GB of SSD storage to the existing 64GB of dynamic RAM cache. In a two-node clustered configuration, the storage system offers up to 124GB of DRAM cache. Full Story

Storage Tradeoffs

Our favorite storage pundit Henry Newman writes that storage costs aren't just about the price of the hardware or software; they're about operating and maintenance costs — and the cost of lost or corrupt data:

"When I am trying to help customers understand the technology tradeoffs, the first thing I do is to try to understand what their requirements are. Usually I get a glazed look or get told to just solve the problem, and sometimes I'm told that the requirement is for storage that's as cheap as possible. Very few people actually understand their requirements, and even fewer know how to apply them." Full Story

Sun Open Storage combines open-source software with industry-standard hardware, enabling you to reduce your reliance on high-priced systems and save up to 90% on storage costs. To learn more, check out this whitepaper: "Guide to Accelerating Performance and Reducing Cost in HPC with Open Storage."

The State of Open Source Storage

Drew Robb looks at the state of Open Source Storage, including recent inroads in HPC:

"Greg Schulz, senior analyst and founder of StorageIO Group, is more upbeat about the state of open source storage offerings.

"There is a wide variety of open source storage solutions and applications from different sources, ranging from volume managers, iSCSI and NAS stacks, file systems, clustered file systems, object-based storage solutions, dedupe and compression, among others, not to mention all of the propriety or commercial solutions that may leverage open source technology embedded into turnkey solutions and products," said Schulz. "Of traditional server and storage vendors, Sun is probably the most notable and vocal around open source storage, along with many smaller startup vendors." Full Story

For more information on Sun's Open Storage offerings, you can download this whitepaper on Revolutionizing the Storage Status Quo.

Video: What Open Storage can do for your HPC environment

In this video, Sun's Chris Martin shows how Amber Road OpenStorage devices can boost your HPC environment. Recorded at the SC08 conference, November, 2008. Download the Open Storage Adoption white paper.

Video: Understanding ZFS Technology

In this video, Tim Thomas presents on ZFS as part of Sun UK's ZFS Discovery Day. Download the MP4 for iPod.

Video: Victor Walker on Open Storage Solutions

In this video, Victor Walker – VP Storage Systems Group, Sun Microsystems, presents on the company's OpenStorage hardware and solutions. Recorded November 15, 2008 at the Sun HPC Consortium in Austin, Texas.

Learn more--Download the Open Storage Adoption white paper. Open storage can be viewed as an architecture built out of general purpose, industry standard servers and open source operating systems and storage management software and commodity disk drives. The white paper discusses the various components and the integration of those components into a system.

Storage Security Performance Challenges

Our favorite storage pundit Henry Newman writes on the performance challenges of Storage Security:

"We all know that the complexity of the calculation of a checksum is far less than the complexity of an encryption algorithm. Complex encryption algorithms are computationally intensive and commodity CPUs are not likely the best place to do this type of algorithm. These types of algorithms are best suited to specialized hardware such as an ASIC or some of the faster FPGAs. CPUs are getting faster and faster but memory bandwidth is not growing at the same rate as CPU performance, and moving data in and out of memory to encrypt it and decrypt it in systems that do a lot of I/O (databases, video and audio applications and backup systems) will not likely be able to keep up with the decryption requirements to ensure that the machine can meet I/O requirements. Full Story

Webinar Replay: SAM-QFS

In this video, Sun's Harriet Coverston provides a technical overview of the Sun StorageTek Storage Archive Manager (SAM) and Sun StorageTek QFS software. The QFS Shared File System provides high-performance, shared access to data over a storage area network (SAN). To further protect and manage data created in a shared access environment, the Sun StorageTek SAM software reduces storage costs and complexity by automating data management and movement across tiered storage based on its business value over time. Download PDF slides or MP4 format for iPod

A Parallel Future For File Systems

Drew Robb continues his look at high performance file systems:

"Sun Microsystems (NASDAQ: JAVA) characterizes Lustre as "the most scalable parallel file system in the world." In evidence of this, it serves six of the top 10 supercomputers and 40 percent of the top 100.

"We have Lustre file systems that scale to petabytes of data in one cohesive name space and deliver in excess of 100 GB/s aggregate performance to 25,000 clients or more," said Peter Bojanic, director of Sun's Lustre Group. "This includes HPC applications at Livermore, Oak Ridge and Sandia National Laboratories, where large-file I/O and sustained high bandwidth are essential." Full Story

Sun Open Storage: Backbone for new data-transfer world record

The California Institute of Technology (Caltech) just announced they have set new records for sustained data transfer among storage systems, using the Sun Fire X4500 and X4540 storage servers. Full Story

The Economics of Data Preservation

A blue ribbon task force, commissioned late last year to identify sustainable economic models to provide access to the ever-growing amount of digital information in the public interest, has issued its interim report. The report calls the current situation urgent, and details systemic pitfalls in developing economic models for sustainable access to digital data. Full Story

Storage Virtualization: Good News for Consultants

Our favorite storage pundit Henry Newman opines that storage virtualization continues to be a bad idea:

"With all the concerns I have about virtualization, I still think it has a place in some environments as long as everyone understand the impact short term (monitoring performance and performance analysis is going to be difficult) and long term (migration to new hardware is going to require careful planning and might not work seamlessly). Looking ahead, I wonder if the whole idea of vitalization of blocks might be overcome by new technologies such as ANSI T10 Object Storage Device (OSD) or NFSv4.1 (pNFS) to allow file systems to manage the virtualization of the storage. What will happen if block-based virtualization systems meet T10 object storage? I suspect virtualization systems won't work because the SCSI commands for object storage are a superset of the standard SCSI commands used today." Full Story

HPC Storage at Sun

In case you missed it, our own Marc Hamilton blogged all about on HPC storage at Sun recently:

"For HPC users, the most widely used open storage software is Sun's Lustre file system. Lustre is used in over half of the world's fastest supercomputers on the Top 500 list. And talk about open, Lustre helped define the world of open storage for HPC as evidenced by its use on four of the world's fastest supercomputers, each from different vendors.

You can't get much more open or higher performance than that. It is a core concept to our open storage strategy. We will offer unified storage systems like the Sun Storage 7000 line that you just plug in and turn on. You can buy our servers and storage and run our open storage software. You can participate in the open storage community and add your own software. And since not everyone will want to buy Sun hardware, you can run our open storage software with other vendors servers and storage as witnessed by the above examples.

If you don't have five PhD students on your IT staff and still want to run Lustre, Sun makes it easy with pre-packaged scaleable storage clusters using Lustre. Expected to see these upgraded next week with even faster performing systems and more ease of use, especially valuable for smaller configurations. Full Story

Three Acronyms That Could Change The Storage World

Our favorite storage pundit Henry Newman explains how the storage networking world could be profoundly changed by three emerging technologies:

"Clearly, disruptive technologies are not an everyday event, nor are they easy to predict. Let's examine some technologies that might significantly change enterprise storage and disrupt the market. I won't adhere to the strict definition, but I am going to suggest some technologies that if adopted could change the enterprise storage market. As I said, I think very few companies are going to be able to create a new technology market from a technology without a standard that others can use." Full Story

Sun Unveils First Storage Blade

Somehow I have scooped the other HPC blogs on this. Go figure ;-)

With today's announcement, Sun is expanding its blades portfolio to provide customers with new options. Sun's blades portfolio has delivered four consecutive quarters of triple digit or better revenue growth year-over-year and seven consecutive quarters of quarter-over-quarter unit shipment growth, according to IDC's Worldwide Quarterly Server Tracker, August 2008(1). The outstanding value and performance of Sun's blade portfolio is recognized by a breadth of enterprise customers who have integrated the Sun Blade products into their infrastructures, across industries including government, energy and utilities, education, media, telecommunications, and Web 2.0. New Sun Blade customers include Citco Technology Management, DLR Case, Western Governors University, Last.fm, and SAP University Competence Center.

“Customers today want performance, efficiency and choice and clearly Sun is delivering. We've nearly tripled our blade server marketshare year-over-year(1),” said Lisa Sieker, vice president, Systems Marketing, Sun Microsystems. “Sun's blades portfolio offers datacenter efficiency at every level and provides the building blocks, x64 and SPARC(R), for infrastructure that costs less from the start and is easy to grow over the years. And now, with the addition of our first storage blade, we're bringing the Open Storage revolution to the blades market.” Full Story

Sun to Shake Up Storage Industry

The New York Times reports on Sun's plans to release new storage hardware and software that takes aim at a part of the storage market dominated by NetApp and EMC. Playing off the same trends that shook its server business — less expensive hardware and software — Sun is looking to undercut the competition and sink their margins. Full Story

Gosh, we can just compete instead of suing each other? Where's the Ferengi profit in that?