Throughput computing series: Defining Throughput Computing

Tuesday Jan 08, 2008

This is the first installment in a series of entries that discuss different aspects of throughput computing. This series aims to improve the understanding of how SPARC CMT servers can be utilized to increase business throughput. Let's start with a definition.

What is throughput computing?

Oxford American defines "throughput" as:

"The amount of materials or items passing through a system or process"

In computer terms, throughput computing is the amount of "work" that can be done in a given period of time. Things like "orders per second", "paychecks per hour", "queries per second", "webpages per minute",... are all metrics of throughput. These measures help define the amount of work a system can complete in a given period of time.

Misguided throughput metrics

• Latency or Response time is not a throughput metric.
• CPU % is not a throughput metric.
• IO wait% is definitely not a throughput metric... or anything other than a measure of idle time
• The "Load average" of a system is not a measure of throughput. OK... you get the idea.

Job level parallelism

Job level parallelism is about taking a single job and breaking it into multiple pieces. Say you have 10,000 letters to put stamps on. If it takes 3 seconds per letter, you would need 30,000 seconds or more than 8 hours to complete the task. Now consider you are a teacher and you bring the letters to class. There are 20 students in the class so each student will place stamps on 500 letters. With only 500 letters to complete per student, the job can be done in only 1500 seconds or 25 minutes.

In terms of throughput, one person processes one letter every 3 seconds or 60/3 = 20 letters per minute... and a class of 20 students can process 20*20 = 400 letters per minute.

Concurrency

Concurrency comes from running multiple jobs or applications together on a system. A job may be single-threaded or use multiple threads of execution as discussed above. These jobs need not be related or even from the same application. To further increase concurrency, virtualization is often used to run multiple concurrent OS images on the same machine in-order to take advantage of modern multi-threaded systems.

Putting it all together with Chip Multi-Threading

Denis Sheanan sums up Sun's throughput computing initiative in his paper on CMT as:

"Sun’s Throughput Computing initiative represents a new paradigm in system design, with a focus toward maximizing the overall throughput of key commercial workloads rather than the speed of a single thread of execution. Chip multi-threading (CMT) processor technology is key to this approach, providing a new thread-rich environment that drives application throughput and processor resource utilization while effectively masking memory access latencies."

The salient point is that you must have an application that has multiple threads of execution in-order to take advantage of CMT. Multiple threads of execution could come from a single job that has been parallelized or from multiple jobs of different types that run concurrently.

Resources

• Developing and Tuning Applications on UltraSPARC T1 Chip Multi-threading Systems
• Improving Application Efficiency Through Chip Multi-Threading
• CMT Tuning and resources.
• Chip Level Multiprocessing.
• Chip Multi-Threading CPU utilization and Calculating CMT utilization with performance counters
• Ali-G discusses Niagara2