Paul Hinker's Weblog

On the benefits of a custom library

For the past five or six years, I've been advocating the repackaging of Perflib into what I call 'Custom Libraries' (pause for musical fanfare here). The idea first surfaced when I received a request from someone in the benchmarking group for a PerflibLite. It started me thinking about the benefits of such a thing with respect to customers and ISVs.

A custom library would contain only those routines necessary for the particular user or user application. If the application in question is expected to run on multiple chips, the library would contain the objects for each of the routines and each of the chips for optimal performance.

A custom library would greatly reduce the mass storage footprint for the Performance Library. I know that disk space is cheap but it's no reason to keep around useless bits. There would also be a (perhaps negligible) benefit during the dynamic linking phase.

We already know that most applications use only a tiny fraction of any given scientific library during computation. That goes not only for Perflib but ATLAS, GOTO, ACML, MKL, NAG, and any number of other offerings. These libraries contain routines for a wide range of computation but any given application accesses only a handful. The libraries also contain routines for various data types (single precision, double precision, complex, and double complex). It's a rare application which makes use of single precision floating point and double precision complex routines.

One of the major benefits from a library provider's view is information about which routines are being used. This allows the developer/tuner some idea as to where to spend scarce tuning resources. Another benefit is the information concerning who is using a particular routine or class of routines. Given this information, users could be sent an e-mail informing them when a routine has had bugs fixed or performance enhanced.

If you think this idea has merit, drop me an e-mail or comment here.

Posted at 07:13AM Sep 19, 2005 by hinkthink in General  |