Darryl Gove's blog

Analyzer probe effects (part 2)

A while back I looked at the probe effect when running the Performance Analyzer on a multi-threaded code. The results showed that the probe effect, in terms of difference in runtime, was most pronounced when using multiple threads and recording the experiment to remote disk.

An alternative metric for probe effect was the amount of distortion in the results. To measure this I took the same code and observed the user time recorded by Analyzer when the experiments were recorded to remote disk or to /tmp. I also compared this to the user time reported by the command time. The results of this are shown in the following figure.

The results show that the runtime when recording the experiment to remote disk was more significantly increased than in my previous experiment. There is also one data point where the runtime was substantially perturbed.

The interesting part of the graph is the recorded user time. For both situations, the recorded user time is pretty much identical to that recorded by the time command. Possibly most critically, this is true even when the runtime is hugely disturbed. Anecdotally, this is what we've been assuming for a while, that the Analyzer data is largely correct even in the situation where there is some activity on the machine. (I'm sure that this would not be the case if the machine were placed under an extreme load, for example running two MT benchmarks.)

There is one rather odd observation, that when recording to the remote file system, the user time is slightly under estimated. But it is apparent from the graph that this is not a large effect.

The conclusion is that recording to a remote file system does not cause large distortion to the collected data, although it does cause the application to run more slowly.

Posted at 10:59AM Dec 15, 2006 by Darryl Gove in Sun  |