Darryl Gove's blog
Other Sun Studio videos
There were two other videos posted to the hpc portal. Marty Itzkowitz talking about the Performance Analyzer and Vijay Tatkar talking about portable applications.
Posted at 03:15PM Nov 26, 2007 by Darryl Gove in Sun |
Multi-threading webcast
A long while back I was asked to contribute a video that talked about parallelising applications. The final format is a webcast (audio and slides) rather than the expected video. This choice ended up being made to provide the clearest visuals of the slides, plus the smallest download.
I did get the opportunity to do the entire presentation on video - which was an interesting experience. I found it surprisingly hard to present to just a camera - I think the contrast with presenting to an audience is that you can look around the room and get feedback as to the appropriate level of energy to project. A video camera gives you no such feedback, and worse, there's no other place to look. Still I was quite pleased with the final video. The change to a webcast was made after this, so the audio from the video was carried over, and you still get to see about 3 seconds of the original film, but the rest has gone. I also ended up reworking quite a few of the slides - adding animation to clarify some of the topics.
The topics covered at a break-neck pace are, parallelising using Pthreads and OpenMP. Autoparallelisation by the compiler. Profiling parallel applications. Finally, detecting data races using the thread analyzer.
Posted at 02:25PM Nov 26, 2007 by Darryl Gove in Sun | Comments[1]
Sun Studio 12 Performance Analyzer docs available
The documentation for the Sun Studio 12 version of the Performance Analyzer has gone live. The Sun Studio 12 docs collection is also available.
Posted at 02:06PM Sep 12, 2007 by Darryl Gove in Sun |
Recording Analyzer experiments over long latency links (-S off)
I was recently collecting a profile from an app running on a machine in Europe, but writing the data back to a machine here in CA. The application normally ran in 5 minutes, so I was surprised that it had made no progress after 3 hours when run under collect.
The Analyzer experiment looked like:
Dir : archives 08/28/07 23:47:10 File: dyntext 08/28/07 23:47:12 File: log.xml 598 KB 08/29/07 03:02:52 File: map.xml 3 KB 08/28/07 23:47:22 File: overview 4060 KB 08/29/07 03:02:51 File: profile 256 KB 08/28/07 23:47:56
Two of the files (log.xml and overview) had accumulated data since the start of the application, the other files had not. truss output showed plenty of writes to these files:
0.0001 open("/net/remotefs/.../test.1.er/log.xml", O_WRONLY|O_APPEND) = 4
0.0000 write(4, " < e v e n t k i n d =".., 74) = 74
0.0004 close(4) = 0
In fact it looked rather like opening and closing these remote files were taking all the time away from running the application. One of the Analyzer team suggested passing -S off to collect to switch off periodic sampling. Periodic sampling is collecting application state at one second intervals. Using this flag, the application terminated in the usual 5 minutes and produced a valid profile.
Posted at 02:02PM Sep 04, 2007 by Darryl Gove in Sun |
Presenting at Stanford HPC conference
I'll be presenting at Stanford next week as part of their HPC conference (Register here). I plan to cover:
- The Sun Studio compilers and tools
- The CMT Developer Tools
- Parallelisation
- Performance tuning
Posted at 11:17AM Aug 16, 2007 by Darryl Gove in Sun |
Comparing analyzer experiments
When performance tuning an application it is really helpful to be able to compare the performance of the current version of the code with an older version. This was one of the motivators for the performance reporting tool spot. spot takes great care to capture information about the system that the code was run on, the flags used to build the application, together with the obvious things like the profile of the application. The tool spot_diff, which is included in the latest release of spot, pulls out the data from multiple experiments and produces a very detailed comparison between them - indicating if, for example, one version had more TLB misses than another version.
However, there are situations where it's necessary to compare two analyzer experiments, and er_xcmp is a tool which does just that.
er_xcmp extracts the time spent in each routine for the input data that is passed to it, and presents this as a list of functions together with the time spent in each function from each data set. er_xcmp handles an arbitrary number of input files, so it's just as happy comparing three profiles as it is two. It's also able to handle data from bit, so comparisons of instruction count as well as user time are possible.
The input formats can be Analyzer experiments, fsummary output from er_print, or output directories from er_html - all three formats get boiled down to the same thing and handled in the same way by the script.
Here's some example output:
% er_xcmp test.1.er test.2.er
2.8 8.8 <Total>
1.8 2.6 foo
1.0 6.2 main
N/A N/A _start
Posted at 03:36PM Aug 15, 2007 by Darryl Gove in Sun |
Locating DTLB misses using the Performance Analyzer
DTLB misses typically appear in the Performance Analyzer as loads with significant user time. The following code strides through memory in blocks of 8192 bytes, and so encounters many DTLB misses
#include<stdlib.h>
void main()
{
double *a;
double total=0;
int i;
int j;
a=(double*)calloc(sizeof(double),10*1024*1024+10001);
for (i=0;i<10000;i++)
for(j=0;j<10*1024*1024;j+=1024)
total+=a[j+i];
}
A profile can be gathered as follows:
cc -g -O -xbinopt=prepare -o tlb tlb.c collect tlb
Viewing the profile for the main loop using er_print produces the following snippet:
Excl.
User CPU
sec.
...
0.230 [11] 10c70: prefetch [%i5 + %i1], #n_reads
0. [11] 10c74: faddd %f4, %f2, %f8
3.693 [11] 10c78: ldd [%i5], %f12
## 7.685 [10] 10c7c: add %i5, %i3, %o3
0. [11] 10c80: faddd %f8, %f0, %f10
4.183 [11] 10c84: ldd [%o3], %f2
## 6.935 [10] 10c88: add %o3, %i3, %o1
0. [11] 10c8c: faddd %f10, %f12, %f4
0. [10] 10c90: inc 3072, %i4
4.123 [11] 10c94: ldd [%o1], %f0
## 7.065 [10] 10c98: cmp %i4, %i0
0. [10] 10c9c: ble,pt %icc,0x10c70
0. [10] 10ca0: add %o1, %i3, %i5
This can be compared with the situation where mpss.so.1 has been preloaded to enable the application to get large pages:
0. [11] 10c70: prefetch [%i5 + %i1], #n_reads 0. [11] 10c74: faddd %f4, %f2, %f8 0. [11] 10c78: ldd [%i5], %f12 ## 7.445 [10] 10c7c: add %i5, %i3, %o3 0. [11] 10c80: faddd %f8, %f0, %f10 0.220 [11] 10c84: ldd [%o3], %f2 ## 6.955 [10] 10c88: add %o3, %i3, %o1 0. [11] 10c8c: faddd %f10, %f12, %f4 0. [10] 10c90: inc 3072, %i4 0.340 [11] 10c94: ldd [%o1], %f0 0. [10] 10c98: cmp %i4, %i0 0. [10] 10c9c: ble,pt %icc,0x10c70 0. [10] 10ca0: add %o1, %i3, %i5
The difference between the two profiles is the appearance of user time attributed directly to the load instruction (and not the normal instruction after the load).
It is possible to confirm that these are DTLB misses using the Performance Analyzer's ability to profile an application using the hardware performance counters:
collect -h DTLB_miss tlb
...
Excl.
DTLB_miss
Events
...
0 [11] 10c70: prefetch [%i5 + %i1], #n_reads
0 [11] 10c74: faddd %f4, %f2, %f8
30000090 [11] 10c78: ldd [%i5], %f12
0 [10] 10c7c: add %i5, %i3, %o3
0 [11] 10c80: faddd %f8, %f0, %f10
## 42000126 [11] 10c84: ldd [%o3], %f2
0 [10] 10c88: add %o3, %i3, %o1
0 [11] 10c8c: faddd %f10, %f12, %f4
0 [10] 10c90: inc 3072, %i4
30000090 [11] 10c94: ldd [%o1], %f0
0 [10] 10c98: cmp %i4, %i0
0 [10] 10c9c: ble,pt %icc,0x10c70
0 [10] 10ca0: add %o1, %i3, %i5
The events are reported on the load instructions that are causing the DTLB misses.
Posted at 09:00AM Apr 19, 2007 by Darryl Gove in Sun |
