Darryl Gove's blog

Second life slides and script

Just completed the Second Life presentation. It appeared well attended, and I got a bundle of great questions at the end. If you were there, thank you! I've uploaded a screen shot that I managed to get before the presentation started. Unfortunately, I didn't get a picture of the stage setup with the life-size books, a very nice touch.

I've uploaded the slides I used in the presentation, and I'm placing the transcript below.

I don't normally write my presentations, I usually use the slides to provide structure and remind me of the key points. However, this was completely different. If I'd been entirely in Second Life, I'd have been facing the audience, which meant I would not be able to see the screen with the slides on it. So I had another window with my slides so I could see them. The other concern was that I was doing a voice chat, and if for some reason there was network congestion or some other technical issue, I wanted to be able to drop back to a transcript rather than reschedule the presentation. Hence I had a third window showing the presentation notes. Therefore, it was a new experience to use notes, and it also means that I can provide both the slides and content here.

Solaris Application Programing
Second Life transcript
Darryl Gove 24th April 2008

Slide 1 – title

My name is Darryl Gove and I've got a few brief slides to share with you about my book Solaris 
Application Programming, which came out in January of this year. I'm also here to answer any 
questions that you might have, either during the presentation or afterwards. The final slide 
of the presentation contains my e-mail address for if you think of something after this 
session has ended, and it also contains the address of  my blog where I reasonably 
regularly write about developer topics.

Slide 2 – outline

First of all I'm going to cover some of the reasons why I ended up writing this book, 
once I've done that I'll give the high level outline of the main sections of the book.

Slide 3 – issues

Solaris has an impressive number of tools available, some will be tools that are used 
regularly, and some are tools that are only used rarely. There are 800+ files listed in /bin. 

One of the hardest problems to solve when developing on a platform is figuring out what 
features are available on that platform. Taking the Solaris tool prstat as an example, 
getting the mapping from the question 'how do I find out what's running on the machine' to 
the tool prstat is no easy task.

A similar issue is when a developer knows the name of a tool on one OS, but doesn't have the 
right name for Solaris. Sometimes they can continue to use their old tool – for example top 
still works on Solaris, but in the case of top the tool is much less efficient than the one 
provided by the OS.

An easier problem to solve is how to use the tools. Once you know the name of the tool it 
is trivial to look up its usage in the man pages. Unfortunately,  the man pages do not 
always provide an adequate explanation of the output of the tool.

These kinds of issues were the ones that I had in mind as I sat down to write the book.

Slide 4 – objectives

I also had a set of what I considered objectives of the book, these were broadly similar to the 
issues, but gave me more general guidelines.

The key objective was to reduce the barrier to entry. When I was writing, I modelled the reader 
as a technically savvy developer who was developing on Solaris, or perhaps porting to Solaris. 
I wanted to provide a resource that could be placed on their desk to help them over the majority 
of the questions that they might have.

When developing, not all tasks  are frequently occurring. There are some tasks  that  occur 
infrequently. An example of this might be looking at the version information for a library. A 
developer might never need to do this, but if they do need to, it is really helpful to have the 
information available. 

As I mentioned previously, there are a bundle of tools provided with the OS. Similarly there are a 
number provided as part of Sun Studio. When searching for a tool to solve a particular problem, its
 not unusual to find multiple options. A great example of this is looking for memory access errors 
– this can be achieved using special malloc libraries, dbx, mdb, or the tool discover. In situations 
where there is choice, it's all to easy to only find one of the options – and perhaps not the most 
appropriate one.

The final objective was to cover the developer angle for Solaris. There is a select group of books 
that cover Solaris, and very few of those are aimed at developing on Solaris.

This covers the motivations for the book, let's move on to cover the contents.

Slide 5 -hardware

As a developer its very easy to regard t hardware as a black box. In an ideal world, perhaps that
is all that should be necessary. But there's been much innovation at hardware level in recent 
years. The two changes I'd pick out are the introduction of 64-bit extensions into the x86 
architecture, and the shift to multicore processors. Both of these represent a huge opportunity 
for programmers.

The move to 64-bits is actually not the big news. The instruction set extensions introduce 
some significant architectural enhancements to the x86. The major change was the introduction 
of a number of additional registers. In many cases it is these registers that give the 
performance boost over a 32-bit binary. In fact the increase in memory footprint that is required
to hold 64-bit data can actual slow down the program.

The other major innovation has been Chip Multi-Threading or CMT. It is almost impossible to buy 
a single core system. The question for developers is how to best use those additional cores. 

Beyond the innovation that is occurring in the processor space, there are some traditional 
features of processors which impact the performance of applications. Consequently  the book 
starts off with an overview of the x86 and SPARC families of processors.

Slide 6 – Solaris

I've already talked about this in some detail. The chapter which covers the tools that are 
part of Solaris is actually available as a free download, the link is on the slide, and the 
pdf is also linked from my blog.

There are generally two types of tools, those that provide static information, and those 
that allow the user to observe what is happening to the system. I talked about prstat 
earlier, and that's really an observational tool. In comparison prtdiag is a tool which 
reports a summary of the system configuration.

The chapter contains information on most of the tools that I use on at least an occasional 
basis,  plus some other tools which are included more for completeness.



Slide 7 – Developer tools

There are a number of chapters in the tools section of the book which focus specifically on 
developer tools. The compilers are the obvious starting point for this. 

Compilers are often treated as another black box where they are all assumed equal. Unfortunately, 
that's an over simplification. To take an example, gcc has three optimisation levels O1 to O3. Sun 
Studio has five – O1 to O5. The net result is that a developer who uses GCC may assume that O3 is 
the most aggressive level of optimisation, and miss out on what the Sun Studio compiler can 
achieve.

Another possibility is that the developer uses no optimisation flags. One of the rules is that 
no optimisation flags mean no optimisation. 

Of course there's plenty more opportunities for the developer to use compiler flags to improve 
the performance of their application. There's a  link in the presentation to the developer portal 
on Sun.com where there are a large number of articles on topics like getting the best out of 
the compiler.

One of the other tools that is of great importance is the Performance Analyzer. This provides 
profiles of applications, indicating where the time is spent, or even where processor events 
(like cache misses) occur. This information can be presented at the level of the functions, 
the lines of source code, or the lines of disassembly. All this makes it a very powerful 
tool when doing performance tuning of applications. One of the messages I hope that comes out 
of the book is that it is imperative to profile an application regularly during development. 
It can be quite enlightening to discover where the time is being spent, and if this fact is 
discovered early enough, it's usually possible to fix the problem. In contrast,  a performance 
issue discovered when the code is more or less finished is often much harder to correct.

Slide 8 – Multi-core/multi-process/multi-thread.

As I mentioned earlier, developers are now expected to be developing applications that utilise 
more processor cores. But there are a variety of ways that an application can be spread over 
multiple cores.  

If there is no need for communication between the different copies, or if the time it takes to 
complete the task is well within the acceptable criteria then most obvious way to achieve 
parallelisation is to run multiple copies of the application.

On the other hand, some applications need to be sped up by using multiple threads that co-operate 
on a single task. In this instance technologies like PThreads or OpenMP are really useful. OpenMP 
is a very helpful specification that allows the developer to parallelise an application by adding 
directives into the source code. This allows the developer to focus on parallelising a single 
part of the code where there is some known benefit. For example making a single loop run in 
parallel. The source code modification needed to do this would be a single line, and an 
additional compiler flag.

In contrast, using PThreads may lead to significant changes to the source code.

The advantage of using PThreads is that this is probably the most flexible way of using threads 
in an application. The cost of the flexibility is the increased complexity of the source, and 
the possibility for introducing bugs.

Sun Studio 12 introduced the Thread Analyzer for detecting parallelisation bugs such as data 
races. A data race is a hard to detect problem that leads to incorrect answers. The situation 
is where multiple threads are accessing a single variable. If one of the threads updates the 
variable, then the value seen by the other threads is unpredictable unless there is some 
synchronisation mechanism that ensures that the appropriate value is seen by all the threads. 
The problem with data races, in common with other forms of memory corruption, is that the 
results of the bug appear arbitrarily far after the problem has occurred – so having a tool 
to detect the problem available is critical.

Slide 9 – Other books.

There are a number of other books available, and in closing I would like to place my book in 
context with these others. The four shown on this slide are very complementary. The two 
Solaris Internals books are largely focussed on the functioning of the Solaris operating 
system. Together with the Solaris source code available from opensolaris.org these enable 
the developer to see what the OS is doing in response to their code.

The next book is Solaris System Programming which covers the APIs. This is more about how 
to use Solaris, or how to call Solaris routines to achieve certain results.

My book is meant to be a developer down view, much more focused on using the tools that are 
provided by the compilers and operating system to develop applications.

Slide 10 – close

I hope that I've given you some flavour of what is covered by Solaris Application Programming. 
My e-mail address and a link to my blog are on the final slide.

I'm going to switch to chat, and I'm looking forward to answering your questions.

Thank you for listening.

Posted at 10:59AM Apr 24, 2008 by Darryl Gove in Sun  |