John Brady's Weblog

Tuesday August 02, 2005

System Activity Snapshot or Performance Profile Baseline

Taking a snapshot of the activity on a system, or a baseline profile of its performance, is a useful tool for dealing with future performance problems. By establishing a baseline, and recording all of its associated details, we have a reference point for comparison at any point in the future. Should any performance problems be reported about the system in the future, then we can compare the current profile of the system to its previous baseline. Any differences will help us identify what has changed and from that the cause of the change in behaviour on the system. How else are we find out what has changed, and is causing the change in performance behaviour of the system? Without a baseline for reference, you literally do not know what has changed since the system was last working properly.

A performance profile consists of saving information about the utilisation of all of the resources on the system (CPU, Memory, Disk, Network, etc.), and all of the processes on the system (the consumers of the resources). This information is normally taken as a series of snapshots over a period of time. Examples range from every hour throughout a 24 hour day, to every minute during the peak hour of the day.

If you have some kind of performance management or monitoring tool, then it should be capable of capturing this data for you over your representative period, and then saving it away somewhere permanently for later use. If you don't have such a tool, then you can achieve something similar using standard UNIX tools like sar, ps, System Accounting and maybe even top (or prstat on Solaris), saving the outputs to a set of files. Of course it will require some manipulation to turn this raw data into a profile of the system and the applications running on it. But it must be better to have some data, no matter how raw it is, than no data at all.

Once this snapshot of the system behaviour has been established, we now have a point of reference for what we consider to be normal activity on the system. Should anything appear to be wrong at any point in the future, then we can compare it to this baseline snapshot, and find out what is different.

Performing this baseline does not require a lot of effort, yet has enormous potential benefits:

• We have a direct comparison between the actual behaviour of the system then and now.
• We can identify the actual changes on the system immediately, and so identify the cause of the effects we are observing.
• We can quickly drill down into a single area that needs further investigation, thereby eliminating the rest of the system.
• Our efforts can be focussed in on one target area only, increasing productivity and improving the time to a resolution.
• Anytime we want to, we can compare the current behaviour of the system to what it was.
• Regular captures of performance profiles can be used for trend analysis, establishing any regular change over time.
• Regular comparison can identify small changes before they become big enough to impact the overall performance of the system.

The cost of this is very little in real terms – some disk space to store the performance profile data, and some software tools to capture that data. Note that these tools would be needed to undertake any performance problem analysis anyway. So if you are serious about performance management on your systems, and have such tools, then there is no real extra expense to using them to baseline your systems.

( Aug 02 2005, 02:03:08 PM BST ) Permalink Comments [0]