Monday Sep 10, 2007

Perceptual blindness -  "the phenomenon of not being able to see things that are actually there. This can be a result of having no internal frame of reference to perceive the unseen objects, or it can be the result of the mental focus or attention which cause mental distractions. The phenomenon is due to how our minds see and process information"

One of the key things to QA as one of my manager used to say is "To be wide awake to see what is happening"  -  though  this appears to be the right thing to do  (and it is)  -  its interpretation varies and is the reason for many  test gaps in my opinion.  To understand more, we need to look at how QA is generally done.

Once a feature is coded and unit tests by the developer are done, they are integrated with the code.  llly, the QA writes a test plan and specifies the procedure that will be used for verifying the functionality of the feature that is being tested.  This may be done either through automated test suites or manual test cases.  Typical test plans have what is the expected behaviour and how it should be verified for the particular test case which could be a fault/load injection or something like configuration check.

Given this structured approach to testing, Engineers tend to fall into the trap of not checking how the changes in the specific module which provides the feature affects the system as a whole. Given the narrow scope of verification procedure, Engineers tend to do exactly the same and if that particular behaviour is inline with what is expected, presume the test has succeeded. This is irrespective of whether it is black box or white box testing.

In a reasonably large piece/complex  software, integrating features is a big challenge and despite the best intentions and processes, slips are inevitable.  So while a feature may work as expected, it can cause trouble in a completely unrelated module which may use a common set of code which could be broken. It is this aspect of testing which is generally ignored.  Lack of knowledge of the complete software results in some obvious bugs being missed because of the narrowly defined verification procedures.

One of the greatest benefits of working in Data Services part of Solaris cluster is it being on top of the stack and hence the engineer needs to have a minimum knowledge of everything in the entire stack.  The most crucial bugs that I have found are mostly in lower stack of the Solaris cluster which got affected due to the introduction of some other functionality. This is not restricted to cluster alone and includes Solaris/any other application that is being tested with cluster. The same applies to any other software as well.

I'd liken it to the following quote " Never send to know for whom the bell tolls; the bell tolls for thee .."

Never brush aside a bug in a different module as unrelated; Never know what caused it or what it will affect!

Tuesday Aug 14, 2007

Just now noted that I have crossed a minor milestone!  I have logged more than 200 valid bugs in just 2 years!!

What is a valid bug?  A bug that is not closed as not reproducible, duplicate or not a defect.  

Now before you suggest that it means our product is bad, let me clarify that bugs are logged across numerous categories like solaris, jes, appserver, webserver, HADB, SAM FS etc.  I feel like treating myself!

My humble contribution towards improving Sun's products! :-)

This blog copyright 2009 by maddy