How do we know we have done enough testing ? I remember "when i'm finished" was a common answer !!
Any test effort will consist of test items (components or systems to be tested), test cases and reports along with a GO/NOGO decision. A significant constraint on any test effort is time, so if a tight deadline means that all testcases cannot be run, then what do we do ? Bring 'risk' in to the decision making process.
Look at the features in the software product/component and rank the importance of each to the customer (in Six-Sigma-speak identify the 'CTQ',, or 'Critical-to-Quality' features). Ask what is the likelihood of each feature failing and what is the impact in terms of severity of the failure from the customers perspective.
How do you know if something is likely to fail ? Thanks for asking.
One way is to ask how stable is the feature - does it have a high fail rate already ? A stable feature, with few code changes would be less likely to fail than a new or heavily modified feature.
How complex is the feature - does it have many interfaces with other parts of the system that could fail ?
Then test the features in order of priority, with most important, riskiest features highest.
There is more to this process but when time and/or resources are tight the message is to test smarter by targeting the riskiest features which matter most to the customer. If testing has to be cut short, at least there will be some level of confidence that important features have been tested already.
