John M's Blog

• Acceptance testing
  

also known as User Acceptance Testing (UAT) is generally performed by customers or user representatives.Sometimes, the product team may perform these tests too. UAT tests can be both functional as well as non-functional.  These tests are performed after Integration & System testing.

UAT is supposed to be the final level of testing and verifies whether the product meets the agreed upon product acceptance criteria. A relatively small set of test cases are selected to be executed as part of the Acceptance tests. These are not meant to unearth new defects, instead they verify meeting of the product acceptance criteria. The tests that are part of UAT should have already been covered as part of the product test team's testing so that no late surprises emerge when the UAT is run by customers. Failure of acceptance tests at the hands of the customer can have serious implications including - the product being rejected by the customers, re-work & delays, penalties and so on.

• Ad hoc testing
  

as the name suggests, this type of testing is generally termed as un-planned testing and does not need to use any formal testing techniques. It may also be termed as monkey testing or random testing in view of its nature and approach. An important attribute in this type of testing is the precedence of test execution over test case development. The tester's gut-feel, intuition, any past experience with similar products, any related domain or subject matter expertise are very much valued and leveraged during this type of test activity.

Ad hoc testing should be a part of a sound overall Test strategy to complement other types of planned test activities. It is not meant to be used in isolation and as a sole determinant of a product's readiness to ship. Some of the attributes of ad hoc testing that tend to endear it to testers include, speed and pace of execution, the range that can be covered and the sheer flexibility inherent in this approach. While all of these may make this seem like a totally unconstrained random activity, it need not necessarily be so. We can introduce some elements of planning into this seemingly unplanned test approach. One of the factors that may be constrained or limited is time. The start and end times for this test activity may be defined. In addition, the general direction of testing, the broad area(s) to be focussed upon may also be specified. The exact actions, the screens and features to be exercised, the sequence of steps to follow are left to the individual tester's judgement.

While an earlier statement indicates that test execution precedes test development, it does not absolve the tester from developing test cases. It is important to translate tests performed into test cases that will then become part of a formal test suite to be run as part of planned testing. The idea is not to end up repeating the same set or type of tests every time ad hoc testing is performed. Once a set of new tests are identified they need to be part of your test suite to be executed as part of your regular planned test cycle. It is recommended that during ad hoc testing, tester's make notes of actions performed, observations and any relevant data that may be used to later develop test cases and help with analyzing any issues that may crop up.

• Aesthetics testing

it is said that, Beauty is in the eye of the beholder. In the case of our products, it is in the eyes of the users of our product. Aesthetics testing involves testing of the User Interface and focussing on the beauty, the product's looks rather than the products functionality alone. It is a part of Usability testing but often gets relegated to the back burner due to focus on attributes such as the ease of use and the user's ability to quickly accomplish tasks. One of the factors that go against any serious focus on product aesthetics is the notion of what engineers think is "good enough" for customers. Added to this is the primary focus on feature functionality and technical wizardry, which is good and essential. However, presentation is definitely important and should not be ignored. Bugs in this type of testing tend to be termed as cosmetic issues or even at times trivial issues which in the heat of approaching deadlines and schedule pressures invariably tend to be deferred or lowered in priority. 

Aesthetics testing covers the various elements of the UI including display styles, fonts, colors, messages, windows, icons, menus, pointers, etc. An application that follows good aesthetic principles helps create a good impression on the users as well as positively influences product acceptance. The product also tends to look and feel like an output of a well designed and professional endeavor. Frankly, how many of us would be inspired to use applications that have odd color schemes and display styles, unsuited fonts and such other cosmetic glitches which may not affect functionality but make the product not pleasing to view.

Considering the subjective nature of beauty and aesthetics, it helps to consult with a representative sample of potential users and folks such as professional UI designers, human-computer interaction experts, graphic artists, and also try to conform to commonly accepted & user expected interface design guidelines.

• Buddy testing

this refers to the practice of pairing up of a developer and tester as buddies to test a piece of code or functionality. This type of testing is best applied in the early development stages after unit testing is complete and prior to checking in of the code. At this stage of product development, buddy testing may employ a variety of white and black box test techniques as appropriate to test the artifact that is available.

Buddy tests aim to avoid duplication of unit test coverage and endeavors to cover areas that may not have been addressed in unit tests, such as the user interfaces. Other important areas that buddy testing is expected to cover include, reviews of coding standards & practices, code walkthroughs, inspection and development of test scenarios to maximize coverage around statements and paths in the code. A beneficial side effect of buddy testing is greater clarity gained around product specifications which help the testers develop better tests and development to gain answers to make any needed design changes early.

Due to the early detection of issues that is possible with this type of testing, Buddy testing is often termed as "preventative" type testing where the aim is to unearth defects early in the development cycle and have them addressed when its generally cheaper to do so than in later phases. Any issues that are raised are discussed and after agreed upon changes are made, the code is checked-in. This buddy tested code is integrated and later delivered to the functional testing team by which time the specifications are clearer and there is an assurance of greater quality in the product.

• Pair testing

involves a pair of testers testing a feature or module together on the same machine. During the actual testing process, while one person tests the feature, the other one makes note of the observations as well as provides additional perspectives. The expectation is that having two people test the same feature together can help leverage their different sets of views, ideas and unique perspectives to have the feature tested better than it would have been if only one tester had tested it. Pair testing may also be used to help quickly ramp-up a new comer by pairing him with a more experienced tester.

While all this sounds pretty good, a point to be remembered is that in some circumstances it may not be possible to derive synergies from this process. Examples include, pairing testers who do not get along well, pairing a junior and senior tester when there's a fast approaching deadline to be met, etc.

Oh ! just one other thing that i thought was obvious but should mention now ... hey, I too am a proud member of this turbo-charged forum ! and am having a fantabulous time being part of this amazing community.

• provides Technical Leadership and Strategic Direction to the Testing Organization (TO)
• is responsible for Test Strategy formulation
• helps Formulate & Develop effective Test Architecture per organizational needs
• is Technically responsible for all the Testing performed by the TO
• is the foremost Technical Authority and is responsible for the overall Quality of deliverables across all parameters, both functional and non-functional including performance, security, usability, etc.
• is expected to pro-actively analyze current processes and practices and suggest/ drive improvements. Also, defines processes as needed
• has wide-reaching scope, impact and influence extending beyond the confines of the TO and spans across the entire product organization
• is the counter part to the development architect
• is involved in driving organization-wide Quality Process initiatives and their implementation to ensure Quality of deliverables
• maintains a “big and complete” picture view of the product, its dependencies, organizational goals, technology arena, etc. and helps guide & direct the functioning of the TO appropriately
• influences the product organization's future direction, strategy and planning
• collaborates effectively & on an on-going basis with all constituents involved in product development & release activity including development, testing, technical publications, marketing, program management and other entities to ensure execution & deliverables per plan
• is involved in customer engagements and provides customer facing organizations with necessary technical product support in making presentations, demos, pocs, etc. Also, receives and analyzes existing customer feedback to identify gaps as well as work with deployment / sustaining organizations as needed. Customer engagement activity also spans alpha / beta trial opportunities and acts as a liaison with customers and partners while ensuring Test strategy is aligned appropriately
• helps with Test plan development
• is responsible for design & development of the TO's Test Automation framework / harness and any in-house tools required. Where tools do not fully meet requirements of the TO, the TA writes code / develops components that can extend available tools or even design & develop tools as needed
• is involved in understanding Business requirements and works with the development architect to translate requirements into solution architecture designs. Reviews requirements and seeks clarity as required, participates in product design reviews and works with the development architect and development team to make any design improvements and refinement as needed. Also helps incorporate Testability requirements into design
• Analyzes competitive products and technologies and makes appropriate suggestions (may use demos, pocs) to influence product / technology direction
• has overall product knowledge and is able to guide both junior and senior team members
• influences Technical direction and use of technologies after making necessary evaluations
• involved in hiring activities for the TO and mentoring of TO team members
• pro-actively seeks to make continuous improvements to Test coverage, execution and automation
• is results oriented and has a high degree of accountability, commitment and responsibility. The expectation is that involving a TA in a project is a guarantee of obtaining positive outcomes
• participates in test planning for all products handled by the TO and owns the test artifacts such as test specs, code, etc.
• Growth upwards from a TA level is towards a more senior role with wider scope of activity & influence across the organization. Needless to state the obvious, there is considerable enhancement in responsibilities and charter as progress is made upwards on the growth path

• Extensive Technical skills covering Product, Technologies and Competitive knowledge. Sound knowledge of domain / areas being handled is essential. Its not sufficient to be a specialist in any one area or technology and requires a wide and fairly deep understanding of a gamut of technologies and tools
• Knowledge of current industry wide Quality & Test processes and practices, Tools and techniques
• Ability to work with teams. This point cannot be emphasized enough since at this level, the last thing that would be acceptable is silo behavior or merely trying to be an individual star performer. Being able to get the team to perform at an outstanding level is absolutely essential here. The “ability to influence” despite not having direct reporting relationships is very key. In this position, a high EQ is as much a necessity as a high IQ. The ability to collaborate and co-operate is important
• Excellent communication skills – within and outside of the TO, across teams, with customers – both horizontally and vertically, is important. Effective negotiation skills are very important too.
• Another facet that is extremely important is an excellent working relationship with the Manager. No i don't say this because i'm viewed as being on the Management side ! The fact is to be a successful TA, requires working in tandem and close co-operation with Management, keeping Management abreast and updated of developments, seeking and providing inputs and feedback, regular reporting, etc. is very important. This attribute cannot be stressed enough
• Ability to focus and prioritize is important. Understanding the distinction between the urgent and the important and effectively prioritizing tasks is key
• Needs to focus on the explicit & implicit customer / user needs
  
• Self-management is a key attribute expected of a TA. Being able to work without the need for follow-up or “too much” management is important. The TA should be self-motivated and a self-starter. No, this does not absolve the Manager of the responsibilities of managing the TA as needed !, but a TA should require very little following up to get things done. The expectation is when a TA is assigned to a product, project or specific area, positive & agreed upon results are guaranteed almost always
  
• Ability to motivate self and others is important. Also, vital is being able to set a good example for the other members of the TO to follow
• Ability to set goals is also key. In many instances, the TA will need to define and set goals including stretch-goals as appropriate
• Patience and a touch of humility is valuable, especially in all dealings with team members. This is especially true when trying to mentor or guide other team members, the ability to articulate in ways that are understood by the listener at their level is necessary while also possessing good listening skills. The humility to acknowledge need for continuous learning and to undertake a program of learning to constantly update skills and keep abreast of current developments in the industry is vital
• Ability to strategize and look ahead and at the big picture
• A great deal of maturity, accountability, high degree of integrity, highest levels of pro-active behavior, ability to take initiative and professional behavior is naturally expected of a TA
• Sound Project Management abilities is important
• Software Analysis & Design knowledge/experience is needed while also having a solid background in Software Quality & Testing. Must have hands-on experience having performed both functional and non-functional testing and be able to review requirements, design and even code as needed

|   Slashdot It! | Submit to del.icio.us

Here's an attempt to talk a little bit about "Usability Testing".

We begin by taking a quick look at the term, Usability. According to the Usability Professionals' Association, "Usability is an approach to product development that incorporates direct user feedback throughout the development cycle in order to reduce costs and create products and tools that meet user needs."

Usability testing provides the opportunity to receive feedback from the very people the application is meant for. And, the consequences of building something without getting user feedback is obvious to anyone who's in the industry !

Whilst planning for Usability Testing, its easy to constrain it to be more of a "Validation" type technique. Usability testing & the information gathered from the exercise, should serve to make informed design & development decisions right from the outset, thereby acting in more of a "preventative" role. The idea in the case of Usability testing is to test early & test often. Usability testing lets the design and development teams identify problems before they are deeply entrenched. The earlier those problems are found and fixed, the less expensive the fixes are. As the project progresses, it becomes more and more difficult and expensive to make major design changes. The more you test and change based on what you learn, the more confident you can be that your application will meet your objectives and your users' needs when it is released.

An iterative process involving developing prototypes, testing it with users, analyzing the results, making needed changes based on the results and repeating the test, analysis and revision cycle is the recommended way to produce applications that are more usable (and acceptable) to users. We could also probably say that, in the initial stages of application development, users would be called to perform tests that are more "Exploratory" in nature. This feedback helps clarify direction for interface design, navigation, etc. In later stages, prior to release, "Validation" type Usability tests are performed to validate that interfaces and design are "usable" and feedback from earlier stages are incorporated.

The folks who should actually be doing Usability Testing (executing tests) should ideally not be anyone associated with the product or organization. A profile of potential subjects (folks who will perform Usability Tests) should be prepared that mimics end user attributes in as fairly and representative manner as is feasible. Based on this profile, subjects may be sourced from market research, temp or contracting agencies.

During Usability testing, representative users try to find information or use functionality on the Web site / Application, while observers, watch, listen, and take notes. The purpose of a usability test is to identify areas where users struggle with the application and make recommendations for improvement.  The most likely goals in Usability Testing or areas that are monitored and measured include, the Time (taken to accomplish specific scenarios or tasks), degree of Accuracy (covers inaccurate menu or navigation choices, errors and lack of clarity or misunderstandings), the Success (in accomplishing the set tasks wherein users are able to complete the scenario they were asked to perform using expected steps) and importantly, Satisfaction of the users (broken down and measured per area such as navigation, information search, etc.)

If you are developing an application, your product must allow users to do their tasks at least as quickly with as few errors and as much success and satisfaction as their current way of working. Ideally, it should let them be more quicker, more accurate, more successful, and more satisfied. Otherwise, there's little chance of customer delight.

That was a brief overview of Usability Testing. While it may not address all aspects and intricacies of this subject, the intent is that it hopefully arouses sufficient curiosity and interest for folks who are not familiar with this subject, to go and make your own exploration.

|   Slashdot It! | Submit to del.icio.us

• Acceptance Testing
• Ad hoc Testing
• Buddy Testing
• Paired Testing
• Exploratory Testing
• Iterative / Spiral model Testing
• Agile / Extreme Testing
• Aesthetics Testing
• Alpha Testing
• Automated Testing
• Beta Testing
• Black Box Testing
• Boundary Testing
• Comparison Testing
• Compatibility Testing
• Conformance Testing
• Consistency Testing (Heuristic)
• Deployment Testing
• Documentation Testing
• Domain Testing
• Download Testing
• EC Analysis Testing
• End-to-End Testing
• Fault-Injection Testing
  
• Functional Testing
• Fuzz Testing
  
• Gray Box Testing
• Guerilla Testing
• Install & Configuration Testing
• Integration Testing
• System Integration
• Top-down Integration
• Bottom-up Integration
• Bi-directional Integration
• Interface Testing
• Internationalization Testing
• Interoperability Testing
• Lifecycle Testing
• Load Testing
• Localization Testing
• Logic Testing
• Manual Testing
• Menu Walk-through Testing
• Performance Testing
• Pilot Testing
• Positive & Negative Testing
• Protocol Testing
  
• Recovery Testing
• Regression Testing
• Reliability Testing
• Requirements Testing
• Risk-based Testing
• Sanity Testing
• Scalability Testing
• Scenario Testing
• Scripted Testing
• Security Testing
• SME Testing
• Smoke Testing
• Soak Testing
  
• Specification Testing
• Standards / Compliance Testing
• 508 accessibility guidelines
• SOX
• FDA / Patriot Act
• Other standards requiring compliance
  
• State Testing
• Stress Testing
• System Testing
• Testability Testing
• Unit Testing
• Upgrade & Migration Testing
• Usability Testing
• White box Testing
• Static Testing Techniques
• Desk checking
• Code walk-through
• Code reviews and inspection
• Structural Testing Techniques
• Unit Testing
• Code Coverage Testing
• Statement
• Path
• Function
• Condition
• Complexity Testing / Cyclomatic complexity
• Mutation Testing

The list above, hopefully covers most of the "Types of Tests" that are generally performed. There may be a few that i might have missed ... let me know if that is the case and I'll be more than glad to add / edit them.

If you are new to Testing, my suggestion is to make it a point to look up each type of Testing (over the course of time at your convenience) listed above and get some information on each of these types.

Few testers end up actually working on all of the above "test types" during their Testing careers, but as a Quality & Testing professional, it helps to know your domain.

|   Slashdot It! | Submit to del.icio.us