Test Levels / Levels of Execution / Phases of Testing
Unit, Integration, System & Acceptance are 4 common test levels.
Unit Testing
Execution of test cases on smallest testable piece of software which is either a module / unit / component / program. Smallest testable piece of software is tested, in isolation, with an intention to uncover & find all the deviations in smallest piece software.
Unit testing is accomplished within the development environment by developer class people.
Functional testing/ Non-Functional testing/ Structural testing/ Testing Related to Changes is possible at Unit Level. However, Unit testing is dominated by structural testing.
Robustness test, Dynamic analysis, Code reviews for control flow analysis, data flow analysis, as well as Code coverage are common tests at Unit level. Stress test is not done at this level.
Component Integration testing
When more than one unit is combined & checked to expose errors that might exist in interfaces or in communication between them.
Integration of components may follow a strategy of either Big Bang Integration or Incremental integration. Big bang integration is when all the units are integrated at once & tested for interfacial & communication errors. Whereas Incremental integration combines available components at the beginning & incrementally combines more units as & when they are available.
Incremental integration strategy is preferable as the isolation of defects is much easier & further time management is better.
Incremental integration may be accomplished either as Top-down integration approach or Bottom-up approach or a mixed approach called Hybrid / Sandwich approach.
Top-down approach is possible when the main module or the modules (at the top of the Control flow / architecture diagram) are available for integration & more units that fall below such top modules are incrementally integrated.
Bottom-up approach is possible when the bottom modules are available & modules that fall above it are incrementally integrated.
A mixed approach may involve random integration involving some of the top & some of the bottom modules incrementally integrated.
Incremental integration requires temporary programs such as Stub or Driver to facilitate test execution.
Stub: is a temporary program that replaces an unfinished Called function or Called module. Stub reacts when called with a message & returns back the control
Driver: is a temporary program that replaces an unfinished Calling function or Calling module. Driver can initiate control
A test execution environment which facilitates test execution with the help of Stub / Driver is referred to as Test Harness. Sometimes a Stub / Driver is referred to as Test Harness.
Integration testing is executed in development environment by developer class people or Specialist integrators. Any type of test is possible at any level.
System Testing
When all the components are integrated and deployed into a customer expected environment & subjected to rigorous testing, such testing is called System testing. System testing is with an intention of uncovering as many defects that exist in the system as a whole. Defects are then referred to the development & once correction is proper then the quality of the system gets improved.
A rigorous evaluation of the system from one end to another involving many negative tests is how system testing can be characterized.
System testing, also called End to End testing, is carried out by Independent testers in a customer expected environment.
Any type of test may be done at this level & must include evaluation of Functional & Non-Functional characteristics.
The source artifacts for System testing can be SRS/ FRS/ Use Cases/ Business processes.
Acceptance Testing
When a system is assessed as to whether it meets user purpose & reasonable expectations such a test is referred to as Acceptance testing.
Acceptance testing is the responsibility of Users / Client & doesn’t involve Negative testing.
The source artifacts for Acceptance testing can be BRS/ Use Cases/ Business processes.
Acceptance testing done at Development company environment is known as Alpha test whereas Acceptance test done at Client / User environment is known as Beta test.
Alpha test is also known as Factory test & Beta test is known as Field test or Site testing.
When acceptance test is done as per contractual criteria such a test is called Contractual acceptance test
When acceptance test is done as per Regulatory standard such a test is called Regulatory acceptance test
UAT: User Acceptance Test is to assess whether the software meets user needs & reasonable expectations.
OAT: Operational Acceptance Test is to assess whether UAT approved software can be made live. In operationalizing a software criteria such as training to users, data backup & restore procedures, security permissions, disaster recovery procedures may need to be defined & achieved. Network admin of the client verifies if such criteria is satisfied before making the application live.
System Integration testing:
When our system combined with a third party component in forming the total solution then an additional test level is involved which is called System Integration Testing.
An external software component called C.O.T.S (Commercial Off The Shelf Software) + our system = Complete Solution
It is required to assess if system of systems has any errors at interfaces or in communication between them.
Maintenance Testing
When a live system (released system) is tested such testing is referred to as Maintenance testing.
A released system may need to be tested because of the following triggers,
1. Modification: to correct failures noticed by the client or to implement change request made by the client or implementation of additional features (enhancement)
2. Migration: change in the environment from one to another
3. Retirement: existing application is being retired & possibly being replaced with a new technology application. Such about-to-be-retired application needs to be evaluated as to whether the data can be safely backed up / restored, data can be migrated to another technology, or data can be retained for longer duration.
Workflow:
Customer reports an issue >> Support staff >> Support staff escalates the issue to higher level called CCB as they are unable to address this issue >> Change Control Board, comprising of the people who have worked on this project, shall analyze for what exactly needs to be changed, extent of the change & impact of proposed change on other aspects of software (called Impact analysis). If CCB approves the changes >> Such changes are implemented by developers >> Such changes are tested by testers >> Patch / Update / Hot fix / New version is released.
Maintenance testing is dominated by Regression testing & influenced by Impact analysis study.
