The goal of the Unit Test and Integration Plan is to outline the software development process which will be used to create the required system. This document introduces software testing strategies, specifies the outline of the Unit Test and Integration Plan document, and provides a detailed description of each section of the Plan document.
It is possible to develop a system that runs nearly perfectly if each component is tested on a standalone basis as it is developed and integrated into the overall system, and if care is taken when integrating components. On the other hand, it is possible to have a system that never or only barely works if testing is overlooked or consciously disregarded. First we look at the purpose of testing, then introduce various testing strategies.
Testing is the opportunity to find errors when they are introduced, when it is the least expensive to correct them, and to deliver an error-free product to the customer. During testing, finding errors is the goal. However, because programmers see their code as an extension of themselves and view criticisms of the code as criticisms of themselves as a person, and because of cognitive dissonance, programmers often make the worst testers of their own code. As a result, independent testing organizations (ITO) or quality assurance (QA) groups are created in organizations to test programs and find errors. In this class, we will rely on the team members to test code as it is developed and integrated.
Testing cannot prove the absence of defects. At best, it can only detect the errors which we anticipate and for which we test. Therefore, testers must be extremely rigorous in enumerating the expected results of execution, and extremely clever in anticipating all of the ways something unanticipated could happen. A tester must execute a program with the intent of finding an error.
A thorough tester will try all possible things that a user can do, particularly the things that a user is not expected to do and should not do. To accomplish this, we write test cases. A good test case is one that has a high probability of finding an as-yet undiscovered error.
Prior to putting an entire program together, it is important to test each piece on a standalone basis. As part of this, you will build drivers and stubs.
A driver calls appropriate functions with reasonable parameters in order to test the functionality of the code being tested. A driver can either be hard-coded or be a simple interface that allows you to try various parameters.
A stub is a piece of code that stands in for another piece of code. We use stubs in order to test a higher level function without being concerned about low-level details. For example, with a chess-playing program it is impossible to test the computer player until the board class exists. However, we can create a board class stub that has hard-coded return values for functions defining what is at each position.
Another test methodology, which is especially useful during the integration testing phase, is
to use a "debug argument" in the calling signature of all functions or methods. The value of
this argument is actually hard-coded to a known parameter. When such a module is called during
the debugging phase, the debugging value is simply returned to the caller as if the method had
actually executed. Later, when the program is executing for real, the debugging argument can
be ignored while the method executes normally. A simple flag, either a global variable or an
instance variable, can control which route execution takes. In this way, the code never needs
modification, but can always be run in a testing
mode.
Black box testing assumes that we do not know how the code we are testing works. It is predicated on testing every possible combination of calls to functions and all boundary conditions to ensure that the code works properly.
White box testing assumes that we do know how the code we are testing works, and we take advantage of this during testing. For example, with white box testing we ensure that all code paths have been traversed. If a code path has not been taken, then we are not sure whether the code on that path works.
Often white box testing is assisted through debugging output generated during the execution of a program. The following is an example:
void Board::AddPiece()
{
#ifdef DEBUG
if (glbDebugOn)
{
glbAddIndent(); // increases indent string length
debugFile << glbIndent << Board::AddPiece()
;
}
#endif
// code for function
#ifdef DEBUG
if (glbDebugOn)
{
debugFile << glbIndent << Board::AddPiece() 2
;
glbRemoveIndent();
}
#endif
}
The contents of the debug file can be checked to ensure that each function as a whole and each part of each function is being reached. Note: these debugging statements can be extremely useful during not only integration tests, but also while testing the system as a whole.
For the purpose of this discussion, a test plan is an approach for testing a software component, and a test suite is an input/expected output pair used to test a software component. For each of the modules in your project, you must provide a test suite consisting of:
The idea is to ensure that each module works as expected before it is integrated with the other modules of the project. The numbering of test suites for your modules will correspond with the number of components of your project.
You are also required to develop test suites for the integration of modules. For example, if a SYSTEM had modules A, B, and C, and the plan was to integrate A and B, then (AB) with C, there would be a plan for standalone testing on A, on B, and on C, then on the integrated AB. The final tests would include all three modules. The tests on ABC constitute the final SYSTEM tests. Again, you should have specific input/output pairs which are used to prove correct operation.
When testing at the integration level, it is often necessary to write test procedures with specific steps. This method is particularly effective in proving correct operation of GUI-driven software, for which no convenient automated test methods exist. Here is an example of what I mean:
10.1.1.5 Help Menu Test
The following procedure tests the menu functions of theHelpmenu:
- Using the mouse, left-click on the
Helpmenu item in the menu bar at the top of the application window. A small sub-menu will appear.- Notice that the sub-menu contains two selections labeled
Help on TempConvAppandAbout. Click on the second of these selections.- A new page should appear in the application window, showing various information about the TempConvApp program, including samples of various temperature conversions, along with a description of the different temperature scales.
- Verify the information is displayed, and enter tester initials and the date of test on the test verification form.
- (A similar set of test steps would exist for whatever operations the
Help on TempConvAppoption will perform.)
In the biz, the SRS will contain a section called the Verification Cross
Reference Matrix, or VCRM. This is usually some sort of table, which is used to
cross-reference the numbered requirements in the SRS with a set of contractual requirements
(usually in the SOW or some such contract document) to show traceability
between
the software requirements and the contract requirements. Then, in the Test Plan, a cross
reference table is also set up, matching each numbered test in the test plan with its numbered
software requirement(s) from the SRS. In this manner, it can be proved (when all the tests
pass) that the software has met its contractual obligations, so the solution provider (you!)
gets the reward (an A for the course!).
For your project, the idea is to model the effects of the VCRM of the SRS, by producing a list of the tests, inputs, expected outputs, and so on. This could be a simple table, for example, with columns for the test number, the module being tested, the name of the test for that module, and the inputs, expected outputs, and any stub code that is required. Inputs should be a set, with different values used to test for "pass" and "fail". For example, if the module is a "times 3" that returns triple the input integer, you would have test inputs for acceptable integer inputs, along with unacceptable alpha and double-precision inputs. Outputs should also be a set, with expected output values corresponding to the inputs; the associations should be obvious. Finally, if there are stubs that are used to mimic operations, the name of the stub should be listed; if there are no stubs required, "None" is fine.
The document should be structured as follows (sections are explained below):
10.0 Testing
10.1 Unit Test Plan
10.1.1 Unit Test Descriptions
10.1.1.1 Unit Test 1
.
.
.
10.1.1.n Unit Test n
10.2 Integration Test Plan
10.2.1 Integration Test Descriptions
10.2.1.1 Integration Test 1
.
.
.
10.2.1.n Integration Test n
10.3 Module Dependencies
Begin by describing the purpose and contents of this section of your document.
This section should include an inventory of the tests to be developed, with a brief (one sentence) description of each. You should have as a minimum one top-level module, one module for each of your high-level components, and one module for each of the subcomponents of your high-level components.
Number and name each component, 10.1.1.1 through 10.1.1.n, as shown in the outline above; you will be referring to them by number in subsequent sections of this Test Plan document.
Do the same thing here as you did in the Unit Test Plan section, but focus on the Integration testing level. These should be higher-level tests than the individual module level described in the previous sections.
Do the same thing here as you did in the Unit Test Plan section, but focus on the Integration testing level. These should be higher-level tests than the individual module level described in the previous sections.
If there is any order-dependence
for any of the set of tests, this section must
include a dependency graph or UML diagram which clearly shows the order in which standalone
testing needs to be completed, and the order in which modules will be integrated. Each module
in section 10.1 and 10.2 for which this situation exists needs to be included.
Entries in the project SDF table of contents should be generated for the Software Development Plan document. These entries should match the section titles listed above and in the table of contents at the bottom of the Course Overview page.
This is the last document in the course documents set. The link below will take you back to the beginning.