Understanding Code
From dis-Emi-A
If a test has no knowledge about the structure, and code, of the software they are testing they are relegated to doing strictly black box testing -- or worse in cases of a highly technical product. This type of limitation leads to highly inefficient prioritization and ultimately results in a poor test coverage.
For this reason it is essential that a test designer attempt to understand the software architecture and not be afraid to look at the code. That would also imply that such resources must also be made available to the test designer.
Contents |
Modules
All non-trivial software is comprised of several modules, each one performing some subset of the software's features. In a perfect world these modules would all be fully independent and could be tested individually. In a more realistic world there are two limitations to that:
- programmers don't always achieve the level of module independence they had hoped for, and thus modules aren't truly independent
- the functions of a module are too low level to be exposed to the tester and thus cannot actually be tested separately
In any case, knowing the modular structure of the code significantly helps in improving code coverage by elinating redundant tests. The goal is that the functionality of a module should only be tested once, regardless of how often in the program that functionality is exposed.
This theory relies on the module basically behaving the same regardless of which other module it is attached to. This is where the problem of modular interdependence comes in. Should the modules not truly be independent this assumption fails, usually to a degree equivalent to that of the interdependence. Exactly how to accomodate this in testing is not always clear.
Libraries
Several of the modules in use in the software should be third-party libraries, or part of a standard API. Existing libraries can both be a blessing, and a curse, for programmers and testers alike.
Should the library be perfect, then the programmer doesn't need to spend much time integrating it, and the tester need not spend much effort testing it. A perfect library has been properly tested and fixed by the vendor of that library.
Should a library be quite faulty, then the programmer will likely make many mistakes in integrating it, and the tester now has to test both that integration and the library itself. In these cases the tester could quite possibly be forced into black box testing, as nobody in development process will truly understand how the library works. Furthermore, the programmer's may have no ability to correct any mistakes a tester finds in the library.
Make sure the programmer's are aware of such problem's when choosing libraries. In some cases the integration of a faulty library might cause so many testing problems, that it is simply not worth the effort to use the library. That is, the programming team should not just be considering their effort when making a library selection, the testing effort needs to also be considered.
Open Specifications
Most products will be required to implement, or obey, certain standards, and will use several known algorithms. Indentifying and understanding this aspects of the software provide a very good test effort reduction.
Standards
Most standards have great testing advantage in that they are well described, and often have a reference implementation. If you're lucky, the standard will even have a certification test suite. In this case, you as a tester, need only use that test suite against your own company's code and see what happens -- the test design is free in this case.
Algorithms
Algorithms are also usually taken directly from known sources, or are adapations of known algorithms. It is quite uncommon for a programming team to be doing large degrees of innovation in this area, and even in projects where such innovation is required, the majority of the code will nonetheless be existing algorithms.
As a test designer it is essential that you can understand the basic nature of the algorithms in use in the software, or their implications for the software. Lacking such knowledge will make the partitioning of the equivalence classes very difficult, and likely wrong.
