Test Prioritization
From dis-Emi-A
This is an approximate overall list of the activities involved in planning the allocation of test resources. In some cases it will be necessary to actually start the test design before moving further, but in many cases you do not need much information, or will need to work on educated guesses in order to best allocate time.
Risk-based Testing is often used as a term to describe test prioritization.
This level of prioritization is very high and deals with the project as a whole. The fine workings of individual features is described under Test Suite Prioritization.
Contents |
Product Selection
Which products to be tested is the first step in the process. This may seem obvious if your company produces only one product, but for a company that has multiple products, varying client bases, and several additional internal products, the choice is all but obvious.
For many testing departments however this decision is not really available, in which case you can skip to the next section.
This is some of the criteria which can be used to determine which products to be tested.
- Available testing resources
- Will you be able to reasonably test more than one product at a time, or should all the time be spent on one product, and then move on to the next product. This depends also on the otherall development cycle, as to how much product release overlap.
- Revenue stream
- How closely if the functioning of the product tied to the revenue of the company. Clearly the more money a product makes the more time should be spent testing it.
- Response time
- How quickly can mistakes be fixed and redeployed upon discovery. The more responsive a deployment can be, the more leniency one has in up-front testing. For customer delivered products this could be a matter of months, if not years, but for server side solutions this could truly come done to a matter of hours.
Feature Assessment
Not all features in a product are equal, especially with respect to revenue stream, therefore it is very important to work with the product managers to get an impression of the value of the features.
As with all priority lists, it is important to keep the number of priorities, or importance levels, relatively small, otherwise you will spend too much time debating placement in the list. The following is a reasonable assessment set for the features.
- Core
- These feature comprise key aspects of the product, such that without this feature the product has no significant value. Virtually all users of the product will be using these features.
- Secondary
- These features offer significant additional value to the core features but may not be used by all users.
- Fringe
- These features have very specific functionality of value to only a limited portion of the users of the product. The product is very likely fully useful even with these features being absent.
- Uncommited
- These features have not yet been commited to development. This is an important class, since there is no point in planning the testing for a feature that may or may not make the release. Generally items in this set will be promoted into another category once they are commited.
Only in extreme cases, when the testing resources fall excessively short of requirements, would it be necessary to do more detailed prioritization. The above sets work on the notion that at least some degree of testing will be possible in all of the categories, and it is only a matter of best assigning that time.
Module Assessment
The structure of the code is important in the next step of priotization. Therefore you should read Understanding Code.
This step is very similar to the previous step, so much so that you can, and should, use the same classification as you did in the previous step.
- Core
- These modules are heavily linked and used throughout the code. They have many dependents, and/or are heavily interdependent themselves.
- Secondary
- These modules are somewhat linked, they rely on core modules, but not too many modules rely on them.
- Fringe
- These modules have no dependents and generally have limited interdependencies.
Time Allotment
Once you have everything categorized you will then need to allot time to testing items in each category. A good recommendation would be 50% of time on Core items, 30% on Secondary, and 20% on fringe. Note that you can't spend too much less time on Fringe items, since they do still need to be tested, and all test design requires a certain minimum effort to be of any use at all.
