Justification of Missing Coverage

In real-life test situations, 100% coverage is hardly achieved for several reasons. You can justify missing coverage with Testwell CTC++, using structured comments in source code or justifications collected in text files.

Justifications always refer to counters relevant for decision coverage, for example:
  • function begins
  • true or false branches of if statements
  • cases of a switch statement
For dependent counters, statements and lines, the property to be justified is derived by ctcreport.

The justifications provided are documented in the HTML report. For all coverage measures, the impact of justifications is calculated. Hence a clear indication is given if "everything" was tested or at least justified.



Structure of Justifications

A justification looks like: 
if (world_still_exists) // CTC++ Justify False | EFFORT: Simulation too expensive, else-branch reviewed instead.
or, inside a justification file: 
182 False | EFFORT: Simulation too expensive, else-branch reviewed instead.
The examples show the common structure of justifications:
type
"False" in example above. Available types are:
  • True: For true counters in if or loop decisions.
  • False: For false counters in if or loop decisions.
  • Function: Whole functions can be justified (Note that this does not refer to unreachable code).
  • empty: Without a specification, single counters like return or case are referred.
  • FullLine: All counters on the line are justified. This can be useful for complex macro code expanded on one line.
tag
"EFFORT": A short text or tag to categorize justifications according to your requirements. Justification information in overview and detail pages is grouped by these tags.
explanation
"Simulation too expensive...": A long explanation for every justification. This explanation is shown in source code view together with the tag for each justified counter.

Separator | is used between type and short text, and : between tag and explanation.