Are Unit Tests without measuring Code Coverage useful?
Code Coverage shows you which part of your software has been tested. If you execute Unit Tests without measuring Code Coverage, you will be unable to tell which parts have and have not been tested. Most likely you will be testing code parts numerous times and thereby waste time and monetary resources. Moreover, other parts of your software, potentially containing serious (and expensive) errors, will be neglected. If later on errors occur at customer level, you will lack insight on where to find them. Without Code Coverage you are unable to tell what you have already tested. Unit Testing without Coverage is like fishing in muddy waters or wandering around in the fog...In safety-critical industries such as aerospace, automotive, or medicine, measuring and proving code coverage is prescribed by norms.
However, even for software tests not regulated by norms, it is quite useful to measure code. Hereby a visible increase of test quality may be achieved. Code Coverage by itself does not guarantee good testing yet. Nevertheless, safety and security are tested only insufficiently and arbitrarily without coverage software. In other words: Your software - not unlike a banana, becomes ripe only when already with the customer.
"Many folks "think" they have good requirements and tests, only to find out later via coverage analysis that those "great" tests only covered 60% of the code. Obviously they had weak requirements, or weak tests of requirements, or more likely: BOTH.
Code Coverage is quite an eye opener. Frankly, even non-safety critical should do structural coverage. Not to attain 100%, not at all: but rather to ensure they’re covering at least 75% thereby affirming they have good requirements and tests of those requirements."
Vance Hilderman, Chief Technical Officer, AFuzion Inc. / Author of the books "Avionics Certification" and "The Avionics Development Ecosystem"
By the way: It is quite motivating for testers to see the progress of their work through Code Coverage.
Code Coverage allows - as already mentioned - to prevent redundancies. Changing the software requires adapting the test cases accordingly. If there are too many test cases - some of which would not have been necessary in the first place if one had measured test coverage - the time necessary to do so drastically increases. This not only leads to high additional costs but is also an unplanned factor within the project which in turn leads to stopping Unit Tests altogether instead of adapting them. Originally intended as an emergency measure, this "makeshift solutions" in the end leads to permanent limited coverage and bad software quality.
Back to FAQ
