Evaluating Test Cases Quality With Mutation Testing

How good are your test cases? Maybe they are good, or maybe you need to add some new ones. How to tell? You can measure things like code coverage to check if some parts of your code were not executed. Still, this does not tell you anything about the quality of your assertions and your software testing results. In the extreme case, a test suite with no assertions might still achieve 100% code coverage, although being of questionable value.

Mutation testing is a technique that is used to automatically inject faults in your code, and then check if your test cases can “kill” those “mutants”. In this way, the quality and effectiveness of your assertions can be evaluated. This talk presents the benefits and challenges of mutation testing, and also shows how to use open source tools like PIT.

Video producer: http://oredev.org/