Assessing and Improving the Mutation Testing Practice of PIT

Thomas Laurent (1) , Anthony Ventresque (2) , Mike Papadakis (3) , Christopher Henard (3) , Yves Le Traon (3, 4)

1 ECN - École Centrale de Nantes
2 UCD - University College Dublin [Dublin]
3 SnT - Interdisciplinary Centre for Security, Reliability and Trust [Luxembourg]
4 CSC - Computer Science and Communications Research Unit [Luxembourg]

Mutation testing is used extensively to support the experimentation of software engineering studies. Its application to real-world projects is possible thanks to modern tools that automate the whole mutation analysis process. However, popular mutation testing tools use a restrictive set of mutants which do not conform to the community standards as supported by the mutation testing literature. This can be problematic since the effectiveness of mutation depends on its mutants. We therefore examine how effective are the mutants of a popular mutation testing tool, named PIT, compared to comprehensive ones, as drawn from the literature and personal experience. We show that comprehensive mutants are harder to kill and encode faults not captured by the mutants of PIT for a range of 11% to 62% of the Java classes of the considered projects.

Mots clés: Software Engineering; Software Testing; Mutation Testing; PIT

Domaines: Génie logiciel [cs.SE]

Thomas Laurent, Anthony Ventresque, Mike Papadakis, Christopher Henard, Yves Le Traon. Assessing and Improving the Mutation Testing Practice of PIT. [Technical Report] Lero@UCD, School of Computer Science, University College Dublin, Ireland; Ecole Centrale de Nantes (ECN); Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg. 2015. ⟨hal-01252099v3⟩