Most higher mutants are useless for method-level mutation operators using weak mutation.

Abstract

Mutation analysis is a popular but costly approach to assess the quality of test suites. One of the attempts to reduce the costs associated to mutation analysis is to identify subsuming higher order mutants (HOMs), i.e., mutants that are harder to kill than the first order mutants (FOMs) from which they are constructed. However, it is not known how many HOMs subsume FOMs. In this paper, we use our previous approach, which discovers redundancy in mutations by proving subsumption relations among method-level mutation operators using weak mutation testing, to encode and prove subsumption relations among FOMs and HOMs. We encode a theory of subsumption relations in the Z3 theorem prover for 27 mutation targets (mutations of an expression or statement). We encode 233 FOMS and 438 HOMs and automatically prove a number of subsumption relations using Z3. Our results indicate that 91% of all mutants could be discarded on average. Moreover, 97.5% of all HOMs could be discarded and HOMs compose only 16.67% of the subsuming mutants sets on average.