Avaliando a qualidade de suítes de teste geradas automaticamente em detectar faltas introduzidas por refatoramentos.

Abstract

Refactorings are the practice where developers alter their code in a way that does not alter the system's behavior. Such a practice is usually accompanied by the use of regression suites to detect unwanted changes in a system's behavior. However, such test suites may not guarantee fault detection, creating a false sense of security during refactorings, since the suite may not detect certain changes. In this work, we propose an approach that aims to evaluate such test suites in their capacity to detect refactoring faults, as well as compare these suites to automated test suites alternatives. For this, a study related to the refactoring faults of the Extract Method refactoring type was carried out, as well as the development of a tool that facilitates the evaluation of a test suite, through a Eclipse IDE plugin made using the Java programming language. From this, refactoring mutants(faults) were created and utilized in a quantitative study, in which we evaluate the regression suites of 3 different open source projects, as well as test suites generated by the generation tool EvoSuite. Our studies show that there is a possible relation between test coverage and refactoring mutant detection in a system, as well as negligence of less common cases during the development of these suites, since about 38.7% of refactoring faults inserted where not detected by the manual test suites, and 45.3% by the automated suites, showing that there is room for improvement in the test suites focused in this context.