Improving traceability recovery between bug reports and manual test cases.

Abstract

During the life cycle of a software there is the production of several types of artifacts, responsible for documenting and organizing not only the software, but also the development process. Many of these artifacts are codependent, but while maintaining consistency between artifacts is an important task, it is not trivial. Since they are produced by different individuals, in different types of details and languages, automation is highly desirable. Some research investigates traceability between bug reports and manual test cases. Since test cases are a popular way of documenting requirements in agile projects, this type of traceability allows, for instance, to analyze how bug reports relate to requirements. Previous research evaluated three Information Retrieval (IR) techniques (LSI, LDA, and BM25) and a Deep Learning (DL) algorithm (Word Vector) used to retrieve links between bug reports and test cases. The results of this research point to the need for improvements. In our research we evaluate a set of improvement techniques applied to a baseline work, which uses artifacts from the open-source Mozilla Firefox project. Improvement techniques are: (i) textual and information cleaning; (ii) spellchecking; (iii) increasing the weight of fields in the bug reports (duplication of title and description); (iv) merge the similarity matrices; and (v) merge the traceability matrices. The evaluation of improvement techniques is made by comparing the results obtained by them and the results obtained by the baseline work, in terms of precision and recall. There was a slight improvement in the precision and recall rates in the techniques of textual and information cleaning, spellchecking, duplication of the title, and in the combination of these three techniques. In one of the strategies of the technique of merging the traceability matrices, which selects traces recovered by at least one of the traceability techniques, it reached a coverage value of 93%.