MILANEZ, A. F.; http://lattes.cnpq.br/2166423222266686; MILANEZ, Alysson Filgueira.
Résumé:
Testing is commonly used to check conformance in contract-based programs, as verification
by formal proofs is hard to scale and static analysis is, sometimes, limited for detecting
general nonconformances. Traditional test cases, with manually-provided data, may be ineffective
in detecting subtle nonconformances that arise only after several instantiations and
modifications in objects under test. Those nonconformances may signalize more subtle bugs,
hindering the benefits of using contract-based programs. Random-generated tests with automatic
test data generation, on the other hand, is a promising approach when more substantial
testing is demanded. In this work, we propose and evaluate an approach, JMLOK 2.0, for
automatically detecting and categorizing nonconformances, in the context of Java Modeling
Language (JML). Our approach aims to help the programmer in the process of nonconformances
correction. The detection is backed by Randomly-Generated Tests (RGT) approach.
And the categorization is backed by heuristics-based approach. We perform two evaluations.
First, we perform an evaluation of our detection approach and our manual categorization
process: we detected 84 nonconformances in over 29 KLOC and 9 K lines of JML contracts
(that we will refer as KLJML henceforth); applying our manual classification system we
got that most detected nonconformances were classified as postcondition errors; we also observed
that a nonconformance is detected after 2.54 top-level test case calls, in average, and
the number of internal calls within the faulty test case call is an average of 2.23, providing
evidence for the need of a more complex generated test structure in nonconformance detection;
furthermore, we compare our approach with JET, an existing test-based approach for
detecting nonconformances in JML programs, using a subset of programs from first study
(6 KLOC and 5 KLJML). JMLOK 2.0 detected 30 nonconformances with Java coverage of
78.44% and JML coverage of 67.67%, while JET detected 9 nonconformances by covering
47.97% of Java and 56.97% of JML. Second, we perform an evaluation of our automatic categorization
approach: we compare automatic and manual categorization and got a matches
value of 0.73 (considering the nonconformances from first evaluation) indicating similarity
between automatic and manual approaches; furthermore, we compare our results with the
categorization performed by voluntary JML experts and we also observed some similarity.