Aumentando a confiança nos resultados de testes de sistemas multi-threaded: evitando asserções antecipadas e tardias.

Abstract

Testing multi-threaded systems is quite a challenge. The inherent non-determinism of these systems makes their implementation and the implementation of their tests far more susceptible to error. It is common to have tests of these systems that may not sometimes pass and whose failures are not caused by application faults (bugs), but by problems with the tests. For instance, thiscan happen when there are asynchronous operations whose corresponding test verifications (assertions) are performed at inappropriate times. Unreliable tests make developers waste their time trying to find non-existing bugs, or else make them search for bugs in the wrong place. Another problem is that developers may cease to believe that certain test failures are caused by software bugs even when this is the case. Scenarios like these have motivated this work. Our main objective is to avoid test failures that are caused, not by application defects, but by test assertions performed either too early or too late. In order to achieve this goal, we present an approach whose basic idea is to use thread monitoring and control in order to support the development of multi-threaded systems tests involving asynchronous operations. This approach is intended to make it easier the development of correct tests for these systems and also to improve developers’ confidence on the results of their tests. The proposed approach has been evaluated through case studies using a tool to support the development of multi-threaded systems tests (developed for this work) and also by formally modeling the approach using theTLA+ language in order to prove that early and late assertions do not occur when this approach isused.