Improving tail latency of interactive cloud microservices through management of background tasks.

Abstract

The increase in use cases with strict latency requirements, e.g., e-commerce and real-time data management, has been pushing towards greater control over performance variability. That is because even infrequent spikes might be unacceptable, as they could dominate the time it takes the response to reach the end-user in modern multi-layered cloud deployments. In this context, the uncoordinated execution of maintenance tasks, e.g., garbage collection and log compactions, might lead to unacceptable performance degradation. Even though researchers and practitioners have been working hard to improve the non-deterministic impact of those tasks on cloud services’ performance, that still unacceptable when strict performance guarantees are required. Furthermore, the proposed solutions are system, application, load, or task-specific. This research goal is to eliminate the negative performance impact of a representative class of tasks. To do so, we start by proposing a taxonomy of those tasks based on their trigger and feasibility to control. Then we formally define the broad class of interactive background tasks composed of controllable activities triggered by request handling. Furthermore, we propose the Background Tasks Controller (BTC), a fully distributed approach to eliminate the negative impact of interactive tasks on cloud microservices. The BTC does so by transparently evicting request handling while executing background tasks. We performed simulated and measurement experiments to thoroughly evaluate BTC efficacy to deal with the automatic garbage collector’s impact, a well-known cause of cloud services’ performance degradation. The results show that BTC effectively eliminates the impact of the garbage collector in real and synthetic workloads; and production and dummy microservices. Furthermore, it leads to a loss in capacity proportional to the frequency and duration of the background tasks, which allows estimating the needed capacity increase to deal with this loss.