Otimizando a utilização de ambientes de nuvem PaaS usando uma abordagem preditiva.

Abstract

Following the trend of many computing areas, Big Data processing has also been moved to the cloud, due to the flexibility and easiness the cloud model provides, specially in the PaaS model, where users can run their Big Data applications in an even easier and cheaper way. Cloud computing data centers need to have proper resource management in order to make best usage of available capabilities and minimize operational cost, as a large portion of the computational resources is idle most of the time. This idleness is mostly caused by the quota management system, which only considers static resource allocation, instead of looking whether the resources are indeed used. In this work, we propose a solution to better manage the resources in PaaS cloud environments, focused on data processing. This management is made through predictive approaches, both in operational level, forecasting the workload of cloud servers in next time windows (based on the current utilization and historical data), and at application level, estimating the makespan of batch data processing applications with a clustering algorithm based on previously executed jobs characteristics. With those data, the proposed solution is able to take a set of non-trivial decisions, such as accelerating the job if more resources than requested are available, postponing the job when resources are only available in next time windows,or rejecting it, when there are not enough resources at the moment, neither in next windows. With this approach, when compared to the usual case, regulated by static resource quotas, we obtained a 10% increase of average CPU and RAM utilization across the cloud, with an operation cost increase of only 1%, considering the non-proportionality of power consuming, also observed in our experiments. Besides that, the system also showed a 20% increase in the average success fully processed jobs, occasioning a profit i ncrement between 10% and 20%.