Validation of a simulation model for FasS performance benchamarking using predictive validation.

Abstract

In the paper Controlling Garbage Collection and Request Admission to Improve Performance of FaaS Applications[30], we veriied and evaluated the impact of memory management mechanics of programming languages in the context of Functions as a Service (FaaS) via simulation experiments. The results of this study pointed to an impact of up to 11.68% on the response time of requests when a garbage collector procedure was executed during the execution of a CPU-bound function. As future work, we listed a few threats to the validity of the results attained, and among them, we cited the validation of the simulation model used. The validation of the model is important because it validates the results generated in the simulation experiments, which ensures realistic results. In this work, we proposed and executed a validation to the simulation model used in the previous work. To do so, we run measurement experiments in a public FaaS platform and simulation experiments of the same scenarios using the same simulator of the previous paper. Then, we validate the simulator by comparing the results obtained in both experiments to ensure that the simulation result and the measurement one are equivalent.