Poda estruturada de redes neurais profundas: um estudo sobre a utilização de métodos de explicabilidade como critério de poda.

Abstract

The advancement of Deep Learning models has provided exceptional results in computer vision and natural language processing tasks. However, the increase in size and complexity of these models brings significant challenges in terms of infrastructure and operational costs. In this context, the technique of structured pruning in deep neural networks emerges as a solution to reduce model size while maintaining similar levels of accuracy. This study investigates the impact of using explainability metrics (Conductance and Layer-wise Relevance Propagation) as pruning criteria, comparing them with magnitude-based pruning and random pruning. Different pruning percentages are evaluated, considering both one-shot pruning and iterative pruning. The results show a positive correlation between the use of explainability metrics and improvement in the quality of pruned models, including higher accuracy, lower variance, and the ability to perform more aggressive pruning without significant loss of accuracy. These promising methods have the potential to enhance operationalization and reduce costs associated with large-scale Deep Learning models.