Avaliação de grandes modelos de linguagem para detecção de tópicos e posicionamentos em debates: um estudo de caso no contexto do Senado Federal.

Abstract

Legislative power in Brazil is one of the three essential functions of the State. However, there is a clear challenge for the population to follow discussions in public bodies. This is due to the considerable length and volume of these meetings, making them inaccessible to many citizens. To address this challenge, this study used the Federal Senate's 2023 tachygraph notes, which are transcripts of parliamentary debates, with the objective of evaluating the potential of Large Language Models (LLMs) to detect relevant topics discussed by parliamentarians and their stance on these topics, classifying them as in for, neutral or against. Experiments were carried out, both using the GPT-3.5-Turbo model, for the tasks mentioned. The first experiment used a data compression technique before providing input to the GPT and covered meetings of different sizes. The second experiment did not involve compression and focused only on small meetings. The results indicate that the model performed better for small meetings. In addition, in a general overview for size-independent meetings, the model performed better in the topic detection task, with an average precision of approximately 70%, while in position detection it performed reasonably well with an average precision of approximately 60%.