Analisando LLMS na resolução de problemas de competições de programação: um estudo com ChatGPT e Bard.

Abstract

With the recent adoption of Generative Artificial Intelligences in the field of Computer Science, questions arise regarding the limits of these tools in code generation and the subsequent performance of this code in solving highly complex programming problems, including challenges involving dynamic programming. In this study, we analyze the performance of two AI models, Bard and ChatGPT, by subjecting them to 83 programming problems of varying complexity levels. Utilizing the prompts from both tools in a comparative case study, we apply the generated code to programming problems obtained from online judges, including Codeforces, Atcoder, CodeChef, and LeetCode. We compare the results of these models in online competitions regularly held by the mentioned platforms, with the problem statements being submitted to the models while the competitions are ongoing, ensuring the problems are novel and unknown to the AIs. This study aims to acquire consistent data to evaluate the capabilities of Bard and ChatGPT 3.5 in solving programming problems with statements unknown to both tools. The results Will contribute to a deeper understanding of the performance of these AIs in programming competitions and future research related to the use of AI in solving complex computational challenges.