SOUZA, D. L. L.; http://lattes.cnpq.br/8530613968137637; SOUZA, Débora Lêda de Lucena.
Résumé:
The generation of programs from natural language aims to transform sentences or commands in
natural language into programming code. ChatGPT is a general-purpose chatbot based on the GPT-3
language model designed to generate text like a human. It has been trained using reinforcement
learning with human feedback in a conversational format. In this scenario, concerns about the
reliability of the responses generated by ChatGPT 3.5 have been raised. Therefore, this study
evaluated the model's performance in solving 100 randomly selected programming problems from
popular platforms such as LeetCode and BeeCrowd. The selected problems were distributed among
the Easy, Intermediate, and Difficult complexity levels. Out of the total of 100 submitted problems,
the language model was able to correctly answer 71 problems across 3 attempts, with 50 of them
from the LeetCode platform and 21 from the BeeCrowd platform. Thus, it can be concluded that
ChatGPT can be used to solve a range of problems, but its use requires careful attention as the
generated result may not always be correct.