Estudo de caso: uso do ChatGPT para resolução de problemas de programação.

Abstract

The generation of programs from natural language aims to transform sentences or commands in natural language into programming code. ChatGPT is a general-purpose chatbot based on the GPT-3 language model designed to generate text like a human. It has been trained using reinforcement learning with human feedback in a conversational format. In this scenario, concerns about the reliability of the responses generated by ChatGPT 3.5 have been raised. Therefore, this study evaluated the model's performance in solving 100 randomly selected programming problems from popular platforms such as LeetCode and BeeCrowd. The selected problems were distributed among the Easy, Intermediate, and Difficult complexity levels. Out of the total of 100 submitted problems, the language model was able to correctly answer 71 problems across 3 attempts, with 50 of them from the LeetCode platform and 21 from the BeeCrowd platform. Thus, it can be concluded that ChatGPT can be used to solve a range of problems, but its use requires careful attention as the generated result may not always be correct.