SOUZA, J. V. M. C.; SOUZA, João Vitor de Melo Cavalcante e.
Abstract:
Large Language Models (LLM) have become popular, with easy to access and more and moreinsightful models, such as Chat GPT, Gemini, Claude, Mistral, among others. With a wide range of possible utilizations in a digital era, LLMs have a growing use in the most diverse fields, specially on human learning. For this reason, it’s important to study and consider such tools and how they can help in solving logical problems found in the Brazilian Olympiad in Informatics (OBI). In this study, we evaluated Chat GPT 3.5 and Le Chat Mistral with 100 logic questions from OBI. Chat GPT answered 23% of them correctly, while Mistral Le Chat answered 36% of the questions correctly, both on the first attempt.
