Quantifying computational thinking abilities

Abstract

Computational Thinking (CT) focuses on the problem-solving process using cognitive abilities in the Computer Science field. In the last decade, several approaches have been created to disseminate CT as well as to assess the development of CT skills in students. Therefore, various instruments have been used to evaluate CT abilities. In most cases, these instruments are based on computer programming, such as quizzes, surveys, and interviews about pro-gramming projects. These approaches are useful for measuring students’ problem-solving skills by means of programming activities. So, these methods depend on programming activities to assess CT abilities, i.e. the students must learn computer programming, and then, a qualitative or quantitative approach assesses the CT abilities exploited in their programming activities. In some scenarios, however, it would be essential to have means to measure such skills without requiring previous programming knowledge from the subjects, because CT is more about the cognitive abilities used to solve problems than about technical skills. The problem is that the strategies for evaluating computational thinking skills are incipient. Although several approaches have been proposed to disseminate CT, little is known about how to measure CT skill reliably. There has been little quantitative analysis of quantifying CT skills. In fact, various studies have been published recently, but few tackle empirical evidence guided by a robust methodology and statistical tools. Our main objective is to investigate strategies and instruments in order to quantify CT skills reliably without manda- tory programming practices. We approach this objective considering the CT skills as latent variables (constructs) explored through the items, i.e. questions, in the instruments. During this research, we have proposed a theoretical CT model based on empirical stud- ies which can be used for assessing CT skills without mandatory programming practices. We have evaluated items designed to explore CT skills using a psychometric approach. This strategy provides us a better understanding of what we should observe to assess CT as well as a proper perspective for measuring CT abilities. Finally, we have gathered a set of lessons learned on providing the items’ characteristics in order to assist future research in design items to measure CT skills reliably. In summary, we have observed that it is possible to quantify CT abilities using a suitable methodology and statistical procedures.