Sumarização de vídeos a partir de dados fisiológicos, atenção visual e unidades de ação facial.

Abstract

The present research had as general objective the development of an approach for the summarization of digital videos, based on the visual attention (eye movements), physiological (skin conductance and heartbeat) and emotional (facial actions) response of the viewer. The monitored data of the viewer was acquired during viewing sessions of the videos. To achieve this general objective, it was necessary to achieve the following specific objectives: (1) to investigate the feasibility of automatic video summarization using viewers’ physiological, facial and visual attention characteristics; (2) to analyze the relevance of the physiological, facial and visual attention characteristics of viewers to the automatic summary of digital videos; and (3) to investigate the relationship between facial expressions and emotional states induced by the presentation of multimedia content. The data collection and analysis took place during three different sets of trials, characterized by different collections of videos, groups of volunteers and improvements identified as necessary after analysis of the previous trial sets. The data from the last set of tests were analyzed according to the following steps: synchronization and normalization, followed by experimental evaluation. In the evaluations, the feasibility of building automatic video summarizers from the monitored characteristics of the viewers was studied, which suggested their viability, including the reduction in the number of characteristics. The relationship between facial expressions and emotional states of the participants was also assessed, which provided experimental evidence of such a link. The results also showed the statistical superiority of the automatic models of affective summarization to a random selection. It was concluded, experimentally, that customized models for summarizing videos can provide results closer to the viewer’s preferences when compared to generic models.