Leitura e compreensão de textos multimodais: análise do gênero infográfico por meio da técnica de rastreamento ocular

Abstract

Research about multimodal reading in infographics highlight the non-linear nature, since the reader would have autonomy to define the route of the reading. Contrary to this idea, this study argues that the selection of information is not a random task among readers. It is proposed that there are patterns in how information is selected. Thus, the study questions how do organization and content influence the patterns of information selection in the cognitive processing of reading the infographic genre? The general objective is to examine the order of information processing in the multimodal infographic genre, by testing, under controlled conditions, how the articulation of linguistic and visual elements contributed to the process of reading and comprehension. Quantitative and experimental in its nature, the study gathers theoretical presuppositions about the brain functioning – Dehaene (2012; 2013); reading and comprehension of texts – Perfetti (1999), Perfetti et al. (2013), Perfetti; Hélder (2022); imagetext integration – Isola-Lanzoni (2022), Kress; Van Leeuwen; multimodal texts reading – Farhat; Gonçalves-Segundo (2022), Paiva (2009; 2011; 2013) and Ribeiro (2016); – and about eye tracking and multimodality – Fragoso (2015), Liang (2013), Majooni et al. (2017) and Ströhl et al. (2017). The role of alignment and the type of predominance of information is analyzed in two experiments done independently. The first utilizes the eye tracking technic in 18 stimuli presented randomly to 76 students at the Federal University in Sergipe. The participants eye movements were monitored with a Tobii T120 eye tracker. In this experiment, five main results were identified: (1) the vertical alignment corresponds to the reading pattern – reading begins from top to bottom and from left to right; (2) infographics with verbal content and vertical alignment showed a higher frequency of staring and time looking at areas of interest, in comparison with other alignments. On the other hand, infographics with visual content present less looking time, no matter the alignment; (3) there is a preferred reading path for readers when interpreting infographics – university students tend to follow a specific order among the different sections of the text, especially in vertical alignment. In texts aligned both horizontally and vertically, there are differences in the fixation time between the sections. However, in radial alignment there is a tendency to distribute attention evenly; (4) the position of the image impacts on the attention of the reader. Nonetheless, these impacts may be influenced depending on the type of image used and the type of infographic alignment; (5) content with intense emotion may alter, in some cases, the reading path; however, the results suggest that vertical alignment is less sensitive to that variation. The second test evaluated the reading comprehension of infographics, in 16 stimuli, distributed in a Latin square, which were presented to 155 graduate students at the Federal University in Sergipe. The experiment was produced by the Lab.Js software (Henninger et al. 2022) and managed by the JATOS software (Lange et al., 2015). The results suggest that the organization and the type of predominant information influence the performance of the participants in the comprehension of the infographics. Moreover, the reading time is longer for infographics with horizontal alignment and those with a predominance of visual information. In this context, the research becomes relevant as it highlights the influence of organization and content on information processing during the reading and comprehension of infographics. The results obtained can be particularly valuable for design professionals, educators and researchers who are interested in the interconnection among cognition, reading and multimodality.