What captures our attention? How communicative and non-communicative actions by human and robot agents influence visual attention under perceptual load

Abstract

This study investigated how agent type (human vs. robot), communicativeness (communicative vs. non-communicative), and perceptual load (low vs. high) interactively influence visual attention and task performance. A 2 (Agent) × 2 (Communicativeness) × 2 (Perceptual Load) within-subjects design was employed with 34 participants. To examine attentional allocation under perceptual load, Areas of Interest (AOIs) were defined for the central letter array and the peripheral agent videos. Behavioral performance (reaction time, accuracy) and multiple eye-tracking metrics (total dwell time, dwell time per fixation, first fixation latency, saccadic velocity, pupil dilation) were analyzed. Behavioral results confirmed the effect of the perceptual load manipulation, with participants being significantly less accurate and slower in high-loaded conditions. In the presence of a distractor, reaction times and accuracy were further modulated by significant interactions between agent, perceptual load, and communicativeness. Eye-tracking analyses revealed that initial orienting was driven by the communicativeness and perceptual load, when communicative cues captured attention faster only under high load. Dwell time per fixation remained stable across the conditions. On the other hand, total dwell time revealed a critical three-way interaction. Under high load, communicative actions performed by humans increase total dwell time, whereas if the action is performed by a robot, dwell time decreases. Pupil data supported this, indicating that under high load, non-communicative actions required more cognitive effort than the communicative ones. Initial orienting of eye metrics was guided by task demand; overall engagement with the nature of the action is a product of a complex and sensitive cognitive process sensitive to perceptual load. Robotic social cues are processed dissimilar compared to human ones.