Visual attention models and applications to 3D computer graphics

Abstract

3D computer graphics, with the increasing technological and computational opportunities, have advanced to very high levels that it is possible to generate very realistic computer-generated scenes in real-time for games and other interactive environments. However, we cannot claim that computer graphics research has reached to its limits. Rendering photo-realistic scenes still cannot be achieved in real-time; and improving visual quality and decreasing computational costs are still research areas of great interest. Recent e orts in computer graphics have been directed towards exploiting principles of human visual perception to increase visual quality of rendering. This is natural since in computer graphics, the main source of evaluation is the judgment of people, which is based on their perception. In this thesis, our aim is to extend the use of perceptual principles in computer graphics. Our contribution is two-fold: First, we present several models to determine the visually important, salient, regions in a 3D scene. Secondly, we contribute to use of de nition of saliency metrics in computer graphics. Human visual attention is composed of two components, the rst component is the stimuli-oriented, bottom-up, visual attention; and the second component is task-oriented, top-down visual attention. The main di erence between these components is the role of the user. In the top-down component, viewer's intention and task a ect perception of the visual scene as opposed to the bottom-up component. We mostly investigate the bottom-up component where saliency resides. We de ne saliency computation metrics for two types of graphical contents. Our rst metric is applicable to 3D mesh models that are possibly animating, and it extracts saliency values for each vertex of the mesh models. The second metric we propose is applicable to animating objects and nds visually important objects due to their motion behaviours. In a third model, we present how to adapt the second metric for the animated 3D meshes. Along with the metrics of saliency, we also present possible application areas and a perceptual method to accelerate stereoscopic rendering, which is based on binocular vision principles and makes use of saliency information in a stereoscopic rendering scene. Each of the proposed models are evaluated with formal experiments. The proposed saliency metrics are evaluated via eye-tracker based experiments and the computationally salient regions are found to attract more attention in practice too. For the stereoscopic optimization part, we have performed a detailed experiment and veri ed our model of optimization. In conclusion, this thesis extends the use of human visual system principles in 3D computer graphics, especially in terms of saliency.