Predicting interest points of virtual characters and accurately simulating their gaze behavior play a significant role for realistic crowd simulations. We propose a saliency model that enables virtual agents to produce plausible gaze behavior. The model measures the effects of distinct saliency features implemented by examining the state-of-the-art perception studies. When predicting an agent's interest point, we compute the saliency scores by using a weighted sum function for other agents and environment objects in the field of view of the agent for each frame. Then, we determine the most salient entity for each agent in the scene; thus, agents gain a visual understanding of their environment. Besides, our model introduces new aspects to crowd perception, such as perceiving characters as groups of people and applying social norms on crowd gaze behavior, effects of agent personality on gaze, gaze copy phenomena, and effects of agent velocity on attention. For evaluation, we compare the resulting saliency gaze model with real-world crowd behavior in captured videos. In the experiments, we simulate the gaze behavior in real crowds. The results show that the proposed approach generates plausible gaze behaviors and is easily adaptable to varying scenarios for virtual crowds.