Automatic determination of navigable areas, pedestrian detection, and augmentation of virtual agents in real crowd videos

Crowd simulations imitate the behavior of crowds and individual agents in the crowd with personality and appearance, which determines the overall model of a multi-agent system. In such studies, the models are often compared with real-life scenarios for assessment. Yet apart from side-by-side comparison and trajectory analysis, there are no practical, out-of-the-box tools to test how a given arbitrary model simulate the scenario that takes place in the real world. We propose a framework for augmenting virtual agents in real-life crowd videos. The framework locates the navigable areas on the ground plane using the automaticallyextracted detection data of the pedestrians in the crowd video. Then it places the three-dimensional (3D) models of real pedestrians in the 3D model of the scene. An interactive user interface is provided for users to add and control virtual agents, which are simulated together with detected real pedestrians using collision avoidance algorithms.