We present a system to simulate the movement of individual agents in large-scale crowds performing the Tawaf. The Tawaf serves as a unique test case. The crowd consists of a heterogeneous set of pilgrims, varying with respect to physical capacity as well as activity. Furthermore, the density of the crowd reaches very high levels. Our approach uses a finite state machine to specify the behavior of the agents at each time step in conjunction with a geometric, agent-based algorithm to specify how an agent interacts with its local neighbors to generate collision-free trajectories. The overall system can model agents with varying age, gender and behaviors, supporting the heterogeniety observed in the performance of the Tawaf, even at high densities.