Ambulance redeployment comprises the problem of deploying ambulances to certain locations in order to minimize the arrival times to possible calls and plays a significant role in improving a country's emergency medical services and increasing the number of lives saved during an emergency. In this study, unlike the existing optimization methods in the literature, the problem is cast as a multi-armed bandit problem. Multi-armed bandit problems are a part of sequential online learning methods and utilized in maximizing a gain function (i.e. reward) when the reward distributions are unknown. In this study, in addition to the objective of maximizing rewards, the objective of minimizing the expected variance of rewards is also considered. The effect of risk taken by the system on average arrival times and number of calls responded on time is investigated. Ambulance redeployment is performed by a risk-averse multi-armed bandit algorithm on a data-driven simulator. As a result, it is shown that the algorithm which takes less risk (i.e. that minimizes the variance of response times) responds to more cases on time.