Management of chronic diseases such as diabetes mellitus requires adaptation of treatment regimes based on patient characteristics and response. There is no single treatment that fits all patients in all contexts; moreover, the set of admissible treatments usually varies over the course of the disease. In this thesis, we address the problem of optimizing treatment regimes under time-varying constraints by using volatile contextual Gaussian process bandits. In particular, we propose a variant of GP-UCB with volatile arms, which takes into account the patient’s context together with the set of admissible treatments when recommending new treatments. Our Bayesian approach is able to provide treatment recommendations to the patients along with confidence scores which can be used for risk assessment. We use our algorithm to recommend bolus insulin doses for type 1 diabetes mellitus patients. We test our algorithm on in-silico subjects that come with open source implementation of the FDA-approved UVa/Padova type 1 diabetes mellitus simulator. We also compare its performance against a clinician. Moreover, we present a pilot study with a few clinicians and patients, where we design interfaces that they can interact with the model. Meanwhile, we address issues regarding privacy, safety, and ethics. Simulation studies show that our algorithm compares favorably with traditional blood glucose regulation methods.