SU‐GG‐T‐445: Sequential Treatment of Brain Metastases Using a Markov Decision Process

Purpose: To use a Markov Decision Process (MDP) to model the sequential treatment of recurrent brain metastases. The purpose of the model is to calculate the optimal set of treatment decisions given the state of the patient. Improved systemic cancer control has resulted in growing numbers of patients who undergo multiple treatments for brain metastases. Given this situation, it is important to determine the optimal treatment decision at any point in time given the condition of the patient and the possibility of future treatments. Methods: Sequential stochastic decisions can be optimized with respect to a given reward using dynamic programming methods. A model of patients with brain metastases was constructed using a definition of state, a set of actions (whole brain RT, surgery, stereotactic radiosurgery, waiting), transition probabilities between states dependent on a given action, and a reward based on quality of life. Transition probabilities were obtained from the literature and clinical data. The problem was solved for both an infinite horizon, resulting in a stationary policy, and a fixed horizon, resulting in optimal decisions for each decision epoch. Results: The model was developed and validated by comparison with clinical judgment. Selection of adequate state variables and suitable rewards was made by comparing optimal policies with standard clinical situations. After the development phase, the model was used to study the role that possible subsequent treatments have on optimal decisions. The effects of knowledge regarding expected lifespan on optimal policies was investigated. In addition, comparisons of stationary policies with finite horizon policies were made. Conclusion: A first attempt at applying MDPs to radiation therapy decision making focussed on improving treatment strategies for metastatic brain cancer. An important application of this model will be to determine the extent to which improvements in our knowledge of a patient's prognosis will alter recommended therapies.