Infinite Horizon Problems

The systems under consideration in (discrete-time) sequential decision problems in operations research and the management sciences often do not have a predetermined time of extinction. Incorporating an arbitrary finite horizon can therefore introduce end-of-study distortions in early decisions. Such problems are therefore typically modeled over an unbounded horizon. A majority of the work in this area focuses on stationary models, which assume that the problem data do not change over time. There is also a considerable body of research on nonstationary problems. We briefly review some of the key concepts in nonstationary infinite horizon sequential decision making problems.