Simulation of population dynamics using continuous-time finite-state Markov chains

This paper is concerned with the simulation of certain types of population dynamics using Markov chains. A brief review of the Markovian property, the Chapman-Kolmogorov equation, and the forward equation together with its solution is given. The essentials in simulating continuous-time Markov chains are provided and two case studies are presented for demonstration. The first is a process of drug delivery for which a closed-form solution of the forward equation can also be obtained. The second one deals with cell population dynamics, for which a Markovian model capturing birth, death, and growth is developed and simulated. The key to such simulation studies is to specify the generator Q, which in turn requires a thorough understanding of the properties of the underlying system. In return, one obtains not only the means and variances, but also the entire probability distributions, and the evolution of the random processes of interest.