Stochastic computing is an alternative approach to conventional real arithmetic. A stochastic computing module is a digital system that operates on random bit streams representing real numbers. The success of stochastic computing relies on the efficient generation of random bit streams encoding real values in the unit interval. We present the design of random bit stream generators based on finite state machines (FSMs) that emulate Reversible Markov chains. We develop a general synthesis method to designs FSMs for generating arbitrary probabilities with finite resolution. We show that our method uses fewer input random sources for the constant random bit streams needed in a computation compared to the previous work. We further show that the output random bit stream quality and convergence times of our FSMs are reasonable.