Stochastic computing using simple logic circuits requires significantly less area and consumes less power compared to traditional computing systems. These circuits are also inherently fault-tolerant. The main drawbacks of these systems include long latency and inexactness in computing. The deviation from exact values increases as the correlation among inputs increases. In many applications, outputs from different sensors may be correlated. Thus, testing correctness of stochastic computing circuits requires generation of correlated stochastic bit streams. While uncorrelated bit streams can be generated using linear feedback shift registers (LFSRs), generation of correlated stochastic bit streams has not yet been fully investigated. This paper presents a general approach to synthesize correlated stochastic bit streams for specified probabilities and specified correlation coefficients. Generation of N correlated stochastic bit streams requires N probabilities and 2N - N - 1 correlation coefficients. Using N LFSRs, N uncorrelated stochastic bit streams are first generated. The N correlated bit streams are then generated one at a time using conditional marginal probabilities. The method is illustrated for generating two and three correlated bit streams. The area and power overheads for two correlated bit streams are 9.09% and 2.12%, respectively, and for three correlated bit streams are 21.03% and 4.80%, respectively. The generated sequences are applied to simple stochastic logic gates and the probability density functions (pdfs) of the outputs are derived. It is shown that these match with the theoretical pdfs of the outputs.