The chronic phase of HIV infection is marked by pathological activation of the immune system, the extent of which better predicts disease progression than either plasma viral load or CD4+ T cell count. Recently, translocation of microbial products from the gastrointestinal tract has been proposed as an underlying cause of this immune activation, based on indirect evidence including the detection of microbial products and specific immune responses in the plasma of chronically HIV-infected humans or SIV-infected Asian macaques. We analyzed tissues from SIV-infected rhesus macaques (RMs) to provide direct in situ evidence for translocation of microbial constituents from the lumen of the intestine into the lamina propria and to draining and peripheral lymph nodes and liver, accompanied by local immune responses in affected tissues. In chronically SIV-infected RMs this translocation is associated with breakdown of the integrity of the epithelial barrier of the gastrointestinal (GI) tract and apparent inability of lamina propria macrophages to effectively phagocytose translocated microbial constituents. By contrast, in the chronic phase of SIV infection in sooty mangabeys, we found no evidence of epithelial barrier breakdown, no increased microbial translocation and no pathological immune activation. Because immune activation is characteristic of the chronic phase of progressive HIV/SIV infections, these findings suggest that increased microbial translocation from the GI tract, in excess of capacity to clear the translocated microbial constituents, helps drive pathological immune activation. Novel therapeutic approaches to inhibit microbial translocation and/or attenuate chronic immune activation in HIV-infected individuals may complement treatments aimed at direct suppression of viral replication.