Bypassing tyrosine kinases responsible for Stat5a/b phosphorylation would be advantageous for therapy development for Stat5a/b-regulated cancers. Here, we sought to identify small molecule inhibitors of Stat5a/b for lead optimization and therapy development for prostate cancer and Bcr-Abl-driven leukemias. In silico screening of chemical structure databases combined with medicinal chemistry was used for identification of a panel of small molecule inhibitors to block SH2 domain-mediated docking of Stat5a/b to the receptor-kinase complex and subsequent phosphorylation and dimerization. We tested the efficacy of the lead compound IST5-002 in experimental models and patient samples of two known Stat5a/b-driven cancers, prostate cancer and chronic myeloid leukemia (CML). The lead compound inhibitor of Stat5-002 (IST5-002) prevented both Jak2 and Bcr-Abl-mediated phosphorylation and dimerization of Stat5a/b, and selectively inhibited transcriptional activity of Stat5a (IC50 = 1.5μmol/L) and Stat5b (IC50 = 3.5 μmol/L). IST5-002 suppressed nuclear translocation of Stat5a/b, binding to DNA and Stat5a/b target gene expression. IST5-002 induced extensive apoptosis of prostate cancer cells, impaired growth of prostate cancer xenograft tumors, and induced cell death in patient-derived prostate cancers when tested ex vivo in explant organ cultures. Importantly, IST5-002 induced robust apoptotic death not only of imatinib-sensitive but also of imatinib-resistant CML cell lines and primary CML cells from patients. IST5-002 provides a lead structure for further chemical modifications for clinical development for Stat5a/b-driven solid tumors and hematologic malignancies.