Background Intraoperative radiation therapy (IORT) has been proposed as a tool to improve local control in patients with locally advanced primary or recurrent colorectal cancer. Methods A retrospective review (1999–2015) of all patients undergoing IORT for locally advanced or recurrent colorectal cancer at a single academic center was performed. Patient demographics, oncologic staging, short-term and long-term outcomes were reviewed. Results There were 77 patients (mean age 63 ± 11 years) identified, of whom 19 had colon cancer, 57 had rectal cancer, and 2 had appendiceal cancers. R0 resection was performed in 53 patients (69%), R1 in 19 (25%) and R2 in 5 (6%). Ten (13%) patients had a local recurrence at 18 ± 14 months and 34 (44%) had a distant recurrence at 18 ± 18 months. Mean survival was 47 ± 41 months. Conclusion IORT resulted in low local failure rates and should be considered for patients with locally advanced or recurrent colorectal cancers.