Few inhibitors exist for CD38, a multifunctional enzyme catalyzing the formation and metabolism of the Ca2+-mobilizing second messenger cyclic adenosine 5′-diphosphoribose (cADPR). Synthetic, non-hydrolyzable ligands can facilitate structure-based inhibitor design. Molecular docking was used to reproduce the crystallographic binding mode of cyclic inosine 5′-diphosphoribose (N1-cIDPR) with CD38, revealing an exploitable pocket and predicting the potential to introduce an extra hydrogen bond interaction with Asp-155. The purine C-8 position of N1-cIDPR (IC50 276 μM) was extended with an amino or diaminobutane group and the 8-modified compounds were evaluated against CD38-catalyzed cADPR hydrolysis. Crystallography of an 8-amino N1-cIDPR:CD38 complex confirmed the predicted interaction with Asp-155, together with a second H-bond from a realigned Glu-146, rationalizing the improved inhibition (IC50 56 μM). Crystallography of a complex of cyclic ADP-carbocyclic ribose (cADPcR, IC50 129 μM) with CD38 illustrated that Glu-146 hydrogen bonds with the ligand N6-amino group. Both 8-amino N1-cIDPR and cADPcR bind deep in the active site reaching the catalytic residue Glu-226, and mimicking the likely location of cADPR during catalysis. Substantial overlap of the N1-cIDPR "northern" ribose monophosphate and the cADPcR carbocyclic ribose monophosphate regions suggests that this area is crucial for inhibitor design, leading to a new compound series of N1-inosine 5′-monophosphates (N1-IMPs). These small fragments inhibit hydrolysis of cADPR more efficiently than the parent cyclic compounds, with the best in the series demonstrating potent inhibition (IC50 = 7.6 μM). The lower molecular weight and relative simplicity of these compounds compared to cADPR make them attractive as a starting point for further inhibitor design.