We present evidence for nuclear spin-lattice relaxation driven by glassy nematic fluctuations in isovalent P-doped BaFe2As2 single crystals. Both the As75 and P31 sites exhibit a stretched-exponential relaxation similar to the electron-doped systems. By comparing the hyperfine fields and the relaxation rates at these sites we find that the As relaxation cannot be explained solely in terms of magnetic spin fluctuations. We demonstrate that nematic fluctuations couple to the As nuclear quadrupolar moment and can explain the excess relaxation. These results suggest that glassy nematic dynamics are a common phenomenon in the iron-based superconductors.