The inflationary paradigm has enjoyed phenomenological success; however, a compelling particle physics realization is still lacking. Axions are among the best-motivated inflaton candidates, since the flatness of their potential is naturally protected by a shift symmetry. We reconsider the cosmological perturbations in axion inflation, consistently accounting for the coupling to gauge fields cφFF, which is generically present in these models. This coupling leads to production of gauge quanta, which provide a new source of inflaton fluctuations, δ φ. For c>102Mp-1, these dominate over the vacuum fluctuations, and non-Gaussianity exceeds the current observational bound. This regime is typical for concrete realizations that admit a UV completion; hence, large non-Gaussianity is easily obtained in minimal and natural realizations of inflation.