Magnetization fluctuations caused by thermal energies are poorly understood, yet they represent a fundamental limit for many magnetic technologies. Here, thermal fluctuations of the magnetization are quantitatively characterized. The model magnetic system is thin square magnetic dots (<10nm) with side lengths down to 200 nm. The magnetic state and the fluctuations are determined in individual dots by measuring the anisotropic magnetoresistance. The magnetization exhibits random telegraph noise with a transition rate determined by the dot size, the applied magnetic field, and the sample temperature. An energy barrier model of the system explains the dependence of the magnetic fluctuations on these parameters and is supported by additional experiments unaffected by the magnetic fluctuations. This independent identification of the mechanism producing magnetic noise, and its agreement with micromagnetic simulations, demonstrates a microscopic understanding of noise extendable to more complex noise sources such as 1/f noise.