Rare earth metal borides have attracted great interest due to their unusual properties, such as superconductivity and f-electron magnetism. A recent discovery attributes the tunability of magnetism in rare earth aluminoborides to the effect of so-called "building defects." In this paper, we report data for the effect of building defects on the thermal conductivities of α-TmAlB4 single crystals. Building defects reduce the thermal conductivity of α-TmAlB4 by ≈30%. At room temperature, the thermal conductivity of AlB2 is nearly a factor of 5 higher than that of α-TmAlB4. AlB2 single crystals are thermally anisotropic with the c-axis thermal conductivity nearly twice the thermal conductivity of the a-b plane. Temperature dependence of the thermal conductivity near and above room temperature reveals that both electrons and phonons contribute substantially to thermal transport in AlB2 with electrons being the dominant heat carriers.