Bedrock incision plays a key role in determining the pace of landscape evolution. Much is known about how bedrock rivers incise vertically, but less is known about lateral erosion. Lateral erosion is widely thought to occur when the bed is alluviated, which prevents vertical erosion and deflects the downstream transport of bedload particles into channel walls. Here we develop a model for lateral erosion by bedload particle impacts. The lateral erosion rate is the product of the volume eroded per particle impact and the impact rate. The volume eroded per particle impact is modeled by tracking the motion of bedload particles from collision with roughness elements to impacts on the wall. The impact rate on the wall is calculated from deflection rates on roughness elements. The numerical model further incorporates the coevolution of wall morphology, shear stress, and erosion rate. The model predicts the undercut wall shape observed in physical experiments. The nondimensional lateral erosion rate is used to explore how lateral erosion varies under different relative sediment supply (ratio of supply to transport capacity) and transport stage conditions. Maximum lateral erosion rates occur at high relative sediment supply rates (~0.7) and moderate transport stages (~10). The competition between lateral and vertical erosion is investigated by coupling the saltation-abrasion vertical erosion model with our lateral erosion model. The results suggest that vertical erosion dominates under near 75% of supply and transport stage conditions but is outpaced by lateral erosion near the threshold for full bed coverage.