We study a sedimentary delta prograding over a fixed adversely sloping bathymetry, asking whether a perturbation to the advancing shoreline will grow (unstable) or decay (stable) through time. To start, we use a geometric model to identify the condition for acceleration of the shoreline advance (auto-acceleration). We then model the growth of a delta on to a fixed adverse bathymetry, solving for the speed of the shoreline as a function of the water depth, foreset repose angle, fluvial top set slope, and shoreline curvature. Through a linearization of this model, we arrive at a stability criterion for a delta shoreline, indicating that auto-acceleration is a necessary condition for unstable growth. This is the first time such a shoreline instability has been identified and analyzed. We use the derived stability criterion to identify a characteristic lateral length scale for the shoreline morphology resulting from an unstable growth. On considering experimental and field conditions, we observe that this length scale is typically larger than other geomorphic features in the system, e.g., channel spacings and dimensions, suggesting that the signal of the shoreline growth instability in the landscape might be "shredded" by other surface building processes, e.g., channel avulsions and alongshore transport.