While many drug delivery systems target constant, or zero-order drug release, certain drugs and hormones must be delivered in rhythmic pulses in order to achieve their optimal effect. Here we describe studies with two model autonomous rhythmic delivery systems. The first system is driven by a pH oscillator that modulates the ionization state of a model drug, benzoic acid, which can permeate through a lipophilic membrane when the drug is uncharged. The second system is based on a nonlinear negative feedback instability that arises from coupling of swelling of a hydrogel membrane to an enzymatic reaction, with the hydrogel controlling access of substrate to the enzyme, and the enzyme's product controlling the hydrogel's swelling state. The latter system, whose autonomous oscillations are driven by glucose at constant external activity, is shown to deliver gonadotropin releasing hormone (GnRH) in rhythmic pulses, with periodicity of the same order as observed in sexually mature adult humans. Relevant experimental results and some mathematical models are reviewed.