This paper presents a novel framework for constrained adaptive learning in Reproducing Kernel Hilbert Spaces (RKHS). A low complexity algorithmic solution is established. Constraints that encode a-priori information and several design specifications take the form of multiple intersecting closed convex sets. A cost function and the training data stream create a sequence of closed convex sets in the RKHS. The resulting recursive solution generates a sequence of estimates which converges to such an infinite intersection of closed convex sets. A time-adaptive beamforming task in an RKHS, rich in constraints, is also established. The numerical results show that the proposed method exhibits a significant improvement in resolution, when compared to the classical linear solution, and outperforms a recently unconstrained online kernel-based regression technique.