Incremental supervised leaming localized updates in nonlocalized networks

We present a novel yet simple approach to incremental learning in neural networks: The problem of updating a mapping based on limited new data. The approach consists of forming a training set by appending to the new data additional training examples generated by exercising the network. This strategy enables the mapping to be updated in the neighborhood of the new data without causing distortions elsewhere in the input space. The approach can be used with any neural network model; it is particularly useful for the popular multilayer sigmoidal networks in which small parameter changes can have nonlocal consequences. Demonstrations and parametric explorations on a toy problem are described.