ASSOCIATIVE MEMORY STORAGE USING A VARIANT OF THE GENERALIZED DELTA RULE.

An n-bit associative memory is constructed using a network of n sigmoid units. Each unit is connected to every other unit with unidirectional weight, and to itself. A threshold is also associated with each unit. The presence of self-weights allows the storage of memories that do not conform to the linear predictability constraint, such as parity-encoded memories. The storage of memories is effected by assigning appropriate values to the weights and thresholds. A variant of the generalized delta rule, which has been used previously in back-propagation, is used for determining these values. The present network is distinct from the back-propagation model in two ways: it consists of one rather than several layers, and it has feedback connections. A consequence of the feedback connections is that a relaxation phase is required in which the unit values are iteratively recomputed until they stabilize. A relaxation procedure analogous to simulated annealing but applicable to networks of real-valued units is described that greatly improves the performance of the memory. Various performance figures are presented.