TY - JOUR

T1 - Permitted Sets and Convex Coding in Nonthreshold Linear Networks

AU - Collazos, Steven

AU - Nykamp, Duane

N1 - Publisher Copyright:
© 2022 Massachusetts Institute of Technology.

PY - 2022/8/16

Y1 - 2022/8/16

N2 - Hebbian theory proposes that ensembles of neurons form a basis for neural processing. It is possible to gain insight into the activity patterns of these neural ensembles through a binary analysis, regarding neurons as either active or inactive. The framework of permitted and forbidden sets, introduced by Hahnloser, Seung, and Slotine (2003), is a mathematical model of such a binary analysis: groups of coactive neurons can be permitted or forbidden depending on the network’s structure. In order to widen the applicability of the framework of permitted sets, we extend the permitted set analysis from the original threshold-linear regime. Specifically, we generalize permitted sets to firing rate models in which ϕ is a nonnegative continuous piecewise C1 activation function. In our framework, the focus is shifted from a neuron’s firing rate to its responsiveness to inputs; if a neuron’s firing rate is sufficiently sensitive to changes in its input, we say that the neuron is responsive. The algorithm for categorizing a neuron as responsive depends on thresholds that a user can select arbitrarily and that are independent of the dynamics. Given a synaptic weight matrix W, we say that a set of neurons is permitted if it is possible to find a stimulus where those neurons, and no others, remain responsive. The main coding property we establish about Pϕ (W ), the collection of all permitted sets of the network, is that Pϕ (W ) is a convex code when W is almost rank one. This means that Pϕ (W ) in the low-rank regime can be realized as a neural code resulting from the pattern of overlaps of receptive fields that are convex.

AB - Hebbian theory proposes that ensembles of neurons form a basis for neural processing. It is possible to gain insight into the activity patterns of these neural ensembles through a binary analysis, regarding neurons as either active or inactive. The framework of permitted and forbidden sets, introduced by Hahnloser, Seung, and Slotine (2003), is a mathematical model of such a binary analysis: groups of coactive neurons can be permitted or forbidden depending on the network’s structure. In order to widen the applicability of the framework of permitted sets, we extend the permitted set analysis from the original threshold-linear regime. Specifically, we generalize permitted sets to firing rate models in which ϕ is a nonnegative continuous piecewise C1 activation function. In our framework, the focus is shifted from a neuron’s firing rate to its responsiveness to inputs; if a neuron’s firing rate is sufficiently sensitive to changes in its input, we say that the neuron is responsive. The algorithm for categorizing a neuron as responsive depends on thresholds that a user can select arbitrarily and that are independent of the dynamics. Given a synaptic weight matrix W, we say that a set of neurons is permitted if it is possible to find a stimulus where those neurons, and no others, remain responsive. The main coding property we establish about Pϕ (W ), the collection of all permitted sets of the network, is that Pϕ (W ) is a convex code when W is almost rank one. This means that Pϕ (W ) in the low-rank regime can be realized as a neural code resulting from the pattern of overlaps of receptive fields that are convex.

UR - http://www.scopus.com/inward/record.url?scp=85136909120&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85136909120&partnerID=8YFLogxK

U2 - 10.1162/neco_a_01523

DO - 10.1162/neco_a_01523

M3 - Letter

C2 - 35896154

AN - SCOPUS:85136909120

SN - 0899-7667

VL - 34

SP - 1978

EP - 2008

JO - Neural computation

JF - Neural computation

IS - 9

ER -