Neural Network Independence Properties with Applications to Adaptive Control

Andrew Lamperski

doi:10.1109/CDC51059.2022.9992994

Neural Network Independence Properties with Applications to Adaptive Control

Andrew Lamperski

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

Neural networks form a general purpose architecture for machine learning and parameter identification. The simplest neural network consists of a single hidden layer connected to a linear output layer. It is often assumed that the components of the hidden layer correspond to linearly independent functions, but proofs of this are only known for a few specialized classes of network activation functions. This paper shows that for wide class of activation functions, including most of the commonly used activation functions in neural network libraries, almost all choices of hidden layer parameters lead to linearly independent functions. These linear independence properties are then used to derive sufficient conditions for persistence of excitation, a condition commonly used to ensure parameter convergence in adaptive control.

Original language	English (US)
Title of host publication	2022 IEEE 61st Conference on Decision and Control, CDC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3365-3370
Number of pages	6
ISBN (Electronic)	9781665467612
DOIs	https://doi.org/10.1109/CDC51059.2022.9992994
State	Published - 2022
Externally published	Yes
Event	61st IEEE Conference on Decision and Control, CDC 2022 - Cancun, Mexico Duration: Dec 6 2022 → Dec 9 2022

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2022-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	61st IEEE Conference on Decision and Control, CDC 2022
Country/Territory	Mexico
City	Cancun
Period	12/6/22 → 12/9/22

Bibliographical note

Funding Information:
This work was supported in part by NSF CMMI-2122856 A. Lamperski is with the department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, USA

Publisher Copyright:
© 2022 IEEE.

Publisher link

10.1109/CDC51059.2022.9992994

Find It

Find It at U of M Libraries

Cite this

Neural Network Independence Properties with Applications to Adaptive Control. / Lamperski, Andrew.
2022 IEEE 61st Conference on Decision and Control, CDC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 3365-3370 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2022-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lamperski, A 2022, Neural Network Independence Properties with Applications to Adaptive Control. in 2022 IEEE 61st Conference on Decision and Control, CDC 2022. Proceedings of the IEEE Conference on Decision and Control, vol. 2022-December, Institute of Electrical and Electronics Engineers Inc., pp. 3365-3370, 61st IEEE Conference on Decision and Control, CDC 2022, Cancun, Mexico, 12/6/22. https://doi.org/10.1109/CDC51059.2022.9992994

@inproceedings{9b8bfbfd96374f98bbccaf76ff4f1c63,

title = "Neural Network Independence Properties with Applications to Adaptive Control",

abstract = "Neural networks form a general purpose architecture for machine learning and parameter identification. The simplest neural network consists of a single hidden layer connected to a linear output layer. It is often assumed that the components of the hidden layer correspond to linearly independent functions, but proofs of this are only known for a few specialized classes of network activation functions. This paper shows that for wide class of activation functions, including most of the commonly used activation functions in neural network libraries, almost all choices of hidden layer parameters lead to linearly independent functions. These linear independence properties are then used to derive sufficient conditions for persistence of excitation, a condition commonly used to ensure parameter convergence in adaptive control.",

author = "Andrew Lamperski",

note = "Funding Information: This work was supported in part by NSF CMMI-2122856 A. Lamperski is with the department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, USA Publisher Copyright: {\textcopyright} 2022 IEEE.; 61st IEEE Conference on Decision and Control, CDC 2022 ; Conference date: 06-12-2022 Through 09-12-2022",

year = "2022",

doi = "10.1109/CDC51059.2022.9992994",

language = "English (US)",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3365--3370",

booktitle = "2022 IEEE 61st Conference on Decision and Control, CDC 2022",

}

TY - GEN

T1 - Neural Network Independence Properties with Applications to Adaptive Control

AU - Lamperski, Andrew

N1 - Funding Information: This work was supported in part by NSF CMMI-2122856 A. Lamperski is with the department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, USA Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - Neural networks form a general purpose architecture for machine learning and parameter identification. The simplest neural network consists of a single hidden layer connected to a linear output layer. It is often assumed that the components of the hidden layer correspond to linearly independent functions, but proofs of this are only known for a few specialized classes of network activation functions. This paper shows that for wide class of activation functions, including most of the commonly used activation functions in neural network libraries, almost all choices of hidden layer parameters lead to linearly independent functions. These linear independence properties are then used to derive sufficient conditions for persistence of excitation, a condition commonly used to ensure parameter convergence in adaptive control.

AB - Neural networks form a general purpose architecture for machine learning and parameter identification. The simplest neural network consists of a single hidden layer connected to a linear output layer. It is often assumed that the components of the hidden layer correspond to linearly independent functions, but proofs of this are only known for a few specialized classes of network activation functions. This paper shows that for wide class of activation functions, including most of the commonly used activation functions in neural network libraries, almost all choices of hidden layer parameters lead to linearly independent functions. These linear independence properties are then used to derive sufficient conditions for persistence of excitation, a condition commonly used to ensure parameter convergence in adaptive control.

UR - http://www.scopus.com/inward/record.url?scp=85146985211&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85146985211&partnerID=8YFLogxK

U2 - 10.1109/CDC51059.2022.9992994

DO - 10.1109/CDC51059.2022.9992994

M3 - Conference contribution

AN - SCOPUS:85146985211

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 3365

EP - 3370

BT - 2022 IEEE 61st Conference on Decision and Control, CDC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 61st IEEE Conference on Decision and Control, CDC 2022

Y2 - 6 December 2022 through 9 December 2022

ER -

Neural Network Independence Properties with Applications to Adaptive Control

Abstract

Publication series

Conference

Bibliographical note

Publisher link

Find It

Other files and links

Fingerprint

Cite this