Mode switching in organisms for solving explore-versus-exploit problems

Debojyoti Biswas, Andrew Lamperski, Yu Yang, Kathleen Hoffman, John Guckenheimer, Eric S. Fortune, Noah J. Cowan

Research output: Contribution to journalArticlepeer-review

Abstract

Trade-offs between producing costly movements for gathering information (‘explore’) and using previously acquired information to achieve a goal (‘exploit’) arise in a wide variety of problems, including foraging, reinforcement learning and sensorimotor control. Determining the optimal balance between exploration and exploitation is computationally intractable, necessitating heuristic solutions. Here we show that the electric fish Eigenmannia virescens uses a salience-dependent mode-switching strategy to solve the explore–exploit conflict during a refuge-tracking task in which the same category of movement (fore-aft swimming) is used for both gathering information and achieving task goals. The fish produced distinctive non-Gaussian distributions of movement velocities characterized by sharp peaks for slower, task-oriented ‘exploit’ movements and broad shoulders for faster ‘explore’ movements. The measures of non-normality increased with increased sensory salience, corresponding to a decrease in the prevalence of fast explore movements. We found the same sensory salience-dependent mode-switching behaviour across ten phylogenetically diverse organisms, from amoebae to humans, performing tasks such as postural balance and target tracking. We propose a state-uncertainty-based mode-switching heuristic that reproduces the distinctive velocity distribution, rationalizes modulation by sensory salience and outperforms the classic persistent excitation approach while using less energy. This mode-switching heuristic provides insights into purposeful exploratory behaviours in organisms, as well as a framework for more efficient state estimation and control of robots.

Original languageEnglish (US)
Pages (from-to)1285-1296
Number of pages12
JournalNature Machine Intelligence
Volume5
Issue number11
DOIs
StatePublished - Nov 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023, The Author(s).

Fingerprint

Dive into the research topics of 'Mode switching in organisms for solving explore-versus-exploit problems'. Together they form a unique fingerprint.

Cite this