Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment

Arif A. Hamid, Michael J. Frank, Christopher I. Moore

Research output: Contribution to journalArticlepeer-review

80 Scopus citations

Abstract

Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control and interacted with reward waves to predict future behavioral adjustments.

Original languageEnglish (US)
Pages (from-to)2733-2749.e16
JournalCell
Volume184
Issue number10
DOIs
StatePublished - May 13 2021

Bibliographical note

Funding Information:
We thank Matthew Nassar, Joshua Berke, Peter Dayan, and Theresa Desrochers for valuable discussion of the project and feedback on an earlier version of the manuscript and members of the Frank and Moore laboratories for feedback at various stages of the project. We also thank Ines Belghiti and Aneri Soni for help with adapting the MoE model implementation to the tone task. GCaMP6f and jRGECO1a were developed and made available by the HHMI Janelia GENIE Project. dLight was developed and made available by Lin Tian. This work was supported by HHMI Hanna Gray Fellowship to A.A.H.; NIH R01MH080066 and NSF grant 1460604 to M.J.F.; and awards from Carney Institute for Brain Sciences and Dean’s office at Brown University to C.I.M.

Publisher Copyright:
© 2021 Elsevier Inc.

Keywords

  • agency learning
  • basal ganglia
  • credit assignment
  • decisionmaking
  • dopamine
  • motivated behaviors
  • reinforcement learning
  • striatum

PubMed: MeSH publication types

  • Journal Article
  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't
  • Research Support, U.S. Gov't, Non-P.H.S.

Fingerprint

Dive into the research topics of 'Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment'. Together they form a unique fingerprint.

Cite this