A reinforcement learning model of precommitment in decision making

Zeb Kurth-Nelson, A. David Redish

    Research output: Contribution to journalArticle

    16 Scopus citations

    Abstract

    Addiction and many other disorders are linked to impulsivity, where a suboptimal choice is preferred when it is immediately available. One solution to impulsivity is precommitment: constraining one's future to avoid being offered a suboptimal choice. A form of impulsivity can be measured experimentally by offering a choice between a smaller reward delivered sooner and a larger reward delivered later. Impulsive subjects are more likely to select the smaller-sooner choice; however, when offered an option to precommit, even impulsive subjects can precommit to the larger-later choice. To precommit or not is a decision between two conditions: (A) the original choice (smaller-sooner vs. larger-later), and (B) a new condition with only larger-later available. It has been observed that precommitment appears as a consequence of the preference reversal inherent in non-exponential delay-discounting. Here we show that most models of hyperbolic discounting cannot precommit, but a distributed model of hyperbolic discounting does precommit. Using this model, we find (1) faster discounters may be more or less likely than slow discounters to precommit, depending on the precommitment delay, (2) for a constant smaller-sooner vs. larger-later preference, a higher ratio of larger reward to smaller reward increases the probability of precommitment, and (3) precommitment is highly sensitive to the shape of the discount curve. These predictions imply that manipulations that alter the discount curve, such as diet or context, may qualitatively affect precommitment.

    Original languageEnglish (US)
    Article number184
    JournalFrontiers in Behavioral Neuroscience
    Volume4
    Issue numberDEC
    DOIs
    StatePublished - Dec 14 2010

    Keywords

    • Addiction
    • Decision making
    • Delay discounting
    • Hyperbolic discounting
    • Impulsivity
    • Precommitment
    • Reinforcement learning

    Fingerprint Dive into the research topics of 'A reinforcement learning model of precommitment in decision making'. Together they form a unique fingerprint.

  • Cite this