Interruptions, tradeoffs, and temporal discounting

SYNOPSIS. Variance in amount of rewards has been the focus of many studies and models of risk sensitivity. However, the timing of rewards has received much less attention. Animals tend to prefer immediate rewards, even when this preference reduces the long term rate of gain. This implies future rewards are devalued, a phenomenpn known as time discounting. The value of a reward can be represented by a function that is decreasing and concave-up with increasing delays until receipt of reward. When a forager is given the choice between an immediate small reward and a delayed larger reward, preference reverses from the smaller to the larger as the delays to both rewards increase. There are several descriptive time discounting models. The exponential and the hyperbolic models both result in the concave-up value function and hence risk sensitivity with respect to timing. However, in the traditional models, only the hyperbolic model predicts preference reversal. We present a model of discounting as a continuous time Markov process. In this model, the discounting rate is determined by the perceived likelihood that the forager will be interrupted from the foraging process. In this continuous interruption rate model, we find both the concave-up value function and preference reversal. In an expanded model that accounts for the variable inter-trial intervals often used in tests of preference, we find that the two portions of the inter-trial interval are weighted differently. Finally, we generate a generalized fitness model to examine the influence of the fitness consequences of a given interruption on foraging choices and find that the type of trade-off involved determines the fitness outcome.