Dopamine plays an outsized role in the public imagination, acting as a ‘happiness’ chemical, the drug that causes psychosis, or the pill that allows frozen people to move again, as in Oliver Sacks’ famous book, Awakenings. But 20 years ago, experiments with monkeys revealed a more specific role for dopamine: comparing outcomes with expectations. When an outcome is better than expected, dopamine neurons increase their activity. When an outcome is completely expected, dopamine neurons do not respond. And when an outcome is worse than expected, dopamine neurons go silent. This pattern of responses is deemed ‘reward prediction error’ and is thought to be a crucial way that we learn from our experiences. Positive prediction errors reinforce actions that lead to reward, while negative prediction errors prevent actions that lead to punishment.
In our new study, published this week in Nature Neuroscience, we explore how individual dopamine neurons make this calculation. Surprisingly, we discover that each neuron calculates prediction error in exactly the same way. Such a system is exceptionally robust and redundant, ensuring that the prediction error signal can be exploited by the broadest possible array of brain circuits to help us learn.
We recorded from neurons deep in the brain while thirsty mice performed simple tasks for water reward. Sometimes we delivered water out of the blue, completely unexpectedly. Other times we presented an odor that predicted water delivery. Every time this odor was presented, the mouse learned to expect water at a particular time in the future. By delivering different amounts of water, with or without the preceding odor, we could measure the precise method that dopamine neurons used to calculate prediction error. We then compared this method from neuron to neuron.
We found that dopamine neurons calculate prediction error through simple subtraction. This is consistent with previous computational theories, but quite rare to find in the brain. In most other settings, neurons appear to work through multiplication or division, rather than addition or subtraction. In this case, though, subtraction is the best method for a precise calculation, and the brain appears to have evolved accordingly.
Moreover, each neuron appears to perform this subtraction in exactly the same way. This is even true for dopamine neurons recorded on different days, from different mice. The only difference between neurons was in the magnitude of their responses to unexpected rewards. Given this information, the rest of that neuron’s response was perfectly predictable. Indeed, even the ‘noise’ in dopamine neurons’ responses—that is, the different activity they exhibit from trial to trial, when the stimuli remain the same—was correlated from neuron to neuron. This has two profound implications: 1) that different dopamine neurons likely have overlapping inputs, and 2) that the targets of dopamine release likely receive similar information, regardless of which dopamine neurons they contact.
The homogeneity of dopamine neuron responses reinforces the idea that dopamine neurons broadcast a common signal to the rest of the brain: namely, prediction error. Even if a group of dopamine neurons were to die, the signal would persist. Thus, the system beautifully ensures our ability to perform one fundamental task: learning from our experience.
Read more in Nature Neuroscience or download PDF
Download PDF of Nature News and Views
