A new study from Naoshige Uchida’s MCB lab provides new insight into how the brain processes contingency during associative learning. Published in Nature Neuroscience (PDF), the study demonstrates that dopamine signals and behavior are shaped by prospective contingency—the information a cue provides about the change in likelihood of receiving a future reward, not simply reward frequency. The concept of contingency is crucial in understanding learning, behavior, and decision-making processes.
Contingency has long been understood to be a key determinant of associative learning. In his famous experiments, Ivan Pavlov paired a cue, a bell, with food, a reward. After repeated pairings, the bell alone triggered salivation in the dog. But later in the 20th century, Robert Rescorla found that this pairing was not sufficient. Working with rats, he found presenting a cue when delivering an electric shock caused the rats to freeze to the cue. But if shocks were delivered both with and without the cue, the rats no longer froze to the cue. He explained this phenomenon using the notion of contingency: it is only when a cue predicts an increase or decrease in an outcome does an animal learn to respond to the cue.
To investigate the neural underpinnings of this phenomenon, the team designed experiments where mice were trained to associate odor cues with water rewards. The key finding was that increasing background rewards—rewards delivered without a preceding cue—diminished both behavioral responses (licking) and dopamine signaling. However, when these additional rewards were paired with their own cue, the decrease in dopamine response was prevented. “It’s the same amount of reward; we’re just adding a new cue and seeing big differences in both behavior and dopamine,” said co-first author Mark Burrell, a postdoctoral fellow in Uchida’s lab. This result suggests that the brain treats cued and uncued rewards differently, highlighting how the mice’s understanding of the task affects their learning. The study examined multiple brain areas, including the ventral striatum and olfactory tubercle, ultimately finding that contingency-related dopamine signals were broadly distributed across all of them.
The study used the temporal difference (TD) model, a long-standing explanation that links dopamine function to learning, to explain not only the main findings but all the observed changes in the behavior and dopamine signals. The temporal difference model, originally developed as a machine-learning algorithm, supposes that prediction errors, the difference between actual and predicted outcomes, drives learning. The activity of dopamine neurons closely resembles these prediction errors.
“This study is really a story about being careful and thinking about the structure of the world—the way we represent state space in our models is crucial,” Burrell emphasized. He pointed out that while some recent theories have proposed fundamentally new roles for dopamine, the findings reaffirm the TD model with only minor modifications. “These models still work, and identifying what you need to add to make them fit the data informs how we should continue thinking about learning and decision-making.” A recent model (Jeong et al., 2022, Science), offered as a replacement to the TD explanation of dopamine function, proposed dopamine signals retrospective contingency. “Our results strongly challenge this model as it could not explain our results” Burrell commented. “Rather, we show that with the right state space, the TD model, which is prospective in nature, is itself a good description of contingency.”
Working with Jay Hennig, then a postdoctoral fellow in Samuel Gershman’s lab and now an assistant professor at Baylor College of Medicine, the study found that small artificial neural networks trained with TD learned similar representations of their task to their mice. This suggests that dopamine TD signals may shape how the brain represents state space.
The implications extend beyond associative learning. The study suggests that the brain learns differently when rewards are attributed to discrete cues rather than being part of the environment. This insight may be relevant to understanding how humans and animals use contingency to infer causal relationships in complex environments. “We are certainly moving in that direction,” Burrell said, though he cautioned against overextending the findings. “I wouldn’t want to say we’ve solved causality, but this is a step toward understanding how the brain links cause and effect.”
This study refines our understanding of how animals—and potentially humans—navigate an uncertain world by demonstrating that contingency, rather than mere reward frequency, shapes learning and dopamine responses. “Humans and animals are good at inferring causal relationships between events, yet how the brain achieves it remains unclear,” adds Professor Uchida. “I hope our study triggers novel inquiries into this direction.”.
The project began as a three-way lab collaboration centered around part of Selina Qian’s PhD work, which was co-supervised by Uchida and MCB’s Venkatesh Murthy, and the lab of Samuel Gershman of the Department of Psychology. Burrell later joined Uchida’s lab to expand the modeling aspect of the project.
