As they navigate their environment, animals readily learn to repeat actions which yield reward and avoid actions which cause punishment. This kind of learning is especially efficient when reward or punishment is surprising. Dopamine neurons in the midbrain, which are excited by reward and inhibited by punishment, are believed to be crucial for learning from surprising outcomes. Dopamine signals are suppressed by expectation, and thus convey information about outcomes to other brain areas only when the outcomes are surprising.
Following this observation, the activity pattern of dopamine is typically formulated as actual outcome value minus expected value, which is called reward prediction error (RPE). How dopamine neurons generate RPE remains unclear. Our recent study (Amo et al., 2024) in the journal Neuron (PDF) revealed how different neurotransmitters collaborate to bring dopamine neurons the information necessary to encode RPE.
In the brain, neurons communicate with each other via excitatory, inhibitory, and modulatory neurotransmitters. Dopamine neurons receive inputs from both glutamate (excitatory) and GABA (inhibitory) neurons. Because dopamine neurons signal RPE, many previous models proposed that glutamate neurons convey actual outcome value while GABA neurons convey expected value, so that dopamine neurons can compute RPE by combining these inputs. However, the exact information glutamate and GABA inputs send to dopamine neurons has not been fully defined due to technical limitations.
Fully characterizing the mechanism for RPE computation in dopamine neurons requires multipronged approach. In our previous study (Tian et al., 2016), we directly recorded the activity of neurons that project to dopamine neurons. We found that information for both actual outcome value and expected value is distributed throughout multiple brain areas rather than localized in specific regions. A complementary approach is to directly examine what net information dopamine neurons receive. The recent construction of genetically encoded glutamate sensors allowed us to take the latter strategy to further understand RPE computation.
We used fiber-fluorometry to simultaneously record bulk glutamate sensor signals in dopamine neurons and dopamine release in behaving mice. We were surprised to find that glutamate inputs send a full RPE signal to dopamine neurons, rather than sending only the actual outcome value as many models proposed. Combined with reports that GABA neurons in some brain areas also send (oppositely signed) RPE to dopamine neurons, our findings indicate that excitatory and inhibitory inputs redundantly provide RPE to dopamine neurons.
Importantly, while reward responses were quite similar between glutamate inputs and dopamine release, we found critical differences in their responses to punishment. While dopamine neurons are inhibited by multiple forms of punishment, including loss of reward and aversive events, glutamate inputs are inhibited by reward loss, but are activated by aversive events. Thus for dopamine neurons to produce inhibitory responses to aversive events, GABA inputs must cancel out glutamate inputs. Taken together, current evidence suggests that excitatory and inhibitory inputs work synergistically to generate reward-related RPE in dopamine neurons, while excitatory and inhibitory inputs compete to signal aversiveness.
This dissociation between reward and aversion signaling suggests that dopamine responses to aversive events can be flexibly modulated without affecting reward signaling, just by altering the weights of excitatory and inhibitory inputs. Indeed, we found that the inhibitory responses of dopamine neurons to aversive events decreased or even flipped to excitation with administration of common opioid analgesics, which inhibit the activity of GABA inputs but do not change the excitatory responses of glutamate inputs. Since inhibitory responses of dopamine neurons are important for learning negative outcomes, suppression of inhibitory responses to aversive events by opioids may block such learning and contribute to analgesia and addiction.
Our study uncovers synaptic mechanisms for computation of dopamine signals. This work will serve as a basis for understanding the neural mechanism of how we learn from reward and punishment and how this learning is disturbed in psychiatric disorders such as addiction. We are currently building on our work by investigating what information is conveyed from site-specific glutamate inputs to dopamine neurons and how excitatory/inhibitory input balance onto dopamine neurons is controlled.
by Ryunosuke Amo and Mitsuko Watabe-Uchida
