As a postdoctoral researcher at Cold Spring Harbor, Naoshige (Nao) Uchida adapted a method for studying decision-making behavior, originally developed for primates, for rodents. (see video below) Rodent experiments would be less expensive and faster than primate experiments, and he could also employ the molecular biology techniques and genetics that were available in rodents.
He felt this new paradigm could allow the precise study of how animals value an experience, how they store information about its reward, and how they regulate decision-making. But at the time, he recalls, there was not much interest in rodent behavior – with the exception of Harvard’s Department of Molecular and Cellular Biology and the Center for Brain Science. Early on, they recognized the promise and impact of Uchida’s research program and decided to move fast to recruit him in 2006. “I think they are visionary,” he says of MCB/CBS. “They took a risk on me because they knew it was important to study behavior in genetically tractable animals. I received so much support from other MCB faculty,” especially from fellow olfactory researchers Catherine Dulac and Venkatesh Murthy, and from his zealous mentor, Markus Meister.
MCB/CBS’s gamble paid off, as Uchida has published high profile papers demonstrating the power of the rodent decision-making paradigm – and has earned a tenured position at Harvard as Professor of Molecular and Cellular Biology. Going forward, one of his plans is to study how animals weigh their options in taking or avoiding risks.
“Many psychiatric disorders have deficits in decision making,” he explains, “and by learning how rewards are valued in animals and how they are modulated by innate biases we can help refine the direction of research into human disorders.”Sniffs and Decisions
Uchida grew up in Osaka, Japan, where he went fishing, played baseball, developed an interest in biology, and aspired to research cancer. He chose a basic science program at Kyoto University Faculty of Science so he could combine the study of biology, physics and chemistry. But during a home stay in Los Angeles, he began discussing philosophical questions with a friend there. Who am I? How does my mind work? Back in Koyto, he switched to brain science.
He joined Masotoshi Takeichi’s lab to study how neurons connect with each other, remaining there for a PhD in biophysics in 1997. After working with fixed brain tissue, he wanted to see how neurons respond in living animals. For post-doctoral work with Kensaku Mori at the Riken Institute from 1998-2000, he studied the brain’s spatial map of odorants and used optical imaging and electrophysiology to measure neural activation patterns in anesthetized animals. But to approach the issues that really interested him, he needed awake and behaving animals. He undertook additional postdoctoral research in the Cold Spring Harbor lab of Zachary Mainen, who at the time was starting a new lab. Together with Mainen, Uchida developed a decision-making paradigm that resembles those used in primates.
Primate tasks typically focus on visual perception, but rodents have a keener sense of smell. Uchida trained rats to sniff an odorant in the center hole of a setup and, depending on the sensory cue, decide to go right or left. The right choice earned a reward. By mixing gradations of two odors, he forced the rats to decide the dominant odor, while he measured the activity from the olfactory area and striatum, which is involved in decision making.Reward Prediction Error
Decision-making involves an animal’s prediction about the reward of taking one course of action versus another. Research has focused on the ventral tegmental area (VTA), a region rich in neurons expressing dopamine, a neurotransmitter that is important in reward and motivation but is dysregulated by addictive drugs. It was thought that dopaminergic neurons increase firing if a reward is better than expected, and decrease firing if the reward is less than expected. But the technology was not available to actually identify the type of neuron that was firing in the VTA, which was critical for understanding the neural basis for the value of the expectation and reward. “These studies were correlational,” Uchida explains. “To explore the function of neurons in the dopamine circuit during decision-making, we needed more biological studies.”
He reflected that in machine learning, decision-making algorithms use error signals derived from the difference between expectation and actual outcome. Also, almost half of the neurons in the VTA expressed GABA, not dopamine, and GABAergic neurons inhibit dopaminergic neurons. Perhaps the decision-making circuit derives the error signal from the relative values of the dopaminergic and GABAergic neurons’ activities.
In what was lauded as an ingenious use of new optogenetics technology, Uchida modified the GABAergic neurons in one group of mice and the dopaminergic neurons in another group to express the light-activated protein Channelrhodopsin 2 (ChR2). He could then discretely identify the two types of neurons in vivo when they responded to brief pulses of light. Then he recorded each type of neuron’s specific activity while mice associate particular odors with reward and punishment.
“We found that the GABA neurons were involved in the reward circuit, and we think that GABA neurons set the prediction of the reward. Dopamine neurons monitor the actual experience, and then subtract the GABA reward prediction from the experienced reward to derive the reward prediction error,” explains Uchida of the results published in the January 18, 2012 issue of Nature.
He thinks this finding is also relevant to the understanding of drug abuse. Opiate drugs and cannabinoids inhibit GABAergic neurons, disrupting the feedback to the dopaminergic neurons and thus changing the calculation of the reward prediction error signal. If so, dopaminergic neurons might keep signaling that drug use is rewarding, despite the many negative consequences of addiction.Tracing the Reward Circuit
While developing this method, Uchida was delving into the decision-making circuit’s “connectome” – the anatomy of the paths that neural connections take – to ask: What are the input-output relationships in the reward pathway?
He adapted a technique that uses the rabies virus, which in nature enters the body through a wound, infects peripheral neurons, and travels in a “retrograde” or reverse flow up to the brain, from post-synaptic to pres-synaptic neuron. Uchida’s lab engineered the virus to only infect dopaminergic neurons, and also to tag the neurons with a fluorescent label to reveal the route a sensory input travels to the dopaminergic neurons in two brain region, the VTA (reward circuit), and the substantia nigra (motor control). In a proof-of-principle experiment reported in the January 7, 2012 Neuron, his team identified the first comprehensive list of inputs to these two regions, which both integrate inputs from more diverse regions than suspected.
Now he plans to combine this method with optogenetics so that the rabies virus also tags the neurons it infects with light-activated ChR2 proteins. “We can go back to the input neurons and shine the light on them to activate them to confirm if they actually connect to the dopamine neurons.”
He eventually wants to learn: How do GABAergic neurons set the prediction of reward? Where are the predictions stored, and how are they recalled and refined? Where do the inputs to dopaminergic neurons about the actual experience come from?
He thinks this method may also aid in diverse avenues of investigation regarding behavior, addiction, and motor function. For example, in Uchida’s recent paper, he discovered that neurons in the subthalamic nucleus preferentially connect to dopamine neurons in the substantia nigra. In the neurodegenerative motor disorder, Parkinson’s Disease (PD), the subthalamic nucleus is the major target for therapeutic Deep Brain Stimulation (DBS), which researchers have speculated that the treatment works by inhibiting the subthalamic neurons. But Uchida’s findings suggest that instead it may activate dopamine neurons in the substantia nigra. The beauty of Uchida’s technique is that it can provide previously-elusive evidence about neural connections that may help researchers better understand the mechanisms underlying brain disorders and develop more specifically targeted interventions.Future Risks in an Enabling Environment
Uchida’s lab now has four postdocs, four graduate students and four undergraduates. “A unique feature of being at Harvard is all the great students and postdocs,” he says. Outside of the lab and classroom, he still enjoys fishing and baseball (he participates in a local Japanese softball tournament), as well as hiking and animal wildlife. But there is less time for that now that his teenage daughter has become a successful competitive swimmer. “We spend a lot of time taking her to swimming meets,” he notes.
Next up for the lab, Uchida plans to study more real-life decision making, including risk taking. What is the neural explanation for what behavioral economists call the “risk aversion” or “framing effect” on a choice between two options that involve the same outcome? Suppose you are given $100 and can then choose between a 50% chance of gaining another $100 versus a 100% chance of gaining another $50. Many people choose the second, sure option. Now suppose you are given $200 and can choose between a 50% chance of losing $100 or losing $50 for sure. Many people now dare to gamble by choosing the first option. Framing the same outcomes in terms of gains (first scenario) or losses (second scenario) dramatically flips people’s choices – the framing effect. Uchida speculates that there may be biological basis for how the risk of gain versus loss is valued in the decision-making circuit, and is trying to develop an “economic value” paradigm in mice for exploring inputs signaling the emotions associated with losses.
He says it is exciting how MCB encourages its faculty to pursue high-risk research, especially as federal grants provide less support to basic research. “It’s very gratifying that many groups have joined the effort to study different sensory modalities and behavioral tasks in rodents,” he adds. “It has changed the way we study the brain in rodents” – including asking the types of questions about how the mind works that intrigued him as an undergraduate.
