Dopamine and Neuromodulation

Some time ago, I suggested that equating dopamine with reward learning was a bad idea. Why?
First of all, because it is a myopic view of the role of neuromodulation in the brain, (and also in invertebrate animals). There are at least 4 centrally released neuromodulators, they all act on G-protein-coupled receptors (some not exclusively), and they all have effects on neural processing as well as memory. Furthermore there are myriad neuromodulators which are locally  released, and which have similar effects, all acting through different receptors, but on the same internal pathways, activating G-proteins.

Reward learning means that reward increases dopamine release, and that increased dopamine availability will increase synaptic plasticity.

That was always simplistic and like any half-truth misleading.

Any neuromodulator is variable in its release properties. This results from the activity of its NM-producing neurons, such as in locus ceruleus, dorsal raphe, VTA, medulla etc., which receive input, including from each other, and secondly from control of axonal and presynaptic release, which is independent of the central signal. So there is local modulation of release. Given a signal which increases e.g. firing in the VTA, we still need to know which target areas are at the present time responsive, and at which synapses precisely the signal is directed. It depends on the local state of the network, how the global signal is interpreted.

Secondly, the activation of G-protein coupled receptors is definitely an important ingredient in activating the intracellular pathways that are necessary for the expression of plasticity. Roughly, a concurrent activation of calcium and cAMP/PKA (within 10s or so) has been found to be supportive or necessary of inducing synaptic plasticity. However, dopamine, like the other centrally released neuromodulators, acts through antagonistic receptors, increasing or decreasing PKA, increasing or reducing plasticity. It is again local computation which will decide the outcome of NM signaling at each site.

So, is there a take-home message, rivaling the simplicity of dopamine=reward?

NMs alter representations (=thought) and memorize them (=memory) but the interpretation is flexible at local sites (=learn and re-learn).

Dopamine alters thought and memory in a way that can be learned and re-learned.

Back in 1995 I came up with the idea of analysing neuromodulators like dopamine as a method of introducing global parameters into neural networks, which were considered at the time to admit only local, distributed computations. It seemed to me then, as now, that the capacity for global control of huge brain areas (serotonergic, cholinergic, dopaminergic and noradrenergic systems), was really what set neuromodulation apart from the neurotransmitters glutamate and GABA. There is no need to single out dopamine as the one central signal, which induces simple increases in its target areas, when in reality changes happen through antagonistic receptors, and there are many central signals.  Also, the concept of hedonistic reward is badly defined and essentially restricted to Pavlovian conditioning for animals and addiction in humans.

Since the only known global parameter in neural networks at the time occurred in reinforcement learning, some people created a match, using dopamine as the missing global reinforcement signal (Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997). That could not work, because reinforcement learning requires proper discounting within a decision tree. But the idea stuck. Ever since I have been upset at this primitive oversimplification. Bad ideas in neuroscience.

Scheler, G and Fellous, J-M: Dopamine modulation of prefrontal delay activity- reverberatory activity and sharpness of tuning curves.  Neurocomputing, 2001.

Scheler, G. and Schumann, J: Presynaptic modulation as fast synaptic switching: state-dependent modulation of task performance. Proceedings of the International Joint Conference on Neural Networks 2003, Volume: 1. DOI: 10.1109/IJCNN.2003.1223347

Non-modifiable synapses

Technical papers sometimes make a distinction between modifiable and non-modifiable synapses. Results on NMDA-NR2A vs. NMDA-NR2B receptors show that a NR2B receptor is required for LTP (learning), and a synapse which contains only NMDA-NR2A and AMPA glutamatergic receptors is not modifiable (except maybe for some LTD).

http://www.ncbi.nlm.nih.gov/pubmed/20164351

“Thus, for LTP induction, the physical presence of NR2B and its cytoplasmic tail are more important than the activation of NR2B–NMDARs, suggesting an essential function of NR2B as a mediator of protein interactions independent of its channel contribution. In contrast, the presence of NR2A is not essential for LTP, and, in fact, the cytoplasmic tail of NR2A seems to inhibit the induction of LTP.”

“The apparent contradiction can be explained by the fact that NR2B, in addition to forming part of a ligand-gated channel, also has a long cytoplasmic tail that binds (directly or indirectly) to a variety of postsynaptic signaling molecules.”

“Our data cannot distinguish whether these collaborating NR2A and NR2B subunits are in the same (triheteromeric) NMDAR complex or in different NMDARs (NR2A–NMDARs and NR2B–NMDARs) that lie near each other.”

“We hypothesize that the NR2A subunit also has a dual function. On one hand, it acts as a channel that facilitates LTP by conducting calcium. On the other hand, it acts as a scaffold that presumably recruits a protein to the synapse that inhibits LTP. Such a protein could act by antagonizing the activation of LTP signaling pathways [e.g., synaptic Ras-GTPase-activating protein (Kim et al., 2005)] or by stimulating the LTD signaling pathways [such as Rap and p38 mitogen-activated protein kinase) (Thomas and Huganir, 2004; Zhu et al., 2005; Li et al., 2006)].”

“Consistent with this idea, changes in plasticity in the visual cortex are correlated with a change in the NR2A/NR2B ratio. Light deprivation lowers the threshold of induction for LTP and is associated with a decrease in synaptic NR2A.”

It is hard to prove a negative, a non-modifiable synapse.  Since NR2B-containing receptors often occur in extrasynaptic positions, stimulation protocols may also recruit them to the synapse.  Triheteromeric receptor complexes do occur, but they may be less common. Also, the role of the NR2D and NR2C subunits, which may also bind to NR1, and which may help in creating Nr2A-type only synapses, is less clear.

Nonetheless it may be justified to make a distinction between “hard-to-modify” synapses and easily modifiable synapses.

This is entirely different from the “silent synapses”, which, because they contain no AMPA receptors, do not contribute (much) to fast signal transmission, but which may be recruited to full synapses, because they do contain NMDA-Nr2B receptors.