How can we model neural plasticity?

A conjecture based on findings of plasticity in ion channel expression is that the level of expression of various ion channels reflects a memory of the cell. This means that we will have networks of neurons with slightly varying dendritic ion channel populations, which influence not only their general intrinsic excitability, but very specifically influence synaptic transmission through their position at synapses. Some channels aid with transmission, others block or reduce transmission, a phenomenon which has been studied as short-term facilitation or depression. Furthermore, ion channels at the synapse have an influence on synaptic plasticity, again, supporting or blocking plastic changes at the synapse.

This makes a model of neural plasticity more complex than synaptic input-dependent LTP/LTD. A single neuron would need variables at synaptic positions for AMPA and NMDA but also for the main potassium and calcium channels (K-A, Kir, Sk, HCN, L-Ca). In addition, a single set of intrinsic variables could capture the density of ion channels in dendritic shaft position. These variables would allow to express the neural diversity which is a result of memorization or learning.

Hypothesis: Neural plasticity is not primarily input-dependent, instead it is guided by a neural internal state which reflects network processes of knowledge building.

How would the variables that define a neural network be learned? The intrinsic variables would be set by neuromodulatory activation and the internal state of the neuron. The synaptic variables would be set from synaptic activation, from neuromodulatory activation, and also from an internal state‘. (In addition, the significant spill-over from synaptic activation which reaches other synapses via the dendrite should also be modeled.)

Let us assume that synaptic stimulation and neuromodulatory activation are well understood. What is the internal state‘ of a neuron?

It has been shown that epigenetic modifications are an important factor in memory. These involve methylation changes in DNA and alterations in histones. Their activation is mediated by protein signaling pathways, encompassing kinases like PKA, PKC, CaMKII, MAPK, and other important protein signaling hubs. Long-term neural plasticity and behavioral memory only happen, when the internal conditions are favorable, many reductions or disruptions of internal processes prevent plasticity and behavioral memory. We do not have the data yet to model these processes in detail. But we may use variables for a neuron‘s internal state which is facilitating or inhibiting plasticity. It is an important theoretical question then to understand the impact of internally-guided plasticity on a neural network, one hypothesis is that it helps with conceptual abstraction and knowledge building.

Hypothesis: Only a few neurons in a target area undergo the permanent, lasting changes underlying long-term memory.

Learning events in rodents lead to epigenetic changes in a targeted area, such as amygdala or hippocampus, but it seems as if there are only a few cells, neurons and non-neurons, involved at a time.These changes begin to appear 30 min after a high-frequency stimulation as in dentate gyrus and last 2-5 hours at least. Some have measured the effects 2 weeks after a learning event. Changes are not widespread, as would be expected in distributed memory, instead they are focal, as if only a few cells suffice to store the memory. It is also possible that the changes are strong only in a focal group of neurons, and present, but much weaker in a more distributed group. Focal learning may be seen as a strategy to build more effective knowledge representations, similar to dropout learning techniques which improve feature representation.

What is the basis for neural plasticity?

The current view of LTP/LTD and specifically AMPA-dependent plasticity, which underlies all theoretical work on neural networks, seems exceedingly narrow. It leaves out levels of plasticity that are well known, like epigenetic modifications, or internal protein signaling, in order to come up with a simple model of use-dependent plasticity, the Hebbian principle, or ‘neurons that fire together, wire together’.

It is interesting and important to tackle the complex task of reducing the large number of individual findings on behavioral memory and neural plasticity into a small set of principles that can be used for models of biologically realistic memorization. Such models should offer capabilities beyond machine learning, namely conceptual abstraction, information filtering, building knowledge.

Here is one such observation: Both AMPA receptor placement and dendritic ion channel expression are regulated by similar, overlapping internal protein pathways. These protein pathways are activated by NM receptors and by NMDA and L-type-calcium channel-based calcium influx. We may conjecture that NM receptors and NMDA-based calcium activation together orchestrate neural plasticity via e.g. the calcium/CaMKII route, and the cAMP/PKA/ERK route, and that these pathways are acting in synergy at the synaptic AMPA sites as well as on the dendritic/synaptic ion channel expression sites.

So what this means is that various forms of intrinsic and synaptic plasticity are guided by the same protein pathways and therefore can be expected to be activated together. Here are specific instances of this synergy:

For instance, strengthening of AMPA could be accompanied by insertion of Sk-channels and reduction of L-type calcium channels, which blocks the synapse from further strengthening (‚overwrite protection‘). Such a mechanism has recently been identified as necessary for the stability and the lasting memorization capabilities of a network. Activation of the cAMP pathway activates Ih (HCN channels) which decreases the intrinsic excitability of the neuron, and allows less synaptic input to be processed. In a way, this kind of activation could be used as a temporal lock to prevent high dendritic excitability after the NM system has become engaged and plasticity in the neuron has started. Vice versa, in the absence of cAMP, dendritic excitability is high and many synaptic inputs are processed by an increase of membrane resistance through a reduction of HCN. Reduction of synaptic activity also reduces dendritic HCN channels. HCN channels may therefore indicate the level of synaptic activation, where more channels limit the parallel processing of synaptic input.

Balanced Inhibition-Excitation

Another idea that I consider ill-conceived is the notion that neural networks need to have balanced inhibition-excitation. This means that with every rise (or fall) of overall excitation of the network, inhibition has to closely match it.

On the one hand, this looks like a truism: excitation activates inhibitory neurons and therefore larger excitation means larger inhibition, which reduces excitation. However, the idea in the present form stems from neural modeling: conventional neural networks with their uniform neurons and dispersed connectivity easily either stop spiking because of a lack of activity, or spike at very high rates and ‘fill up’ the whole network to capacity. It is difficult to tune them to the space in-between, and difficult to keep them in this space. Therefore it was postulated that biological neural networks face the same problem and that here also excitation and inhibition need to be closely matched.

First of all inhibition is not simple. Inhibitory-inhibitory interactions make the simplistic explanation unrealistic, and the many different types of inhibitory neurons that have evolved again make it difficult to implement the balanced inhibition-excitation concept.

Secondly, more evolved and realistic neural networks do not face the tuning problem, they are resilient even with larger and smaller differences between inhibition and excitation.

Finally, there are a number of experimental findings showing that it is possible to tune inhibition in the absence of tuning excitation. In a coupled negative feedback model this simply means that the equilibrium values change. But some excitatory neurons may evolve strong activity without directly increasing their own inhibition. Inhibition needs not to be uniformly coupled to excitation, if a network can tolerate fairly large fluctuations in excitation.

Ubiquitous interneurons may still be responsible for guarding lower and upper levels of excitation (‘range-control’). This range may still be variably positioned.

In the next post I want to discuss an interesting form of regulation of inhibitory neurons, which also does not fit well  with the concept of balanced inhibition-excitation.

 

Dopamine and Neuromodulation

Some time ago, I suggested that equating dopamine with reward learning was a bad idea. Why?
First of all, because it is a myopic view of the role of neuromodulation in the brain, (and also in invertebrate animals). There are at least 4 centrally released neuromodulators, they all act on G-protein-coupled receptors (some not exclusively), and they all have effects on neural processing as well as memory. Furthermore there are myriad neuromodulators which are locally  released, and which have similar effects, all acting through different receptors, but on the same internal pathways, activating G-proteins.

Reward learning means that reward increases dopamine release, and that increased dopamine availability will increase synaptic plasticity.

That was always simplistic and like any half-truth misleading.

Any neuromodulator is variable in its release properties. This results from the activity of its NM-producing neurons, such as in locus ceruleus, dorsal raphe, VTA, medulla etc., which receive input, including from each other, and secondly from control of axonal and presynaptic release, which is independent of the central signal. So there is local modulation of release. Given a signal which increases e.g. firing in the VTA, we still need to know which target areas are at the present time responsive, and at which synapses precisely the signal is directed. It depends on the local state of the network, how the global signal is interpreted.

Secondly, the activation of G-protein coupled receptors is definitely an important ingredient in activating the intracellular pathways that are necessary for the expression of plasticity. Roughly, a concurrent activation of calcium and cAMP/PKA (within 10s or so) has been found to be supportive or necessary of inducing synaptic plasticity. However, dopamine, like the other centrally released neuromodulators, acts through antagonistic receptors, increasing or decreasing PKA, increasing or reducing plasticity. It is again local computation which will decide the outcome of NM signaling at each site.

So, is there a take-home message, rivaling the simplicity of dopamine=reward?

NMs alter representations (=thought) and memorize them (=memory) but the interpretation is flexible at local sites (=learn and re-learn).

Dopamine alters thought and memory in a way that can be learned and re-learned.

Back in 1995 I came up with the idea of analysing neuromodulators like dopamine as a method of introducing global parameters into neural networks, which were considered at the time to admit only local, distributed computations. It seemed to me then, as now, that the capacity for global control of huge brain areas (serotonergic, cholinergic, dopaminergic and noradrenergic systems), was really what set neuromodulation apart from the neurotransmitters glutamate and GABA. There is no need to single out dopamine as the one central signal, which induces simple increases in its target areas, when in reality changes happen through antagonistic receptors, and there are many central signals.  Also, the concept of hedonistic reward is badly defined and essentially restricted to Pavlovian conditioning for animals and addiction in humans.

Since the only known global parameter in neural networks at the time occurred in reinforcement learning, some people created a match, using dopamine as the missing global reinforcement signal (Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997). That could not work, because reinforcement learning requires proper discounting within a decision tree. But the idea stuck. Ever since I have been upset at this primitive oversimplification. Bad ideas in neuroscience.

Scheler, G and Fellous, J-M: Dopamine modulation of prefrontal delay activity- reverberatory activity and sharpness of tuning curves.  Neurocomputing, 2001.

Scheler, G. and Schumann, J: Presynaptic modulation as fast synaptic switching: state-dependent modulation of task performance. Proceedings of the International Joint Conference on Neural Networks 2003, Volume: 1. DOI: 10.1109/IJCNN.2003.1223347

Theories, Models and Data

In the modern world, a theory is a mathematical model, and a mathematical model is a theory. A theory described in words is not a theory, it is an explanation or an opinion.

The interesting thing about mathematical models is that they go far beyond data reproduction. A theoretical model of a biological structure or process may be entirely hypothetical, or it may use a certain amount of quantitative data from experiments, integrate it into a theoretical framework and ask questions that result from the combined model.

A Bayesian model in contrast is a purely data-driven construct which usually requires additional quantitative values (‘priors’) which have to be estimated. A dynamical model of metabolic or protein signaling processes in the cell assumes only a simple theoretical structure, kinetic rate equations, and then proceeds to fill the model with data (many estimated) and analyses the results. A neural network model takes a set of data and performs a statistical analysis to cluster the patterns for similarity, or to assign new patterns to previously established categories. Similarly, high-throughput or other proteomic data are usually analysed for outliers and variance with statistical significance with respect to a control data set. Graph analysis of large-scale datasets for a cell type, brain regions, neural connections etc. also aim to reproduce the dataset, to visualize it, and to provide quantitative and qualitative measures of the resulting natural graph.
All these methods primarily attempt to reproduce the data, and possibly make predictions concerning missing data or the behavior of a system that is created from the dataset.

Theoretical models can do more.

A theoretical model can introduce a hypothesis on how a biological system functions, or even, how it ought to function. It may not even need detailed experimental data, i.e. experiments and measurements, but it certainly needs observations and outcomes. It should be specific enough to spur new experiments in order to verify the hypothesis.
In contrast to Popper, a hypothetical model should not be easily falsifiable. If that were the case, it would probably be an uninteresting, highly specific model, for which experiments can be easily performed to falsify the model. A theoretical model should be general enough to explain many previous observations and open up possibilities for many new experiments, which support, modify and refine the model. The model may still be wrong, but at least it is interesting.
It should not be easy to decide which of several hypothetical models covers the complex biological reality best. But if we do not have models of this kind, and level of generality, we cannot guide our research towards progress in answering pressing needs in society, such as in medicine. We then have to work with old, outdated models and are condemned to accumulate larger and larger amounts of individual facts for which there is no use. Those facts form a continuum without a clear hierarchy, and they become quickly obsolete and repetitive, unless they are stored in machine-readable format, where they become part of data-driven analysis, no matter their quality and significance. In principle, such data can be accumulated and rediscovered by theoreticians which look for confirmation of a model. But they only have significance after the model exists.

Theories are created, they cannot be deduced from data.

Bad Ideas in Neuoscience

balanced excitation inhibition

dopamine=reward learning

hidden layers

explaining attention by top-down and bottom-up processes

I should collect some more. Why are they bad? Because they are half-truths. There is “something” right about these ideas, but as scientific concepts, the way they are currently defined, I think they are wrong. Need to be replaced.

The LTP/LTD hypothesis of memory storage

In a classical neural network, where storage relies only on synapses, all memory is always present. Synaptic connections have specific weights, and any processing event uses them with all of the memory involved.

It could be of course that in a processing event only small regions of the brain network are being used, such that the rest of the connections form a hidden structure, a reservoir, a store or repository of unused information. Such models exist.

There is a real issue with stable pattern storage of a large number of patterns with a realistic amount of interference in a synaptic memory model. Classification, such as recognition of a word shape can be done very well, but storing 10,000 words and using them appropriately seems difficult. Yet that is still a small vocabulary for a human memory.

Another solution is conditional memory, i.e. storage that is only accessed when activated, which otherwise remains silent. Neurons offer many possibilities for storing memory other than at the strength of a synapse, and it should be worthwhile investigating if any of this may be exploited in a theoretical model.

Neural Coding

To understand neural coding, we have to regard the relationship of synchronized membrane potentials (local field potentials) and the firing of the individual neuron. We have two different processes here, because the firing of the single neuron is not determined simply by the membrane potential exceeding a fixed threshold. Rather, the membrane potential’s fluctuation does not predict the individual neuron’s firing, because the neuron has a dynamic, flexible firing threshold that is determined by its own internal parameters. Also, the membrane potential is subject to synchronization by direct contact between membranes, it is not necessarily or primarily driven by synaptic input or neuronal spiking. Similarly, HahnG2014(Kumar) have noted that membrane synchronization cannot be explained from a spiking neural network.
The determination of an individual neuron’s firing threshold is a highly dynamic process, i.e. the neuron constantly changes its conditions for firing without necessarily disrupting its participation in ongoing membrane synchronization processes. In other words, membrane potential fluctuations are determined by synaptic input as well as local synchronization processes, and spikes depend on membrane potentials filtered by a dynamic, individually adjustable firing threshold.

coding1

The model for a neural coding device contains the following:

A neuronal membrane that is driven by synaptic input and synchronized by local interaction (both excitatory and inhibitory)
A spiking threshold with internal dynamics, possibly within an individual range, which determines spiking from membrane potential fluctuations.

In this model the neural code is determined by at least three factors: synaptic input, local synchronization, and firing threshold value. We may assume that local synchronization acts as a filter for signal loss, i.e. it unifies and diminishes differences in synaptic input. Firing thresholds act towards individualization, adding information from stored memory. The whole set-up acts to filter the synaptic input pattern towards a more predictable output pattern.