internal memory, Synaptic Plasticity

What is the most important untested computational prediction in neuroscience?

by

The Organization for Computational Neuroscience has started a survey, asking people for their submissions, and here is my contribution:

Prediction: The basis for learning and memory exists primarily within the single neuron.

Rationale: (A) Dendrites/axons are adaptive, in particular the expression and contribution of ion channels adapts to use. This also extends to synaptic channels. (B) The decision on transforming a transient calcium signal into a permanent trace lies within the single neuron, within its protein signaling network and DNA readout mechanisms. The neuron’s memory traces are both use-dependent (dependent on shape and size of calcium signals received) and subject to additional internal computations, e.g. involving kinases/phosphatases, early genes, histones etc.

Remote memories are not coded by current synaptic connectivity, but internally by clusters of neurons, which become activated under certain conditions.

Conclusion: Memory research has to focus on the cellular (neuronal) basis of adaptation, synaptic connectivity will be predictable from adequate neuron models.

The statement in italics is extra. I have no papers, no references on that. For the rest, cf.

Scheler G. Regulation of neuromodulator receptor efficacy–implications for whole-neuron and synaptic plasticity. Prog Neurobiol. 2004 Apr;72(6):399-415. PMID: 15177784

Scheler G. Learning intrinsic excitability in medium spiny neurons. F1000Res. 2013 Mar 14 2:88. doi: 10.12688/f1000research.2-88.v2. eCollection 2013. PMID: 25520776

Scheler, G: Logarithmic distributions prove that intrinsic learning is Hebbian. F1000Res. 2017 (August).

 

internal memory, Neuromodulation ,

Dendritic computation

by

A new paper  Universal features of dendrites through centripetal branch ordering published: July 3, 2017) shows more or less the opposite of what it cites as common wisdom: „neuronal computation is known to depend on the morphology of dendrites”

Namely, since all dendrites follow general topological principles, it is probably not the dendritic morphology that matters in a functional sense. To make a dendrite functional, i.e. let it participate in adaptive information processing, we have to refer to the ion channels and GPCRs that populate the spines and shafts and shape the generation of action potentials.

Compare:
Dendritic integration: 60 years of progress. (Stuart GJ, Spruston N.) Nat Neurosci. 2015 Dec;18(12):1713-21. doi: 10.1038/nn.4157. Epub 2015 Nov 25. Review. PMID:26605882.

Plasticity of dendritic function. Magee JC, Johnston D. Curr Opin Neurobiol. 2005 Jun;15(3):334-42. Review. PMID:15922583

Activation-dependent learning rule for GPCR localization – 5ht2AR regulation in prefrontal cortical neurons

Gabriele Scheler BMC Neurosci. 2013; 14(Suppl 1): P344. Published online 2013 Jul 8. doi: 10.1186/1471-2202-14-S1-P344. PMCID: PMC3704850

Neuromodulation of circuits with variable parameters: single neurons and small circuits reveal principles of state-dependent and robust neuromodulation. Marder E1, O’Leary T, Shruti S. Annu Rev Neurosci. 2014;37:329-46. doi: 10.1146/annurev-neuro-071013-013958.
internal memory, Synaptic Plasticity

Input-dependent vs. Internally guided Memory

by

A  wealth of experimental data supports the idea that synaptic transmission can be potentiated or depressed, depending on neuronal stimulation, and that this change of at the synapse is a long-lasting effect responsible for learning and memory (LTP/LTD paradigm).What is new is to assume that synaptic and neural plasticity also requires conditions which are not input-dependent (i.e. dependent on synaptic stimulation, or even neuromodulatory activation) but instead dependent on the ‚internal state‘ of the pre- or postsynaptic neuron.

Under such a model, for plasticity to happen after stimulation it must meet with a readiness on the part of the cell, the presynapse or the postsynaptic site. We may call this conditional plasticity to emphasize that conditions deriving from the internal state of the neural network must be met in order for plasticity to occur. A neural system with conditional plasticity will have different properties from current neural networks which use unconditional plasticity in response to every stimulation. The known Hebbian neural network problem  of  ‘preservation of the found solution’ should become much easier to solve with conditional plasticity. Not every activation of a synapse leaves a trace. Most instances of synaptic transmission have no effect on memory. Existing synaptic strengths in many cases remain unaltered by transmission (use of the synapse), unless certain conditions are met. Those conditions could be an unusual strength of stimulation of a single neuron, a temporal sequence of (synaptic and neuromodulatory) stimulations that match a conditional pattern, a preconditioned high readiness for plastic change, e.g. by epigenetics.  However, it is an open question whether using conditional plasticity in neural network models will help with the tasks of conceptual abstraction and knowledge representation in general, beyond improved memory stability.

The ‚readiness‘ of the cell for plastic changes is a catch-all term for a very complex internal processing scheme. There is, for instance, the activation state of relevant kinases in protein signalling, such as CAMKII, PKA, PKC, and ERK, the NF-κB family of transcription factors, CREB, BDNF, factors involved in glucocorticoid signaling, c-fos etc., which can all be captured by dynamical models (Refs), but which generate prohibitively complex models with hundreds of species involved and little opportunity for generalization. That is not even sufficient. There are epigenetic effects like histone modification, which play a role in transcription, and which would require a potentially large number of data to be dynamically modeled as well. However it is known that non-specific drugs like HDAC inhibitors, enhancing gene transcription via increasing histone acetylation, improve learning in general.

This underscores the idea that neurons may receive a ‘readiness potential’ or threshold, a numeric value, which indicates the state of the cell as its ability to engage in plasticity events. Plasticity events originate from the membrane, by strong NMDA or L-type channel mediated calcium influx. Observations have shown that pharmacological blockade of L-type VSCCs as well as chelation of calcium in close proximity to the plasma membrane inhibits immediate-early gene induction, i.e. activation of cellular plasticity programs. If we model readiness as a threshold value, the strength of calcium input may be less than or more than this threshold to induce transcription plasticity. If we model readiness as a continuous value, we may integrate values over time to arrive at a potentially more accurate model.

To simplify we could start with a single scalar value of plasticity readiness, specific for each neuron, but dynamically evolving, and explore the theoretical consequences of a neural network with internal state variables. We‘d be free to explore various rules for the dynamic evolution of the internal readiness value.

internal memory, Synaptic Plasticity

How can we model neural plasticity?

by

A conjecture based on findings of plasticity in ion channel expression is that the level of expression of various ion channels reflects a memory of the cell. This means that we will have networks of neurons with slightly varying dendritic ion channel populations, which influence not only their general intrinsic excitability, but very specifically influence synaptic transmission through their position at synapses. Some channels aid with transmission, others block or reduce transmission, a phenomenon which has been studied as short-term facilitation or depression. Furthermore, ion channels at the synapse have an influence on synaptic plasticity, again, supporting or blocking plastic changes at the synapse.

This makes a model of neural plasticity more complex than synaptic input-dependent LTP/LTD. A single neuron would need variables at synaptic positions for AMPA and NMDA but also for the main potassium and calcium channels (K-A, Kir, Sk, HCN, L-Ca). In addition, a single set of intrinsic variables could capture the density of ion channels in dendritic shaft position. These variables would allow to express the neural diversity which is a result of memorization or learning.

Hypothesis: Neural plasticity is not primarily input-dependent, instead it is guided by a neural internal state which reflects network processes of knowledge building.

How would the variables that define a neural network be learned? The intrinsic variables would be set by neuromodulatory activation and the ‘internal state’ of the neuron. The synaptic variables would be set from synaptic activation, from neuromodulatory activation, and also from an ‘internal state‘. (In addition, the significant spill-over from synaptic activation which reaches other synapses via the dendrite should also be modeled.)

Let us assume that synaptic stimulation and neuromodulatory activation are well understood. What is the ’internal state‘ of a neuron?

It has been shown that epigenetic modifications are an important factor in memory. These involve methylation changes in DNA and alterations in histones. Their activation is mediated by protein signaling pathways, encompassing kinases like PKA, PKC, CaMKII, MAPK, and other important protein signaling hubs. Long-term neural plasticity and behavioral memory only happen, when the internal conditions are favorable, many reductions or disruptions of internal processes prevent plasticity and behavioral memory. We do not have the data yet to model these processes in detail. But we may use variables for a neuron‘s internal state which is facilitating or inhibiting plasticity. It is an important theoretical question then to understand the impact of internally-guided plasticity on a neural network, one hypothesis is that it helps with conceptual abstraction and knowledge building.

Hypothesis: Only a few neurons in a target area undergo the permanent, lasting changes underlying long-term memory.

Learning events in rodents lead to epigenetic changes in a targeted area, such as amygdala or hippocampus, but it seems as if there are only a few cells, neurons and non-neurons, involved at a time.These changes begin to appear 30 min after a high-frequency stimulation as in dentate gyrus and last 2-5 hours at least. Some have measured the effects 2 weeks after a learning event. Changes are not widespread, as would be expected in distributed memory, instead they are focal, as if only a few cells suffice to store the memory. It is also possible that the changes are strong only in a focal group of neurons, and present, but much weaker in a more distributed group. Focal learning may be seen as a strategy to build more effective knowledge representations, similar to dropout learning techniques which improve feature representation.

internal memory Leave a comment

What is the basis for neural plasticity?

by

The current view of LTP/LTD and specifically AMPA-dependent plasticity, which underlies all theoretical work on neural networks, seems exceedingly narrow. It leaves out levels of plasticity that are well known, like epigenetic modifications, or internal protein signaling, in order to come up with a simple model of use-dependent plasticity, the Hebbian principle, or ‘neurons that fire together, wire together’.

It is interesting and important to tackle the complex task of reducing the large number of individual findings on behavioral memory and neural plasticity into a small set of principles that can be used for models of biologically realistic memorization. Such models should offer capabilities beyond machine learning, namely conceptual abstraction, information filtering, building knowledge.

Here is one such observation: Both AMPA receptor placement and dendritic ion channel expression are regulated by similar, overlapping internal protein pathways. These protein pathways are activated by NM receptors and by NMDA and L-type-calcium channel-based calcium influx. We may conjecture that NM receptors and NMDA-based calcium activation together orchestrate neural plasticity via e.g. the calcium/CaMKII route, and the cAMP/PKA/ERK route, and that these pathways are acting in synergy at the synaptic AMPA sites as well as on the dendritic/synaptic ion channel expression sites.

So what this means is that various forms of intrinsic and synaptic plasticity are guided by the same protein pathways and therefore can be expected to be activated together. Here are specific instances of this synergy:

For instance, strengthening of AMPA could be accompanied by insertion of Sk-channels and reduction of L-type calcium channels, which blocks the synapse from further strengthening (‚overwrite protection‘). Such a mechanism has recently been identified as necessary for the stability and the lasting memorization capabilities of a network. Activation of the cAMP pathway activates Ih (HCN channels) which decreases the intrinsic excitability of the neuron, and allows less synaptic input to be processed. In a way, this kind of activation could be used as a temporal lock to prevent high dendritic excitability after the NM system has become engaged and plasticity in the neuron has started. Vice versa, in the absence of cAMP, dendritic excitability is high and many synaptic inputs are processed by an increase of membrane resistance through a reduction of HCN. Reduction of synaptic activity also reduces dendritic HCN channels. HCN channels may therefore indicate the level of synaptic activation, where more channels limit the parallel processing of synaptic input.

internal memory, Signal Transduction 1 Comment

Internal Memory: An Example

by
A protein Er81 which is present in about 60% of parvalbumin interneurons (parvalbumin is a calcium buffer which is fast, in contrast to calbindin) in layer II/III in the cortex of mice has been found to have an effect on the latency of spiking in these interneurons. (Er81 is also found in layer V pyramidal cells, there are also publications about that). This is mediated by the expression of the Kv1.1 potassium channel. Neurons with low Er81 expression have less Kv1.1, and these neurons, fast spiking basket cells, respond without latency. These neurons receive both E and I input. Neurons with high Er81 expression have more Kv1.1 channels, and these neurons (primarily basket cells again) have noticeable latencies. In slices it was found that cells of this kind have mostly E input and much less I input.
It then was shown that these ‘types’ of neurons actually undergo adult plasticity. A simple experiment – stimulation with kainate, and inhibition with nifedipine, a L-type calcium blocker – showed that Er81 expression was regulated inversely proportional to total network activity, and that this was observable after approximately two hours. So this is a kind of internal plasticity on the same time scale as LTP/LTD.
Additional experiments showed that Er81 plasticity was mediated by calcium entry into the cell (as so many other forms of plasticity), so we have evidence for a cell-specific regulation of Er81.
More precisely, the internal memory is the level of Er81. This can be a long-term storage element and remain constant over long time periods. The plasticity is intrinsic, i.e. in the expression of ion channels. The internal memory sets a parameter on the membrane (µKAs cf. Scheler 2013). When the internal memory changes – a new value emerges, the old value is overwritten – then there is a read-out at the membrane in terms of the µKAs parameter. So in this particular case, it seems as if the internal value is superfluous, and the µKAs is identical to epsilon Er81. But this is a mistake, in reality, µKAs is set by a number of factors, and epsilon ER81 very probably has other effects in the system as well.
It is not clear from this work, why the innervation by E and I neurons is different,  and also how and whether this changes, on the same time scale, or at all.
A surprising observation from this paper is also that high activity causes latencies of interneurons to appear, but low activity abolishes them. One might think that with less latency, there is more inhibition in the network, and high activity abolishes latencies to upregulate inhibition. That is not the case.
Without a simulation, I’d guess that inhibitory latencies reduce excitatory pressure; where activation is stored in the membrane potential of I neurons without letting them spike. There is then reduced spiking of I neurons, but still a reduction of overall excitation in the network, since the capacity of the I neuron to buffer synaptic input is enhanced. These neurons receive mostly E input, because they have this buffer capacity, no-latency neurons in contrast participate in disinhibition – they respond to the level of inhibition as well and adjust their activity. There is more activity stored in the network with longer latencies but less spiking. This is just a guess concerning the behavior of a real network.
Summarizing: A cytoplasmic protein Er81 regulates the density of Kv1.1 channels, which is a form of intrinsic plasticity that is set by a cell-internal calcium-related parameter. Neuronal activation of course increases calcium entry, so the internal parameter is influenced by external signals. The density of Kv1.1 channels influences spike latency and overall spike frequency. There is no synaptic plasticity in this scenario.
 Tuning of fast-spiking interneuron properties by an activity-dependent transcriptional switch
Nathalie Dehorter etal.
Science  11 Sep 2015: Vol. 349, Issue 6253, pp. 1216-1220
DOI: 10.1126/science.aab3415
Neural Networks

Balanced Inhibition-Excitation

by

Another idea that I consider ill-conceived is the notion that neural networks need to have balanced inhibition-excitation. This means that with every rise (or fall) of overall excitation of the network, inhibition has to closely match it.

On the one hand, this looks like a truism: excitation activates inhibitory neurons and therefore larger excitation means larger inhibition, which reduces excitation. However, the idea in the present form stems from neural modeling: conventional neural networks with their uniform neurons and dispersed connectivity easily either stop spiking because of a lack of activity, or spike at very high rates and ‘fill up’ the whole network to capacity. It is difficult to tune them to the space in-between, and difficult to keep them in this space. Therefore it was postulated that biological neural networks face the same problem and that here also excitation and inhibition need to be closely matched.

First of all inhibition is not simple. Inhibitory-inhibitory interactions make the simplistic explanation unrealistic, and the many different types of inhibitory neurons that have evolved again make it difficult to implement the balanced inhibition-excitation concept.

Secondly, more evolved and realistic neural networks do not face the tuning problem, they are resilient even with larger and smaller differences between inhibition and excitation.

Finally, there are a number of experimental findings showing that it is possible to tune inhibition in the absence of tuning excitation. In a coupled negative feedback model this simply means that the equilibrium values change. But some excitatory neurons may evolve strong activity without directly increasing their own inhibition. Inhibition needs not to be uniformly coupled to excitation, if a network can tolerate fairly large fluctuations in excitation.

Ubiquitous interneurons may still be responsible for guarding lower and upper levels of excitation (‘range-control’). This range may still be variably positioned.

In the next post I want to discuss an interesting form of regulation of inhibitory neurons, which also does not fit well  with the concept of balanced inhibition-excitation.

 

Neural Networks, Neuromodulation, Synaptic Plasticity

Dopamine and Neuromodulation

by

Some time ago, I suggested that equating dopamine with reward learning was a bad idea. Why?
First of all, because it is a myopic view of the role of neuromodulation in the brain, (and also in invertebrate animals). There are at least 4 centrally released neuromodulators, they all act on G-protein-coupled receptors (some not exclusively), and they all have effects on neural processing as well as memory. Furthermore there are myriad neuromodulators which are locally  released, and which have similar effects, all acting through different receptors, but on the same internal pathways, activating G-proteins.

Reward learning means that reward increases dopamine release, and that increased dopamine availability will increase synaptic plasticity.

That was always simplistic and like any half-truth misleading.

Any neuromodulator is variable in its release properties. This results from the activity of its NM-producing neurons, such as in locus ceruleus, dorsal raphe, VTA, medulla etc., which receive input, including from each other, and secondly from control of axonal and presynaptic release, which is independent of the central signal. So there is local modulation of release. Given a signal which increases e.g. firing in the VTA, we still need to know which target areas are at the present time responsive, and at which synapses precisely the signal is directed. It depends on the local state of the network, how the global signal is interpreted.

Secondly, the activation of G-protein coupled receptors is definitely an important ingredient in activating the intracellular pathways that are necessary for the expression of plasticity. Roughly, a concurrent activation of calcium and cAMP/PKA (within 10s or so) has been found to be supportive or necessary of inducing synaptic plasticity. However, dopamine, like the other centrally released neuromodulators, acts through antagonistic receptors, increasing or decreasing PKA, increasing or reducing plasticity. It is again local computation which will decide the outcome of NM signaling at each site.

So, is there a take-home message, rivaling the simplicity of dopamine=reward?

NMs alter representations (=thought) and memorize them (=memory) but the interpretation is flexible at local sites (=learn and re-learn).

Dopamine alters thought and memory in a way that can be learned and re-learned.

Back in 1995 I came up with the idea of analysing neuromodulators like dopamine as a method of introducing global parameters into neural networks, which were considered at the time to admit only local, distributed computations. It seemed to me then, as now, that the capacity for global control of huge brain areas (serotonergic, cholinergic, dopaminergic and noradrenergic systems), was really what set neuromodulation apart from the neurotransmitters glutamate and GABA. There is no need to single out dopamine as the one central signal, which induces simple increases in its target areas, when in reality changes happen through antagonistic receptors, and there are many central signals.  Also, the concept of hedonistic reward is badly defined and essentially restricted to Pavlovian conditioning for animals and addiction in humans.

Since the only known global parameter in neural networks at the time occurred in reinforcement learning, some people created a match, using dopamine as the missing global reinforcement signal (Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997). That could not work, because reinforcement learning requires proper discounting within a decision tree. But the idea stuck. Ever since I have been upset at this primitive oversimplification. Bad ideas in neuroscience.

Scheler, G and Fellous, J-M: Dopamine modulation of prefrontal delay activity- reverberatory activity and sharpness of tuning curves.  Neurocomputing, 2001.

Scheler, G. and Schumann, J: Presynaptic modulation as fast synaptic switching: state-dependent modulation of task performance. Proceedings of the International Joint Conference on Neural Networks 2003, Volume: 1. DOI: 10.1109/IJCNN.2003.1223347

General , Leave a comment

Theories, Models and Data

by

In the modern world, a theory is a mathematical model, and a mathematical model is a theory. A theory described in words is not a theory, it is an explanation or an opinion.

The interesting thing about mathematical models is that they go far beyond data reproduction. A theoretical model of a biological structure or process may be entirely hypothetical, or it may use a certain amount of quantitative data from experiments, integrate it into a theoretical framework and ask questions that result from the combined model.

A Bayesian model in contrast is a purely data-driven construct which usually requires additional quantitative values (‘priors’) which have to be estimated. A dynamical model of metabolic or protein signaling processes in the cell assumes only a simple theoretical structure, kinetic rate equations, and then proceeds to fill the model with data (many estimated) and analyses the results. A neural network model takes a set of data and performs a statistical analysis to cluster the patterns for similarity, or to assign new patterns to previously established categories. Similarly, high-throughput or other proteomic data are usually analysed for outliers and variance with statistical significance with respect to a control data set. Graph analysis of large-scale datasets for a cell type, brain regions, neural connections etc. also aim to reproduce the dataset, to visualize it, and to provide quantitative and qualitative measures of the resulting natural graph.
All these methods primarily attempt to reproduce the data, and possibly make predictions concerning missing data or the behavior of a system that is created from the dataset.

Theoretical models can do more.

A theoretical model can introduce a hypothesis on how a biological system functions, or even, how it ought to function. It may not even need detailed experimental data, i.e. experiments and measurements, but it certainly needs observations and outcomes. It should be specific enough to spur new experiments in order to verify the hypothesis.
In contrast to Popper, a hypothetical model should not be easily falsifiable. If that were the case, it would probably be an uninteresting, highly specific model, for which experiments can be easily performed to falsify the model. A theoretical model should be general enough to explain many previous observations and open up possibilities for many new experiments, which support, modify and refine the model. The model may still be wrong, but at least it is interesting.
It should not be easy to decide which of several hypothetical models covers the complex biological reality best. But if we do not have models of this kind, and level of generality, we cannot guide our research towards progress in answering pressing needs in society, such as in medicine. We then have to work with old, outdated models and are condemned to accumulate larger and larger amounts of individual facts for which there is no use. Those facts form a continuum without a clear hierarchy, and they become quickly obsolete and repetitive, unless they are stored in machine-readable format, where they become part of data-driven analysis, no matter their quality and significance. In principle, such data can be accumulated and rediscovered by theoreticians which look for confirmation of a model. But they only have significance after the model exists.

Theories are created, they cannot be deduced from data.