Memory and the Volatility of Spines

Memory has a physical presence in the brain, but there are no elements which permanently code for it.

Memory is located – among other places – in dendritic spines. Spines are being increased during learning and they carry stimulus or task-specific information. Ablation of spines destroys this information (Hayashi-Takagi A2015). Astrocytes have filopodia which are also extended and retracted and make contact with neuronal synapses. The presence of memory in the spine fits to a neuron-centric view: Spine protrusion and retraction are guided by cellular programs. A strict causality such that x synaptic inputs cause a new spine is not necessarily true, as a matter of fact highly conditional principles of spine formation or dissolution could hold, where the internal state of the neuron and the neuron’s history matters. The rules for spine formation need not be identical to the rules for synapse formation and weight updating (which depend on at least two neurons making contact).

A spine needs to be there for a synapse to exist (in spiny neurons), but once it is there, clearly not all synapses are alike. They differ in the amount of AMPA presence and integration, and other receptors/ion channels as well. For instance, Sk-channels serve to block off a synapse from further change, and may be regarded as a form of overwrite protection. Therefore, the existence or lack of a spine is the first-order adaptation in a spiny neuron, the second-order adaptation involves the synapses themselves.

However, spines are also subject to high variability, on the order of several hours to a few days. Some elements may have very long persistence, months in the mouse, but they are few. MongilloGetal2017 point out the fragility of the synapse and the dendritic spine in pyramidal neurons and ask what this means for the physical basis of memory. Given what we know about neural networks, for memory to be permanent, is it necessary that the same spines remain? Learning allows to operate with many random elements, but memory has prima facie no need for volatility.

It is most likely that memory is a secondary, ’emergent’ property of volatile and highly adaptive structures. From this perspective it is sufficient to keep the information alive, among the information-carrying units, which will recreate it in some form.

The argument is that the information is redundantly coded. So if part of the coding is missing, the rest still carries enough information to inform the system, which recruits new parts to carry the information. The information is never lost, because not all synapses, spines, neurons are degraded at the same time, and because internal reentrant processing keeps the information alive and recreates new redundant parts at the same time as other parts are lost. It is a dynamic cycling of information. There are difficulties, if synapses are supposed to carry the whole information. The main difficulty is: if all patterns at all times are being stored in synaptic values, without undue interference, and with all the complex processes of memory, forgetting, retrieval, reconsolidation etc., can this be fitted to a situation, where the response to a simple visual stimulus already involves 30-40% of the cortical area where there is processing going on? I have no quantitative model for this. I think the model only works if we use all the multiple, redundant forms of plasticity that the neuron possesses: internal states, intrinsic properties, synaptic and morphological properties, axonal growth, presynaptic plasticity.

Balanced Inhibition/Excitation (2) – The I/E ratio

Some time ago, I suggested that the theoretical view on balanced inhibition/excitation (in cortex and cortical models) is probably flawed. I suggested that we have a loose regulation instead, where inhibition and excitation can fluctuate independently.

The I/E balance stems from the idea that the single pyramidal neuron should receive approximately equal strength of inhibition and excitation, in spite of the fact that only 10-20% of neurons in cortex are inhibitory (Destexhe2003, more on that below). Experimental measurements have shown that this conjecture is approximately correct, i.e. inhibitory neurons make stronger contacts, or their influence is stronger relative to excitatory inputs.

The E-I balance in terms of synaptic drive onto a single pyramidal neuron is an instance of antagonistic regulation which allows gear-shifting of inputs, and in this case, allows very strong inputs to be downshifted by inhibition to a weaker effect on the membrane potential. What is the advantage of such a scheme? Strong signals are less prone to noise and uncertainty than weak signals. Weak signals are filtered out by the inhibitory drive. Strong signals allow unequivocal signal transmission, whether excitatory synaptic input, (or phasic increases of dopamine levels, in other contexts), which are then gear-shifted down by antagonistic reception. There may also be a temporal sequence: a strong signal is followed by a negative signal to restrict its time course and reduce impact. In the case of somatic inhibition following dendritic excitation the fine temporal structure could work together with the antagonistic gear-shifting exactly for this goal. Okun and Lampl, 2008 have actually shown that inhibition follows excitation by several milliseconds.

But what are the implications for an E/I network, such as cortex?

Here is an experimental result:

During both task and delay, mediodorsal thalamic (MD) neurons have 30-50% raised firing rates, fast-spiking (FS) inhibitory cortical neurons have likewise 40-60% raised firing rates, but excitatory (regular-spiking, RS) cortical neurons are unaltered. Thus there is an intervention possible, by external input from MD, probably directly to FS neurons, which does not affect RS neuron rate at all (fig. a and c, SchmittLIetal2017)

Untitled 1

Mediodorsal thalamic stimulation raises inhibition, but leaves excitation unchanged.

At the same time, in this experiment, the E-E connectivity is raised (probably by some form of short-term synaptic potentiation), such that E neurons receive more input, which is counteracted by more inhibition. (cf. also Hamilton, L2013). The balance on the level of the single neuron would be kept, but the network exhibits only loose regulation of the I/E ratio: unilateral increase of inhibition.

There are several studies which show that it is possible to raise inhibition and thus enhance cognition, for instance in the mPFC of CNTNAP2 (neurexin, a cell adhesion protein) deficient mice, which have abnormally raised excitation, and altered social behavior (SelimbeyogluAetal2017, cf. Foss-FeigJ2017 for an overview). Also, inhibition is necessary to allow critical period learning – which is hypothesized to be due to a switch from internally generated spontaneous activity to external sensory perception (ToyoizumiT2013) – in line with our suggestion that the gear-shifting effect of locally balanced I/E allows only strong signals to drive excitation and spiking and filters weak, internally generated signals.

 

Some thoughts about Language Evolution

Judging by ontogenetic development, language is derived from two separate streams (and a third one later on).
One of these is the development of object and action concepts by visuomotoric handling as opposed to background, space or situation. This may well be specific to humans with infants developing eye-hand-coordination in contrast to other species which do not undergo such a phase and may not develop strong concepts for objects. Infants also form a concept for actions when they experience their own agency and extend it to other agents.
The other is the sound-making ability, which appears innately pleasurable to human infants, and contributes to a long period of babbling.
At around 10-12 months of age these developments combine (‘naming insight’ in child language literature) and articulated words which refer to simple concepts arise. These refer to objects at first, sometimes actions and are articulated by phonological sequences. The one and two word (‘pivot grammar’) stages follow.
I see no reason to assume that phylogenetic development should have been different – the development of articulatory abilities, considered beautiful for their own sake (‘music/song’) running in parallel with solid conceptual structuring of the environment before one and two word communication became commonplace.
What about communication and communicative needs? I believe they popped up after the first concept naming skills took root. Suddenly there must have been a drift towards higher information content messages rather than just a string of words denoting objects and actions. Communication then is what must have driven grammar. Note that grammar remains a highly social accomplishment – like phonological sophistication it even requires a critical period. Grammar like phonology probably has a strong striatal component – habit learning – and it spontaneously appears in sign language as well. For this we would require a more detailed theory on how grammar arose from communication, when each grammatical system is distinct from any other.

What is the most important untested computational prediction in neuroscience?

The Organization for Computational Neuroscience has started a survey, asking people for their submissions, and here is my contribution:

Prediction: The basis for learning and memory exists primarily within the single neuron.

Rationale: (A) Dendrites/axons are adaptive, in particular the expression and contribution of ion channels adapts to use. This also extends to synaptic channels. (B) The decision on transforming a transient calcium signal into a permanent trace lies within the single neuron, within its protein signaling network and DNA readout mechanisms. The neuron’s memory traces are both use-dependent (dependent on shape and size of calcium signals received) and subject to additional internal computations, e.g. involving kinases/phosphatases, early genes, histones etc.

Remote memories are not coded by current synaptic connectivity, but internally by clusters of neurons, which become activated under certain conditions.

Conclusion: Memory research has to focus on the cellular (neuronal) basis of adaptation, synaptic connectivity will be predictable from adequate neuron models.

The statement in italics is extra. I have no papers, no references on that. For the rest, cf.

Scheler G. Regulation of neuromodulator receptor efficacy–implications for whole-neuron and synaptic plasticity. Prog Neurobiol. 2004 Apr;72(6):399-415. PMID: 15177784

Scheler G. Learning intrinsic excitability in medium spiny neurons. F1000Res. 2013 Mar 14 2:88. doi: 10.12688/f1000research.2-88.v2. eCollection 2013. PMID: 25520776

Scheler, G: Logarithmic distributions prove that intrinsic learning is Hebbian. F1000Res. 2017 (August).

 

Dendritic computation

A new paper  Universal features of dendrites through centripetal branch ordering published: July 3, 2017) shows more or less the opposite of what it cites as common wisdom: „neuronal computation is known to depend on the morphology of dendrites”

Namely, since all dendrites follow general topological principles, it is probably not the dendritic morphology that matters in a functional sense. To make a dendrite functional, i.e. let it participate in adaptive information processing, we have to refer to the ion channels and GPCRs that populate the spines and shafts and shape the generation of action potentials.

Compare:
Dendritic integration: 60 years of progress. (Stuart GJ, Spruston N.) Nat Neurosci. 2015 Dec;18(12):1713-21. doi: 10.1038/nn.4157. Epub 2015 Nov 25. Review. PMID:26605882.

Plasticity of dendritic function. Magee JC, Johnston D. Curr Opin Neurobiol. 2005 Jun;15(3):334-42. Review. PMID:15922583

Gabriele Scheler BMC Neurosci. 2013; 14(Suppl 1): P344. Published online 2013 Jul 8. doi: 10.1186/1471-2202-14-S1-P344. PMCID: PMC3704850

Neuromodulation of circuits with variable parameters: single neurons and small circuits reveal principles of state-dependent and robust neuromodulation. Marder E1, O’Leary T, Shruti S. Annu Rev Neurosci. 2014;37:329-46. doi: 10.1146/annurev-neuro-071013-013958.

Input-dependent vs. Internally guided Memory

 wealth of experimental data supports the idea that synaptic transmission can be potentiated or depressed, depending on neuronal stimulation, and that this change of at the synapse is a long-lasting effect responsible for learning and memory (LTP/LTD paradigm).What is new is to assume that synaptic and neural plasticity also requires conditions which are not input-dependent (i.e. dependent on synaptic stimulation, or even neuromodulatory activation) but instead dependent on the ‚internal state‘ of the pre- or postsynaptic neuron.

Under such a model, for plasticity to happen after stimulation it must meet with a readiness on the part of the cell, the presynapse or the postsynaptic site. We may call this conditional plasticity to emphasize that conditions deriving from the internal state of the neural network must be met in order for plasticity to occur. A neural system with conditional plasticity will have different properties from current neural networks which use unconditional plasticity in response to every stimulation. The known Hebbian neural network problem  of  ‘preservation of the found solution’ should become much easier to solve with conditional plasticity. Not every activation of a synapse leaves a trace. Most instances of synaptic transmission have no effect on memory. Existing synaptic strengths in many cases remain unaltered by transmission (use of the synapse), unless certain conditions are met. Those conditions could be an unusual strength of stimulation of a single neuron, a temporal sequence of (synaptic and neuromodulatory) stimulations that match a conditional pattern, a preconditioned high readiness for plastic change, e.g. by epigenetics.  However, it is an open question whether using conditional plasticity in neural network models will help with the tasks of conceptual abstraction and knowledge representation in general, beyond improved memory stability.

The ‚readiness‘ of the cell for plastic changes is a catch-all term for a very complex internal processing scheme. There is, for instance, the activation state of relevant kinases in protein signalling, such as CAMKII, PKA, PKC, and ERK, the NF-κB family of transcription factors, CREB, BDNF, factors involved in glucocorticoid signaling, c-fos etc., which can all be captured by dynamical models (Refs), but which generate prohibitively complex models with hundreds of species involved and little opportunity for generalization. That is not even sufficient. There are epigenetic effects like histone modification, which play a role in transcription, and which would require a potentially large number of data to be dynamically modeled as well. However it is known that non-specific drugs like HDAC inhibitors, enhancing gene transcription via increasing histone acetylation, improve learning in general.

This underscores the idea that neurons may receive a ‘readiness potential’ or threshold, a numeric value, which indicates the state of the cell as its ability to engage in plasticity events. Plasticity events originate from the membrane, by strong NMDA or L-type channel mediated calcium influx. Observations have shown that pharmacological blockade of L-type VSCCs as well as chelation of calcium in close proximity to the plasma membrane inhibits immediate-early gene induction, i.e. activation of cellular plasticity programs. If we model readiness as a threshold value, the strength of calcium input may be less than or more than this threshold to induce transcription plasticity. If we model readiness as a continuous value, we may integrate values over time to arrive at a potentially more accurate model.

To simplify we could start with a single scalar value of plasticity readiness, specific for each neuron, but dynamically evolving, and explore the theoretical consequences of a neural network with internal state variables. We‘d be free to explore various rules for the dynamic evolution of the internal readiness value.

How can we model neural plasticity?

A conjecture based on findings of plasticity in ion channel expression is that the level of expression of various ion channels reflects a memory of the cell. This means that we will have networks of neurons with slightly varying dendritic ion channel populations, which influence not only their general intrinsic excitability, but very specifically influence synaptic transmission through their position at synapses. Some channels aid with transmission, others block or reduce transmission, a phenomenon which has been studied as short-term facilitation or depression. Furthermore, ion channels at the synapse have an influence on synaptic plasticity, again, supporting or blocking plastic changes at the synapse.

This makes a model of neural plasticity more complex than synaptic input-dependent LTP/LTD. A single neuron would need variables at synaptic positions for AMPA and NMDA but also for the main potassium and calcium channels (K-A, Kir, Sk, HCN, L-Ca). In addition, a single set of intrinsic variables could capture the density of ion channels in dendritic shaft position. These variables would allow to express the neural diversity which is a result of memorization or learning.

Hypothesis: Neural plasticity is not primarily input-dependent, instead it is guided by a neural internal state which reflects network processes of knowledge building.

How would the variables that define a neural network be learned? The intrinsic variables would be set by neuromodulatory activation and the internal state of the neuron. The synaptic variables would be set from synaptic activation, from neuromodulatory activation, and also from an internal state‘. (In addition, the significant spill-over from synaptic activation which reaches other synapses via the dendrite should also be modeled.)

Let us assume that synaptic stimulation and neuromodulatory activation are well understood. What is the internal state‘ of a neuron?

It has been shown that epigenetic modifications are an important factor in memory. These involve methylation changes in DNA and alterations in histones. Their activation is mediated by protein signaling pathways, encompassing kinases like PKA, PKC, CaMKII, MAPK, and other important protein signaling hubs. Long-term neural plasticity and behavioral memory only happen, when the internal conditions are favorable, many reductions or disruptions of internal processes prevent plasticity and behavioral memory. We do not have the data yet to model these processes in detail. But we may use variables for a neuron‘s internal state which is facilitating or inhibiting plasticity. It is an important theoretical question then to understand the impact of internally-guided plasticity on a neural network, one hypothesis is that it helps with conceptual abstraction and knowledge building.

Hypothesis: Only a few neurons in a target area undergo the permanent, lasting changes underlying long-term memory.

Learning events in rodents lead to epigenetic changes in a targeted area, such as amygdala or hippocampus, but it seems as if there are only a few cells, neurons and non-neurons, involved at a time.These changes begin to appear 30 min after a high-frequency stimulation as in dentate gyrus and last 2-5 hours at least. Some have measured the effects 2 weeks after a learning event. Changes are not widespread, as would be expected in distributed memory, instead they are focal, as if only a few cells suffice to store the memory. It is also possible that the changes are strong only in a focal group of neurons, and present, but much weaker in a more distributed group. Focal learning may be seen as a strategy to build more effective knowledge representations, similar to dropout learning techniques which improve feature representation.

What is the basis for neural plasticity?

The current view of LTP/LTD and specifically AMPA-dependent plasticity, which underlies all theoretical work on neural networks, seems exceedingly narrow. It leaves out levels of plasticity that are well known, like epigenetic modifications, or internal protein signaling, in order to come up with a simple model of use-dependent plasticity, the Hebbian principle, or ‘neurons that fire together, wire together’.

It is interesting and important to tackle the complex task of reducing the large number of individual findings on behavioral memory and neural plasticity into a small set of principles that can be used for models of biologically realistic memorization. Such models should offer capabilities beyond machine learning, namely conceptual abstraction, information filtering, building knowledge.

Here is one such observation: Both AMPA receptor placement and dendritic ion channel expression are regulated by similar, overlapping internal protein pathways. These protein pathways are activated by NM receptors and by NMDA and L-type-calcium channel-based calcium influx. We may conjecture that NM receptors and NMDA-based calcium activation together orchestrate neural plasticity via e.g. the calcium/CaMKII route, and the cAMP/PKA/ERK route, and that these pathways are acting in synergy at the synaptic AMPA sites as well as on the dendritic/synaptic ion channel expression sites.

So what this means is that various forms of intrinsic and synaptic plasticity are guided by the same protein pathways and therefore can be expected to be activated together. Here are specific instances of this synergy:

For instance, strengthening of AMPA could be accompanied by insertion of Sk-channels and reduction of L-type calcium channels, which blocks the synapse from further strengthening (‚overwrite protection‘). Such a mechanism has recently been identified as necessary for the stability and the lasting memorization capabilities of a network. Activation of the cAMP pathway activates Ih (HCN channels) which decreases the intrinsic excitability of the neuron, and allows less synaptic input to be processed. In a way, this kind of activation could be used as a temporal lock to prevent high dendritic excitability after the NM system has become engaged and plasticity in the neuron has started. Vice versa, in the absence of cAMP, dendritic excitability is high and many synaptic inputs are processed by an increase of membrane resistance through a reduction of HCN. Reduction of synaptic activity also reduces dendritic HCN channels. HCN channels may therefore indicate the level of synaptic activation, where more channels limit the parallel processing of synaptic input.

Internal Memory: An Example

A protein Er81 which is present in about 60% of parvalbumin interneurons (parvalbumin is a calcium buffer which is fast, in contrast to calbindin) in layer II/III in the cortex of mice has been found to have an effect on the latency of spiking in these interneurons. (Er81 is also found in layer V pyramidal cells, there are also publications about that). This is mediated by the expression of the Kv1.1 potassium channel. Neurons with low Er81 expression have less Kv1.1, and these neurons, fast spiking basket cells, respond without latency. These neurons receive both E and I input. Neurons with high Er81 expression have more Kv1.1 channels, and these neurons (primarily basket cells again) have noticeable latencies. In slices it was found that cells of this kind have mostly E input and much less I input.
It then was shown that these ‘types’ of neurons actually undergo adult plasticity. A simple experiment – stimulation with kainate, and inhibition with nifedipine, a L-type calcium blocker – showed that Er81 expression was regulated inversely proportional to total network activity, and that this was observable after approximately two hours. So this is a kind of internal plasticity on the same time scale as LTP/LTD.
Additional experiments showed that Er81 plasticity was mediated by calcium entry into the cell (as so many other forms of plasticity), so we have evidence for a cell-specific regulation of Er81.
More precisely, the internal memory is the level of Er81. This can be a long-term storage element and remain constant over long time periods. The plasticity is intrinsic, i.e. in the expression of ion channels. The internal memory sets a parameter on the membrane (µKAs cf. Scheler 2013). When the internal memory changes – a new value emerges, the old value is overwritten – then there is a read-out at the membrane in terms of the µKAs parameter. So in this particular case, it seems as if the internal value is superfluous, and the µKAs is identical to epsilon Er81. But this is a mistake, in reality, µKAs is set by a number of factors, and epsilon ER81 very probably has other effects in the system as well.
It is not clear from this work, why the innervation by E and I neurons is different,  and also how and whether this changes, on the same time scale, or at all.
A surprising observation from this paper is also that high activity causes latencies of interneurons to appear, but low activity abolishes them. One might think that with less latency, there is more inhibition in the network, and high activity abolishes latencies to upregulate inhibition. That is not the case.
Without a simulation, I’d guess that inhibitory latencies reduce excitatory pressure; where activation is stored in the membrane potential of I neurons without letting them spike. There is then reduced spiking of I neurons, but still a reduction of overall excitation in the network, since the capacity of the I neuron to buffer synaptic input is enhanced. These neurons receive mostly E input, because they have this buffer capacity, no-latency neurons in contrast participate in disinhibition – they respond to the level of inhibition as well and adjust their activity. There is more activity stored in the network with longer latencies but less spiking. This is just a guess concerning the behavior of a real network.
Summarizing: A cytoplasmic protein Er81 regulates the density of Kv1.1 channels, which is a form of intrinsic plasticity that is set by a cell-internal calcium-related parameter. Neuronal activation of course increases calcium entry, so the internal parameter is influenced by external signals. The density of Kv1.1 channels influences spike latency and overall spike frequency. There is no synaptic plasticity in this scenario.
 Tuning of fast-spiking interneuron properties by an activity-dependent transcriptional switch
Nathalie Dehorter etal.
Science  11 Sep 2015: Vol. 349, Issue 6253, pp. 1216-1220
DOI: 10.1126/science.aab3415

Balanced Inhibition-Excitation

Another idea that I consider ill-conceived is the notion that neural networks need to have balanced inhibition-excitation. This means that with every rise (or fall) of overall excitation of the network, inhibition has to closely match it.

On the one hand, this looks like a truism: excitation activates inhibitory neurons and therefore larger excitation means larger inhibition, which reduces excitation. However, the idea in the present form stems from neural modeling: conventional neural networks with their uniform neurons and dispersed connectivity easily either stop spiking because of a lack of activity, or spike at very high rates and ‘fill up’ the whole network to capacity. It is difficult to tune them to the space in-between, and difficult to keep them in this space. Therefore it was postulated that biological neural networks face the same problem and that here also excitation and inhibition need to be closely matched.

First of all inhibition is not simple. Inhibitory-inhibitory interactions make the simplistic explanation unrealistic, and the many different types of inhibitory neurons that have evolved again make it difficult to implement the balanced inhibition-excitation concept.

Secondly, more evolved and realistic neural networks do not face the tuning problem, they are resilient even with larger and smaller differences between inhibition and excitation.

Finally, there are a number of experimental findings showing that it is possible to tune inhibition in the absence of tuning excitation. In a coupled negative feedback model this simply means that the equilibrium values change. But some excitatory neurons may evolve strong activity without directly increasing their own inhibition. Inhibition needs not to be uniformly coupled to excitation, if a network can tolerate fairly large fluctuations in excitation.

Ubiquitous interneurons may still be responsible for guarding lower and upper levels of excitation (‘range-control’). This range may still be variably positioned.

In the next post I want to discuss an interesting form of regulation of inhibitory neurons, which also does not fit well  with the concept of balanced inhibition-excitation.