Memristive synapses

A synapse keeps its strength where it is used: the signal crossing it is scaled by that strength, and what arrives at its two sides changes it. Nothing is fetched, computed elsewhere and written back. Digital hardware carries its weights from memory to the arithmetic and pays more for the trip than for the sum. A memristive synapse is a two-terminal resistor whose conductance depends on the history of the current through it: a small voltage reads the weight, a larger pulse changes it.

A resistor with a history

In 1971 Leon Chua, at Berkeley, noted that of the six ways to pair current, voltage, charge and flux linkage (the time integral of voltage), two are fixed already and the resistor, capacitor and inductor supply three. For the sixth he proposed the memristor, whose resistance depends on the charge that has passed; no passive one was known, so he built it from active circuits. In 1976 he and Sung Mo Kang generalised it to memristive systems, naming unrecognised examples that included the Hodgkin–Huxley model of the axon membrane.

In 2008 Dmitri Strukov, Gregory Snider, Duncan Stewart and Stanley Williams at HP Labs supplied a physical model: a titanium dioxide film a few nanometres thick, in which positively charged oxygen vacancies drift with the current and move the boundary between conducting and insulating oxide. The film acts as two resistors in series whose proportions follow the charge:

$$M(q) = R_{\text{OFF}} \left( 1 - \frac{\mu_V R_{\text{ON}}}{D^2} \, q \right)$$

Here $M$ is the memristance, $q$ the charge passed, $R_{\text{ON}}$ and $R_{\text{OFF}}$ the film’s resistance fully doped and fully undoped ($R_{\text{ON}} \ll R_{\text{OFF}}$), $\mu_V$ the vacancies’ mobility and $D$ the thickness. The $1/D^2$ is the point: for a given material the memory term is a million times larger at the nanometre scale than at the micrometre scale, so it grows important as devices shrink.

What moves varies — silver or copper ions, or oxygen vacancies — and so does where (Choi and colleagues, 2023). A thin filament heats fast when switched and freezes its ions as it cools: fast and non-volatile, but variable between devices and cycles. Devices that change across the whole electrode interface, their current scaling with area, are more uniform but slower, and forget. Strukov’s moving boundary is the uniform, idealised case; in titanium dioxide cells, Kwon and colleagues imaged switching filaments of a reduced oxide phase in 2010.

What was borrowed from the synapse

The label across the axis, the synapse that keeps its weight where it is used, names an arrangement more than a mechanism: a strength that is a state of the connection itself, read by the signal crossing it and changed by signals present at its two ends. Hebbian learning stated the locality, and spike-timing-dependent plasticity gave it a shape: in Bi and Poo’s cultured rat hippocampal neurons, inputs up to 20 ms before a postsynaptic spike were strengthened and those up to 20 ms after it weakened.

In 2010 Sung Hyun Jo and colleagues in Wei Lu’s group at Michigan built a synapse for that rule from silicon co-sputtered with silver, in which pulses pushed silver back and forth. Driven by CMOS neurons, as Greg Snider at HP had proposed, it changed with the order and spacing of pre- and postsynaptic pulses, in a curve they set beside one from rat hippocampal neurons. No arithmetic is needed: a device that sees only the voltage between its terminals, with a threshold neither spike crosses alone, changes only when spikes overlap (Zamarreño-Ramos and colleagues, 2011).

What crossed was the curve, not the chemistry: Bi and Poo’s synapses needed NMDA receptors for both halves of the window, while the device’s coincidence detector is a voltage threshold.

Reading without writing

The arrangement had been built before, in copper. In October 1960 Bernard Widrow at Stanford described the “memistor”, a resistor with memory — a name one letter short of Chua’s, eleven years earlier — made by electroplating copper onto a pencil lead, which he and Ted Hoff developed as the adjustable weight of their Adaline. Plating current through a third terminal set the resistance between the other two by its time integral; the weight was read with alternating current and written with direct, plating on or off by the sign of error times input. Among Widrow’s future directions was memistor action from phenomena in solids.

Two terminals must separate reading from writing with physics instead. A state that followed every coulomb would shift with every read, and, Di Ventra and Pershin argued, could not protect itself from fluctuations. Working devices have a threshold: in Alibart and colleagues’ titanium dioxide, Joule heating made ionic mobility rise super-exponentially with voltage, so 0.2 V reads spared a weight that pulses above about 0.7 V could move.

Whether these are Chua’s memristors is disputed. Vongehr and Meng argued in 2015 that the device implied in 1971 would, like the inductor, need magnetism, has not been found and probably cannot exist; the oxide switches work without it. Chua’s answer, in 2011, was that every two-terminal resistance-switching memory is a memristor, known by its fingerprint, a current–voltage loop pinched at the origin. Either way, the useful device is the non-ideal one.

Ohm’s law multiplies, Kirchhoff’s law adds

Two terminals make the arrangement dense: a device at every crossing of two sets of wires is a synapse from a row to a column. Drive the rows with voltages $V_i$ and hold the columns at virtual ground; with $G_{ij}$ the conductance at row $i$ and column $j$, Ohm’s law gives each device’s current $G_{ij} V_i$, and Kirchhoff’s current law sums them on the column wire as $\sum_i G_{ij} V_i$. That is a vector–matrix product in one step, where the weights are stored; a signed weight takes two devices.

The weights never travel, and travel is what digital hardware pays for in energy: in Horowitz’s figures for a 45 nm process, a 32-bit multiply costs 3.1 pJ and a 64-bit fetch from off-chip DRAM 1.3–2.6 nJ. The cost moves rather than vanishes, to sensing and converting column currents at the edge of the array, but it is paid per column, not per weight.

Writing happens in place too: half the write voltage on a row and half, of opposite sign, on a column add up to the full voltage only at their crossing, and every other device on those lines sees half, below its threshold. Strukov’s group, by then at Santa Barbara, ran a 12 × 12 oxide crossbar this way in 2015 with no transistor at any crossing. Floating-gate synapses in neuromorphic circuits also adjust in place, but have three terminals and must confine electrons, which Alibart and colleagues judged harder to scale.

What it costs

Variability starts with forming. In Prezioso and colleagues’ account, variation above all in the voltage needed to form each device was why earlier memristive networks had formed every device in isolation, through off-chip wiring or a transistor at each crossing; Alibart’s group confined forming to a volume about 20 nm across and still saw significant dispersion in switching. Training in place absorbs some of it, as Widrow had found.

Updates are neither linear nor symmetric. In Prezioso’s crossbar the same ±1.3 V pulses added about 60 µS and removed about 5 µS at 20 µS, but added 24 µS and removed 55 µS at 65 µS. Phase-change synapses, which showed spike-timing rules in 2012 (Kuzum and colleagues), are lopsided by construction: crystallising pulses raise the conductance in steps, but the melt-and-quench pulse drops it abruptly, so each weight becomes two devices, one per direction, refreshed before both saturate (Boybat and colleagues, 2018). The resistance of their amorphous phase also drifts upward after writing (Pirovano and colleagues, 2004).

Every update is a write: Prezioso’s devices showed a switching endurance of at least 5,000 cycles, while tantalum oxide bilayers have passed $10^{12}$ (Lee and colleagues, 2011). And a passive crossbar is one resistor network, so a read meant for one device also finds sneak paths through its neighbours (Linn and colleagues, 2010). A transistor at each crossing fixes that by spending the density that justified two terminals; the alternative is a device that passes over ten times the current at its switching voltage as at half of it, as Prezioso’s did.