The Reichardt motion detector

A single photoreceptor cannot tell which way the world is moving: brightening at one point looks the same whichever side the edge came from. Direction exists only in the relation between two points over time. In 1956 Bernhard Hassenstein and Werner Reichardt inferred, from nothing but a beetle’s tendency to turn, the simplest circuit that extracts it — delay the signal from one receptor, multiply it by its neighbour’s, and subtract the same product formed the other way round. They recorded no neurons. Fifty-seven years later a reconstruction of a fruit fly’s optic lobe turned up a candidate for the wiring.

What the beetle’s turns gave away

The animal was a weevil, Chlorophanus viridis, tethered to walk on a featherweight globe of bamboo twigs so that its tendency to turn could be measured while the experimenters controlled what it saw. Hassenstein made the measurements; the analysis asked what simplest circuit could link stimulus to response, and drew it as topology, not anatomy.

Each stimulus was a change of brightness confined to one ommatidium, given to two of them in turn. Changes about 90 ms apart turned the beetle hardest, both shorter and longer intervals less, and reversing the order reversed the turn. The strengths of the two stimuli combined multiplicatively, and so did their signs: two brightenings or two dimmings turned the beetle with the sequence, one of each turned it against. Because a firing rate cannot go negative, they argued, brightening and dimming must travel on separate lines — the ON and OFF channels that many eyes were known to have.

They saw that this made a poor clock, two-valued and swayed by the stimuli’s strength, and defended it. The intervals that matter to a beetle lie on the long side of the peak; and the detector is not a clock but the error sensor of a feedback loop whose set point is no motion relative to the surroundings — a job in which the sign of the signal matters more than its calibration.

Delay, multiply, subtract

In the form that became standard, each half of the detector passes one receptor’s signal through a first-order low-pass filter, whose phase lag serves as the delay, and multiplies it by the neighbour’s unfiltered signal; the other half mirrors it, and the output is the difference. An edge moving from the first receptor to the second excites them in turn, and the filter holds back the early signal until it meets the late one; motion the other way pulls them further apart. Direction is read from relative timing — a temporal code across two points — and the multiplier turns it into an amplitude.

Reichardt showed in 1957 that the averaged output is a correlation of the moving pattern with itself. To an engineer each half is a mixer used as a phase detector: a drifting grating gives the two receptors one sinusoid at two phases, the filter adds a lag, and the product averages to the cosine of the net phase, plus a ripple at twice the frequency. Subtracting the mirror half cancels the ripple and every term that does not change sign with direction, leaving, with input filtering ignored,

$$\bar{R} = \Delta I^{2} \, \sin\!\left( \frac{2\pi \Delta\varphi}{\lambda} \right) \frac{\omega \tau}{1 + \omega^{2} \tau^{2}}$$

where $\bar{R}$ is the mean output, $\Delta I$ the amplitude of the grating’s brightness modulation, $\lambda$ its spatial period, $\Delta\varphi$ the angle between the receptors, $\tau$ the filter’s time constant, and $\omega = 2\pi v / \lambda$ the temporal frequency at each receptor for a grating moving at angular speed $v$. Reichardt and Varjú’s more general result of 1959 has the same two factors.

The sign gives the direction. Speed enters only through $\omega$, so the detector is tuned to temporal frequency, peaking where $\omega\tau = 1$, not to speed. Contrast enters squared. And a grating whose period lies between one and two receptor spacings turns the sine negative: the detector aliases and reports the wrong direction.

Found in the fly

In a single 2013 issue of Nature, Shin-ya Takemura and colleagues reconstructed a repeating module of the Drosophila medulla from electron micrographs and found that two cell types, Mi1 and Tm3, supply more than 80% of the input to direction-selective T4 cells, from points offset by less than one ommatidial spacing along each cell’s preferred direction — the two arms of a candidate correlator. Matthew Maisak and colleagues, imaging calcium, found four subtypes each of T4 and T5 tuned to the four cardinal directions, with T4 responding only to moving brightenings and T5 only to dimmings: the separate ON and OFF lines of 1956.

Anatomy could show neither the delay nor the multiplication. Takemura’s group put the conduction delays along both arms at about a millisecond, an order of magnitude too short, and raised metabotropic receptors as a possible source of the lag instead. The fly also proved richer than the model. Recording from single T4 cells while stimulating single columns, Juergen Haag and colleagues found in 2016 that they combine amplification of motion in the preferred direction, as in the Hassenstein–Reichardt scheme, with suppression of motion in the opposite one — the scheme Barlow and Levick proposed for the rabbit’s retina in 1965 — in different parts of the receptive field.

Delay-and-correlate in silicon

Every part of the detector is a cheap primitive in analog CMOS. In Reid Harrison and Christof Koch’s neuromorphic version, the delay was the phase lag of a $g_mC$ low-pass filter, run at currents small enough to give 10–100 ms from capacitors of about a picofarad, and the multiplier a Gilbert cell — originally a bipolar circuit — built from MOS transistors below threshold, where they too are exponential. A complete detector with its photoreceptor took 61 × 199 µm in a 1.2 µm process and drew 50 nW in indoor light.

In 1999 they closed the optomotor loop around the detector. A chip carrying 144 of them faced forward on a two-wheeled robot whose wheels were geared 5:1 against each other, so that it drove in tight circles; the summed, low-pass-filtered output was added to one motor command and subtracted from the other — negative feedback, as in the fly. In a cluttered laboratory the mean turning rate fell from 116°/s to 3.7°/s, a factor of 31, with the detector array drawing under 5 µW. The CCD imagers on the Sojourner Mars rover, they noted, had drawn 0.75 W.

What crossed was not correlation itself — Reichardt and Varjú had taken its mathematics from control engineering’s theory of random processes — but an architecture: a small correlator for each pair of neighbouring pixels, opponent subtraction, summation over the field, and a loop that needs the sign more than the size. Tobi Delbrück’s 1993 silicon retina had already put correlation-based motion detection in every pixel, along delay lines.

Temporal frequency, not speed

The cost is the one the equation states. With the 80 ms filter fitted to Harrison and Koch’s chip the peak falls near 2 Hz, so a grating with a 10° period gives almost the same output at 10°/s as at 40°/s, and a grating twice as coarse must move twice as fast to give its largest response. Contrast enters squared, which they counted among the detector’s biggest drawbacks and blunted by letting the multipliers saturate. And a blank view gives nothing to correlate: they put a 45° turn in one robot run down to the sensor facing a featureless part of the room.

Natural scenes soften this: their statistics are regular enough that a realistic correlator’s mean response tracks velocity in practice (Dror, O’Carroll and Laughlin, 2001).

Where speed itself is needed, something else does the work. Honeybees centre themselves in a tunnel by balancing the walls’ angular speeds, largely regardless of the stripes’ period or contrast, by a mechanism Srinivasan and colleagues judged different in kind from the optomotor one. Nicolas Franceschini, an engineer by training who had worked in Reichardt’s institute, built his robots’ motion sensors to time an edge across two photodiodes, giving an output that rises with angular speed and barely depends on contrast; his 1991 Robot Fly avoided obstacles at 50 cm/s with 114 of them, and a later 100 g tethered helicopter followed terrain with one. He is explicit that they are not correlators. Judging distance from optic flow needs a speed that holds whatever the texture and contrast, which a correlator does not deliver.