The silicon retina
Analog VLSI built to copy the retina's habit of reporting change rather than absolute brightness, which is the ancestor of every event camera shipping today.
A conventional image sensor answers a question nobody asked: what is the brightness of every pixel, right now, thirty times a second, whether or not anything moved. The retina does not work this way, and Mahowald and Mead’s silicon retina was an argument that the difference matters for engineering and not only for biology.
What it is
An analog CMOS array in which each pixel contains a photoreceptor with logarithmic response, a local adaptation stage, and comparison against its own recent average. A pixel that sees no change says nothing. A pixel whose log-intensity shifts past a threshold emits an event, immediately and asynchronously, and resets its reference.
There is no frame, and no clock defining one. The output is a stream of events in continuous time.
Why it matters
Three consequences fall out of that design, all of them things frame-based sensors struggle with:
- Dynamic range. Because adaptation is local and logarithmic, the sensor handles a scene with bright sky and deep shadow simultaneously — well past 120 dB in modern parts, where a conventional sensor must choose an exposure and lose one end.
- Latency. An event is emitted when the change happens, not when the next frame is read out. Microseconds instead of milliseconds.
- Bandwidth and power. A static scene costs almost nothing to report. Data rate scales with activity in the world, not with pixel count and frame rate.
That last point is sparse coding doing exactly what it does in biology: spend the power budget on the channels that carry information and leave the rest quiet. The retina did not arrive at this because change detection is elegant; it arrived there because the optic nerve is a bandwidth bottleneck and metabolism is expensive.
The engineering lesson generalises past vision, and this is the part worth taking seriously: the constraint that shaped the biology — limited channel capacity, limited energy — is the same constraint on an embedded sensor. Where the constraints match, the biological solution is not a metaphor. It is a design.
Origins & further reading
- Carver Mead & Misha Mahowald, 1988. A silicon model of early visual processing. Neural Networks. paper · doi
- Misha Mahowald & Carver Mead, 1991. The silicon retina. Scientific American. paper · doi
- Patrick Lichtsteiner et al., 2008. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits. paper · doi
Concepts
Related