I am a camera with its shutter open, quite passive, recording, not thinking..
Christopher Isherwood, Goodbye to Berlin
As we stare at a computer screen, admire a landscape or otherwise observe the world around us, our eyes are anything but still. Even when we are unaware of their motion, they actively scan our surroundings with continuous microscopic movements, and it is these movements that enable us to see. Scientists once thought that in the course of this scanning, the eyes worked like cameras in a one-way process: Differences in light intensity are conveyed from the eye lens to the retina, which relays signals to the brain. But in the past few years an alternative concept has emerged, in which our vision has a sort of feedback cycle, or closed loop: As our eyes scan the world, the brain simultaneously takes in this information and directs the eye movements in response. Researchers at the Weizmann Institute of Science have now performed a study that supports this latter view.
“The brain doesn’t merely receive visual information but becomes, in a way, integrated with the observed reality,” says Prof. Ehud Ahissar of the Neurobiology Department, who mentored the study conducted by doctoral student Liron Gruber.
The study’s findings contribute to the fundamental understanding of the way our vision works, but they may also have practical implications. They clarify the way that human vision differs from how cameras work: our vision relies on an interactive, or closed-loop, mechanism; cameras, in contrast, use the one-way, or open-loop, method. Taking this difference into account may help explain why the human eye outperforms robotic vision, particularly when the viewing conditions are imperfect, and it may ultimately make it possible to bridge this gap. Moreover, knowing exactly how our vision is affected by our eye movements may help in developing ways of overcoming visual problems by, for example, designing training methods that would take these movements into consideration.
Gruber and Ahissar explored the prevailing assumptions concerning two major types of eye movement that make vision possible. Rapid jumps called saccades are followed by scanning movements called drifts, which typically last 0.3-0.4 seconds and are more than ten times slower than the saccades. The saccades have been commonly regarded as part of a closed-loop system, in which the brain optimizes the direction of the gaze based on the information it receives. But the drifts were thought to work in an open-loop, one-way manner, so that once the saccade directs the eye to a region of interest, the drift was assumed to produce snapshots that are not affected by what the eye sees at any given moment, much like a camera that “mindlessly” records whatever is placed in front of its shutter.
The Weizmann scientists designed experiments to test whether that is indeed what happens in the course of the drifts. Study participants were asked to watch a computer screen displaying a series of geometric objects that varied in size and were presented under different conditions: to be viewed naturally or through a virtual tunnel that eliminated all the details outside of a small area. The participants wore helmets equipped with cameras that tracked what exactly they were looking at, as well as their eyes’ saccades and drifts. This experiment aimed, in particular, at analyzing the speed and trajectory of the drifts. The scientists hypothesized that if the eyes make snapshots in a one-way, open-loop manner, then the drifts would be unaffected by the viewing conditions or the viewed details. If, on the other hand, the drifts are part of an interactive, closed-loop process, they would be expected to vary according to sensory input and duration in time, as the visual system adapts to the task.
Rather than drifting in one direction, the eye lingered on the region of interest
The experiments showed that, contrary to the prevalent belief, the drifts clearly operate in a closed-loop manner – that is, as a dynamic, interactive process. When the sensory details were more interesting to the visual system, providing more complex information, the drifts slowed down. Moreover, their trajectory changed: Rather than drifting in one direction, the eye lingered on the region of interest, staying closer to the starting point and moving around it so as to continue exploring it. Furthermore, during the scanning of each region of interest, the speed of the drift also gradually dropped, on average, ultimately converging on a value that was typical for each set of viewing conditions. Finally, for each viewing condition and image size, the drifts converged on a certain balance between speed and trajectory that was – the scientists speculate – optimal for these conditions. None of this would have happened had the drifts been a camera-like, open-loop affair.
“Our findings show that not only the saccades, but the drifts too, operate in a closed-loop manner, continuously reacting to any sort of input the eyes receive – which means that the entire visual system works in a way that’s dynamic and interactive,” says Gruber.
“And this conclusion,” adds Ahissar, “has a bearing on a long-standing philosophical debate: Is vision, and perception in general, direct or indirect? Namely, do we perceive objects in an immediate, non-mediated manner, or are we doomed to always perceive mediated representations of the world? The finding that vision works in a closed loop suggests that the visual brain is continuously trying to achieve a direct, non-mediated perception of the external world.”