“Seeing eye to eye” is an expression of harmony, but do different people literally see the same thing when they view the same external world? “The short answer is – no,” says Dr. Liron Gruber. “Even the same person sees the same thing differently each time they look at it,” adds Prof. Ehud Ahissar.
Gruber and Ahissar, of the Brain Sciences Department at the Weizmann Institute of Science, arrived at these conclusions after conducting a study in which they investigated intriguing discrepancies between human and computer vision uncovered by Weizmann mathematicians. Those researchers, headed by Prof. Shimon Ullman of the Computer Science and Applied Mathematics Department, had found that a computer algorithm, no matter how clever, was much worse than humans at interpreting image fragments known as minimally recognizable configurations, or MIRCs (middle row in the figure above) – that is, at telling from which objects (top row) these fragments had been derived. Moreover, when the researchers gradually cropped or blurred the MIRCs, recognition by the computer decreased in a linear fashion, whereas among human participants it dropped abruptly at a certain cut-off point (bottom row).
Gruber realized that experiments involving MIRCs could provide a wealth of data about the workings of the human visual system. In an earlier study, she and Ahissar, her PhD advisor, had already shown that contrary to the widely accepted view, the human eye doesn’t work like a camera that takes passive snapshots. In the new study, she and Ahissar teamed up with computer scientist Ullman to put human vision to the test.
Identifying MIRCs typically takes people a relatively long time – over 2 seconds, which is more than six times longer than the 300 or so milliseconds needed to recognize whole objects. The researchers recorded the eye movements of people attempting to recognize MIRCs and, using a computational model, simulated the resulting activities of neurons in the retina. These activity patterns not only varied with different eye movements, they differed depending on whether or not people managed to recognize the object in the picture. On average, recognition took four sets of scanning by the eyes of different points in the picture; at each point the eyes drifted locally in all directions for several hundred milliseconds.
“”The eyes of different people follow different paths when viewing the same thing, and even the eyes of the same person never copy the same trajectory, so in a way, each time we look at something, it’s a one-off experience”
The results indicated that the interactions between eye movements and the object are critical to recognition. In fact, when the researchers canceled out the interactions between the objects and the eye movements – for example, by moving the pictures in step with the eyes – study participants failed to recognize the objects.
“The retina doesn’t create copies of the outside world – unlike a camera, which reproduces external patterns on film or digitally. Rather, human vision is an active process that involves interactions between the external objects and eye movements,” Ahissar says. “The eyes of different people follow different paths when viewing the same thing, and even the eyes of the same person never copy the same trajectory, so in a way, each time we look at something, it’s a one-off experience.”
So how does the brain encode visual reality? More precisely, how does this encoding result from the interactions between the eye movements and the object? Says Gruber: “When we look at an object or scene, the light picked up by each receptor in the retina changes in intensity with every eye movement. The resultant patterns of neuronal activity can be interpreted and perhaps stored by the brain.”