The fourth room · Behind the eyeChapter 19 of 21
The light from a surface depends on two things at once: what the surface is, and what is shining on it. Your eye receives only their product. Your brain hands you the surface, and you never see the sum it did to get there.
Chapter 19 · Behind the eye
The Best Guess
The same light can come from many different worlds. How the brain picks one, and how to catch it choosing.
Look first
One light, many worlds
A grey tile sends your eye a certain amount of light. That amount is the light falling on the tile multiplied by the share of it the tile reflects. From the product alone there is no way back to the two things multiplied.
The patch on the left below is the light that arrives: one fixed grey. Move the slider to say how much light you suppose is falling on the surface. The patch on the right is the surface that must then be there.
Bright light, dark tile. Dim light, pale tile. The light at the eye is the same in every case.
One arriving light, and the surface it implies under each assumption about the lighting.
Commit to a prediction
Two tiles, the same light
On the floor at the top of this page, one tile out in the open and one tile inside the shadow have been given exactly the same value in every pixel. Not nearly the same: the same number.
When they are pointed out to you, will the two tiles look the same?
The two marked tiles have the same value in every pixel. Join them with a strip of that same grey, or hide the rest of the floor, and check.
To almost everyone the tile in the open looks dark and the tile in the shadow looks light, and no amount of knowing better changes it. Edward Adelson published a picture of this kind in 1995. It has been fooling people who know exactly how it works ever since.
It is not an error. A shadow that lets through 36% of the light makes a light tile send just what a dark tile sends in the open. Faced with that light, the brain reports the two surfaces most likely to be there: a dark tile here, a light tile in shadow there. It is right about the floor. It is wrong only about the pixels, which it was never trying to tell you about.
Why the first guesses made sense
You do not see light
If you said yes, you took seeing to be a measurement of the light. Chapters 17 and 18 could be read that way: three numbers, edited, sent to the brain. But what you are aware of is not the three numbers. It is tiles, shadows, a floor.
The brain’s business is the world, and the light is only its evidence. Where the evidence fits two worlds equally, it gives you the likelier one, whole and finished, with no sign that a choice was made.
The rule
Weigh each world by how likely it was
Hermann von Helmholtz, who had measured the speed of a nerve signal and invented the instrument for looking into the living eye, gave this a name in 1867: unconscious inference. Seeing, he argued, has the form of a conclusion drawn from evidence, though we draw it without knowing and cannot undo it by knowing better.
The modern form of his idea is a rule from the theory of chances. For each world that could have sent this light, take how well it fits the light, and multiply by how likely that world was before you looked. The world with the largest product is the best guess.
What the brain expects beforehand is called a prior. The floor of tiles relies on one: shadows are common, and paint that happens to change exactly at a shadow’s edge is rare.
What was expected, what the evidence says, and their product: the best guess lies between them, nearer whichever is more certain.
A case you have not seen
A white page under two lamps
Read a page of white paper by a filament lamp in the evening, a glow at 2,800 kelvin. Read the same page the next day by a window, in daylight at 6,500 kelvin. It is a white page both times.
How different is the light that leaves the page?
The light leaving a white page under each lamp, from chapter 13’s law, and the two lights as one screen shows them side by side.
For each unit of red light, the page by the window sends your eye 1.35 units of blue. The page under the lamp sends 0.19: seven times less. Side by side on one screen the two are white and amber. Yet in each room the page looks white.
The brain does not take the cone signals at face value. In effect it divides each by what the lighting as a whole is giving that kind of cone, a rule named for the physiologist Johannes von Kries. Do that sum for the white page and the answer is the same under both lamps. Do it for a green leaf and the two answers agree to within 16%, where the raw signals differed threefold in their balance of blue.
A camera has no such rule built in. It must be told the lamp, or guess it, before it can show you a white page.
One more case
Bumps, turned over
A flat picture of round shapes, each shaded from light to dark across its width. Shaded light at the top, a disc looks like a bump standing out of the page.
Turn the picture upside down. What do the bumps become?
Twelve discs: six shaded light on top, six shaded light underneath. Nothing changes as the discs turn except which way is up.
Most people see six bumps and six dents, and when the picture is turned over, each bump becomes a dent and each dent a bump. Turn it a quarter of the way and they waver.
The picture cannot say which is which. A bump lit from above and a dent lit from below give the same shading, point for point. The brain settles it with a prior a lifetime under the sky has earned: light comes from above. Turn the picture, and the same prior gives the opposite answer.
That prior is not fixed. In 2004 Wendy Adams, Erich Graf and Marc Ernst gave people active experience of a scene in which the usual assumption was wrong. Their sense of where light comes from shifted, and the shift carried over to a different task.
The bench
A guesser you can adjust
Two ties in the light, and the assumption that breaks each. Change the assumption and watch the answer change, while the picture stays as it is.
Things to try
- With the shaded disc, set the expectation of light from above to nothing. What is the best guess now, at any angle?
- Turn the disc until it is lit from the side. Does a strong expectation help?
- Switch to the striped cloth. Find a light under which it is white and gold, and one under which it is blue and dark. Which pixels changed?
What you can now say
The light at the eye fits many different worlds. Seeing is the brain’s best guess at which one is there: the world that fits the light and was likeliest beforehand. You are shown the guess, never the light, and never the choosing.
You have now caught it choosing three times: a tile, a page, a bump.
For the determined reader: the rule, the models and the limits
Bayes’s rule
For a world W and the light received:
chance of W, given the light ∝ (chance of that light, given W) × (chance of W beforehand)
For the shaded disc there are two worlds that give the light equally well, so the answer is the ratio of their priors alone. The bench models the expectation of light from above as a smooth preference round the circle, strongest straight up, of adjustable strength. For two uncertain measurements of one thing, the same rule says to weight each by how certain it is. Marc Ernst and Martin Banks showed in 2002 that people judging the height of a block by eye and by hand combine the two in very nearly this way.
The lamp and the page
The lamps are the ideal glowing bodies of chapter 13, the cone signals are chapter 17’s, and the green surface is a model: it reflects 8% of all light and up to 60% in a band around 550 nanometres. Von Kries’s division is the simplest rule that works, and it is not the whole of what the visual system does.
Where this stops
Bayes’s rule says what the best guess is. It does not say how a brain could compute it, and whether brains really work this way, or merely end up close to it, is argued over. The priors used here are chosen for the page. Nobody has measured a person’s priors completely, and the three demonstrations work less well on some people than on others.
The dress
The idea that seeing involves judgement is old. Ibn al-Haytham, whose Book of Optics stands first on this exhibition’s timeline, argued a thousand years ago that much of what we take ourselves to see is concluded, not received. Helmholtz gave the idea its modern form and its name.
In 2015 a photograph of a striped dress went round the world, because people could not agree what colour it was. Some saw white and gold. Others, looking at the same picture, saw blue and black. The dress itself was blue and black.
Rosa Lafer-Sousa, Katherine Hermann and Bevil Conway surveyed 1,401 people. Most gave one of those two answers, and some said blue and brown. The pixels themselves, they noted, are brown and blue: the colours of ordinary light, the yellow of lamps and the blue of the sky. A viewer who takes the dress to be in blue shade discounts the blue and sees white and gold. One who takes it to be under a warm lamp discounts the yellow and sees blue and black. The bench works the same sum on a cloth of its own.
Nothing in the picture says which light it was. Each viewer’s brain made a different guess, and showed its owner the result as plain fact.
Where this explanation stops
A guess that feels like a fact
There is something odd in all three demonstrations. You were told the tiles were the same, and shown that they were, and they went on looking different. Knowing does not reach down to where the guess is made.
So the world you see is the brain’s best guess, held in check by the light. Which raises a question. If the guessing machinery is that strong, what does it show you when the light is taken away?
End of chapter 19
Added to your notebook
The light at the eye fits many different worlds. Seeing is the brain’s best guess at which one is there: the world that fits the light and was likeliest beforehand. You are shown the guess, never the light, and never the choosing.
Take the light away, and leave the machinery running. What does it show?
Chapter 20 · Seeing Without Light