The Shape of LightAn interactive exhibition about light

The fourth room · Behind the eyeChapter 19 of 21

Computed: a tiled floor with a shadow across it. Somewhere on it are two tiles, one light and one dark, that are exactly the same grey.

The light from a surface depends on two things at once: what the surface is, and what is shining on it. Your eye receives only their product. Your brain hands you the surface, and you never see the sum it did to get there.

Chapter 19 · Behind the eye

The Best Guess

The same light can come from many different worlds. How the brain picks one, and how to catch it choosing.

Look first

One light, many worlds

A grey tile sends your eye a certain amount of light. That amount is the light falling on the tile multiplied by the share of it the tile reflects. From the product alone there is no way back to the two things multiplied.

The patch on the left below is the light that arrives: one fixed grey. Move the slider to say how much light you suppose is falling on the surface. The patch on the right is the surface that must then be there.

Bright light, dark tile. Dim light, pale tile. The light at the eye is the same in every case.

calculated
One arriving light, and the surface it implies under each assumption about the lighting.

Commit to a prediction

Two tiles, the same light

On the floor at the top of this page, one tile out in the open and one tile inside the shadow have been given exactly the same value in every pixel. Not nearly the same: the same number.

When they are pointed out to you, will the two tiles look the same?

Why the first guesses made sense

You do not see light

If you said yes, you took seeing to be a measurement of the light. Chapters 17 and 18 could be read that way: three numbers, edited, sent to the brain. But what you are aware of is not the three numbers. It is tiles, shadows, a floor.

The brain’s business is the world, and the light is only its evidence. Where the evidence fits two worlds equally, it gives you the likelier one, whole and finished, with no sign that a choice was made.

The rule

Weigh each world by how likely it was

Hermann von Helmholtz, who had measured the speed of a nerve signal and invented the instrument for looking into the living eye, gave this a name in 1867: unconscious inference. Seeing, he argued, has the form of a conclusion drawn from evidence, though we draw it without knowing and cannot undo it by knowing better.

The modern form of his idea is a rule from the theory of chances. For each world that could have sent this light, take how well it fits the light, and multiply by how likely that world was before you looked. The world with the largest product is the best guess.

What the brain expects beforehand is called a prior. The floor of tiles relies on one: shadows are common, and paint that happens to change exactly at a shadow’s edge is rare.

calculated
What was expected, what the evidence says, and their product: the best guess lies between them, nearer whichever is more certain.

A case you have not seen

A white page under two lamps

Read a page of white paper by a filament lamp in the evening, a glow at 2,800 kelvin. Read the same page the next day by a window, in daylight at 6,500 kelvin. It is a white page both times.

How different is the light that leaves the page?

One more case

Bumps, turned over

A flat picture of round shapes, each shaded from light to dark across its width. Shaded light at the top, a disc looks like a bump standing out of the page.

Turn the picture upside down. What do the bumps become?

The bench

A guesser you can adjust

Two ties in the light, and the assumption that breaks each. Change the assumption and watch the answer change, while the picture stays as it is.

Things to try

  1. With the shaded disc, set the expectation of light from above to nothing. What is the best guess now, at any angle?
  2. Turn the disc until it is lit from the side. Does a strong expectation help?
  3. Switch to the striped cloth. Find a light under which it is white and gold, and one under which it is blue and dark. Which pixels changed?

What you can now say

The light at the eye fits many different worlds. Seeing is the brain’s best guess at which one is there: the world that fits the light and was likeliest beforehand. You are shown the guess, never the light, and never the choosing.

You have now caught it choosing three times: a tile, a page, a bump.

For the determined reader: the rule, the models and the limits

Bayes’s rule

For a world W and the light received:

chance of W, given the light ∝ (chance of that light, given W) × (chance of W beforehand)

For the shaded disc there are two worlds that give the light equally well, so the answer is the ratio of their priors alone. The bench models the expectation of light from above as a smooth preference round the circle, strongest straight up, of adjustable strength. For two uncertain measurements of one thing, the same rule says to weight each by how certain it is. Marc Ernst and Martin Banks showed in 2002 that people judging the height of a block by eye and by hand combine the two in very nearly this way.

The lamp and the page

The lamps are the ideal glowing bodies of chapter 13, the cone signals are chapter 17’s, and the green surface is a model: it reflects 8% of all light and up to 60% in a band around 550 nanometres. Von Kries’s division is the simplest rule that works, and it is not the whole of what the visual system does.

Where this stops

Bayes’s rule says what the best guess is. It does not say how a brain could compute it, and whether brains really work this way, or merely end up close to it, is argued over. The priors used here are chosen for the page. Nobody has measured a person’s priors completely, and the three demonstrations work less well on some people than on others.

The dress

The idea that seeing involves judgement is old. Ibn al-Haytham, whose Book of Optics stands first on this exhibition’s timeline, argued a thousand years ago that much of what we take ourselves to see is concluded, not received. Helmholtz gave the idea its modern form and its name.

In 2015 a photograph of a striped dress went round the world, because people could not agree what colour it was. Some saw white and gold. Others, looking at the same picture, saw blue and black. The dress itself was blue and black.

Rosa Lafer-Sousa, Katherine Hermann and Bevil Conway surveyed 1,401 people. Most gave one of those two answers, and some said blue and brown. The pixels themselves, they noted, are brown and blue: the colours of ordinary light, the yellow of lamps and the blue of the sky. A viewer who takes the dress to be in blue shade discounts the blue and sees white and gold. One who takes it to be under a warm lamp discounts the yellow and sees blue and black. The bench works the same sum on a cloth of its own.

Nothing in the picture says which light it was. Each viewer’s brain made a different guess, and showed its owner the result as plain fact.

Where this explanation stops

A guess that feels like a fact

There is something odd in all three demonstrations. You were told the tiles were the same, and shown that they were, and they went on looking different. Knowing does not reach down to where the guess is made.

So the world you see is the brain’s best guess, held in check by the light. Which raises a question. If the guessing machinery is that strong, what does it show you when the light is taken away?

End of chapter 19

Added to your notebook
The light at the eye fits many different worlds. Seeing is the brain’s best guess at which one is there: the world that fits the light and was likeliest beforehand. You are shown the guess, never the light, and never the choosing.

Take the light away, and leave the machinery running. What does it show?

Chapter 20 · Seeing Without Light