Attention scan: How our minds shift focus in dynamic settings

A person’s capacity for attention has a profound impact on what they see, dictating which details they glean from the world around them. As they walk down a busy street, the focus of their attention may shift to a compelling new billboard advertisement or a shiny Lamborghini parked on the curb.

That attention, however, can be fleeting. When that person reaches a busy intersection, details of that billboard or sports car disappear from their minds. Their attention instead becomes focused on approaching or stationary traffic, the flashing walk sign, and other pedestrians they’ll need to avoid in the crosswalk.

Most research on attention has concentrated on what happens when we notice the new billboard or shiny car. But in a new study, Yale psychologists instead focus on what happens when our attention shifts to a specific goal, such as navigating the busy intersection.

In an article for the journal Psychological Review, they unveil a human attention model that explains how the mind evaluates what is task-relevant in complex, dynamic scenarios—and apportions computational capacity in response.

“We have a limited number of resources with which we can see the world,” said Ilker Yildirim, assistant professor of psychology in Yale’s Faculty of Arts and Sciences and senior author of the study. “We think of these resources as elementary computational processes; each percept we experience, such as the position of an object or how fast it’s moving, is a result of exerting some number of these elementary perceptual computations.”

For the study, they developed a system which they call “adaptive computation,” which is essentially a software program that rations these elementary computations in order to more deeply process goal-relevant objects. For example, when a person crosses a busy street, adaptive computation would prioritize the pedestrian walk sign over the shiny car.

“Our model reveals a mechanism by which human attention identifies what things in a dynamic scene are relevant to the goal at hand, and then rations perceptual computations accordingly,” Belledonne said.

In one experiment, they presented volunteer participants with eight identically colored circles on a computer screen. The researchers then highlighted a group of four circles and asked the participants to track the highlighted circles as all eight circles moved randomly across the screen. Such tracking of multiple objects at the same time elicits a complex, dynamic ebb and flow of attention among the participants.

Researchers measured these shifts of attention, at sub-second thresholds, by asking subjects to hit the space bar whenever they noticed a flashing dot appear very briefly on a specific object. The frequency with which these flashing dots were noticed indicates where and when people are attending, and the adaptive computation model successfully predicted these momentary, fine-grained patterns of attentional deployment.

In another experiment, participants again were asked to track four objects, but in this case the researchers varied how many more identically colored “distractor” objects were on the screen and how fast the objects were moving. When the objects stopped moving, researchers asked participants to rate how difficult it was to track them.

The researchers showed that the adaptive computation model also explains these subjective difficulty ratings: The more computational resources the model exerted on a trial, the more difficult it was rated by participants. In this way, the researchers’ model provided a computational signature of the feeling of exertion that occurs when a person has to focus attention on the same task for a prolonged period, Yildirim said.

“We want to work out the computational logic of the human mind, by creating new algorithms of perception and attention, and comparing the performance of these algorithms to that of humans,” he said.

The model also helps make sense of what’s sometimes considered a “human quirk”: the ability to make perceptions of non-task-oriented objects—such as the billboard or the sports car—disappear while crossing the busy street.

“We think this line of work can lead to systems that are a bit different from today’s AI, something more human-like,” Yildirim said. “This would be an AI system that when tasked with a goal might miss things, even shiny things, so as to flexibly and safely interact with the world.”