Mirror in the brain

When people work together, they have to coordinate their actions very closely. Wolfgang Prinz, Director at the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, and his colleagues are investigating precisely what goes on in their heads in the process.

Text: Peter Zekert

Playing in a duet or a musical ensemble means harmonizing with others in more than one sense. The sounds produced by the musicians will combine to form a unified voice only if each note played by the individual members complements those produced by the rest of the group. And musicians are not the only ones who need an acute awareness of the other people around them. We all need this in the course of our everyday lives, which consist of a series of major and minor social interactions in which we repeatedly and intuitively adapt to other people.

Whether giving way as pedestrians to other oncoming pedestrians, shaking hands with another person, helping someone carry a sofa up a staircase, dancing or playing basketball, all of our actions must be coordinated with those of other people. But how is it that we always realize so quickly what they are going to do? Wolfgang Prinz is interested in the basic processes involved in this kind of joint action. The Director of the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig conducts research into social behavior on the micro level. The fact is that a great deal more cognitive activity lies behind our everyday interaction with other people than we realize.

The way in which the actions we perform in conjunction with others form a unified whole is something that usually functions automatically and something we notice mainly when things go wrong – when the musical ensemble produces the wrong note, when a pass to a team member goes awry, or when a foot lands on a tango partner’s toes.

Thought processes extend the connection

On closer examination, the effortless and silent nature of this coordination, which is accomplished in mere fractions of a second, is truly astonishing. The possession of sensitive “antennae” and the ability to respond quickly is not sufficient to explain it. “People can cooperate with others so smoothly because they usually know in advance what the other person is going to do,” says Wolfgang Prinz.

Behind this capacity lie mechanisms that remain largely unconscious and that have become an object of research only in recent years. For a long time, psychologists assumed that such processes followed a linear scheme. “The sequence perception – cognition – action was accepted as the classic model,” explains the Max Planck researcher. Based on this model, actions we perceive in others would first have to undergo a complex thought process to be understood.

We would then have to weigh up the different action options and initiate the corresponding movements in order to eventually react. “This process would simply be too slow for many of the lightning-fast interactions we engage in every day,” says Prinz.

He quickly came to the conclusion that there had to be a shortcut that leads directly from the perception of an action carried out by another to our own action. Back in the early 1990s, the scientist formulated the theory of common coding, according to which perception and action are linked, at least in part, by shared cognitive and neuronal resources.

Initially, he was rather isolated in this view. However, this situation came to an abrupt end when brain cells were found in Macaque monkeys that did exactly what Prinz had predicted theoretically: the so-called mirror neurons became active in the primates both when they themselves reached for a piece of fruit and when they merely observed another individual carrying out the same action. Neurons with this dual function are still being sought in humans. The fact that they exist is viewed as a certainty as, in humans too, the mere perception of an action can activate the same motoric areas that are also responsible for its implementation.

The consequences of this phenomenon can be observed in everyday life: everyone has experienced how contagious the effect of yawning or laughing can be. Also, when we observe the sitting posture and movement of a partner in a conversation, we often imitate both actions involuntarily. It is suspected that the hub of the human mirror system is located in the premotor cortex. Numerous studies have demonstrated that this area is the pivotal point between sensory perception and action control.

Running through it all in your head first

As the immediate neighbor of the motor cortex, the movement center of the brain, the premotor cortex connects audio- visual input with the planning and implementation of our movements. As a result, it is assigned a key role in our interaction with other people.

It is now assumed that simulation processes that take place in the premotor cortex help us understand other people’s actions. “The behavior of others is translated into action goals there,” explains Wolfgang Prinz. “By acting out what other people do internally, we can understand it better than through logical comprehension. It is then possible to deduce from the internal simulation what a person is likely to do next.”

Natalie Sebanz, a former doctoral student of Wolfgang Prinz, was the first to study this imperceptible influence of such processes on our actions. Today, she conducts research into joint action with her own research group at Radboud University Nijmegen in the Netherlands. Sebanz designed a stimulus-response experiment for this purpose, which was carried out both by one person alone and by two people together.

The test subjects were presented with images of a hand that had either a red or a green ring on its index finger. Depending on the color of the ring, they were instructed to press a button on either their left or their right side as quickly as possible. An additional challenge was incorporated into the experiment in that the finger was shown not only wearing the ring in question but also pointing to the left or right. “The direction in which the finger is pointing is actually irrelevant to the test subjects, as they are supposed to react only to the color stimulus,” says the researcher. However, it is impossible to ignore spatial information. This phenomenon is known as the Simon effect.

Thus, if the finger on the screen pointed left although the button to be pressed was on the right, this delayed the test subject’s reaction. “This effect was of interest in the context of social processes,” says Natalie Sebanz, “because it arises only when one person is responsible for both buttons.” If the test subject operates only one of the two buttons, the Simon effect disappears.

It emerged from Sebanz’s experiment that the effect returns immediately when a partner is positioned at the side of the test subject. “When the color stimulus tells me that it’s my turn, but the finger displayed on the screen is pointing to the person beside me, again, it takes me a moment longer to react,” explains Sebanz. The two people together acted like one person who had two hands to coordinate.

This is due to the fact that both participants not only had their own tasks on their minds, but also the element for which the other person was responsible. This phenomenon is known as co-representation: even if a task is involved in which it is more of a hindrance than not, you keep the other person on your radar – and constantly, at that. Even when it was not an individual test subject’s turn, their brain simulated the other person’s action.

Because the process of observing or imagining the other person’s action already activates the subject’s own motor areas for this action, an impulse arises to complete the action oneself. In order to prevent us from immediately imitating what we observe with others, this action impulse must be suppressed. It was possible to measure the increased brain activity necessary for this via EEG.

Natalie Sebanz suspects that co-representation may be of such fundamental importance for life in social groups that, over the course of evolution, it became automatic and etched in the human brain. “It is in our nature to cooperate with others.”

Always keeping the actions of the others in mind

The Sebanz experiment created the basis for wide-ranging research on joint action. The effect of co-representation on shared tasks is now being examined from a number of different perspectives. One of these concerns making music in a group. Studies carried out with the help of functional magnetic resonance imaging have now shown that simulation activity in the human brain is strongest if the actions we perceive in others also belong to our own action repertoire. For instance, if a sonata is played to both non-musicians and experienced pianists, the motor areas of the latter’s brains are significantly more active – they play along internally.

“Musicians that play in a duet or ensemble are a good example of complex and real-time coordination during joint action,” says Peter Keller, head of the Independent Junior Research Group “Music Cognition and Action” at the Max Planck Institute in Leipzig. Keller, who was born in Australia, comes from a musical family. His sister is a renowned jazz pianist back home and he himself studied both trombone and musicology. “During the many hours we practiced back then, it became increasingly clear to me that playing together is not merely a matter of mastering one’s own instrument, it is an elaborate form of social behavior.”

Bringing oneself in accord with other musicians is an extremely challenging process, explains Keller, because on the one hand, everything must be very accurately coordinated in terms of tempo, while on the other hand, musicians repeatedly deviate from regularity. “Otherwise, their playing would seem mechanical and devoid of individual expression.”

If they hold onto a note for a bit longer, spontaneously change tempo, or play louder or softer, everyone else in the group must adapt to these new departures. This necessitates permanent concentration on the part of the musicians – and in numerous respects: not only must each of the players pay attention to the sounds they produce, but they must simultaneously listen to those produced by the other players and monitor the overall sound. “This requires highly developed cognitive resources,” says Peter Keller.

In order to find out how musicians synchronize with one another and what kind of similarities and differences exist between various forms of joint music-making, such as piano duos, mixed-instrument ensembles and choirs, musicians are regularly invited to perform miniature concerts in the Leipzig laboratory. The focus in this research is on nuances that are barely perceptible for the most part. The researchers use electronic keyboards for their tests, as this allows them to obtain extremely accurate data on the duration and intensity of key strokes, and on the minutest asynchronies in the performances. These are always present and, on average, typically range between 30 and 50 milliseconds in good musical ensembles.

Each one is their own best partner in a duet

The scientists recently demonstrated for the first time that action simulation is also important for the temporal coordination of action. In the study, pianists each practiced one part of several duets that were previously unknown to them. A few months later, they were invited back to play the complementary second part. It was revealed that they could synchronize best with a recorded version of the first part if they had played it themselves.

“The simulated timing coincides best with the actual behavior if both are the product of the same cognitive-motor system,” says Keller. Each pianist was thus his or her own ideal duet partner. Moreover, as Keller and his colleagues observed, pianists playing duets manage to synchronize better with their partners the more similar their movements are while playing. The researchers measured the small differences in the forward and backward swaying of the upper body, which helps keep the player in time, by attaching motion capture markers – which are usually used for lifelike 3-D animation – to the musicians’ backs.

The researchers are currently investigating whether this link also exists for other instruments and larger groups. As a result, loud metallic rhythms can now regularly be heard emanating from their laboratory. Inside, music students from Leipzig can be found sitting in a semicircle on the floor playing Gamelan – a form of music particularly common in Java, Bali and Indonesia – which involves beating small pot-like bronze gongs.

As was the case with the pianists, the researchers also use motion capture techniques here to evaluate the subtlest nuances of movement later at the computer. “This way, you can see details that you might otherwise miss – for example, who orients themselves to whom within the group, who tends to lead the others when playing, and who adopts a subordinate role,” says Keller. It is interesting to note that such relationships often seem to emerge spontaneously. Precisely what lies behind this in cognitive terms is something he can only speculate about at this stage.

This is one of many questions about the social side of our brain that will be investigated in the near future. The researchers at the Max Planck Institute for Human Cognitive and Brain Sciences are studying, among other things, the conditions under which co-representation takes place and whether it is more pronounced if one person knows the other person involved. The scientists would also like to identify the point at which the idea of other people’s action arise in early childhood development.

Although this work is currently classified as basic research, when the mechanisms of joint action are better understood, the outcomes will be of interest for many applied fields, including, for example, cognitive robotics. Researchers in this field have been working for some time on programming artificial social intelligence with a view to endowing industrial robots with soft skills and knowledge of human nature at some point in the future. Music and dance education could also benefit from such a development. And finally, the new insights could also provide a better understanding of disorders that affect people’s capacity for empathy, for example in the case of autism and certain brain injuries, and lead to the development of better approaches in future treatment. 

 

GLOSSARY

Common coding
The theory of common coding states that perception and action are based on the same cognitive processes. This prompts the assumption that they are connected and can interact directly with each other.

Mirror neurons
Nerve cells that become active both when carrying out an action and when only observing an action. They were first described by Italian neurologists Giacomo Rizzolatti and Vittorio Gallese.

Premotor cortex
An area of the cerebral cortex that is responsible for the planning of actions.

Electroencephalography (EEG)
During an EEG, changes in brain activity are measured using electrodes, which are attached to the scalp. Co-representation The phenomenon whereby the actions of one person activate the same neurons in an observer. Functional magnetic resonance imaging (fMRI) An imaging process that renders visible the areas of the brain that become active for certain tasks and stimulus conditions.