Using Virtual Reality to make experiments more realistic

Avatars are all around us: they represent real people online and colonise new worlds in the movies. In science, their role has been more limited. But avatars can be extremely useful in linguistics, new research shows. Scientists from the Max Planck Institute for Psycholinguistics use virtual avatars to investigate how real people behave in interaction. The method makes it possible to study with great precision how people adjust to each other in conversation.

An exciting question in psycholinguistics is how people adapt their speech to each other in conversation. The research method of choice has long been the “confederate”: a conversational partner who, without the other participant knowing, has been instructed to speak in certain ways. However, some aspects of speaking cannot be studied in this way, for instance minute changes in speech rate or intonation. Evelien Heyselaar, Peter Hagoort and Katrien Segaert have exchanged the human confederate for a virtual reality avatar, opening up exciting new possibilities for studying the dynamics of dialogue.

Best of both worlds

To get around the problems of human confederates, some studies have used pre-recorded sound files instead of confederates. However, speaking to a sound recording can feel very different from speaking in a face-to-face context, and indeed  studies have shown that language behaviour is different when interacting with an sound recording compared to a video. Heyselaar and colleagues set out to determine whether virtual reality could be used to combine the best of the confederate and sound-only methods.

Virtual reality makes it possible to create a physical presence, an avatar, whose behaviour and speech patterns we can control to the tiniest detail. Heyselaar and colleagues decided to test the effectiveness of this avatar using syntactic priming, a phenomenon in which participants adapt their grammar usage to match that of the other. Because syntactic priming is well understood, it provides the ideal testing ground for an implementation using virtual reality.

Human-like avatars

Recent work has suggested that the degree to which people adapt can be influenced by their opinion of the person they are interacting with. Heyselaar and colleagues invited people to do a card game with either a human confederate, a human-like avatar, or a computer-like avatar. The two avatars looked identical and differed only in their behaviour: the human-like avatar showed realistic facial expressions such as smiles, eyebrow raises and eye contact, while the computer-like avatar had none of these.

As predicted, the human-like avatar worked just as well as the human confederate: both resulted in the same amount of grammatical adaptation, whereas the computer-like avatar did not. These results suggest that future conversational experiements can use virtual reality avatars and investigate behaviour in ways that were previously impossible.

MD/MEZ