Center for Interdisziplinary Research

Embodied Communication I

Date: October 5 - 8, 2005
Organizers: Ipke Wachsmuth (Bielefeld), Günther Knoblich (Newark)

Opening Conference of the ZiF: Research Group 2005/2006 "Embodied Communication in Humans and Machines"

The response to the organizers' invitation was overwhelming and so was the program: 31 talks were held by highly acclaimed speakers in four days on topics such as the rediscovery of the body in cognitive science, the physiology of embodiment, the connection of gestures, words and thoughts, dynamical systems, verbal communication, communication with machines, action and interaction, and social embodiment. About 60 researchers from the USA, Canada, Japan, Austria, France, Italy, England, Sweden, and Germany contributed to the success of the conference. A pointlight dance performed by Bielefeld sports students and a steelband provided examples of embodied communication in joint action coordination.

Embodied Communication

A "silent hero" which stood in the background of many talks was the mirror. That there are neurons in the brain of the monkey and most probably in that of humans as well that do not only engage in performing actions but also in perceiving them, gave rise to multiple speculations about the mechanism underlying action comprehension and social understanding. Leonardo Fogassi from the neuroscience laboratory in Parma where the mirror neurons were discovered provided an overview of the state of mirror neuron research. He emphasized that what we perceive when watching an action is not movement but goal-directed behaviour. The mirror system may also be responsible for the astounding capability of humans to detect human motion from very subtle cues, psychologist Bennett Bertenthal (Chicago) added. It may be the case that observing human motion "resonates" in the motor system of the observer which makes human motion special to humans. But it is a likely myth, brain scientist Michael Arbib (Los Angeles) warned, that the mirror neurons are sufficient to understand human understanding. For an embodied neurolinguistics multiple mirror systems in the broader context of the brain are required. Psychologist Wolfgang Prinz (Leipzig) discussed two kinds of mirrors: those inside and those outside the individual. The inside mirror is provided by representational devices for matching action production and action perception, whereas the outside mirror is provided by social practices and interactive habits in shared action. Inner and outer mirrors make it possible that individuals can perceive their own acting through the action of others.
The mirror system hypothesis provides one argument for the fundamental claim of the research year, namely that an abstract process of encoding, transmitting and decoding information is by far not the whole story about communication. More arguments came from the large field of gesture studies. Adam Kendon (Naples), one of its pioneers, laid out the physical characteristics that makes gestures stand out as meaningful body movements: they have a sharp beginning and end by the hands returning to their starting point without acting on the environment. David McNeill (Chicago), the other gesture authority, took an inside view on gestures and highlighted the connection of gesture and thought. He introduced the "growth point" as a minimal mental unit in which imagery and linguistic content are combined to unfold in a multimodal utterance. The growth point, McNeill emphasized, may also be shared in interacting partners. Philosopher Joëlle Proust (Paris) tried to relate gestures to the distinction of an unconscious body schema and a conscious body image. While it appears that most gestures are not consciously monitored and executed, gestures may also be planned, controlled and conscious.
A promising way to find out how isolated aspects are important for the whole behaviour is to build a robot or a simulation of an agent and see how people react to them. Computer scientists Stefan Kopp and Timo Sowa (both Bielefeld) reported on how they provided the artificial agent Max with the capability to imitate natural gestures presented by a human demonstrator which is one step on the way to build a competent conversational agent. Another step in this direction is taken by computer scientist Catherine Pelachaud (Paris) who described how to extract descriptions of emotions from real world data and how to implement them in a virtual embodied conversational agent.
Switching from virtual agents to real robots physicist Shuji Hashimoto (Tokyo) presented a comprehensive survey on what he calls KANSEI communication. Kansei is the Japanese word for sensibility, sensuality, and emotion. A humanoid robot, Hashimoto explained, needs physics, logic and Kansei, and gesture is one of the most direct ways to express Kansei. Toyoaki Nishida (Kyoto) drew the attention to unexpected human reactions when he demonstrated an automated chair that followed the person around in the office, and a robot that asks a human not to interrupt rudely a conversation.
Emotion is one of the most important fields where the body comes in for communication. Emotions change the whole body and thus its reaction to a situation, explained psychologist Arthur Glenberg (Wisconsin-Madison). And because emotion systems differ across the genders, creating systems capable of human-computer communication may be even harder then suspected.
Gregor Schöner (Bochum) and J. Scott Jordan (Normal, Illinois) reminded the audience of the tools it takes to formally describe human communication. They may not be found in classical mathematics but more likely in dynamic systems theory which sometime fits quite naturally to what we all seem to be: "wild cognitive systems".
All together communication is a highly social endeavour. This became apparent when linguists Elisabeth Ahlsén and Jens Allwood (both Göteborg) presented a theory of own communication management where turn-taking and turn-holding is analysed in face-to-face interaction. Psychologists Natalie Sebanz and Günther Knoblich (both Newark) found that the social context shapes the perception of physical objects. For instance, a box full of potatoes is rated less heavy when another person is present. Uta and Chris Frith (both London) asked a simple question that nevertheless entails most of the issues discussed in the conference: How does an idea get from my mind into yours? They suggested three principles - prediction, contagion by mirroring, and mutuality - by which the social loop in two-person interactions is closed.

ZiF - Center for Interdisciplinary Research - Homepage > List of ZiF Workshops > 2005 >