If you ask someone directions and they point LEFT but say RIGHT which way do you think you’d go? In this situation gesture generally takes precedence over speech – you’re likely to go left. However, in face-to-face conversation, gesture is not closely monitored by speakers of the co-produced language, and as such, it offers new insights to the social and cognitive processes involved in linguistic production and the structuring of discourse interaction. A single gesture may have a content meaning or an interactional and discourse-regulating function, depending on context.
In this talk we lay out the kinds of gestures found in face-to-face conversation (emblems, propositional gestures, spontaneous gestures) and highlight some of the primary non-verbal social cues found in conversation. In particular, we will focus on the role of gesture in embodied face-to-face human interaction. We will highlight issues that an absence of gesture cues might raise for virtual interactions, particularly in the areas of general dialogue management across distributed environments and more localized dialogue management across conversational function. This includes the role of gesture and conversational behaviours (such as eyebrow raising), in turn-taking, and in miscommunication and repair management. For example, head movements such as nods may replace or augment verbal content and are typically contiguous with gaze and particular eyebrow movements. Other seemingly unconscious movements such as head tilts often indicate that processing time is needed.
In concluding, we will ground the discussion of nonverbal behaviours seen in face-to-face conversation by looking at those that would be most useful when implemented in embodied conversational agents.
Authors: Barbara Kelly and Alexis Tabensky
Event: SF08: Embodied Interaction in Mobile, Physical and Virtual Environments Workshop