© Universität Bielefeld
Laughter in Conversation: Form, Function and Multimodality (LAUGH)
The LAUGH project investigates laughter in spontaneous conversations, from the prism of its form, its function, as well as its multimodal realization. It aims to offer a more connected view of the use of laughter in conversation, one which takes into account these three aspects, together with their interplay, in an interactive setting. First, it will investigate the laughter dimensions in which interlocutors become more similar during their interaction, whether this process varies with conversational context, and whether it has a functional role in conversation. Next, the project will complement existing knowledge on multimodal laughter production with an analysis of the co-occurring gestures and their relation to the type of laughter. Then, it will study acoustic-prosodic cues able to discriminate laughter from speech, which are robust to variability sources present in laughter. Finally, an exploratory investigation will performed to evaluate the findings of the project, by means of interaction experiments with a dialogue system implementing laughter.
The project is funded by the German Research Foundation - DFG, for the period 2021-2024.
Findings
The project first explored the process of laughter entrainment at a previously unstudied level, the segmental level. The conducted acoustic studies of conversational laughter showed that interlocutors become more similar throughout the conversation in the way they produce laughter vowels (in terms of their formant values) [LSW22] and consonantal segments (with respect to several spectral characteristics) [SWL23]. However, we also found that laughter entrainment is modulated by the type of conversation considered (free vs. task-based dialogue) [LW23], raising the question whether entrainment at other linguistic levels might be influenced by conversation type. The project also examined the use of laughter to mark discourse structure [LS22]. It revealed that laughter is one of the cues speakers may chose to employ for marking speaker turns, the odds of laughter events at turn ends being double than their odds at transition relevance places. Moreover, laughter seems to favour turn changes rather than turn holds in conversation.
Next, we investigated the acoustic characteristics of laughter, by looking at two prosodic cues, intensity and voice quality. Our findings showed that both measures are able to differentiate between laughter and speech, as well as to distinguish between laughs, speech-laughs and speech [LW22]. Employing only features which have been shown to discriminate laughter from speech, we developed a laughter detection system [L23] which obtained similar performances on the SSPNet corpus to approaches using more complex feature sets or learning algorithms. This study also revealed that less-commonly used features for laughter detection, characterizing the voice quality and the speech modulation spectrum, are more useful for speech/laughter classification than intensity and f0.
The analysis of laughter form was extended to include its multimodal realization, by examining the co-occurrence of laughter and gestures [LSRW23]. Our findings point towards a specific role for the head gestures preceding laughter, based on the fact that their distribution differs from overall gesture distribution. Moreover, we observed a link between gestures and laughter form, with the presence of head gestures increasing the odds of producing speech-laughs, rather than laughs. The multimodal expression of laughter was explored also for instances marking listener feedback [RSLZ23], noticing differences in form between laughter and other backchannel items, both acoustically and in terms of head gesture co-occurrence.
Publications
[LW23] Ludusan, B. and Wagner, P. (2023). The effect of conversation type on entrainment: Evidence from laughter, in Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 168-174, Prague, Czechia.
[L23] Ludusan, B. (2023). The usefulness of phonetically-motivated features for automatic laughter detection, in Proceedings of the Disfluency in Spontanous Speech (DiSS) Workshop, pp. 33-37, Bielefeld, Germany.
[LSRW23] Ludusan, B., Schröer, M., Rossi, M., and Wagner, P. (2023). The co-use of laughter and head gestures across speech styles, in Proceedings of the Annual Conference of the International Speech Communication Association – INTERSPEECH. Dublin, Ireland.
[RSLZ23] Rossi, M., Schröer, M., Ludusan, B., and Zellers, M. (2023). A multimodal account of listener feedback in face-to-face interactions, in Proceedings of the International Congress of Phonetic Sciences, pp. 4120-4124, Prague, Czechia.
[LSW22] Ludusan, B., Schröer, M., and Wagner, P. (2022). Investigating phonetic convergence of laughter in conversation, in Proceedings of the Annual Conference of the International Speech Communication Association – INTERSPEECH, pp. 1332-1336, Incheon, South Korea.
[LS22] Ludusan, B. and Schuppler, B. (2022). To laugh or not to laugh? The use of laughter to mark discourse structure, in Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 76-82, Edinburgh, United Kingdom.
[LW22] Ludusan, B. and Wagner, P. (2022). ha-HA-hha? Intensity and voice quality characteristics of laughter, in Proceedings of the International Conference on Speech Prosody, pp. 560-564, Lisbon, Portugal.
Other outputs
[SWL23] Schröer, M., Wagner, P. and Ludusan, B. (2023). Do consonants converge in laughter?, Poster presented at P&P 2023, Bern, Switzerland.
Ludusan, B. (2023). Speech prosody: from language characterization to paralinguistic use, Invited talk at the Laboratoire Interdisciplinaire des Sciences du Numérique, Orsay, France.
Ludusan, B. (2023). Laughter as a communicative element, Invited talk at the Institute for Scandinavian Studies, Frisian Studies and General Linguistics, Kiel University, Germany.
Ludusan, B. (2023). Laughter function annotation (agreement), Practical session at the 2nd Graz-Wien Speech Workshop, Austria.
Ludusan, B. (2023). Why should a virtual agent use laughter?, Invited talk at the Joint Computational & Digital Linguistics Groups Meeting, Bielefeld University, Germany.
Ludusan, B. (2022). Communicative and linguistic aspects of laughter in conversation, Invited talk at the Phonetics Colloquium, Saarland University, Germany.