How multimodal communication influences explanations

A group of scientists from the Transregio took part in the 1st International Multimodal Communication Symposium (short: MMSYM). At the conference at the Pompeu Fabra University in Barcelona, the researchers of two sub-projects from the project area "A - Explaining" presented their research results on the influence of multimodal forms of communication on explanations.

The linguists Stefan Lazarov and Dr. Angela Grimminger from sub-project A02 "Monitoring the understanding of explanations" presented a poster for a study: The team examined how transitions to elaborations by the explaining person are related to the multimodal behavior of the person to whom something is being explained. Computational linguists Yu Wang and junior professor Dr. Hendrik Buschmeier, also from sub-project A02, presented a poster about a method for detecting head movements. In a session on multimodal annotation and corpora, Dr. Olcay Türk presented the corpus created in collaboration with the other researchers of project A02 to study the comprehension of explanations. Linguistics PhD student Vivien Lohmer presented a poster on the use of gestures in everyday explanations from her collaboration with Lutz Terfloth and Professor Dr. Friederike Kern. All three are conducting research in subproject A04 "Integrating the technical model into the partner model in explanations of digital artifacts".

The first international symposium in Barcelona emerged from a series of symposia that have dealt with the topic of multimodal communication since 1997, first on a regional level in the Nordic region, and since 2013 on a European level. This year, the symposium was held in Spain for the first time. The second edition of this international conference is planned for 2024.

Further Information

Dr. Olcay Türk presents the corpus created by A02 on multimodal communication
Stefan Lazarov presents the poster prepared by him and Dr. Angela Grimminger
Vivien Lohmer at the poster session