Multimodal communication: resources and application University of Debrecen Department of General and Applied Linguistics HuComTech Research Group February 11-13, 2015

Our interest in multimodal communication derives from a EU-Hungary joined financed project, HuComTech that has been running at our university since 2009. One of its aim is to learn those aspects of human-human communication that are relevant and implementable for human-machine interaction systems. With this interest, in addition to it being highly relevant to several areas in the humanities and social sciences (linguistics, psychology, sociology etc.)  the project expresses the importance of the contribution of digital humanities for the enhancement of engineering and information science as well. With this, DH emphasizes its ever increasing role in the well-being and development of the wider society, a purpose it has been actively and successfully pursuing in the past decades.

Communication is essentially multimodal: it encompasses both verbal and non-verbal aspects of communication. Therefore, in order to better understand communication, one needs to collect, describe and understand communication data from multiple sources and modalities. The fact that none of the components of communication are obligatory and that sequences of events can not be strictly predefined being relatively optional represents a rather unexplored field in our knowledge. Therefore, the present workshop was expected to yield important scientific and educational results for the participants and those beyond the workshop as well.

The invited speakers represented several geographic regions in Europe, and the participants were also selected to represent both several countries and degrees of expertise. Special care was given to the sponsoring of PhD- and MA-students from across Europe who, in addition to being able to attend the valuable talks and discussion, were also given the chance to present their own work (which they did with much enthusiasm).

Special highlights of the scientific program included presentations of such resources and applications as the new and groundbreaking research methodology embedded in the software Theme and the virtual environment for collaborative research, VirCA. 

The program of the workshop was as follows:

February 11, Wednesday: arrival

February 12, Thursday

10:00        Opening remarks (Laszlo Hunyadi, University of Debrecen):
10:00    -11:30: Anna Esposito ((UNINA2/IIASS, Caserta): Emotions in context
11:30-11:45: Coffee break
11:45-13:15: Jens Edlund (KTH, Stockholm): Humanlikeness in spoken dialogue systems
13:15-14:00: Lunch
14:00-15:30: Gábor Prószéky (Péter Pázmány Catholic University, Budapest): Computational Parsing Using Psycho-linguistic Principles
15:30-17:00: Péter Baranyi (SZATI and Technical University, Budapest)): Demonstration of the VirCa collaboration platform
19:00: Dinner

February 13, Friday

9:00-9:10: Lorna M. Hughes (School of Advanced Studies, University of London): Inter- and multidisciplinarity in Digital Humanities
9:10-11:25: Presentations by PhD-students
9:10-9:30: Ágnes Abuczki (University of Debrecen):
9:30-9:50: Simon Alexanderson (KTH, Stockholm): Multimodal corpus collection for data-driven animation
9:50-10:10: Ghazale Esfandiari Baiat (University of Debrecen): The observer Experiment: A view at dynamics of conversation from the outside
10:10-10:25: Coffee break
10:25-10:45: Anna Lambrechts (UCL, London): Temporal Dynamics of Speech and Gesture in Autism Spectrum Disorder
10:45-11:05: István Szekrényes (University of Debrecen):
11:05-11:25: Gergely Varjasi: (t.b.a.)
11:25-11:40: Coffee break
11:40-13:10: Gudberg Jonsson (Human Behavior Laboratory, University of Iceland): DETECTION OF T-PATTERNS: The T-language and THEME
13:10-14:00: Lunch
14:00-16:00: Gudberg Jonsson: Hands-on demonstration of THEME
16:00: Closing of the workshop (Laszlo Hunyadi)

The workshop proved to be highly successful: it generated much interest among the participants, both professors and novices (with special emphasis on students). At least two direct outcomes can be mentioned here: a project proposal submitted for the multimodal analysis of communication using Theme and the established network of PhD-students who may be convening in the coming months to exchange their ideas in the issues covered during the workshop.
We are grateful for NeDiMAH for having supported the event and given us the opportunity to further develop our research connections in this rapidly growing field of digital humanities.

Laszlo Hunyadi

Groups audience: