Open Science - Shaping science together
With the “Open Science” movement, scientists make their research processes visible to the public by clearly presenting their methods and sharing their data. This allows for fellow scientists and interested parties to understand the underlying projects and processes the scientist originally used in their research, and if necessary, collaborate on improvements beyond the confines of institutions.
“We raise the quality of our research by bringing transparency to the process” says Professor Dr. Katharina Rohlfing, spokesperson of TRR 318. “Within the Transregio, we are already working across disciplines: researchers from various scientific fields come together, exchange ideas and thus enhance each other’s knowledge. With Open Science, we are looking for exactly this kind of exchange with experts outside of the project.”
Resources
This data publication contains the annotations of MUNDEX (MUltimodal UNderstanding of EXplanations), a multimodal corpus for studying the understanding of explanations created in project A02.
This is a multi-class classification dataset aimed at predicting players’ positions based on various features such as height, shots per match, in-game minutes per match, and others.
The ADEX linguistic coding scheme is used to annotate the utterances of the interlocutors as they explain a game. This annotation includes the spoken and its content classification. TRR researcher Josephine Fisher developed the coding scheme in project A01, Adaptive explanation generation.
Additional coding schemes and instruments of the project are available on OSF: osf.io/daqv9
Researchers of the INF project created a corpus of dialogical explanations to enable NLP research on how humans explain and how AI can learn to imitate this process. The corpus comprises 65 transcribed English dialogues with 1,550 manually labeled dialogue turns from the Wired video series "5 Levels", explaining 13 topics to five explainees of different proficiency.
More information in the research article
To study how well current Large Language Models can lead co-constructive explanation dialogues, researchers from the INF project collected a dataset of human-LLM explanation interactions, as well as pre- and post-questionnaires regarding the users' understanding. The data was subsequently analyzed and published in a research paper. To foster further analysis and evaluations on this topic, the data were made publicly available.
Link to the data: GitHub - webis-de/sigdial25-co-constructive-llms-data
Data set of metaphors of explaining and understanding produced and explained by approx. 300 participants. German and English data. Verb metaphors and nominalized verb metaphors are coded. MIPVU (Steen et al., 2010) was used for metaphor identification.
The data set is part of ongoing research on the function and usage of metaphors in explaining. The data set contains transcripts from 24 "5 Levels" videos series from Wired. In these videos, an expert explains an abstract and complex concept to people with 5 different levels of expertise. Metaphors in these dialogues are coded.
This corpus is an extension upon the existing FLUTE dataset, with addition of source and target domains. Originally built for figurative natural language inference tasks, the addition of the source and the target domains, besides the already existing explanations, are designed to leverage the information of the domains in downstream NLI tasks.
This corpus is an annotation of 300 news editorials from New York times, with their respective metaphors, and related source and target domains. The metaphors and the domains correlate significantly with persuasiveness of the reader's stance on the topic in play, depicting how metaphors and their domains are used in everyday explanatory language.
The short questionnaire was develped in Project A01. It assesses co-constructive behavior of the explainer reported by the explainer and by the explainee.
Scale for the assessment of partner-model dimensions of explainers and explainees.
Scale for the assessment of partner-model dimensions of explainers and explainees.
Decisions made by Artificial Intelligence are not based on a single attribute, but on many. Most attributes interact with each other. Researchers of Subproject C03 developed the openscoure-tool shapiq to better understand these interactions. The tool allows the efficient computation of the attributes and can visualize the interactions.
More information in the GitHub-Repository.
The software was developed by project A05. The main focus of the system is to support the work in interdisciplinary teams and to enable also non-computer scientists to create, discuss and generate adaptive scaffolding strategies for robots in human-robot-interaction.
More information: MAS / Projects / SFB TRR318 / A05 / SHIFT · GitLab
To investigate the influence of stress on social signals related to understanding and confusion, project A06 has designed and pre-registered a study to record and compare these signals between two board game explanation scenarios, one under stress and one without stress.
The TRR Wiki contains a glossary of terms from the field of explainable AI that are used in the subprojects of TRR 318. The Wiki was launched by members of the INF subproject and is continuously being updated and expanded.
Link: The TRR Wiki
Demonstrators from the workshops
The Demeter AI is characterised by the fact that it works according to the principle of reinforcement learning. The AI learns to play a game that is similar to the mobile game Flappy Bird. Users also have the option of playing the game themselves. Otherwise, they can let the AI play and observe how it learns and gets better and better.
Do Science Yourself! - Instructions & tutorials on scientific methods
Git is a free distributed version control system for software projects, released in its first version in 2005. The program allows multiple developers to work on a project simultaneously, regardless of their location.
Version control makes it easy to independently add changes to the project from anywhere, to log and track those changes, and to access older states of the project at a later time. Git is platform independent and can therefore be used in almost any environment.
TRR researchers Vivien Lohmer (A04) and Amelie Sophie Robrecht (A01) have summarised in a tutorial and cheatsheet how Git and sustainable data storage works: