Scaffolding behavior has a multifaceted property (e.g., sharing goal, the exaggerated verbal/nonverbal instruction, and direct support in some cases). Here, I introduce our previous study focusing on the minimal component of the scaffolding in an imitation learning situation and the analytical technique to disentangle it. This will help understand what the scaffolding is and provide insights to develop XAI at the first step.

In dyadic explanations, interlocutors adapt cognitively as well as verbally to one another as the interaction unfolds. Crucial for cognitive adaptation is the so-called partner model, which refers to the process of inferring the partner’s mental states. Initial partner models are formed via stereotypes, cues or prior assumptions and continuously get updated during the interaction. To reveal how these two forms of adaptivity serve scaffolding, we hypothesize that specific patterns of cognitive adaptivity will co-occur with verbal adaptivity.

Can explainees’ reactions scaffold the structure of explanations during the moment of conversation? In our study, we investigate whether explainees’ multimodal behavior can trigger a change in the episodic structure of doctor’s explanations of an upcoming surgery. Adapting previous research on explanation episodes (Roscoe & Chi, 2007), we analyze patients’ multimodal behavior (gaze, gestures and backchannelling) during transitions between different blocks of explanations.

Whereas scaffolding is mostly comprised as clear guidance for a task and what to do, the use of negation suggests what not to do. We investigated whether negation has an influence on basic processes such as the distribution of attention across the visual field. In an experiment with pairs of colored items, instructions for “now red!” caused a large attentional weight. When the instruction was negated “not green!”, the attentional weights were more balanced. This points toward the possibility to scaffold attention to a more balanced distribution by negation and will be tested in a human-robot interaction.

Emotions play an important role in tutoring situations and affect understanding processes in different ways. Whereas in parent–child interactions, the display of ostensive affective signals such as surprise or joy can support the understanding of action explanations, in other interactions such as with decision support systems, emotions may hinder the processes of understanding and action taking. We therefore develop an interactive system that is capable of scaffolding emotional processes in order to avoid such negative effects.