Multimodality and AI in Online Education

Multimodality and AI in Online Education

Thursday 26/05 14:00-15:30h

Main Hall

Abstract

Improving the quality and addressing the shortcomings of online education is a research field with roots as deep as its creation that due to the pandemic consequences took centre stage. In traditional physical classrooms, students’ learning states and performances can be observed and interventions can take place in real-time. However, this is hardly the case in remote settings, particularly in the domains of affective (emotions, feelings) and psychomotor (physical movements). Multimodality comes from the theory of embodied communication, in which humans use their bodies to communicate with each other, applying various channels to exchange messages such as gestures, facial expressions, etc. Sensors and AI technology are able to replicate multimodal sensory information (e.g, visual, audio), recognize user inputs, and provide timely feedback in a multimodal manner. In this workshop we are aiming to get information from the participants via group-work sessions that will help tackle our research questions.

Needs Analysis

The topic of this workshop is to explore ways that information acquired from utilising sensors and AI methods can be effectively presented to different stakeholders (teachers and students). From the side of the PhD candidates, this workshop is an opportunity to gather information on two critical aspects, which serve as our main research questions. Namely:

1. What content of the feedback has the higher influence in enhancing education (appropriate modalities, design)?
2. How can the feedback be provided in a meaningful manner to the different stakeholders e.g, students, teachers (frame of interaction – real-time, after-the-fact)?

The purpose of enhancing online learning experiences with the aforementioned technology makes this topic a TEL/EdTech fit. This workshop will give the participating PhD candidates in the field of TEL the opportunity to learn about the benefits of developing new tools that assist online learning by incorporating AI as well to participate in the design phase of such applications.

Learning Objectives

1. They will learn about various aspects of AI in synchronous online education/training.
2. New opportunities introduced by emotions/movements recognition and multimodal feedback.
3. Through group-work they learn about the learning-centered affective states (LCAS)/psychomotor training and their role in education and the design of a “future online classroom/training environment”.

Pre-activities

The participants do not need to follow any particular material before the workshop.

Session Description

Presentation about multimodality in online education and its challenges. (15 min)
Session 1: Content of feedback (35 min)
Group work 1: LCAS
Activity: Card Sorting
Participants will work in small groups. Firstly, the groups will choose which stakeholder they will represent, teacher/student. Secondly, each group will perform a card-sorting task, ranking cards describing different LCAS, e.g., anger, boredom. The ranking criteria will be the effectiveness of the knowledge of each LCAS for each stakeholder. The purpose of this activity will be to gain insights from the participants on the LCAS in an online learning environment.
Group work 2: Design of training environment
Activity: Motion data collection pt. 1
Participants will be asked to perform psychomotor tasks (e.g., yoga) in front of a screen and sensors (e.g., depth cameras), collecting the movement data. A live 3D avatar will be visualized. This session will focus on the selection of suitable sensors and exploration of more performance metrics as a significant step into implementing the feedback system.
Session 2: Communication of feedback (40 min)
Group work 3: Feedback user interface (UI)
Activity: Co-designing
The setup will be similar to Group work 1. This activity will focus on the modalities and design of the UI that will communicate the feedback in real-time to the different stakeholders. Each group will be given a board of wireframe of a prototype online lecture UI. The participants will be asked to design a UI that will be used to provide feedback. The purpose of this activity is to collect insights on how the participants prefer the feedback to be presented to them.
Group work 4: Feedback components
Activity: Motion data collection pt. 2
The set-up will be similar to group work 2 but this session will discuss on the selection of modalities and frame of interaction that are to be given as feedback to the learner. Questionnaires will be provided for the collection of qualitative data.

Post-activities

1. References of the topics will be provided.
2. Contact information of the PhD students for future collaboration.
3. Questionnaires.