Multimodal Technologies for Skills Development (mute4skid)

Multimodal Technologies for Skills Development (mute4skid)

Monday 13/05 10:30-12:00h (Part 1)

Monday 13/05 14:00-15:30h (Part 2)

Workshop Space B

Note: this workshop is divided into two parts:

• Part 1 Monday 13/05 10:30-12:00h – Multimodal Technologies for Skills Development (mute4skid)
• Part 2 Monday 13/05 14:00-15:30h – Capturing Expertise: Recording and Analyzing Psychomotor Skills

Needs Analysis

Multimodal and immersive systems, such as virtual and augmented reality, powered by AI and sensors, are changing how we interact and learn with technology. This has implications for human-computer interaction and technology-enhanced learning, enabling personalised, contextual and deliberate practice of skills in real settings. This can foster holistic learning that integrates cognitive, affective and psychomotor domains. Sensor data and computational approaches can reveal complex learning phenomena and enable seamless learning interventions in authentic settings. Multimodal and immersive systems can support embodied learning, considering context, psychomotor, affective and physiological aspects. However, multimodality still faces some challenges.

Learning Objectives

This workshop will consist of two parts.

The first part (in the morning) is a lecture-style presentation introducing the rationale of AI in Education for skills development. While presenting concrete examples from research, we will discuss our research on multimodal technologies for skill development. We will also alternate the presentations with a brainstorming/group-work phase in which groups of participants work on aspects to push forward skills development using multimodal technologies. Participants will learn about various aspects of AI in education, including the main concepts, historical and future development of AI in education, and the new opportunities introduced by multimodal interfaces, sensors, and advanced machine-learning environments.

The second part (in the afternoon) is a hands-on session titled “Capturing Expertise: Recording and Analyzing Psychomotor Skills”, which offers a practical approach to understanding psychomotor skills. Participants will be introduced to the differences in skill execution between experts and novices. The session includes activities such as sharing personal psychomotor skills, selecting skills for recording, and planning the recording process. Group recordings will be conducted to gather data, with repeated sessions for accuracy. The workshop concludes with an analysis of the recordings to better understand skill mastery. This event provides a straightforward opportunity for participants to learn about psychomotor skills in a collaborative setting.

Pre-activities

There are no prerequisites for participants to attend this workshop.

Session Description

The first part will be a lecture-style presentation about multimodal technologies 4 Skills development.

• Introduction to Multimodal Learning Experience (by Roland Klemke)
• Theoretical Grounding of Multimodal Learning Experience (by Bibeg Limbu)
• Sensor Applications (by Jan Schneider)
• Immersive Learning Environments (by Khaleel Asyraaf)
• Automated Feedback with AI Tutors (by Daniele Di Mitri and Deniz Iren)

The session will be intervalled by prototype demonstrations, interactive activities, and open group discussions on using Multimodal technologies for skill development.

The second part will be a hands-on workshop, and the activities will be:

• Expertise Roll Call: Share your psychomotor skills with peers in a dynamic group activity, setting the stage for practical recording sessions.

• Skill Selection Symposium: Collaboratively choose the most insightful skills to record, ensuring a focus on valuable learning experiences.

• Designing the Recording Process: Strategize the recording setup, deciding tasks and smartphone placement for effective skill capture.

• The Recording Sessions: Perform and record your psychomotor task 20 times in small groups, creating a comprehensive dataset.

• Repetition for Precision: Repeat sessions 2-3 times to ensure data reliability and capture the subtleties of each skill.

• Data Visualization and Analysis: Analyze plots from recordings and brainstorm analytical methods to interpret skill mastery.