Design and Evaluation of Personalized and Adaptive Learning

Design and Evaluation of Personalized and Adaptive Learning

Monday 05/06 10:30-12:00h

Plenary Hall

Abstract

The design and evaluation of personalized and adaptive Learning (PAL) includes (1) the digital platforms, (2) didactic design criteria learning activities, and (3) pedagogical design for integrating in course/subjects and study routine. The personalization processes are claimed to increase autonomy and offer many other advantages, and the analytics dashboards are claimed to provide insights that are claimed to contribute with extrinsic motivation, reward, and reflective learning. The workshop will present (1) broad overview of methods for design and evaluation of the analytics and impact assessment of PAL, (2) Kano’s two-factor theory for prioritizing analytics dashboards (3) example of making action-guiding analytics and conflicting evaluation outcomes, (4) and research design and outcome from empirical impact assessment study. The workshop facilitation will include participants’ reflection on methods for studying personalization and adaptive elements in own project, potential use of two-factor theory in design and evaluation of analytics dashboards, reflection on dilemmas in the design of analytics dashboards and design for impact assessment study.

Needs Analysis

The design and evaluation of personalized and adaptive Learning (PAL) includes (1) the digital platforms, (2) didactic design criteria learning activities, and (3) pedagogical design for integrating in course/subjects and study routine. Learning analytics and the designs for learning are central for designing the use and evaluating the quality and impact of PAL from commercial, educational, and research perspectives. LA and AL are the two most common elements in all digital learning technology PhD projects. The workshop will present (1) methods for design and evaluation of the analytics and impact assessment of PAL, (2) Kano’s two-factor theory for prioritizing analytics dashboards (3) example of making action-guiding analytics and conflicting evaluation outcomes, (4) and research design and outcome from empirical impact assessment study. Participants will reflect on methods for studying PAL elements in own project, potential use of two-factor theory, reflection on dilemmas.

Learning Objectives

1. Broad overview of methods for design and evaluation of the analytics and impact assessment of personalized and adaptive learning (PAL),
2. Kano’s two-factor theory for prioritizing analytics dashboards,
3. Considerations in making action-guiding analytics and conflicting evaluation outcomes,
4. Research design and defining desired outcomes from empirical impact assessment study on adaptive learning & learning analytics,

The desired outcomes for each of the participants are:

• Making informed choice regarding the selection methods for the design and evaluation of personal and adaptive learning including learning analytics
• Considering Kano’s two-factor as an alternative to one-factor instrument for prioritizing design criteria and perceived evaluation of the technology features
• Ideating protocol for investigating the different user personalities providing conflicting evaluation outcomes
• Documenting protocol for experimental and exploratory design setup

Pre-activities

Materials shared before the session: Four papers/manuscripts on adaptive learning, learning analytics dashboard design, and the impact assessment of the AL. Glance through 1-3, if possible. Selected content from the sources will be presented during the workshop.
https://doi.org/10.1007/978-3-031-06675-7_3
https://doi.org/10.1007/978-981-19-7528-8_6
https://www.researchgate.net/publication/368721259_Prototyping_the_Learning_Analytics_Dashboards_of_an_Adaptive_Learning_Platform_Faculty_Perceptions_versus_Designers’_Intentions
https://doi.org/10.13140/RG.2.2.35751.42405

Before: Considering your own interest and project on learning analytics, dashboard designs, and the research design on personalized and adaptive learning, write some problems & dilemmas, and/or research design related research questions (3-5) you would like to discuss during the workshop.

Pre-Requisite:
Interest and potentially have a project on a digital learning platform.

Session Description

Activities:

1. Short presentations on methods identified in reviews and methods applied in empirical project on the design and evaluation of personalized and adaptive learning including their learning analytics.
2. Two pair-wise discussion and documentation on the application of methods and/or experimental/exploratory research design on learning analytics, personalized and adaptive learning.
3. A Kahoot for gamified reflection on the central topics.
4. Exit ticket on evaluation of the workshop and interest in attending in extend PhD course.

Example of task:
Considering your own interest and project on learning analytics, dashboard designs, and the research design on personalized and adaptive learning, write some problems & dilemmas, and/or research design related research questions (3-5) you would like to discuss during the workshop.