Design and Evaluation of Personalized and Adaptive Learning

Design and Evaluation of Personalized and Adaptive Learning

Friday 17/05 17:30-19:00h

Workshop Space B

Needs Analysis

The design and evaluation of personalized and adaptive Learning (PAL) includes (1) the digital platforms, (2) didactic design criteria learning activities, and (3) pedagogical design for integrating in course/subjects and study routine. Learning analytics and the designs for learning are central for designing the use and evaluating the quality and impact of PAL from commercial, educational, and research perspectives. LA and AL are the two most common elements in all digital learning technology PhD projects. The workshop will present (1) methods for design and evaluation of the analytics and impact assessment of PAL, (2) Kano’s two-factor theory for prioritizing analytics dashboards (3) example of making action-guiding analytics and conflicting evaluation outcomes, (4) and research design and outcome from empirical impact assessment study. Participants will reflect on methods for studying PAL elements in own project, potential use of two-factor theory, reflection on dilemmas.

Learning Objectives

(1) Broad overview of methods for design and evaluation of the analytics and impact assessment of personalized and adaptive learning (PAL),
(2) Kano’s two-factor theory for prioritizing analytics dashboards
(3) Considerations in making action-guiding analytics and conflicting evaluation outcomes,
(4) Research design and defining desired outcomes from empirical impact assessment study on adaptive learning & learning analytics

The desired outcomes for each of the participants are:
1. Choosing effective methods for designing and evaluating personalized learning, including learning analytics.
2. Exploring Kano’s two-factor model as an alternative for prioritizing design criteria and evaluating technology features.
3. Developing a protocol to investigate conflicting user personalities and their impact on evaluation outcomes.
4. Documenting experimental and exploratory design setups based on workshop examples.

Pre-activities

Materials shared before the session: Four papers on adaptive learning, learning analytics dashboard design, and the impact assessment of the AL. Glance through 1-3, if possible. Selected content from the sources will be presented during the workshop.

Recommended Activities Before the Session: Considering your own interest and project on learning analytics, dashboard designs, and the research design on personalized and adaptive learning, write some problems & dilemmas, and/or research design related research questions (3-5) you would like to discuss during the workshop.

Pre-Requisite to Fully Understand the workshop content: The participant is expected to have interest and potentially have a project related to personalized/adaptive learning or learning analytics or design or evaluation of a digital learning platform. The methods and research design covered in the project are from adaptive learning platform but applicable in other scenarios of use of digital learning technologies.

Session Description

Activities

1. 1. Short presentations on methods identified in reviews and methods applied in empirical projects on the design and evaluation of personalized and adaptive learning including their learning analytics.
   2. Two pair-wise discussions and documentation on the application of methods and/or experimental/exploratory research design on learning analytics, personalized and adaptive learning.
   3. A Kahoot for gamified reflection on the central topics.
   4. Exist ticket on evaluation of the workshop and interest in atten