Session: Metacognition, Sentiments and Feedback

★ Best paper candidate

Julien Douady, Christian Hoffmann and Nadine Mandran
[1] Université Grenoble Alpes, France

Abstract: When entering higher education, students must become more autonomous in their learning, particularly know how to take stock of their ways of learning: identify what they know, and also what they do not know, then adapt their learning strategies. They must therefore develop metacognitive skills. This article analyzes the responses of 3830 newly arrived undergraduate students through a pre-requisites test including confidence levels. Focus is given on both their success rate, i.e., their achievement at the test, and their realism, i.e., if they were predictive in their confidence judgement. To compute a relevant realism index, previous work by Prosperi [1] is extended to our context. First, an expected course effect is observed: one of the seven proposed courses reveals a lower realism index, and at the same time, its success rate is lower too. Moreover, a gender impact is highlighted: females reach a higher realism index than males and this gap fluctuates over the 4 last years. This gender effect is probably different from the course effect because success rates of males and females remain equivalent, thus success rate and realism seem to be dissociated in this case. Finally, students who perform poorly on the pre-requisites test and choose to take a second session after a remediation period improve their results: both gaps of success rate and realism are closed. That could prove the relevance of the remediation, and/or the effect of metacognition feed-back provided just at the end of the pre-requisites test.

📄 Read More: https://link.springer.com/chapter/10.1007/978-3-031-16290-9_7

Elien Sijmkens, Mieke De Cock and Tinne De Laet
[1] KU Leuven, Belgium

Abstract: One of the main goals of science and engineering education is to guide students in becoming proficient problem solvers. Metacognitive abilities play an important role here, since they help students to regulate their own solving process. The Disciplinary Learning Companion (DLC) is an online tool that aims at developing these abilities through discipline- and topic-specific reflection on the solving process. In this contribution, we report on the results of the implementation of the DLC in a first-year Newtonian mechanics course. We studied the interplay between students’ interaction with the DLC (online learning traces), their metacognitive abilities (pre and post self-reported questionnaire), academic achievement (final exam score and particular exam problem score), and conceptual understanding (coding exam problem). We found no significant relationship between students’ interaction with the DLC and their metacognitive abilities as measured by the self-reported questionnaire. The results, however, show that students that used the tool more frequently obtain a higher final exam score and have a better conceptual understanding of the exam problem considered. Moreover, the results suggest that the topic-specificity of the reflection questions plays a role in the improvement in academic achievement.

📄 Read More: https://link.springer.com/chapter/10.1007/978-3-031-16290-9_26

Hagit Gabbay and Anat Cohen
Tel Aviv University, Israel

Abstract: Automated Testing and Feedback (ATF) systems are widely applied in programming courses, providing learners with immediate feedback and facilitating hands-on practice. When it comes to Massive Open Online Courses (MOOCs), where students often struggle and instructors’ assistance is scarce, ATF appears to be particularly essential. However, the impact of ATF on learning in MOOCs for programming is understudied. This study explores the connections between ATF usage and learning behavior, addressing relevant measures of learning in MOOCs. We extracted data of learners’ engagement with the course material, code-submissions and self-reported questionnaire in a Python programming MOOC with an ATF system embedded, to compile an overall and unique picture of learning behavior. Learners’ response to feedback was determined by sequence analysis of code submission, identifying improved or feedback-ignored re-submissions. Clusters of learners with common learning behaviors were identified, and their response to feedback was compared. We believe that our findings, as well as the holistic approach we propose to investigate ATF impact, will contribute to research in this field and to effective integration of ATF systems to maximize learning experience in MOOCs for programming.

📄 Read More: https://link.springer.com/chapter/10.1007/978-3-031-16290-9_9

Felix Böttger[1], Ufuk Cetinkaya[2], Daniele Di Mitri[2], Sebastian Gombert[2], Krist Shingjergji[1], Deniz Iren[1] and Roland Klemke[1]
[1] Open University of The Netherlands, The Netherlands
[2] DIPF - Leibniz Institute for Research and Information in Education, Germany

Abstract: The recent pandemic has forced most educational institutions to shift to distance learning. Teachers can perceive various non-verbal cues in face-to-face classrooms and thus notice when students are distracted, confused, or tired. However, the students’ non-verbal cues are not observable in online classrooms. The lack of these cues poses a challenge for the teachers and hinders them in giving adequate, timely feedback in online educational settings. This can lead to learners not receiving proper guidance and may cause them to be demotivated. This paper proposes a pragmatic approach to detecting student affect in online synchronized learning classrooms. Our approach consists of a method and a privacy-preserving prototype that only collects data that is absolutely necessary to compute action units and is highly scalable by design to run on multiple devices without specialized hardware. We evaluated our prototype using a benchmark for the system performance. Our results confirm the feasibility and the applicability of the proposed approach.

📄 Read More: https://link.springer.com/chapter/10.1007/978-3-031-16290-9_4