Methodology Workshops

Abstract

Much research in TEL is design work – i.e., the research team designs an intervention that is intended to support learning. This intervention needs to be evaluated to show the extent to which this goal has been reached; and to gain additional insights that are sought for. Field studies are one main type of evaluations. They are challenging to set up; and in case of a bad study design cannot be easily repeated due to the effort and cost of running a field study. The goal of this lecture and workshop is

• To provide a blueprint for field studies in technology enhanced learning
• To present a hierarchical principle of evaluating learning interventions– based on Kirkpatrick & Kirkpatrick: Usage/observable activities – Learning – Impact on task/work performance – Impact on organization (in workplace learning/applicable to settings in which individual learning impacts a wider social entity)
• To have students plan a field study for their own PhD in rough lines individually
• To discuss their plans with peers and the lecturer, as well as other senior researchers who may be present – i.e., students will get feedback on their own plan

The blueprint for field studies is to evaluate in a hierarchy of research questions/evaluation level: First, one assesses the observable (learning) activities that are carried out – in particular how and whether participants adhered to the prescribed intervention; this helps understand the success of the intervention and it is possible to identify problems. Second, one assesses concrete learning outcomes – insights that are generated. Thirdly, one assesses a change in task performance. In parallel, a mix of qualitative and quantitative methods should be used – this allows on the one hand statistical comparison (pre/post; between groups). On the other hand, one can get in depth explanatory insights. Sometimes, also the impact on a wider social entity (e.g., an organization in which the intervention takes place) is relevant.

Speaker

Viktoria Pammer-Schindler Graz University of Technology (Austria)

Abstract

Designing learning opportunities with educational technologies is challenging for teachers and lecturers in traditional educational institutions. Despite the popularity of MOOCs and learning analytics it is important to remember that the educational practice heavily relies on blending conventional teaching and technology-enhanced learning approaches. In order to successfully blend different modes of learning and teaching it is important to structure learning processes and align appropriate tools with learning activities. However, the adoption of technology-enhanced learning into the individual teaching practice is very often constrained by limited time for rethinking course designs and teaching concepts. Therefore, institutions need to support their educational and training staff for selecting and integrating appropriate technologies into their teaching concepts, efficiently. As most educational institutions are drawing on an existing face-to-face teaching practice, such support needs to consider the transitions and transformation from conventional teaching to technology-enhanced learning and reverse. This contribution presents a pattern-based approach for preparing and analyzing complex learning and teaching fast and allows subject matter experts in talent development and training to build their teaching concepts on top of tested didactics. Our solution provides a toolkit and framework that helps to conceptualize, analyze, and communicate learning designs of different scales and speed-up the deployment of richer learning experiences using off-the-shelf LMS functions and features.

Speakers

Christian Glahn Delft University of Technology (NL)	Marion R. Gruber University of Zurich (Switzerland)

Abstract

As PhDs and researchers at some point, we need to report our research. In this workshop attendees will create a draft of a paper while learning important steps about academic writing; including the purpose of an academic paper, structure of a paper, objectives, and content of each of the sections. The workshop will include small rounds for reviewing each other’s work and discussions about best practices for writing papers (dealing with writer’s block, things that make reviewers angry, etc.).

Speakers

Jan Schneider Leibniz Institute for Research and Information in Education (Germany)	Maren Scheffel Open University of the Netherlands (NL)

Abstract

Opening up TEL research has to do with making more transparent and accessible each stage of the scientific cycle (research design, data collection, analysis, and publication) with open approaches to science, such as pre-registration, data sharing, transparent analyses, and open access publication (Zee & Reich, 2018). This initiative is very relevant from a perspective of Responsible Research and Innovation and to facilitate a more credible and robust evolution of the knowledge. However, it conveys several challenges. One important challenge has to do with ethical issues and data literacy challenges in the training of TEL researchers (making data readable and usable) and the end users (reading and using data). ata handling and visualization is crucial at the time of making science not only open, but also accessible by other researchers and the end users, normally unaware of being engaged in such processes. A subtle aspect of this situation relates the poor appropriation and utilization of scientific research products in the pedagogical field by potential users (stakeholders in the education and training system)(Jamil Salmi, 2015). In fact, approach to a more fair science promoted by the European Commission has been developing internationally for about ten years, but in the field of TEL, in spite of the easiness of massive data collection, the difficulties for opening up research have led to intrinsic, ongoing debates (Ho et al., 2014). This workshop will address this problem by introducing essential information on the Open Science movement. Moreover, since some instruments and areas of Open Science are better known, it will focus the emergent problem of Open Data in TEL research.

Speakers

Juliana Elisa Raffaghelli Fundacio per la Universitat Oberta de Catalunya (Spain)	Davinia Hernández-Leo Universitat Pompeu Fabra (Spain)

Abstract

PhD studies are at the highest level of education and the process can be discouraging and exhausting at times, but the final outcome is at the same time a major achievement in life and a rewarding event. Conducting
academic research and writing a PhD thesis may lead students to unhappiness and to an unhealthy lifestyle, often combined with lack of exercise, unhealthy diet, anxiety etc. Financial problems, incomplete data, lack of motivation and creativity, limited social life are among common problems faced by PhD students. Therefore, it is necessary that universities foster health promotion policies in order to improve their students’ quality of life and wellbeing.

According to the World Health Organisation (WHO), health promotion refers to the process that enables people to increase control over their health and to improve it. The holistic approach to health spans across the five dimensions of health: physical, mental, emotional, societal, social and spiritual health. Factors such as anxiety and lifestyle, financial problems and academic obligations may hinder research students from adopting this approach. A university that promotes health holistically has to create a sustainable and healthy learning and living environment for its students. academic obligations may hinder research students from adopting this approach. A university that promotes health holistically has to create a sustainable and healthy learning and living environment for its students.

Speaker

Dimitra Spyropoulou University of Patras (Greece)

Abstract

Public Speaking is one of the 21st century skills. As PhDs and researchers giving presentations is one very important aspect of our careers that is usually overlooked. In this workshop, participants will learn and practice some principles that will make their presentations more engaging and interesting.

Speakers

Jan Schneider Leibniz Institute for Research and Information in Education (Germany)	Maren Scheffel Open University of the Netherlands (NL)

Abstract

The aim of this workshop is to bring most of the summer school participants together encouraging and discussing new working collaborations. The idea behind this workshop is the need to give the chance to early and advanced TEL doctoral students and senior researchers to collaborate on ideas for (a) possible commercialization of research and / or (b) writing their own innovative project proposals and apply for national or EU funding. Being able to work in the field of your interest with people that you trust and admire is the dream of many young researchers. This workshop can support this dream and offer the opportunity to TEL experts to share their valuable knowledge with the new generation of TEL researchers creating the basis of promising collaborations.

Speakers

Endrit Kromidha University of Birmingham (UK)	Daniele Di Mitri Open University of the Netherlands (NL)	Maria Perifanou University of Macedonia; Aristotle University of Thessaloniki (Greece)	Mikhail Fominykh Norwegian University of Science and Technology (Norway)	Ana Torres Instituto Politécnico de Santarém (Portugal)

Abstract

The Summer School is just in time for the second EC-TEL Doctoral Consortium Deadline. This year the EC-TEL Doctoral Consortium introduces a new “Ideas” Track for early stage Ph.D. candidates. This creates the opportunity for getting valuable feedback in your initial ideas and offers the opportunity to grow your network in the TEL Research community.

This workshop allows you to get an insight on how the EC-TEL Doctoral Consortium works and how your research benefits most from participating to it. The DC’s call has many points to consider for a submission, because we have found that considering these points helps improving your submission and is the key for gaining the biggest benefit for your research. At the same time, we often hear that candidates struggle answering them appropriately. In this workshop we will work on hardening your research question and expressing the innovative potential of your ideas and refine them into a suitable submission for the EC-TEL Doctoral Consortium. This workshop helps you to give your DC submission the final touch or just to align your ideas to the community. If you plan to submit to this year’s EC-TEL DC, make sure that you already have a draft submission going and your formal recommendation letter signed by your supervisor/adviser. The workshop addresses Ph.D. candidates at all stages of their PhD project.

Speakers

Christian Glahn Delft University of Technology (NL)	Esther Tan Delft University of Technology (NL)

Abstract

The trouble with statistics is that it is easy to run statistical tests using software, but the interpretation of these results is a different animal all together. All too often, results are reported that upon closer scrutiny are not as trustworthy as suggested by the outcome of statistical tests.
In this workshop, we’ll look at common mistakes and misinterpretations in statistics. The focus won’t be on what test to use on what sort of data, but on understanding why this is the wrong question to begin with. No formulae (well, a few) but more on correct interpretation of results and methodology.
We’ll look at things like the correct interpretation of p-values and hypotheses, the power of experiments, the base rate fallacy, and more.

Speaker

Peter Ebus Open University of the Netherlands (NL)

Abstract

Research that involves human subjects raises ethical and legal issues and requires ethical approval. Ethical approval has the following objectives: 1. to protect participants; 2. to ensure that research serves interests of individuals, and/or society as a whole; 3. to examine research activities, looking specifically at the risk management plan, data protection strategies and the informed consent procedure. The last point is of particular concern since we know that current consent practices often fail to make consent really informed, especially, when personal data is involved. Even if people find protection of their personal data important, they tend to ignore the information provided in regard to their personal data use (privacy paradox). Informed consent is a complex matter: on the one hand, it should comply with the legal requirements; on the other hand, it should also meet the data subjects’ needs and expectations (usability requirements). We conducted a study on the requirements for informed consent: legal requirements established by the new European Data Protection Regulation (GDPR), that came into force as of May 2018; and usability requirements extracted from different usability studies. During the workshop we will discuss these requirements and present a template developed to support data controllers in creating usable consent procedures. Besides, participants of the workshop will be given an opportunity to work in small groups on designing an informed consent form (for a specific scenario) using the template.

Speakers

Ekaterina Muravyeva Open University of the Netherlands (NL)	Maren Scheffel Open University of the Netherlands (NL)

Abstract

I am a PhD student but I hardly sleep for 3-4 hours. Too many deadlines! Probably, I will take a small break after this article submission!? Why is my supervisor so demanding? All these questions come to every PhDs’ mind at some point of their PhD journey. But, is everything always answered or does every PhD try to find the answer to some of these questions? Apart from answering similar questions, this workshop tries to use life storytelling and Q&A from both the workshop organizers and the participants (both as individuals and in groups) to give everyone an opportunity to devise a strategy to learn from each other and try to see the current challenges or opportunities during a PhD from a different perspective.

Speakers

Sambit Praharaj Open University of the Netherlands (NL)	Roland Klemke TH Köln (Germany)	Maren Scheffel Open University of the Netherlands (NL)	Bibeg Limbu Open University of the Netherlands (NL)	Jan Schneider Leibniz Institute for Research and Information in Education (Germany)

Abstract

This interactive workshop will consider ways that will encourage PhD researchers towards finishing the writing process of their PhD thesis and highlight aspects of successfully defending it. In particular, the workshop will be an opportunity to reflect on ways of organizing time and data, on creating a diary and sketch a PhD thesis structure for writing. Hands-on tips on the use of reference management systems will be given. It will also focus on the use of other PhD theses which can guide our own writing, on thesis formatting and on publishing. A PhD journey and particularly thesis writing can be emotional. Therefore, apart from the practical and technical aspects of PhD thesis, this workshop will focus on these aspects of writing such as the writer’s block and how to overcome it. Moreover, it will discuss ways of taking time off the thesis, on handling writing with other commitments (full-time work and family) and will give the opportunity for PhD researchers to pose questions and share challenges and good practices that will enable them to finish their PhD thesis. Further, the end goal of writing a thesis is to successfully defend it. Thus this workshop deals with the decisions that PhD researchers need to make at the writing process which will make an examiner avoid asking tricky questions. The session will initially introduce the topic and will give time to PhD Researchers to work both in groups and individually on drafting their own structure for their PhD thesis writing and identify critical aspects that need to be taken into account when thinking long term goals of defending their thesis.

Speaker

Tina Papathoma The Open University (UK)

Abstract

From simple visualisations that clearly communicate research results to different stakeholders, to more complicated user interfaces that visualise learning data and directly support teachers and learners to make informed decisions about the learning process, visualising data is a crucial skill for any TEL researcher.

During the workshop, we will dive into the art of storytelling with data and and apply the principles presented by Roberto Martínez-Maldonado’s in his keynote “From Oysters to Pearls: Key Strategies for Communicating Learning Analytics Insights” in order to build visualisations that communicate insights supported by data.

Speakers

Roberto Martinez-Maldonado University of Technology Sydney (Australia)	Ioana Jivet Open University of the Netherlands (NL)

Abstract

Presenting and sharing research is, in most cases, a secondary task for a researcher. However, its importance cannot be underestimated, as obtaining research results is not enough – they should be contributed to the society. Social Media can support this process. Nowadays emerging technologies have changed the way that everyone finds, presents and communicates information opening up new forms of undertaking and disseminating research. Researchers have now the possibility to work in collaboration and communicate their research results in a highly efficient way making use of the power of Web 2.0 tools.

It is though important to know which tools you will choose for each purpose, in which way and to what extent, in order to achieve successfully your goals. In fact, there is no single ’right way’ for researchers to use Web 2.0 tools in order to improve the way they work. How you use them will depend on you, your discipline, those around you and the research you are doing. There are though some important and useful “key-actions” that could support this process.

Speakers

Maria Perifanou University of Macedonia; Aristotle University of Thessaloniki (Greece)	Mikhail Fominykh Norwegian University of Science and Technology (Norway)	Daniele Di Mitri Open University of the Netherlands (NL)