Become a Member

LASI20 Workshops & Tutorials

We are pleased to present a great LASI20 program that includes 8 Workshops and 8 Tutorials!

Based on feedback collected from prior years, this year's organizing committee has developed the following program. LASI participants will now take a deep dive into two workshops of 5 hours each on Tuesday and Wednesday. In addition, each participant will take part in two short 2-hour tutorials, enabling participants to get a flavor of a range of topics.

Each individual can choose 2 workshops and 2 tutorials. The organizing committee will strive to have all participants attend their top choices but based on attendance and popularity of certain sessions, your choices are not guaranteed. Register early to ensure your spot in your top choices!

<strong>Workshops</strong><strong>Tutorials</strong>

W1. Leadership and Deployment of Learning Analytics

Title: Leadership and Deployment of Learning Analytics

Description: Research into the field of Learning Analytics (LA) has made much progress over the past decade. There are numerous and extensive works ranging from predictive models of learning outcomes to semantic analyses with an increasing focus on multi-modal data sets. Arguably innovations in LA have largely plateaued as the field establishes itself as a major contributor to advancing the learning sciences and the process of education. To date, the complexities associated with the deployment and translation of LA based findings has diminished the scale of impact of the field. To transition LA from a novel small scale research approach to system wide impact requires different thinking and different approaches towards institutional adoption. In order to make this transition, an exploration of the role of leadership is required. This workshop introduces SPARK – a leadership framework to deploy and evaluate the impact of LA at the institutional level. The framework accounts for the complexities of learning environments, organizational policies, and external environment practices while guiding leadership teams to deploy and scale LA initiatives to achieve organizational impact.

Activities

The objective of the workshop is to explore the elements that influence how learning analytics are adopted at the institutional level and devise a strategy to promote the appearance of productive tensions to transition the use of learning analytics from small research projects to fully deployed operational functions in an institution.

The workshop is divided into the following activities

  1. Discussion of the contextual model of LA and the SPARK framework:
    1. Elements to be considered in the context of LA (data infrastructure, pedagogical approach, data-supported decision making)
    2. Stakeholder groups influencing institutional adoption
    3. Leadership models and context for LA deployments.
    4. The SPARK framework.
  2. Review successful and unsuccessful case studies of LA institutional adoption.
    1. Attendees will explore case studies of institutional adoption of LA processes. The activity requires mapping the case context onto SPARK dimensions.
    2. Groups will diagnose problems and propose changes in the context that would improve the institutional uptake.
  3. Contextualization of the model in your institution
    1. Identify the elements in the models and framework that map into your institution
    2. Map processes, stakeholder groups, relationships and institutional structures required for institutional adoption

Target Audience

The workshop is ideal for researchers, practitioners and administrators engaged in and responsible for the deployment of learning analytics tools and outcomes at the institutional level. The workshop will be of interest specially to those developing leadership skills in the area.

Takeaways

  1. A strategic view of the elements and processes that influence how educational institutions effectively adopt LA.
  2. A framework to articulate actions, policies, teams and initiatives to transition from small research initiatives to institutional adoption.

Preparations

The attendees are encouraged to read the following two documents:

  • Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A., & Gasevic, D. (2018). Rethinking learning analytics adoption through complexity leadership theory. Paper presented at the International Conference on Learning Analytics and Knowledge – LAK ’18, Sydney, Australia. doi:10.1145/3170358.3170375
  • Uhl-Bien, M., Marion, R., & McKelvey, B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. The Leadership Quarterly, 18(4), 298-318.

The workshop will be conducted in a highly interactive and exploratory fashion. Attendees are expected to approach the problem of adoption through their institution’s lens and role, sharing their insights and contextual information, but at the same time to explore alternative scenarios, contexts and initiatives.

>Instructor: Shane Dawson and Abelardo Pardo
University of South Australia

W2. Grounding Learning Analytics in Learning Theory

Title: Grounding Learning Analytics in Learning Theory

Description:

Theory strongly influences what data are sought, analyses used to probe relations among data, interpretations of analyses and designs for learning analytics. Topics this workshop will survey include: What is the current landscape of theories of learning? How are data grounded in and biased by theory? What major findings have been generated by research in learning science? What do these theories and research findings imply for learning analytics?

Learners have and use their own theories to choose how they engage in learning and why they believe that is the way they should learn. So, answers to questions like these need to acknowledge learners are self-regulating learners. But, unlike well-funded labs where highly trained leanring scientists use multiple state-of-the-art technologies to investigage and improve theories, learners “au naturel” have practically no training and hardly any resources to pursue self-directed, personally-focused, progressive learning science. Can modern learning technologies and fledgling learning analytics offset these disdvantages? What data can be gathered online and offline? How should data be analyzed and interpreted? What analytics best describe learners, how they learn and what they have learned? How might learning analytics help learners progressively and productively self-regulate learning?

Activities

  1. Theoretical perspectives and research findings from learning science will be surveyed, discussed and organized in the context of online learning.
  2. In working groups, participants will draw on this background to:
    1. propose and justify operational definitions for data to gather about learners, learning activities, content learners study and conditions under which learners work;
    2. sketch designs for instruments that gather those data, and
    3. propose learning analytics developed from those data and warranted by theories in learning science.
  3. In whole-group mode:
    1. working groups will present products of their collaboration, and
    2. participants will discuss, refine, extend and amalgamate working groups’ proposals to sketch future technologies (e.g., query systems, chatbots) that help learners use learning analytics to progressively improve self-regulated learning across time spans ranging over one study session, a project, a semester, and an academic year.

Target Audience

People interested to develop or extend basic understandings of learning science as a tool for designing trace data and learning analytics for self-regulating learners.

Prerequisites

None.

Takeaways

  1. Perspectives on (a) what theory is and is for, (b) foundations of learning science, and (c) self-regulated learning.
  2. Deeper understandings about operational definitions for trace data and implications of theoretically-shaped accounts of a learner’s cognition, metacognition and motivation.
  3. Experience in developing and justifying learning analytics grounded in learning science.
  4. Seeding collaborations among participants to push forward theoretically-grounded work in learning analytics and learning technologies.

Preparation

  1. Readings:

Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise & D. Gašević (Eds.), Handbook of learning analytics (pp. 241-249). Beaumont, AB: Society for Learning Analytics Research. https://solaresearch.org/wp-content/uploads/2017/05/chapter21.pdf

Winne, P. H., Teng, K., Chang, D., Lin, M. P-C., Marzouk, Z., Nesbit, J. C., Patzak, A., Raković, M., Samadi, D., & Vytasek, J. (2019). nStudy: Software for learning analytics about processes for self-regulated learning. Journal of Learning Analytics, 6, 95-106. https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/6194/7179

  1. Bring a laptop with the Google Chrome browser installed. We may play with nStudy (see Winne, Teng, Chang et al., 2019) as an illustrative technology designed from a perspective about self-regulated learning to gather and process trace data for learning analytics.

Instructor: Phil Winne,
Simon Fraser University

W3. Python Bootcamp for Learning Science Practitioners

Title: Python Bootcamp for Learning Science Practitioners

Description: The hands-on workshop for learning analytics practitioners will introduce the Python language and data science libraries. It is intended for beginners who have no previous experience with Python or data science methodologies. The workshop consists of four modules: a) Python Foundations; b) Exploratory Data Analysis; c) Statistics; d) Machine Learning.

Python Foundations. Python is the de facto language for scientific computing and one of the principal languages, along with R, for data science and machine learning. Python is also an accessible computing language, ideal for beginners. The Python Foundations module will serve as an introduction to the Python language along with core concepts such as data structures, functions, conditions and iteration. We will also introduce key intermediate concepts such as comprehensions, collections, and object orientation. Special emphasis will be given to coding in “idiomatic Python”.

Exploratory Data Analysis. Pandas is the de facto Python library for data science. In the Exploratory Data Analysis module we use Pandas along with Numpy, Matplotlib, and Seaborn libraries to analyze and visualize datasets. We will emphasize EDA as a foundational step for deriving insights from data. EDA is also a prelude to building formal models and simulations.

Statistics. In the statistics module we will use Python for performing basic descriptive and inferential statistics. Special emphasis will be given to using Python to calculate p-values, confidence intervals, and effect sizes.

Machine Learning. In the ML section we will introduce basic machine learning concepts and their application using the scikit-learn library. Under supervised learning we will use Python to predict continuous and categorical outcomes using linear and logistic regression, respectively. Under unsupervised learning we will use k-means as a data mining technique to discover clusters.

Deep Learning (optional): Deep learning is one of the fastest growing areas of machine learning and is particularly well suited for very large datasets. If we have time, we will introduce the five foundational concepts of deep learning and build a simple Deep Learning model in Python.

Activities:
  1. Participants will learn to code Python in the Google Colab environment using Jupyter Notebooks
  2. Each module will begin with a brief lecture. The bulk of the time will be dedicated to coding in small groups.
  3. Each participant will have full access to a portal environment, which includes tutorial videos and instructional materials. Participants will receive access to the online environment four weeks before the workshop and will retain access indefinitely.

Takeaways:

  1. The ability to code in Python at a basic level.
  2. Basic understanding of the core Python libraries for data science and learning analytics.
  3. Examples in learning analytics of how to apply the Python libraries.
  4. Review of basic concepts in descriptive and inferential statistics.
  5. Basic introduction to Machine Learning for learning analytics.

Preparations

  1. Participants are expected to spend approximately 4 hours of pre-work prior to the workshop. The pre-work will consist of brief video lectures, reading material, and exercises. The pre-work will be made available online a month prior to the workshop through the portal.
  2. Participants are expected to bring their own laptops with a functioning Web browser, preferably Chrome.

Instructors: Alfred Essa, Macmillan Learning
Not Pictured: Lalitha Agnihotri, McGraw-Hill Education

W4. Systematizing analytics with serious games

Title: Systematizing analytics with serious games

Description:

Serious Games (SG) have already proven their advantages in different educational environments. Combining SGs with Game Analytics (GA) techniques, we can further improve the lifecycle of serious games, from design to deployment, using a data-driven approach to make more effective SGs and, therefore, foster SGs large-scale adoption. With Game Learning Analytics (GLA), the goal is to obtain an evidence-based methodology based on in-game user interaction data that can provide insight about the game-based educational experience promoting aspects such as a better assessment of the learning process of players.

The current barrier to obtain such a methodology is that GA and GLA are usually done from scratch and ad-hoc for each SG. In this workshop, we will approach the systematization of the GLA process from the in-game user interaction data acquisition to the data analysis. One of the key open issues in learning analytics with SGs is the standardization of the data collected. To address this aspect, we use the Experience API Profile for Serious Games: a profile developed to align with the most common data collected in the serious games domain. Using the xAPI-SG profile allows the production of default analysis for SGs and simplifies the creation of ad-hoc analysis and the sharing of the GLA data. These analyses can be performed using common frameworks usually applied in learning analytics (e.g. Jupyter notebooks with Python).

Following this process with a simple SG, we will illustrate different purposes of GLA and how different stakeholders (e.g. designers, developers, researchers) can benefit from the application of systematized GLA.

Activities:

  1. Introduction to Game Learning Analytics: from serious game design to implementation, evaluation and deployment
  2. The xAPI Serious Games Application Profile: the standard to collect interaction data from serious games
  3. From xAPI-SG data to analyzable data
  4. Default and ad-hoc analytics for serious games
  5. Hands-on workshop: small working groups will work in relating analysis with game design and deployment in a simple serious game or by working with a pre-existing serious game user-interaction data in xAPI-SG

Target Audience:

Anyone interested in applying learning analytics techniques to serious games, for different purposes including assessment. Python and Jupyter notebooks knowledge is beneficial but not a strong requirement.

Takeaways:

  1. Perspective on how learning analytics for serious games can be systematized using standards.
  2. Understanding how in-game user interaction data can be used to obtain more effective SGs.
  3. Systematization of the data collection for other people with previous experience in data analysis.

 

Preparations or references

Instructors: Baltasar Fernandez-Manjon and the eUCM research group, Universidad Complutense de Madrid

W5. Learning Analytics in Physical Spaces (Multi-modal Analytics)

Title: Learning Analytics in Physical Spaces (Multi-modal Analytics)

Description: Learning does not only occur over Learning Management Systems or digital tools. It tends to happen in several face-to-face, hands-on, unbounded and analog learning settings such as classrooms and labs. Multimodal Learning Analytics (MMLA) emphasizes the analysis of natural rich modalities of communication during situated learning activities. This includes students’ speech, writing, and nonverbal interaction (e.g., movements, gestures, facial expressions, gaze, biometrics, etc.). A primary objective of multimodal learning analytics is to analyze coherent signal, activity, and lexical patterns to understand the learning process and provide feedback to its participants in order to improve the learning experience. This workshop is posed as a gentle introduction to this new approach to Learning Analytics: its promises, its challenges, its tools and methodologies. To follow the same spirit of MMLA, this workshop will include a hands-on learning experience analyzing different types of signals captured from real environments.

Activities:

  • A brief introduction to Multimodal Learning Analytics
  • Review and testing of different sensors and analysis techniques
  • The capture of a learning activity
  • Multimodal analysis of recordings
  • Designing a feedback dashboard
  • Final discussion

Target Audience:

Researchers that already have some experience with traditional Learning Analytics that want to expand their capabilities to physical learning spaces.

Takeaways:

  • Knowing the current state-of-the-art of Multimodal Learning Analytics
  • Being able to select sensors to capture learning activities
  • Being able to select analysis techniques to extract multimodal features
  • Fuse different multimodal features to estimate learning constructs
  • Feedback the participants of the learning activities

Prerequisite Skills/Experience: Basic statistics, programming in R.

Advanced preparation: Students should read Chapter 11 (Multimodal Learning Analytics) from the Handbook of Learning Analytics (https://www.solaresearch.org/hla-17/hla17-chapter11/) that is openly available.


Instructor: Xavier Ochoa, New York University

W6. Building predictive models of student success using Amazon Web Services

Title: Building predictive models of student success using Amazon Web Services

Description: In this workshop participants will use AWS SageMaker to build classification, regression, and clustering models. Supervised machine learning algorithms (classification and regression) are often used in learning analytics to make predictions of student outcomes in support of timely interventions, while unsupervised (clustering) algorithms are often used to uncover common patterns of student behaviour. Participants will learn to apply these methods on supplied datasets.

Activities:

  • Build, train, evaluate models using AWS SageMaker
  • Create reproducible and shareable machine learning workflows in Jupyter Notebooks
  • Deploy models to make predictions via a REST API

Target Audience:

Anyone interested in learning new skills for modern, cloud based machine learning workflows. Participants should have a basic understanding of:
– Common terminology and concepts in machine learning
– Python programming
Note: prior experience combining the above (Machine Learning in Python) is not expected/required.
Takeaways:
By the end of the workshop, participants will have:
1. An understanding of features and capabilities the AWS SageMaker platform
2. The ability to generate, apply, and evaluate predictive models using AWS services

Advanced preparation:
Participants should have created an AWS account in preparation for for this workshop. Additional “getting started” instructions will be provided closer to LASI 2020.

Instructor: Craig Thompson,
University of British Columbia

W7. Data visualisation and effective dashboard design for LA

Title: Data visualisation and effective dashboard design for LA

Description: Using visualisations for communication purposes is very common. In the educational field, individual visualisations or even whole dashboards are often used to show the results of data analysis in an easily digestible form, e.g. to learners or teachers. Often, however, what was intended to be communicated by a visualisation and how it is then interpreted by people differs. Similarly, results of data analysis are often visualised without a clear purpose or reason, but simply because the data is available. In this workshop, we will look at principles and guidelines of data visualisation and work on a structured approach to the why, what and how of effective dashboard design for learning analytics. Participants will be introduced to guidelines of data visualisation, will analyse and interpret examples of LA dashboards, and will design their dashboard.

Activities:

  • Participants will be introduced to world of data visualisation.
  • They will get to see examples from the field of LA as well as other fields.
  • Based on the analysis of examples and mock-ups, principles and guidelines will be formulated on how to design effective LA dashboards.
  • In small groups, participants will then design a learning analytics dashboard: they will first explore educational problems (i.e. what problem do they want to solve with a dashboard, how can it be grounded in theory and practice) and then identify relevant information and data to work on the problem. Based on this, participants will draft dashboard mock-ups using the principles. Finally, they will prioritise design features and sketch evaluation criteria and plans.
Target Audience:
Anyone interested in the visualisation of learning data and the design of learning analytics dashboards, anyone from students to teachers to practitioners to educational institution managers.
An understanding of what learning analytics is, how and where it can be used and who its stakeholders are is beneficial.
Takeaways:

  • Understanding of principles and guidelines for data visualisation
  • Getting a glimpse into the art of storytelling with data
  • Understanding the importance of grounding dashboard designs in theory and practice
  • Learning the process of designing dashboards in educational contexts

Advanced preparation:

  • Bring your own “problem” that you would like to solve with a learning analytics dashboard
  • Reading list coming soon

Instructor: Maren Scheffel,
Open Universiteit

W8. Temporal and sequential analysis for learning analytics

Title: Temporal and sequential analysis for learning analytics

Description:

Data in learning analytics research (e.g. SIS, clickstream, log-files) are often rich in temporal features that could allow us to explore the dynamic changes in learning behavior at different time granularities (e.g. seconds, days, weeks, semesters). This workshop will introduce participants to several common temporal/sequential analysis methods and techniques using R. During the workshop, we will cover some basic concepts in temporal analysis (i.e. sequences, trends, stationary, seasonality, autocorrelation). Next, we will go through some techniques to explore and visualize temporal data. Participants will learn and apply two types of temporal models: a) explanatory models using statistical techniques, such as Sequence Analysis, ARIMA and b) predictive models using modern machine learning techniques, such as long-short term memory (LSTM). Finally, we will brainstorm and discuss some applications of temporal analysis in educational research.   

Activities:

  • Learn the foundation and intuition of temporal/sequential analysis
  • Apply temporal/sequential analysis on educational datasets using R
  • Discuss in groups how to use temporal analysis to answer research questions in education

Target Audience:

This workshop is designed for anyone interested in temporal/sequential analysis. No experience in temporal and sequential analysis is required. To get the best learning experience, participants should familiarize themselves with basic statistics and machine learning concepts (e.g. regression, variance, autocorrelation, classification, cross-validation, overfitting).  

Takeaways:

  1. Bring a laptop with RStudio installed. More information on the packages will be provided before the workshop
  2. Special issues worth checking out:

Knight, S., Friend Wise, A., & Chen, B. (2017). Time for Change: Why Learning Analytics Needs Temporal Analysis. Journal of Learning Analytics, 4(3), 7–17.

https://doi.org/10.18608/jla.2017.43.2

Chen, B., Knight, S., & Wise, A. F. (2018). Critical Issues in Designing and Implementing Temporal Analytics. Journal of Learning Analytics, 5(1), 1–9. https://doi.org/10.18608/jla.2018.53.1

Instructor: Quan Nguyen, University of Michigan

T1: LA101: Introduction to Learning Analytics

Title: LA101: Introduction to Learning Analytics

Description:

Learning Analytics (LA) is the development and application of data science methods to the distinct characteristics, needs, and concerns of educational contexts and the data streams they generate for the purpose of better understanding and supporting learning processes and outcomes. It is an exciting technology for making concrete improvements within educational systems by enabling data-informed decision-making by students and educators, but it also raises questions about how to address data rights, ensure inference validity, and design appropriate implementations. This tutorial provides an introduction to learning analytics, where participants will develop foundational knowledge in the processes and techniques of LA, integration of LA with educational systems, and data ethics and policy issues. Participants will have the opportunity to experience the LA cycle of development and application through concrete examples and hands-on activities. The tutorial will focus on developing knowledge of the kinds of questions that learning analytics are helpful to address and the types of data and analyses that can be used to answer these questions. These two main topics will be situated in the larger context of the practical, ethical and policy issues that must be addressed to make learning analytics actionable.

Activities:

Participants will:

  1. Work with a real-life education scenario to develop valuable learning analytics questions and identify the types of data needed to answer them.
  2. Explore the kinds of insight that different learning analytics techniques offer and walk through the use of one technique to experience the analytical decision-making process.
  3. Examine the practical and ethical issues involved in implementing learning analytics.

Target Audience:

Anyone who is interested in Learning Analytics and is curious to know more about what it is and how it can be used. No prior knowledge or experience with Learning Analytics is required.

Takeaways:

At the end of this tutorial participants will be able to:

  1. Identify and define questions that can be answered by learning analytics to inform decision making.
  2. Describe the range of learning analytics techniques available and the purposes for which they are useful.
  3. Follow the life-cycle of learning analytics from question development to solution implementation.
  4. Take an informed position on issues related to the use of learning analytics in practice.

Advanced preparation:

  1. Read: Wise, A. F. (2019). Learning Analytics: Using Data-Informed Decision-Making to Improve Teaching and Learning. In Contemporary Technologies in Education (pp. 119-143). Palgrave Macmillan, Cham.
  2. Reflect: (1) What do you know about learning analytics going in to this tutorial? (2) What do you hope to learn by the end of it? (3) How do you envision learning analytics to be useful for you? You will be asked to share these reflections with the tutorial organizers one week before LASI.
  3. Optional: If you want to try running the demo code yourself during the tutorial, you should install R and RStudio on your machine. Pre-written code by the tutorial leaders will be shared the week before LASI. This is optional as all code will be run by the tutorial as a demonstration during the session.

Instructors: Alyssa Wise and Qiujie Li, New York University

T2: Closing the Loop in Learning Analytics with AI

Title: Closing the Loop in Learning Analytics with AI

Description:

One of the primary goals of Learning Analytics is to identify opportunities for the improvement of educational systems. In this tutorial, participants will learn how collected data from educational technologies can be used to 1) show and track learning, 2) identify opportunities for improvement, 3) implement changes to improve, and 4) close the loop with experimental testing.

Participants will explore existing datasets in DataShop and LearnSphere to interpret learning curves with tools provided and learn to apply different statistical and machine learning models to show student learning over time.

Activities:

  1. Participants will learn to login and view data from educational technologies stored in DataShop.
  2. Participants will use the tools in DataShop and LearnSphere to explore learning curves and apply predictive models to identify opportunities for improvement.
  3. Participants will work in small groups to suggest possible improvements to the improvement opportunities that were identified.
  4. Participants will analyze an existing close the loop dataset and see how learning was improved.

Target Audience:

Learning Analytics practitioners interested in using data from educational technologies to improve those technologies. Anyone interested in learning curve analysis and prediction models used to track learning over time.

Takeaways:

  1. How to track and visualize fine-grained learning over time.
  2. Identification of opportunities to improve models of learning and learning technologies.
  3. Conducting “close-the-loop” experiments using data driven improvemnt.

Advance Preparation: 

Read: Stamper, John C., and Koedinger, Kenneth R. (2011).  Human-Machine Student Model Discovery and Improvement Using DataShop. In Artificial Intelligence in Education. 353-360.  Springer Berlin Heidelberg. (http://dev.stamper.org/publications/AIED2011_stamper_koedinger.pdf)

Read: Koedinger, Kenneth R., Stamper, John C., McLaughlin, Elizabeth A., and Nixon, Tristan (2013).  Using Data-Driven Discovery of Better Student Models to Improve Student Learning. In Artificial Intelligence in Education. 421-430.  Springer Berlin Heidelberg.

(http://dev.stamper.org/publications/AIED_2013_Koedinger_et_al.pdf)

Participants should bring a laptop.


Instructor: John Stamper, Carnegie Mellon University

T3: Introduction to Writing Analytics: Fundamental Principles and Data Exploration

Title: Introduction to Writing Analytics: Fundamental Principles and Data Exploration

Description:

Broadly defined, learning analytics involves the collection and analysis of data to understand and promote learning. Writing analytics focusses specifically on the collection and analysis of student writing to investigate patterns in writing process (e.g., timestamp) and product (e.g., essays) data. These data can provide insights about students’ writing achievement at a specific point in time, or over a period time. Writing analytics research helps us to explore relationships between process and product feature data (drawn from student writing samples) and sociocognitive writing achievement framework components. These components include writing domain knowledge (e.g., vocabulary choice), general skills (e.g. critical thinking), and intra- (e.g., motivation) and interpersonal (e.g., collaboration) factors. For instance, writing analytics helps us to address research questions, such as: Is there a relationship between writing domain knowledge and student interest, or are there relationships between writing features and student outcomes, such as Grade Point Average (GPA)? Our knowledge about these relationships can feed education technology innovation. For instance, we can build systems supporting embedded writing curriculum development and instruction (e.g., innovative feedback reports) and assessment (e.g. innovative, technology-rich item types and score reports).

Activities:

In this tutorial, we will discuss key concepts in writing analytics (e.g., process vs. product data, including measurement of writing achievement through subfactors), data collection mechanisms, educational motivations for writing analytics research, and example writing analytics research projects and research studies, including descriptions of data modelling and findings. Efficient writing analytics research relies on technology, so we will also discuss relevant automated writing evaluation tools, data modelling methods, and applications in writing analytics research.

Participants will have the opportunity (1) to try out the Writing MentorÒ Google Docs add-on and explore process and product `analytics` collected from the app event logs that are associated with the user writing submissions data; and, (2) to review and explore a public, postsecondary writing data sets available on the ETS GitHub site: https://github.com/EducationalTestingService/ies-writing-achievement-study-data. We will explore these data, and (3) to participate in illustrative exercises to consider meaningful exploratory directions, produce research questions, and think about how to address research questions using these exemplary data. Data exploration may be illustrated with publicly-available tools, such as Jupyter Notebook for visualizations.

Target Audience:

This is an introductory tutorial.  No prerequisite knowledge about writing analytics is required. It is for anyone interested in learning writing analytics fundamentals and pursuing a writing analytics research project.

Takeaways:

Participants will learn the fundamental principles of writing analytics research. After participating in this tutorial, you will have a fundamental understanding of how to approach a writing analytics research project – including, motivating factors relevant to educational settings, advantages of writing construct-related analysis, types of research questions that can be addressed, and essential considerations for data collection and analysis.

Advanced Preparation:

  • Recommended Reading (At a minimum, read at least 3 papers — one paper from each category)

Writing Analytics: Process Data and Writing Domain Knowledge

  1. Allen, L. K., Jacovina, M. E., Dascalu, M., Roscoe, R. D., Kent, K. M., Likens, A. D., & McNamara, D. S. (2016). {ENTER} ing the Time Series {SPACE}: Uncovering the Writing Process through Keystroke Analyses. International Educational Data Mining Society, 22-29.
  2. Zhu, M., Zhang, M., & Deane, P. (2019) Analysis of Keystroke Sequences in Writing Logs. ETS Research Report Series (ETS-RR-19-11), 1-14.

Writing Analytics: Product Data, and General Skills and Broader Outcomes

  1. Allen, L. K., Snow, E. L., & Crossley, S. A., Jackson, G. T., & McNamara, D. S. (2014).Reading components and their relation to the writing process. L’année psychologique/Topics in Cognitive Psychology, 114, 663-391.
  2. Allen, L. K., Perret, C., & McNamara, D. S. (2016). Linguistic signatures of cognitive processes during writing. In the Proceedings of the 38th Annual Meeting of the Cognitive Science Society (Cog Sci 2016). Philadelphia, PA.
  3. Burstein, J., McCaffrey, D., Beigman Klebanov, B., & Ling, G. (2017). Exploring Relationships between Writing and Broader Outcomes with Automated Writing Evaluation. In Proceeding of the 12th Workshop on Innovative Use of NLP for Building Educational Applications (BEA), EMNLP 2017, Copenhagen, Denmark, 101-108.
  4. Burstein, J. Elliot, N., Beigman Klebanov, B., Madnani, N., Napolitano, D., Schwartz, M., Houghton, P. & Molloy, H. (2018). Writing Mentor: Writing Progress Using Self-Regulated Writing Support. Journal of Writing Analytics. Vol. 2: 285-313.
  5. Burstein, J., McCaffrey, D., Beigman Klebanov, B., Ling, G. & Holtzman, S. (2019). Exploring Writing Analytics and Postsecondary Success Indicators. In Companion Proceedings 9th International Conference on Learning Analytics & Knowledge (LAK19), 213-214.

Writing Analytics: Product Data and Intrapersonal Factors

  1. Perin, D., Lauterbach, M., Raufman, J. & Santikian Kalamkarian, H. (2016, online first). Text- basedwriting of low-skilled adults: Relation to comprehension, self-efficacy and teacher judgments. Reading and Writing: An Interdisciplinary Journal. doi: 10.1007/s11145-016- 9706-0.  Full text can be accessed from here: https://psycnet.apa.org/record/2016-54019-001.
  2. Beigman Klebanov, B., Burstein, J., Harackiewicz, J., Prinski, S., Mullholland, M. (2017) Reflective writing as a tool for increasing STEM motivation and retention — can AI help scale it up? Special Issue for the International Journal of Artificial Intelligence in Education: MARWIDE: Multidisciplinary Approaches to Reading and Writing Integrated with Disciplinary Education (Eds. D. McNamara, S. Muresan, R. J. Passonneau, & D. Perin), 27(4), 791-818, doi: 10.1007/s40593-017-0141-4.
  • Bring a Laptop
  • Install the Writing Mentor Google Docs add-on: https://mentormywriting.org/. Watch the video and try it out!
  • (1) Review ETS GitHub public data: https://github.com/EducationalTestingService/ies-writing-achievement-study-data. The repository data can be cloned or simply downloaded; and (2) Bring potential research questions to the tutorial that can be discussed during the tutorial. Try to bring questions that diverge from or are completely different from those previously addressed in the Burstein et al (2017), Burstein et al (2018), and Burstein et al (2019).


Instructor: Jill Burstein,
Educational Testing Service

T4: Scaling Learning Analytics—A people-centred Approach

Title: Scaling Learning Analytics—A people-centred Approach

Description:

Learning analytics promises to provide useful insights into learning and teaching, thereby leveraging decision-making and enhancing learning experiences. However, the adoption of learning analytics is often found to be sporadic rather than systematic at an institutional scale. This tutorial introduces the audience to the social factors associated with learning analytics adoption, e.g., political contexts, data culture, and multi-stakeholder views. While examples presented in the tutorial will be drawn from cases in higher education, the adoption frameworks introduced will be applicable to other contexts and the participants will be encouraged to contribute perspectives from contexts beyond higher education. The tutorial will enable participants to take first steps towards scaling learning analytics by considering different needs and priorities among key stakeholders through policy and strategy formation.

Activities:

  1. Presentations and discussion of the social factors that influence institutional adoption of learning analytics.
  2. Applying a complexity leadership model to identify challenges and strategic approaches to move learning analytics innovations to operation.
  3. Applying the SHEILA framework to develop a learning analytics policy and strategy.

Target Audience:

Anyone interested in exploring the social complexities of learning analytics and approaches to scale learning analytics at an institutional level through strategic leadership and policy development.

Takeaways:

  1. Understanding the disparities among different stakeholders regarding drivers and concerns about learning analytics.
  2. A draft of institutional policy or strategy for learning analytics

Advanced preparation:

Read:

Tsai, Y. S., Poquet, O., Gašević, D., Dawson, S., & Pardo, A. (2019). Complexity leadership in learning analytics: Drivers, challenges and opportunities. British Journal of Educational Technology.

Author accepted manuscript is available here: http://bit.ly/lasi2020_reading

Instructor: Yi-Shan Tsai,
University of Edinburgh

T5: Power to the User: Using Participatory Design to develop Learning Analytics tools

Title: Power to the User: Using Participatory Design to develop Learning Analytics tools

Description:

How often do Learning Analytics tools disappoint developers and users? As designers and researchers we tend to have a clear idea of how our LA tools will change the lives of faculty, administrators and students. Then, as they are rolled out, we are often shocked by the reality of users who have a different idea of what the tools are for, who fail to see implications for their practice, or use it in unforeseen ways –sometimes ones we consider negative. For decades, Participatory Design processes have been used in the field of human-computer interaction to address some of these challenges. Involving stakeholders in design processes can empower them and  render tools which are adjusted to their needs, routines and values.

In this workshop, we will present a number of processes, frameworks and examples for engaging in  participatory design processes with students, faculty or administrators, emphasizing both opportunities and challenges. In the tutorial participants will work with  a current or past project to develop first steps to an outline for a participatory design process.

Activities:

  • Participants will be introduced to the concept of Participatory Design and the reasoning (practical and ethical) behind the implementation of this design methodology.
  • Participants will be introduced to different processes, frameworks, and examples for Participatory Design.
  • We will discuss key elements of designing experiences that lend themselves to participation of stakeholders, discussing issues of power, shared knowledge, encouragement of participation, and designing for creativity.
  • They will use an example to begin to develop an outline for a process of participatory design for LA.
    • They will:
      • Choose a design process and decide where stakeholders should be involved.
      • Identify potential risks of stakeholder involvement and co-design solutions to these challenges.
      • Reflect on the potential impact of participatory design of LA tools.

Target Audience:

Anyone interested in the design of Learning Analytics tools using Human-Centered Design frameworks and particularly Participatory processes that involve the stakeholders in the processes of design.

Takeaways:

Participants will:

  • Identify reasons for the use of Participatory Design in Learning Analytics tool development.
  • Learn about Human Centered Design and Participatory Design.
  • Take first steps to designing a participatory process applicable to their own projects.

Prerequisite Skills/Experience:

  • Ideally, participants should have a specific project (either ongoing, past or future) that can be used as a scenario to work and redesign using participatory practices. A scenario will be provided for new researchers and designers.
  • For context, we have provided a recent paper from LAK2020 to read before class, and will send a worksheet to complete before the session.

Instructors: Juan Pablo Sarmiento and Fabio Campos, New York University

T6: Publication Demystified

Title: Publication Demystified

Description:

The phrase “Publish or Perish” is attributed to Kimball C. Atwood (Columbia University, ~1950) and refers to the relationship between scholarly publications and faculty promotion. The extent to which this adage holds true depends on many factors, including both the academic institution and the department granting the faculty position.

In addition, how many publications are expected, what counts as a publication, and the relative weight of specific publication venues (e.g., print journals or open journals; peer reviewed, published conference proceedings) will also vary according to the specifics of the promotion case requirements.

As junior faculty positions are typically quite competitive, graduate students and post-doctoral scholars also have pressure to enter the job market with publications already in on their resume. However, many students & post docs have limited opportunities to submit publications before completing their dissertation, and have little experience with the review processes that lead to successful submissions. This tutorial is aimed at de-mystifying the publication game. Because the field of Learning Analytics is both international and interdisciplinary, participants should not expect to learn a magic formula for getting their papers accepted. Instead we will discuss what makes a good submission and how reviewers make their recommendations to editors and program chairs. We will also discuss how to make strategic decisions about where & when to submit manuscripts.

Activities:

The first half of the tutorial will be devoted to a general discussion of the publication & reviewing process, focusing on the do’s and don’ts of a successful submission. We will focus on the participants’ current level of understanding about the important factors related to the faculty positions they aspire to and talk about how to address the specifics of those hiring departments. In the second half of the tutorial, participants will be asked to develop an individualized plan for the steps needed to produce their next successful writing project. These plans will be shared with each other for feedback and support.

Target Audience:

Graduate students and post-doctoral scholars

Takeaways:

Participants will have a better general understanding of how to make decisions about submitting to conferences and journals, and feel better prepared to craft successful manuscripts. They should leave the tutorial with a plan for their next writing project.

Advanced preparation:
None.


Instructor: Stephanie Teasley,
University of Michigan

T7: An Introduction to Methodologies in Learning Analytics

Title: An Introduction to Methodologies in Learning Analytics

Description:

Learning analytics is a bridge discipline, drawing on methods from education and learning sciences, computing and data science, psychology, statistics, linguistics, etc. Newcomers to the field can be easily overwhelmed. They might, however, be comforted to know that critical thinking and reasoning can go a long way in compensating for a shortage of technical proficiency, which will come with time and practice. In this tutorial, we explore the structure of learning analytics arguments and outline some best practices from the framing of research questions to selection of methods to collaboration on and communication of analytic results.

Activities

Participants will:

  1. Design a research study (in 5 minutes!) using the Heilmeier catechism.
  2. Explore data sets to identify types of variables, missingness, outliers, and other irregularities.
  3. Discuss levels of analytics from description to prediction, explanation, and causal inference.
  4. Identify research questions and methods in published LAK/JLA papers and enter results into the MLA Airtable.
  5. Learn to use R markdown notebooks (and GitHub) to produce (and collaborate on) reproducible research.

Target Audience:

This tutorial is for anyone just getting started in learning analytics.

Takeaways:

  1. A high-level view of the methods and methodology of learning analytics
  2. A close look at a few published examples
  3. A framework for planning learning analytics explorations
  4. A minimal toolbox for communicating and collaborating effectively.

Advanced preparation:

  1. Software and technology:
    1. Bring a laptop with R and RStudio installed
    2. (optional) Create a GitHub account, verify a working installation of git on your laptop, and install Github Desktop.
  2. Read:
    1. Bergner, Y., Gray, G., & Lang, C. (2018). What does methodology mean for learning analytics? Journal of Learning Analytics, 5(2), 1-8. https://epress.lib.uts.edu.au/index.php/JLA/article/view/6164
    2. (optional) Bergner, Y. (2017). Measurement and its uses in learning analytics. Handbook of learning analytics, 35-48. https://www.solaresearch.org/wp-content/uploads/2017/05/chapter3.pdf


Instructor: Yoav Bergner, New York University

 

T8: Hierarchical Cluster Analysis Heatmaps in R

Title: Hierarchical Cluster Analysis Heatmaps in R

Description:

Throughout learning analytics and much of education research, researchers and practitioners are faced with the task of describing and understanding high dimensionality data, such as clickstream logfile data with hundreds to thousands of rows (students) and columns (variables), yet have few tools to visualize and describe their datasets. Hierarchical Cluster Analysis (HCA) heatmaps are a recently applied technique in learning analytics to visualize and understand high dimensionality data, and importantly include visualizations of relationships and clustering both for the rows (students) as well as the columns (variables) in your dataset. HCA heatmaps have been used extensively across multiple fields of research. For example, in molecular biology and cancer research HCA heatmaps are used to display thousands of patients’ tumors across tens of thousands of different genes, visualizing gene transcript level and activity, correlated to tumor malignancy to pinpoint the most important genes to target for cancer interventions and treatments. In comparison to supervised clustering techniques such as k-nearest neighbors (KNN) in which the number of clusters must be provided a priori, hierarchical cluster analysis is unsupervised and builds the clusters from the data provided, and importantly for education research, is highly robust to missing data. When combined with heatmap visualization techniques, HCA heatmaps provide a useful means to visualize and describe a complex dataset. Recently, HCA heatmaps have been applied to visualizing student clickstream logfile patterns, visually identifying interesting correlations across clusters of students and variables. A central aspect of HCA heatmaps are their ability to cluster and visualize the relationships across a dataset while displaying each datapoint for each individual, without aggregating the data to summary statistics, thus “seeing” each individual across an entire dataset, in context with others who have the most similar data patterns.

Activities:

  1. Participants will be introduced to the research on cluster analysis, hierarchical cluster analysis, and HCA heatmaps, especially as it is applied in education research and learning analytics.
  2. Participants will generate HCA heatmaps using code provided in R. Participants can bring their own data, or a dataset will be provided.

Target Audience:

Researchers and practitioners who are looking for innovative ways to visualize and describe high dimensionality data.

Takeaways:

  1. Understanding the different types of cluster analysis and how they compare for use in education research, especially KNN versus hierarchical cluster analysis.
  2. Understanding how heatmaps can be used to visualize high dimensionality data.
  3. Create an HCA heatmap in R to visualize a dataset.

Advanced preparation:

  1. Read:
    1. Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18. http://pareonline.net/pdf/v15n7.pdf
    2. Lee, J., Recker, M., Bowers, A.J., Yuan, M. (2016). Hierarchical Cluster Analysis Heatmaps and Pattern Analysis: An Approach for Visualizing Learning Management System Interaction Data. Presented at the annual International Conference on Educational Data Mining (EDM), Raleigh, NC: June 2016. http://www.educationaldatamining.org/EDM2016/proceedings/paper_34.pdf
    3. Overview and skim the online R code manual book for the ComplexHeatmap() package by: Zuguang Gu (2019) ComplexHeatmap Complete Reference. https://jokergoo.github.io/ComplexHeatmap-reference/book/
  2. Bring a laptop with the latest R and R-studio installed. During the session we will install all needed packages and walk through examples.


Instructor: Alex Bowers, Teachers College, Columbia University

 
×

Register | Lost Password