Handbook of Learning Analytics

Chapter 27

Handbook of Learning Analytics
First Edition

A Critical Perspective on Learning
Analytics and Educational Data Mining

Rita Kop, Helene Fournier & Guillaume Durand


In our last paper on educational data mining (EDM) and learning analytics (LA; Fournier, Kop & Durand, 2014), we concluded that publications about the usefulness of quantitative and qualitative analysis tools were not yet available and that further research would be helpful to clarify if they might help learners on their self-directed learning journey. Some of these publications have now materialized; however, replicating some of the research described met with disappointing results. In this chapter, we take a critical stance on the validity of EDM and LA for measuring and claiming results in educational and learning settings. We will also report on how EDM might be used to show the fallacies of empirical models of learning. Other dimensions that will be explored are the human factors in learning and their relation to EDM and LA, and the ethics of using “Big Data” in research in open learning environments.

Export Citation: Plain Text (APA)     BIBTeX     RIS

Supplementary Material

No Supplementary Material Available

References (40)

Atkinson, S. P. (2015). Adaptive learning and learning analytics: A new learning design paradigm. BPP Working paper. https://spatkinson.files.wordpress.com/2015/05/atkinson-adaptive-learning-and-learning-analytics.pdf

Bates, T. (2014). Two design models for online collaborative learning: Same or different? http://www.tonybates.ca/2014/11/28/two-design-models-for-online-collaborative-learning-same-or-different/

Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19, 206–220.

Bista, K. (2014). Is Twitter an effective pedagogical tool in higher education? Perspectives of education graduate students. Journal of the Scholarship of Teaching and Learning, 15(2), 83–102. http://files.eric.ed.gov/fulltext/EJ1059422.pdf

Bouchard, P. (2013). Education without a distance: Networked learning. In T. Nesbit, S. M. Brigham, & N. Taber (Eds.), Building on critical traditions: Adult education and learning in Canada. Toronto: Thompson Educational Publishing.

Boyd, D. (2010). Privacy and publicity in the context of big data. Paper presented at the 19th International Conference on World Wide Web (WWW2010), 29 April 2010, Raleigh, North Carolina, USA. http://www.danah.org/papers/talks/2010/WWW2010.html

Boyd, D., & Crawford, K. (2012). Critical equations for Big Data. Information, Communication & Society, 15(5), 662–679. doi:10.1080/1369118X.2012.678878

British Science Council. (2009). What is science? http://www.sciencecouncil.org/definition

Cavoukian, A., & Jonas, J. (2012). Privacy by design in the age of big data. https://privacybydesign.ca/content/uploads/2012/06/pbd-big_data.pdf

Cormack, A. (2015). A data protection framework for learning analytics. Community.jisc.ac.uk. http://bit.ly/1OdIIKZ

Christopher, J. C., Wendt, D. C., Marecek, J., & Goodman, D. M. (2014) Critical cultural awareness: Contributions to a globalizing psychology. American Psychologist, 69, 645–655. http://dx.doi.org/10.1037/a0036851

Denzin, N., & Lincoln, Y. (Eds.) (2011). The Sage handbook of qualitative research (4th ed.). Thousand Oaks, CA: Sage.

Durand, G., Belacel, N., & Goutte, C. (2015) Evaluation of expert-based Q-matrices predictive quality in matrix factorization models. Proceedings of the 10th European Conference on Technology Enhanced Learning (EC-TEL ʼ15), 15–17 September 2015, Toledo, Spain (pp. 56–69). Springer. doi:10.1007%2F978-3-319-24258-3_5
Ed Tech Review (2016). Educational Data Mining (EDM). http://edtechreview.in/dictionary/394-what-is-educational-data-mining

El Emam, K. (1998). The predictive validity criterion for evaluating binary classifiers. Proceedings of the 5th International Software Metrics Symposium (Metrics 1998), 20–21 November 1998, Bethesda, MD, USA (pp. 235–244). IEEE Computer Society. http://ieeexplore.ieee.org/document/731250/

European Data Protection Supervisor. (2015). Leading by example: The EDPS strategy 2015–2019. https://secure.edps.europa.eu/EDPSWEB/edps/site/mySite/Strategy2015

Fenwick, T. (2015a). Things that matter in education. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw

Fenwick, T. (2015b). Professional responsibility in a future of data analytics. In B. Williamson (Ed.) Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw

Fournier, H., & Kop, R. (2015). MOOC learning experience design: Issues and challenges. International Journal on E-Learning, 14(3), 289–304.

Gergen, J. K., Josselson, R., & Freeman, M. (2015). The promise of qualitative inquiry. American Psychologist, 70(1), 1–9.

Gonzalez-Brenes, J., & Huang, Y. (2015). Your model is predictive — but is it useful? Theoretical and empirical considerations of a new paradigm for adaptive tutoring evaluation. In O. C. Santos, J. G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. M. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, & M. Desmarais (Eds.), Proceedings of the 8th International Conference on Education Data Mining (EDM2015), 26–29 June 2015, Madrid, Spain (pp. 187–194). International Educational Data Mining Society.

Gupta, A. (2013). Fraud and misconduct in clinical research: A concern. Perspectives in Clinical Research. 4(2), 144–147. doi:10.4103/2229-3485.111800.

Hardt, M. (2014). How big data is unfair: Understanding sources of unfairness in data driven decision making. https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de

Jones, C., Dirckinck-Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer Supported Collaborative Learning, 1(1), 35–56.

Kansanen, P., & Meri, M. (1999). The didactic relation in the teaching–studying–learning process. TNTEE Publications, 2, 107–116. http://www.helsinki.fi/~pkansane/Kansanen_Meri.pdf

Kitchin, R. (2015). Foreword: Education in code/space. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw

Kop, R. (2012). The unexpected connection: Serendipity and human mediation in networked learning. Educational Technology & Society, 15(2), 2–11.

Kop, R., Fournier, H., & Durand, G. (2014). Challenges to research in massive open online courses. Merlot Journal of Online Learning and Teaching, 10(1). http://jolt.merlot.org/vol10no1/fournier_0314.pdf

Long, Y., & Aleven, V. (2014). Gamification of joint student/system control over problem selection in a linear equation tutor. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014), 5–9 June 2014, Honolulu, HI, USA (pp. 378–387). New York: Springer. doi:10.1007/978-3-319-07221-0_47

Long, C., & Siemens, G. (2011, September 12). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5). http://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education

Meyer, J. H. F., & Land, R. (2006). Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge. London: Routledge.

Mischel, W. (1968). Personality and assessment. London: Wiley.

Oblinger, D. G. (2012, November 1). Analytics: What we’re hearing. EDUCAUSE Review. http://er.educause.edu/articles/2012/11/analytics-what-were-hearing

Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450.

Siemens, G., Dawson, D., & Lynch, L. (2013). Improving the quality and productivity of the higher education sector: Policy and strategy for systems-level deployment of learning analytics. SoLAR. https://www.itl.usyd.edu.au/projects/SoLAR_Report_2014.pdf

Stewart, B. (2015). Open to influence: What counts as academic influence in scholarly networked Twitter participation. Learning, Media and Technology, 40(3), 287–309. doi:10.1080/17439884.2015.1015547

Weller, M. (2011). A pedagogy of abundance. Spanish Journal of Pedagogy, 249, 223–236.

Wen, M., Yang, D., & Rosé, C. (2014). Sentiment analysis in MOOC discussion forums: What does it tell us? In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM2014), 4–7 July, London, UK (pp. 130–137). International Educational Data Mining Society.

Xu, D., & Smith Jaggars, S. (2013). Adaptability to online learning: Differences across types of students and academic subject areas. CCRC Working Paper No. 54. Community College Research Center, Teachers College, Columbia University. http://anitacrawley.net/Reports/adaptability-to-online-learning.pdf

About this Chapter

A Critical Perspective on Learning Analytics and Educational Data Mining

Book Title
Handbook of Learning Analytics

pp. 319-326




Society for Learning Analytics Research

Rita Kop1
Helene Fournier2
Guillaume Durand2

Author Affiliations
1. Faculty of Education, Yorkville University, United Kingdom
2. Information and Communications Technologies, National Research Council of Canada, Canada

Charles Lang3
George Siemens4
Alyssa Wise5
Dragan Gašević6

Editor Affiliations
3. Teachers College, Columbia University, USA
4. LINK Research Lab, University of Texas at Arlington, USA
5. Learning Analytics Research Network, New York University, USA
6. Schools of Education and Informatics, University of Edinburgh, UK


Register | Lost Password