Handbook of Learning Analytics
Chapter 27
Handbook of Learning Analytics
First Edition
A Critical Perspective on Learning
Analytics and Educational Data Mining
Rita Kop, Helene Fournier & Guillaume Durand
Abstract
In our last paper on educational data mining (EDM) and learning analytics (LA; Fournier, Kop & Durand, 2014), we concluded that publications about the usefulness of quantitative and qualitative analysis tools were not yet available and that further research would be helpful to clarify if they might help learners on their self-directed learning journey. Some of these publications have now materialized; however, replicating some of the research described met with disappointing results. In this chapter, we take a critical stance on the validity of EDM and LA for measuring and claiming results in educational and learning settings. We will also report on how EDM might be used to show the fallacies of empirical models of learning. Other dimensions that will be explored are the human factors in learning and their relation to EDM and LA, and the ethics of using “Big Data” in research in open learning environments.
Export Citation: Plain Text (APA) BIBTeX RIS
No Supplementary Material Available
Atkinson, S. P. (2015). Adaptive learning and learning analytics: A new learning design paradigm. BPP Working paper. https://spatkinson.files.wordpress.com/2015/05/atkinson-adaptive-learning-and-learning-analytics.pdf
Bates, T. (2014). Two design models for online collaborative learning: Same or different? http://www.tonybates.ca/2014/11/28/two-design-models-for-online-collaborative-learning-same-or-different/
Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19, 206–220.
Bista, K. (2014). Is Twitter an effective pedagogical tool in higher education? Perspectives of education graduate students. Journal of the Scholarship of Teaching and Learning, 15(2), 83–102. http://files.eric.ed.gov/fulltext/EJ1059422.pdf
Bouchard, P. (2013). Education without a distance: Networked learning. In T. Nesbit, S. M. Brigham, & N. Taber (Eds.), Building on critical traditions: Adult education and learning in Canada. Toronto: Thompson Educational Publishing.
Boyd, D. (2010). Privacy and publicity in the context of big data. Paper presented at the 19th International Conference on World Wide Web (WWW2010), 29 April 2010, Raleigh, North Carolina, USA. http://www.danah.org/papers/talks/2010/WWW2010.html
Boyd, D., & Crawford, K. (2012). Critical equations for Big Data. Information, Communication & Society, 15(5), 662–679. doi:10.1080/1369118X.2012.678878
British Science Council. (2009). What is science? http://www.sciencecouncil.org/definition
Cavoukian, A., & Jonas, J. (2012). Privacy by design in the age of big data. https://privacybydesign.ca/content/uploads/2012/06/pbd-big_data.pdf
Cormack, A. (2015). A data protection framework for learning analytics. Community.jisc.ac.uk. http://bit.ly/1OdIIKZ
Christopher, J. C., Wendt, D. C., Marecek, J., & Goodman, D. M. (2014) Critical cultural awareness: Contributions to a globalizing psychology. American Psychologist, 69, 645–655. http://dx.doi.org/10.1037/a0036851
Denzin, N., & Lincoln, Y. (Eds.) (2011). The Sage handbook of qualitative research (4th ed.). Thousand Oaks, CA: Sage.
Durand, G., Belacel, N., & Goutte, C. (2015) Evaluation of expert-based Q-matrices predictive quality in matrix factorization models. Proceedings of the 10th European Conference on Technology Enhanced Learning (EC-TEL ʼ15), 15–17 September 2015, Toledo, Spain (pp. 56–69). Springer. doi:10.1007%2F978-3-319-24258-3_5
Ed Tech Review (2016). Educational Data Mining (EDM). http://edtechreview.in/dictionary/394-what-is-educational-data-mining
El Emam, K. (1998). The predictive validity criterion for evaluating binary classifiers. Proceedings of the 5th International Software Metrics Symposium (Metrics 1998), 20–21 November 1998, Bethesda, MD, USA (pp. 235–244). IEEE Computer Society. http://ieeexplore.ieee.org/document/731250/
European Data Protection Supervisor. (2015). Leading by example: The EDPS strategy 2015–2019. https://secure.edps.europa.eu/EDPSWEB/edps/site/mySite/Strategy2015
Fenwick, T. (2015a). Things that matter in education. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw
Fenwick, T. (2015b). Professional responsibility in a future of data analytics. In B. Williamson (Ed.) Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw
Fournier, H., & Kop, R. (2015). MOOC learning experience design: Issues and challenges. International Journal on E-Learning, 14(3), 289–304.
Gergen, J. K., Josselson, R., & Freeman, M. (2015). The promise of qualitative inquiry. American Psychologist, 70(1), 1–9.
Gonzalez-Brenes, J., & Huang, Y. (2015). Your model is predictive — but is it useful? Theoretical and empirical considerations of a new paradigm for adaptive tutoring evaluation. In O. C. Santos, J. G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. M. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, & M. Desmarais (Eds.), Proceedings of the 8th International Conference on Education Data Mining (EDM2015), 26–29 June 2015, Madrid, Spain (pp. 187–194). International Educational Data Mining Society.
Gupta, A. (2013). Fraud and misconduct in clinical research: A concern. Perspectives in Clinical Research. 4(2), 144–147. doi:10.4103/2229-3485.111800.
Hardt, M. (2014). How big data is unfair: Understanding sources of unfairness in data driven decision making. https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de
Jones, C., Dirckinck-Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer Supported Collaborative Learning, 1(1), 35–56.
Kansanen, P., & Meri, M. (1999). The didactic relation in the teaching–studying–learning process. TNTEE Publications, 2, 107–116. http://www.helsinki.fi/~pkansane/Kansanen_Meri.pdf
Kitchin, R. (2015). Foreword: Education in code/space. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK. http://bit.ly/1NdHVbw
Kop, R. (2012). The unexpected connection: Serendipity and human mediation in networked learning. Educational Technology & Society, 15(2), 2–11.
Kop, R., Fournier, H., & Durand, G. (2014). Challenges to research in massive open online courses. Merlot Journal of Online Learning and Teaching, 10(1). http://jolt.merlot.org/vol10no1/fournier_0314.pdf
Long, Y., & Aleven, V. (2014). Gamification of joint student/system control over problem selection in a linear equation tutor. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014), 5–9 June 2014, Honolulu, HI, USA (pp. 378–387). New York: Springer. doi:10.1007/978-3-319-07221-0_47
Long, C., & Siemens, G. (2011, September 12). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5). http://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
Meyer, J. H. F., & Land, R. (2006). Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge. London: Routledge.
Mischel, W. (1968). Personality and assessment. London: Wiley.
Oblinger, D. G. (2012, November 1). Analytics: What we’re hearing. EDUCAUSE Review. http://er.educause.edu/articles/2012/11/analytics-what-were-hearing
Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.
Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450.
Siemens, G., Dawson, D., & Lynch, L. (2013). Improving the quality and productivity of the higher education sector: Policy and strategy for systems-level deployment of learning analytics. SoLAR. https://www.itl.usyd.edu.au/projects/SoLAR_Report_2014.pdf
Stewart, B. (2015). Open to influence: What counts as academic influence in scholarly networked Twitter participation. Learning, Media and Technology, 40(3), 287–309. doi:10.1080/17439884.2015.1015547
Weller, M. (2011). A pedagogy of abundance. Spanish Journal of Pedagogy, 249, 223–236.
Wen, M., Yang, D., & Rosé, C. (2014). Sentiment analysis in MOOC discussion forums: What does it tell us? In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM2014), 4–7 July, London, UK (pp. 130–137). International Educational Data Mining Society.
Xu, D., & Smith Jaggars, S. (2013). Adaptability to online learning: Differences across types of students and academic subject areas. CCRC Working Paper No. 54. Community College Research Center, Teachers College, Columbia University. http://anitacrawley.net/Reports/adaptability-to-online-learning.pdf
Title
A Critical Perspective on Learning Analytics and Educational Data Mining
Book Title
Handbook of Learning Analytics
Pages
pp. 319-326
Copyright
2017
DOI
10.18608/hla17.027
ISBN
978-0-9952408-0-3
Publisher
Society for Learning Analytics Research
Authors
Rita Kop1
Helene Fournier2
Guillaume Durand2
Author Affiliations
1. Faculty of Education, Yorkville University, United Kingdom
2. Information and Communications Technologies, National Research Council of Canada, Canada
Editors
Charles Lang3
George Siemens4
Alyssa Wise5
Dragan Gašević6
Editor Affiliations
3. Teachers College, Columbia University, USA
4. LINK Research Lab, University of Texas at Arlington, USA
5. Learning Analytics Research Network, New York University, USA
6. Schools of Education and Informatics, University of Edinburgh, UK