Handbook of Learning Analytics
Chapter 15
Handbook of Learning Analytics
First Edition
Epistemic Network Analysis:
A Worked Example of Theory-Based
Learning Analytics
David Williamson Shaffer & Andrew R. Ruis
Abstract
In this article, we provide a worked example of a theory-based approach to learning analytics in the context of an educational game. We do this not to provide an ideal solution for others to emulate, but rather to explore the affordances of a theory-based — rather than data-driven — approach. We do so by presenting 1) epistemic frame theory as an approach to the conceptualization of learning; 2) data from an epistemic game, an approach to educational game design based on epistemic frame theory; and 3) epistemic network analysis (ENA), a technique for analyzing discourse and other data for evidence of complex thinking based on the same theory. We describe ENA through a specific analytic result, but our aim is to explore how this result exemplifies what we consider a key “best practice” in the field of learning analytics.
Export Citation: Plain Text (APA) BIBTeX RIS
No Supplementary Material Available
Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired, 16(7). https://www.wired.com/2008/06/pb-theory/
Andrist, S., Collier, W., Gleicher, M., Mutlu, B., & Shaffer, D. (2015). Look together: Analyzing gaze coordination with epistemic network analysis. Frontiers in Psychology, 6(1016). journal.frontiersin.org/article/10.3389/fpsyg.2015.01016/pdf
Arastoopour, G., Shaffer, D. W., Swiecki, Z., Ruis, A. R., & Chesler, N. C. (2016). Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. International Journal of Engineering Education, 32(2), in press.
Atkinson, R., Derry, S., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.
Bagley, E. A., & Shaffer, D. W. (2009). When people get in the way: Promoting civic thinking through epistemic game play. International Journal of Gaming and Computer-Mediated Simulations, 1(1), 36–52.
Bagley, E. A., & Shaffer, D. W. (2015a). Learning in an urban and regional planning practicum: The view from educational ethnography. Journal of Interactive Learning Research, 26(4), 369–393.
Bagley, E. A., & Shaffer, D. W. (2015b). Stop talking and type: Comparing virtual and face-to-face mentoring in an epistemic game. Journal of Computer Assisted Learning, 31(6), 606–622.
Baker, R. S. J. d., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.
Chesler, N. C., Ruis, A. R., Collier, W., Swiecki, Z., Arastoopour, G., & Shaffer, D. W. (2015). A novel paradigm for engineering education: Virtual internships with individualized mentoring and assessment of engineering thinking. Journal of Biomechanical Engineering, 137(2), 024701:1–8.
Dorogovtsev, S. N., & Mendes, J. F. F. (2013). Evolution of networks: From biological nets to the Internet and WWW. Oxford, UK: Oxford University Press.
Gee, J. P. (1999). An introduction to discourse analysis: Theory and method. London: Routledge.
Goffman, E. (1974). Frame analysis: An essay on the organization of experience. Cambridge, MA: Harvard University Press.
Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606–633.
Cancho, R. F., & Solé, R. V. (2001). The small world of human language. Proceedings of the Royal Society of London B: Biological Sciences, 268(1482), 2261–2265.
Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (2007). Handbook of latent semantic analysis. Mahwah, NJ: Erlbaum.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, MA: Cambridge University Press.
Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203–208.
Mislevy, R. J. (2006). Issues of structure and issues of scale in assessment from a situative/sociocultural perspective. CSE Technical Report 668. Los Angeles, CA.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (1999). Evidence-centered assessment design. Educational Testing Service. http://www.education.umd.edu/EDMS/mislevy/papers/ECD_overview.html
Mislevy, R., & Riconscente, M. M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of Test Development (pp. 61–90). Mahwah, NJ: Erlbaum.
Nash, P., Bagley, E. A., & Shaffer, D. W. (2012). Playing for public interest: Epistemic games as civic engagement activities. American Educational Research Association Annual Conference (AERA). Vancouver, BC, Canada.
Nash, P., & Shaffer, D. W. (2011). Mentor modeling: The internalization of modeled professional thinking in an epistemic game. Journal of Computer Assisted Learning, 27(2), 173–189.
Nash, P., & Shaffer, D. W. (2012). Epistemic youth development: Educational games as youth development activities. American Educational Research Association Annual Conference (AERA). Vancouver, BC, Canada.
Nash, P., & Shaffer, D. W. (2013). Epistemic trajectories: Mentoring in a game design practicum. Instructional Science, 41(4), 745–771.
Newman, M. E. J. (2004). Analysis of weighted networks. Physical Review E, 70(5), 56131.
Rohde, M., & Shaffer, D. W. (2004). Us, ourselves and we: Thoughts about social (self-) categorization. Association for Computing Machinery (ACM) SigGROUP Bulletin, 24(3), 19–24.
Rupp, A. A., Gustha, M., Mislevy, R., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. Journal of Technology, Learning and Assessment, 8(4), 4–47.
Rupp, A. A., Sweet, S., & Choi, Y. (2010). Modeling learning trajectories with epistemic network analysis: A simulation-based investigation of a novel analytic method for epistemic games. In R. S. J. d. Baker, A. Merceron, P. I. Pavlik Jr. (Eds.), Proceedings of the 3rd International Conference on Educational Data Mining (EDM2010), 11–13 June 2010, Pittsburgh, PA, USA (pp. 319–320). International Educational Data Mining Society.
Shaffer, D. W. (2004). Epistemic frames and islands of expertise: Learning from infusion experiences. Proceedings of the 6th International Conference of the Learning Sciences (ICLS 2004): Embracing Diversity in the Learning Sciences, 22–26 June 2004, Santa Monica, CA, USA (pp. 473–480). Mahwah, NJ: Lawrence Erlbaum.
Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers and Education, 46(3), 223–234.
Shaffer, D. W. (2007). How computer games help children learn. New York: Palgrave Macmillan.
Shaffer, D. W. (2012). Models of situated action: Computer games and the problem of transfer. In C. Steinkuehler, K. D. Squire, & S. A. Barab (Eds.), Games, learning, and society: Learning and meaning in the digital age (pp. 403–431). Cambridge, UK: Cambridge University Press.
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.
Shaffer, D. W., Hatfield, D. L., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E. A., … Frank, K. (2009). Epistemic Network Analysis: A prototype for 21st century assessment of learning. International Journal of Learning and Media, 1(1), 1–21.
Svarovsky, G. N. (2011). Exploring complex engineering learning over time with epistemic network analysis. Journal of Pre-College Engineering Education Research, 1(2), 19–30.
Tannen, D. (1993). Framing in discourse. Oxford, UK: Oxford University Press.
Wise, A. F., & Shaffer, D. W. (2016). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.
Title
Epistemic Network Analysis: A Worked Example of Theory-Based Learning Analytics
Book Title
Handbook of Learning Analytics
Pages
pp. 175-187
Copyright
2017
DOI
10.18608/hla17.015
ISBN
978-0-9952408-0-3
Publisher
Society for Learning Analytics Research
Authors
David Williamson Shaffer
Andrew R. Ruis
Author Affiliations
Wisconsin Center for Education Research, University of Wisconsin–Madison, USA
Editors
Charles Lang1
George Siemens2
Alyssa Wise3
Dragan Gašević4
Editor Affiliations
1. Teachers College, Columbia University, USA
2. LINK Research Lab, University of Texas at Arlington, USA
3. Learning Analytics Research Network, New York University, USA
4. Schools of Education and Informatics, University of Edinburgh, UK