Chapter 16 : Multilevel Analysis of Activity and Actors in Heterogeneous Networked Learning Environments

actors, and identifying relationships between actors. The approach outlined here, the Traces framework, has been used for discovery oriented research, but can also support hypothesis testing research that requires variables at these higher levels of description, or live monitoring of production learning settings using such descriptions. The Traces framework involves a set of concepts for thinking about and modelling interaction in sociotechnical systems, a hierarchy of models with corresponding representations, and computational methods for translating between these levels by transforming representations. These meth-

This chapter is most relevant to readers who will be more digital settings (which may be of multiple media types), and wish to computationally derive higher levels of description of what is happening, possibly across description include identifying sessions of interaction, identifying groups or communities of learners across actors, and identifying relationships between actors.The approach outlined here, the Traces framework, has been used for discovery oriented research, but can also support hypothesis testing research that requires variables at these higher levels of description, or live monitoring of production learning settings using such descriptions.The Traces framework involves a set of concepts for thinking about and modelling interaction in sociotechnical systems, a hierarchy of models with corresponding representations, and computational methods for translating between these levels by transforming representations.These methand tested on data from a heterogeneous networked learning environment.
The purpose of this chapter is to introduce the reader to the conceptual and representational aspects of the framework, with brief descriptions of how it can be used for multilevel analysis of activity and actors our implementation and research are not included 1 . 1 and Suthers and Rosen (2011) for the development of our analytic representations; see Suthers, Fusco, Schank, Chu, and Schlager (2013) for community detection applications; see Suthers (2015) for activity reporter for monitoring a large networked learning environment.Suthers et al. (2013) and Suthers (2015) describe the data from the Tapped In network of educators we used as a case study hawaii.edu.
Motivations for the Traces framework derive in part from phenomena such as the emergence of Web 2.0 (O'Reilly, 2005) and its adoption by educational practitioners and learners for formal and informal learning, including more recent interest in MOOCs (massive open online courses) (Allen & Seaman, 2013).In these environments, learning is distributed across time and virtual place (media), and learners may participate in multiple settings.We focus on networked learning sociotechnical network that involves mediated interac-& Shumar, 2002) and cMOOCs (connectivist MOOCs) (Siemens, 2013).The framework is not applicable to which large numbers of individuals interact primarily with courseware or tutoring systems.
Learning and knowledge creation activities in these networked environments are often distributed across multiple media and sites.As a result, traces of such activity may be fragmented across multiple logs.For clude a mashup of threaded discussion, synchronous resource sharing.Events may be logged in different formats and locations, disassociating actions that for gration of multiple sources of trace data into a single transcript may be needed to reassemble data on the interaction.Also, the granularity at which events are media-level events may be the wrong ontology for analyses concerned with relationships between acts, persons, and/or media rather than individual acts.levels of description may be required to begin the primary analysis.
tivated by theoretical accounts of learning as a comlearning takes place in social settings vary regarding the agent of learning, including individual, small group, network, or community; and in the process of learn-mentation, intersubjective meaning-making, shifts in participation and identity, and accretion of cultural taneously at all of these levels of agency and with all of these processes, potentially at multiple time scales (Lemke, 2000).A multi-level approach is also motivated by our theoretical stance that social regularities arise from how myriad individual acts are aggregated (Latour, 2005), and the methodological implication that to understand phenomena such as actor relationships or community structures, we also need to look at the stream of individual acts out of which these phenomena are constructed.Thus, understanding learning in its full richness requires data that reveal the relationships between individual and collective levels of agency and potentially coordinating multiple & Contractor, 2003;Suthers, Lund, Rosé, Teplovs, & Law, 2013).This section covers the levels of description and corresponding representations underlying the Traces potential applications.To preview the approach, logs of events are abstracted and merged into a single abstract transcript of events, which is then used to derive a series of representations that support levels of analysis of interaction and of relationships.Three kinds of graphs model interaction.Contingency graphs record how events such as chatting or posting a message are observably related to prior events by temporal Uptake graphs aggregate the multiple contingencies between each pair of events to model how each given act may be Session graphs are abstractions of uptake graphs: they cluster events into spatio-temporal sessions with uptake relationships between sessions.Relationships between actors and artifacts are abstracted from interaction graphs to obtain sociograms that we call associograms.The representations used at various levels of analysis are shown schematically

About Transcript
We begin with various traces of activity (such as log from different media (e.g., chats, threaded discussion, chat, chat contribution, post message, read message, stamps, actors, content (e.g., chat content), and locations (e.g., chat rooms) involved in the event where relevant.The result is an abstract transcript of the distributed sentations of activity to the abstract transcript, we integrate hitherto fragmented records of activity into TRACES ANALYTIC FRAMEWORK MOTIVATIONS one analytic artifact.

Contingency Graph
We then compute contingencies between events (arrows upon its setting in diverse ways: computational methods can capture some of the contingencies amenable called proximal event occurring close together in time and space are related.installed to prior contributions in the same room that occur within an adjustable time window but not too recently.Address and reply contingencies are installed between an utterance mentioning a user by name and ticipant within a time window, using a parser/matcher Same actor contingencies are installed to prior acts of a participant over a larger purpose.Overlap in content as represented by sets lexical overlap contingency weighted by the number of overlapping stems.Further contingencies could be computed based on natural language processing methods for analysis of interactional structure (Rosé et al., 2008).action model.In this graph, vertices are events, and contingencies are typed edges between vertices multiple edges between any two vertices (e.g., two will have at least three contingencies between them).

Uptake Graph
It is necessary to collapse the multiple edges between vertices into single edges for two reasons.First, most graph algorithms assume at most only one edge between any two vertices.Second, we are interested in uptake, the relationship between events in which a human action takes up some aspects of prior events damental building block of interaction (Suthers et al., 2010), uptake is a basic unit for analysis of how learning takes place in and through interaction.Replying to prior of uptake, but uptake is not limited to replies: one can appropriate a prior actor's contribution in other in different media, and cross media (Suthers et al., 2010).Contingencies are of interest only as collective evidence for uptake, so we abstract the contingency graph to an uptake graph.contingency graphs in that they also relate events, but they collect together bundles of the various types of contingencies between a given pair of vertices into a single graph edge, weighted by a combination of the strength of evidence in the contingencies and optionstructure of sessions, constructing sociograms).Importantly, we do not throw away the contingency the nature of the uptake relation, and, once aggregated into sociograms, of the tie between actors.We can do several interesting things with uptake graphs, but want to handle separately, as they represent sessions.

Sessions
computed to identify sessions (indicated by rounded discussed later.For intra-session analysis, the uptake graph for a session is isolated.Several paths are pos-stand the development of group accomplishments: for graph structure analysis can be applied, such as cluster detection, or tracing out thematic threads (Trausan-Matu & Rebedea, 2010).For inter-session representing the session, but retain the inter-session time and space from one session to another.

Sociograms
Sociotechnical networks are commonly studied using the methods of social network analysis, using sociogram or sociomatrix representations of the presence or strength of ties between human actors, and graph algorithms that leverage the power of these reprenon-local (network) social structures (Newman, 2010; Wasserman & Faust, 1994).Either within or across tie strength between actors is the sum of the strength of uptake between their contributions.If we want to be stricter about the evidence for relations between the two actors, we may use a different weighting that of orientation to the other actor.These sociograms identify key actors.

Associograms
The sociogram's singular tie between two actors tween the actors on which the tie is based, as well as the media through which they interacted.To retain the advantages of graph computations on a summary representation while retaining some of the information about how the actors interacted, we use bipartite, multimodal, directed weighted graphs, similar to bipartite because all edges go strictly between actors and artifacts and multimodal because the artifact (arcs) indicate read/write relations or their analogs: an arc goes from an actor to an artifact if the actor has read that artifact (e.g., opened a discussion message or was present when someone chatted), and from an (e.g., posted a discussion message or chatted).The direction of the arc indicates a form of dependency, the arcs indicate the number of events that took place between the corresponding actor/artifact pair in the nature we call these graphs associograms (Suthers & Rosen, 2011).This term is inspired by Latour's (2005) concept that social phenomena emerge from dynamic networks of associations between human and non-human actors.representing asymmetric interaction between two another writing to most of the discussions.A sociogram consisting of a single link between these two actors would fail to capture this information.The associogram retains information about the distribution of activity across media.Network analytic methods can then simultaneously tell us how both human actors and artifacts participate in generating the larger phenomena of interest, such as the presence of communities of actors and the media through which they are technologically embedded (Licoppe & Smoreda, 2005).Although interaction is not directly represented, the associogram also provides a bridge to the interaction level of analysis (Suthers et al., 2010), allowing us to The Traces framework provides multiple pathways for analysis.In the following sections we illustrate various analyses that can be supported by this framework implementation.

Identifying Sessions of Interaction
interaction graphs.If interaction is not clearly demarcated by periods of non-interaction and one wishes to discover clusters of high activity, we have found that algorithms (Fortunato, 2010) such as modularity par-2008) applied to uptake graphs are useful (Suthers, 2017).If (as in our Tapped In data) activity is distributed across rooms and the activity within a room almost always has periods of non-activity between sessions, construct a contingency graph (it can be constructed later for other purposes).Activity is tracked in each every time there is a gap of S seconds of no activity.S is a tunable parameter, such as 240 seconds.Suthers (2017) discusses these options further.

Tracing Inf luences Between Sessions
sessions across time and space.Uptake relations between events in different sessions can be aggregated into weighted uptake relations between sessions (Figure session graph from Suthers ( 2015) is weighted links between them.Reading the edges in reverse order (uptake points backwards in time), we session 755 (Teaching Teachers room), which in turn rooms and participants involved showed that many participants logged into or met in the Reception room, then went to Teaching Teachers for session 755 on mentoring in the schools.Then the facilitator of 755 announced that she had another session on teacher training in another room: several participants in the mentoring session followed her to NTraining for session 848.Further details are in Suthers (2015).

Identifying Actor Roles and Tracking Change in Participation Over Time
Educators or NLE facilitators may want to identify the key participants in their online learning communities, whether for assessment in formal educational settings, to encourage volunteers in participant-driven settings, or for research purposes such as to study what drives participants.It is also important to know who is disengaged.Some of these needs can be met through social network analysis.We can generate sociograms for any granularity of the uptake graph (e.g., within a session, or across sessions over a time period) by folding uptake relations between events into ties in session 755, the session on mentoring teachers led Sociograms add information over mere counts of number of contributions because some sociometrics are hence potentially took up a given act.Aggregating these acts for an actor is an estimate of how much an actor's contributions are taken up by others.This metric is sensitive to both the level of activity of the actor and that activity's relation to others' activity.Weighted out-degree is an estimate of how much an actor takes up others' contributions.Eigenvector centrality (and ties) is a non-local metric that takes into account the to others who are themselves central.Betweeness centrality is an indicator of actors who potentially play brokerage roles in the network: high betweeness centrality means that the node representing an actor is on relatively more shortest paths between other actors (Newman, 2010, p. 185), so potentially controls actors.Betweeness is of particular interest when generally have different actors, so an actor attending multiple sessions will have high betweeness.

EXAMPLES OF ANALYTIC OPTIONS
Analyses on longer time scales may be of interest to researchers as well as practicing educators.One can trace the development of actors' roles over time in one might aggregate uptake for all actors in the network into sociograms at one week intervals, and then graph the sociometrics on a weekly basis, looking for trends.One can see some of these trends in Figure guide), and for those who return for periodic events facilitated monthly events).Steadily increasing or decreasing metrics indicate persons becoming more many sociometric analyses found in the literature, so we should highlight what the Traces framework has added.Our implementation of the Traces framework derived these latent ties from automated interaction analysis of streams of events, by identifying and then aggregating multiple contingencies between events, and then folding the resulting uptake relations between sis or the use of surveys to derive tie data, which are latent ties in actual interaction between the persons in question.Another advantage is described below.

Identifying Relationships Between Actors
The Traces framework derives ties between actors by aggregating multiple contingencies between their contributions.The contingencies indicate the qualitative nature of the relationship between these contributions, e.g., being close in time and space, using the same words, and addressing another actor by name.When contingencies are aggregated into uptake relations, we keep track of what each type of contingency contributed to the uptake relation.This record keeping is continued when folding uptakes into ties, so that for any given pair of actors we have a vector of weights that provides information about the of the relationship in terms of the underlying given time period in terms of how often they chatted chat contents, and how often they addressed each other by name in each direction.Relational information might be of interest to educators or researchers who are managing collaborative learning activities amongst students.The Traces framework makes this possible by retaining information about the interactional origins of ties (see Suthers, 2015).as evidence for the latter, particularly when studying networked societies (Castells, 2001;Wellman et al., 2003).and then provides a linked abstraction hierarchy using observable contingencies between events to build models of interaction and ties.Contingencies are applied to events in the abstract transcript to produce a contingency graph.Contingencies are then aggregated into uptake between the same events.Uptake that crosses partitions can be used to identify structure of a session.Uptake graphs can be folded into networks where nodes are actors rather than events, to which sociometrics are applied.Events ciated with each other via mutual read and write of media objects.The framework addresses the need to tested with data from a heterogeneous networked learning community.

Identifying Groups or Communities of Learners Across Sessions
Other authors have noted the need to combine multiple analysis in networked learning environments.For of combining social network analysis with various qualitative and quantitative methods in the study of participation networks.Others have constructed and folded interaction graphs into sociograms of ties from references and names.The Traces framework is in the same spirit, but is arguably more mature.We consider multiple kinds of relations between events subsequent analysis of activity and actors within sessions, and have automated these analyses and tested them on a rich historical data corpus where diverse many features of today's distributed interaction.use of multiple contingencies, but has only recently been abstracted to higher levels of analysis.A thesis by Charles (2013) has provided an alternative implecontingencies.Our approach dovetails with work that applies natural language processing methods for analysis of interactional structure, and indeed rules for generating additional contingencies could be derived from such research.Although our software is

Figure 16
Figure 16.1.Levels of analysis and their representations.

-
Figure 16.2.An associogram from the Tapped In data.Actors represented by nodes on the right have read by differently coloured nodes on the left.

Figure 16 . 5 .
Figure 16.5.Eigenvector Centrality of several actors in Tapped In over a 14 week period.
intuition that individuals in a sociological community are more closely associated with each other than they are with individuals outside of their community.Algorithms based on the modularity metric are widely used in the literature for this purpose.The modularity metric (Newman, 2010, p. 224) compares the density of weighted links inside (non-overlapping) partitions best possible partition under a modularity metric is computationally impractical on large networks, but a fast algorithm known as the Louvain method (Blondel -Once partitions have been obtained, one can characone large community, or does the network contain does the use of different media vary with community and the media through which they interact to interpret each partition.See Suthers, Fusco, et al. (2013) for This chapter introduced the Traces analytic framework, which integrates traces of activity distributed across media, places, and time into an abstract transcript,