Beyond Threaded Discussions: Learning with Knowledge Building Analytics

We need better analytics for student discourse

Online discussion systems are a standard feature in learning platforms. They’re used for a range of purposes, including deepening student’s engagement with curriculum concepts, team communication in project investigations, and community building. However, such tools provide no support to help students understand how to make better contributions for knowledge building, and they provide very limited forms of analytics for educators to understand the quality of the conversation. Number of forum posts, replies, and wordcount may be useful for site administrators, and provide a very rough indicator of “student engagement” for teachers — but could we provide more insights into the quality of contributions, that would be beneficial for both teachers and students?

We’ve studied analytics in forums and are convinced we need analytics that are more meaningful and impactful in improving the quality of the discourse. Here are 3 reasons:

1. Teaching and assessment remain heavily dependent on visual cues during face-to-face interactions but with online instruction necessitated by emergent situations (e.g., the pandemic), we can no longer rely on such cues. Learning analytics can help fill this gap and provide greater insights into student discourse.
2. Students’ development of important 21st century competencies such as collaboration and communication skills may still be captured if teachers choose to conduct online discussions with students. With learning analytics, teachers can also remotely assess these attributes and students’ development.
3. Apart from using online discussions to achieve learning objectives, students also engage in problem-sharing and idea-building. Discourse-centric learning analytics can play a greater role in providing insights into these processes for meaningful learning and knowledge construction.

Knowledge Building

Knowledge Building is a long-term process that equips communities of students to create knowledge in a way we can capture. Knowledge building is guided by the 12 principles of knowledge building and is a collaborative effort directed towards the creation and improvement of theories, designs, and ideas for problem solving.

Knowledge Forum web platform

The principles underlying the theory and pedagogy of knowledge building have been realized in a well-designed online platform, Knowledge Forum, purpose-built to support knowledge building in school age students, through synchronous and asynchronous discourse. Knowledge Forum provides students with scaffolds for their online contributions, such as "I need to understand” and “This idea doesn’t explain”. These sentence-starters help students to think through what and how they want to build on and share their ideas with their peers.

For example, this screenshot shows how students who have difficulty in starting off a note on the Knowledge Forum can choose one of the scaffolds. They’ve chosen Theory Building and “I need to understand” as the opening to their question.

In contrast to traditional forum posts that are undifferentiated in type and a simple linear sequence/branch, Knowledge Forum conversations are displayed as spatial networks to convey at a glance a sense of the context and depth of discussions. Some of the affordances from the Knowledge Forum are highlighted when we compare a single threaded discussion that can be found on most conventional forums and the potentially wider range of ideas and discussions that can be generated from a multi-threaded discussion.

In this example, primary school students were discussing whether a historical figure was a good or bad emperor. It is usually difficult to get students to discuss in-depth, especially when students are asked to continue online. The teacher in this lesson, however, captured conclusions from the classroom discussions and reposted them onto the Knowledge Forum (note the handwritten notes on the right of the ruler’s portrait), which students were able to reference and continue their exploration of the topic. This action helps to provide a seamless transition between offline and online discussions.

Knowledge Building Analytics

Knowledge Forum’s discourse structure and mechanisms for encouraging contributions (e.g., scaffolds and build-on) and consolidations (e.g., group summaries) are what distinguishes Knowledge Forum from other conventional platforms. A suite of novel Learning Analytics tools are integrated into Knowledge Forum to help teachers make sense of students’ work, e.g.

• how well are students resolving problems?
• where are the knowledge gaps in the learning community?
• how should this inform face-to-face lesson plans?

This barchart provides summary statistics of student activity, but goes beyond merely charting the number of posts. Now the teacher can see which scaffolds and buildons are being used, to inform interventions.

Curriculum-ideas-Analytics (CiA) show whether students’ contributions are mentioning syllabus concepts within and beyond their grade levels. In Knowledge Forum, it facilitates teacher insight of student discourse in a visual manner. The CiA firstly mines a wide range of textual syllabuses to obtain multiple sets of syllabus-related keywords from various grades, and then compares these sets of keywords with those that are found in the student discourse. Depending on the set of keywords chosen for comparison, the final result is a word cloud containing keywords highlighted with different colors, indicating key curriculum words that students are currently using, have yet to use, or are using beyond their current grade levels. This has proved to be very useful for teachers who are looking for an automated method of gauging curriculum coverage and the potential of ideas from student discourse.

In this third example, Knowledge Building is connected to online documents and discourse through direct annotation. Hypothes.is an open-source web annotation tool, now integrated into the Knowledge Forum platform in the “Idea Magnet” tool. First, the student annotates a website:

Finally, Idea Thread Mapper connects multiple students’ ideas across multiple discourse spaces on Knowledge Forum. Students are able to share discussions and create “wondering areas” as major directions of inquiry, based on shared interests and emergent problems. Within each wondering area, students can then focus on idea threads as a line of inquiry. As students build-on each other’s notes, the responses are visualized as lines that criss-cross between different students on a timeline. Each row is a student, and each dot is a note on the timline.