Predicting course outcomes with digital textbook analytics

By Reynol Junco, associate professor, Iowa State University; and faculty associate, Berkman Center for Internet and Society1.

Over the last years, I’ve been researching how real-time behavioral data, collected unobtrusively through technology, can predict learning outcomes. As part of this line of research, I’ve recently published the paper Predicting course outcomes with digital textbook usage data in The Internet and Higher Education.

The study used data collected from student engagement with digital textbooks in order to predict course grades. Two measures of student engagement with the texts were analyzed: an engagement index that was calculated through a linear combination of the number of pages read, number of times a student opened their textbook, number of days the student used their textbook, time spent reading, number of highlights, number of bookmarks, and number of notes. The second analysis included the individual components of the engagement index.

Major Findings
The engagement index was significantly predictive of final course grades and was a stronger predictor of course outcomes than previous academic achievement. However, time spent reading, one of the variables that make up the engagement index, was more strongly predictive of course grades than the entire engagement index.

« Prediction based on digital textbook analytics can help identify students at risk of poor performance in real time »

The effects of course level and instructor were controlled as well as the effects of gender, race/ethnicity, and previous academic achievement. In other words, data collected unobtrusively through the use of digital textbooks can be used to predict student outcomes even when using it for courses with different levels of reading, with textbooks that require varying levels of reading comprehension, with instructors who have different teaching styles, and for subject areas that vary in their technical nature.

Students who read more, do better in their courses. While that might be a “no duh” kind of conclusion, what is noteworthy from this research is the knowledge that behavioral academic data collected unobtrusively can predict how well a student will do in a course better than previous academic performance (which is typically the single best predictor of course outcomes).

Commonly, students are categorized as “at risk” based on previous academic performance (using high school GPA or SAT scores, for instance); however, this method of focusing interventions casts an overly broad net and misses students who might be struggling for other reasons.

Prediction based on digital textbook analytics can help identify students at risk of poor performance in real time, even before a student submits any gradable material to the faculty member. In the future, digital textbook data can be added to other data sources — like learning analytics from learning and course management systems — in order to provide even more precise prediction of student success and to be better able to target interventions for those students most at need.

Note: A longer version of this article was first published at Junco’s blog: Social Media in Higher Education, which is released under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. The text and accompanying picture are hereby adapted and reproduced with permission from its author, and released under the same mentioned license.

Tagged with: , , , , , , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *


About the Question
Are Big Data & Analytics shaping a smarter society?

Every day we generate a huge amount of big data, but we need to resort to analytics to make abstract information meaningful and get valuable knowledge from it. In education, learning platforms let us easily gather an immense quantity of data regarding students’ behaviour, interactions, preferences and opinions. When properly analysed — through learning analytics — all these data might provide useful insight on how to make learning processes more adaptive, attractive and efficient.

Are these techniques allowing us to provide better support to our students? Are we taking advantage of big data and analytics to help shape the citizens of the future?

Big Data and Simheuristics