Home » Uncategorized » “It’s Big Brother, sort of, but with a good intent”

“It’s Big Brother, sort of, but with a good intent”

Back in the day, when professors wanted to know whether students had done the required reading, they would ask questions in class and engage students in discussions that would flow from their answers. They would draw upon nonverbal cues signalling enthusiasm, confusion, boredom, and numerous other feelings the material might evoke. From time to time, they would give exams to measure student understanding. But now, the good folks at CourseSmart, the product of a consortium of textbook publishers, are promising to improve our teaching lives through technology. Now, through the use of digital textbooks, we can “know when students are skipping pages, failing to highlight significant passages, not bothering to take notes — or simply not opening the book at all.”

With this technology, instructors receive an “engagement index” for each student based on how and how much students interact with their electronic textbooks. From CourseSmart’s perspective, this ongoing data mining will improve the educational experience for everyone. Students who are struggling will get prompt, targeted feedback. Instructors, and especially those teaching online courses, will get advance notice of which concepts students understand and which ones confound them. And publishers will be able to use the data to revise sections of textbooks to address student problems, or even perhaps to eliminate sections that aren’t being assigned or read.

Color me skeptical, though. Measuring student engagement through the use of this tracking technology would appear to produce results that are both over-inclusive and under-inclusive. They are over-inclusive because, as the Times noted, students can readily find ways to game the system by behaving in ways that reflect engagement in the most superficial way, much as scholars have been able to trick GradeBot into awarding high scores to essays featuring gibberish. Even leaving the textbook open without actually reading it contributes positively to the engagement score. At the same time, the scores are under-inclusive because handwritten notes, or notes stored to a computer file not being tracked, don’t register as engagement.

Another concern has to do with the assumptions made by CourseSmart about how people interact with their source material. As a student, when I encountered material that was new and complex, I would make a point of identifying central themes in the text. But when the material was less challenging, when the central argument was intuitive and easily remembered, I didn’t bother to note or highlight it. Instead, I would flag, with a “tell me something I don’t already know” attitude, things that were novel, thought-provoking, or counter-intuitive. This approach served me well. But I suspect that the CourseSmart tracker would have reported that I wasn’t understanding the main ideas, or that I was getting distracted by minor points. Maybe I’m weird this way (and yes, friends and family, I can hear you snickering from over here). But my larger point is that each student is weird in his or her own way, and an assessment method that compels a standardized approach to reading will fail students by denying them the opportunity to have their own weirdness work to their benefit.

Finally, I question what effect CourseSmart will have on the books themselves. Yes, as CourseSmart chief executive Sean Devine put it, before the software existed, “the publisher never knew if Chapter 3 was even looked at.” The implication is that the publisher would respond to this neglected Chapter 3 by asking the textbook’s authors to revise, or even to remove, it. But while it is possible that the chapter isn’t being read because professors aren’t assigning it, it’s also possible that professors are assigning it, but students aren’t reading it, or aren’t marking it in ways that register in the engagement score. Should the content of textbooks be driven by what students want, rather than by what educators deem to be valuable? There’s a kind of focus-group mentality at work here, the same kind that leads Hollywood studios to override filmmakers’ visions in order to satisfy the presumed audience’s desire to see more explosions, more boobs, and more happy endings even if they feel as fake as the gratuitously added boobs. Students aren’t customers, and treating them as such does no favors for authors who want to develop, and instructors who want to assign, texts that challenge students and engage them without making their immediate gratification a central concern.

There’s this underlying theme running throughout these technological innovations, be they EdX’s software (what I’ve dubbed GradeBot above) or CourseSmart’s Great Big Data Miner: that professors should welcome these shortcuts because they make our jobs easier and free up time for other tasks. As the comments sections in the Times attest to, my fellow professors are not oblivious to the long-term implications of technology that could make education–“education,” more accurately–available without having to hire so many of us. More gratifyingly, they, and I, also express puzzlement and annoyance at the assumption that we’re seeking shortcuts, that we’d settle for the simulacrum of educational engagement instead of relying on our expertise and experience to determine whether, and how effectively, students are learning.




  1. Lisa says:

    Great comments, Jeremy. I don’t use Coursesmart, but even if I did, I would not use these capabilities and their features would not change the way I teach. While it is frustrating to try to facilitate a discussion with students who haven’t read the material, I don’t have the time nor the interest to police student study habits. I’d like to think that my lecturers and activities are challenging enough to show me when students have read and when they haven’t. For the most part, students get out of a course what they put into it. If they don’t find the content interesting and it doesn’t motivate them to learn more about the topic/field, then the best I can hope for is that they leave my classroom more informed. For example, no amount of thought provoking questions or “fun” activities any math teacher/professor could have done would have made me more interested in learning math concepts. If they aren’t motivated/ interested in the material, the only other motivator is their grade. If students don’t read and don’t engage in class sessions, they will not do well in the course. And a student who receives a low grade on the first quiz/test will either realize that to s/he has to do the work and engage with the material, or else accept a low grade as what s/he will most likely earn. Honestly, if I had wanted to hover over my students, I would have gone into elementary education. I don’t think policing them at the university level does anything to promote autonomy or responsibility, and over the past decade, I have seen students demonstrate these qualities with less and less frequency. I’m a professor, not a campus extension of a helicopter parent.

  2. Jeremy Buchman says:

    Lisa–You have the honor of leaving the first non-test comment at the blog, and I can only hope that future commenters will bring such high quality to the site.

    I appreciate that you’ve picked up on two other dimensions of CourseSmart that rub me the wrong way. The first is its training students for life in the surveillance state, where every action is monitored and reviewed for how well it serves the employer’s ends. The second is the replacement of intrinsic motivation with reward-driven motivation. We’re training young people to encounter the real world, and one thing students should learn is that not every task they face can be repurposed as fun. Some students figure this out faster than others, but all students have to learn the lesson for themselves.Otherwise, they will spend their adult lives waiting for the helicopter to bring rescuers.

  3. This made me think of a marketing professor I had whose class instruction consisted of, literally, opening the book and reading verbatim what was written aloud to the class. That’s it. I hated that class and it really turned me off to marketing (which looking back I wish it hadn’t because I could use some knowledge in that area today!) If that professor used this technology, the whole text would be seen as “usable” with a very high engagement rate because we all sat in the class, turning pages along with the professor. But of course, those scores would be wrong!
    I guess as with just about any new technology, there are bugs and unintended consequences that need to be worked out. History shows that technology is sought out and used in classrooms so I’d be interested in seeing if this ends up panning out!

    • Jeremy Buchman says:

      Great example! Even if nobody were trying to game the system, you’d still end up with a very misleading indicator of engagement. And, of course, there are so many incentives on everyone’s part to game the system.

      I am saddened to hear that students would be subjected to any course centered around the instructor reading from the textbook, but it’s especially ironic to have it happen in a subject like marketing, where learning to connect with one’s audience is so crucial. I shudder to think that one of my colleagues was the guilty party. (Not asking for names.)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: