How Do We Define Success in an Open Course
In June I attended the Society for Teaching and Learning In Higher Education (STLHE) conference in Kingston, Ontario. As part of the conference I presented, along with Nancy Turner and Jaymie Koroluk (University of Ontario Institute of Technology), a poster about the Introduction to Learning Technologies (ILT) open course that the GMCTE offered earlier this year. During discussions around our poster as well as in other sessions related to open courses, I had a number of conversations with colleagues about just what is “success” in an open course.
Completion rates are often used as measures of success by administrators and the media, but is that really a fair measurement? Open courses, whether we call them MOOCs (Massive Open Online Courses) or the TOOCs (Truly Open Online Courses) that we’re advocating at the GMCTE, aren’t like traditional face-to–face or distance courses in that students don’t pay tuition, there are no prerequisites for entry into the courses and no formal credit is given to students. Why do we try to measure success in open courses using the same metrics that we use for traditional courses when they are so different (of course the argument can absolutely be made that rates of attrition in traditional courses shouldn’t be measures of success either)?
While I was at the conference, an article appeared in The Chronicle of Higher Education about a new paper out from a study conducted jointly by researchers at Cornell University and Stanford University looking at types of engagement in “Massive Online Courses”. The authors of the study argue that there are five types of participants in open courses including Viewers (watch the videos), Solvers (complete assignments without watching videos or reading lecture notes), All-Rounders (do at least some of both), Collectors (download for viewing materials later) and Bystanders (they registered, but there’s no evidence that they did anything in the course). I think that these categories have merit and provide a more nuanced picture of participants, taking us beyond simply grouping everyone into those who complete and those who don’t.
Very few people completed all of the assignments in ILT, so if we looked at completion rates as the measure of success, then this course was a failure. If, however, we look at different metrics another picture emerges. After the course ended (it’s a truly open course so all of the materials are still open) we sent a survey to the 300 participants and 15 percent completed the surveys (yes, I know it’s a very low response rate, but it’s an open course and most people may have been ignoring my emails by the end). Of those who completed the survey, 81.3% said that they applied what they had learned for their own professional development and 69.6 percent said that they shared what they learned with colleagues and / or students.
Learning technologies are constantly changing and as such, I saw it as important that there should bean increase in participant comfort and skill in using a variety of types of tools rather than developing expertise in use of specific ones. A key success of the course for me was therefore the response to the survey question regarding the effect the course had on their comfort level with learning technologies; 55.3 percent reported a moderate increase and 21.3 percent said they experienced a considerable increase.
Of course the low rate of response does mean we have to interpret these results with caution, but the data does add to the argument that success for these courses shouldn’t be measured by how many students do all of the work. I’m currently completing an overall program review of the course for one of my Ph.D. courses and will then be revising the course for another offering next January (watch for details about the course dates and registration to appear on Educatus in the Fall). We’re also working with Ken Coates, the Canada Research Chair in Regional Innovation at the Johnson-Shoyama Graduate School of Public Policy and the Director of the International Centre for Northern Governance and Development on an open course that he’ll be teaching early in 2015. Both courses will provide us with valuable information on what students actually do in an open course, as well as how they define success for themselves.
You May Also Like
