Multilitteratus Incognitus
Pondering what to learn next 🤔
MOOC Completion...according to whom?
22-01-2020, 12:22 accomplishment, Badges, cMOOC, completion, coursera, CV, defunct, documentation, edx, futurelearn, learning, MOOC, Open2Study, xMOOX
The other day I had an interesting (but brief exchange) with Kelvin Bentley on twitter about MOOC completion. This isn't really a topic that I come back to often, given that completion-rates for MOOCs, as a topic, seems to have kind of died down, but it is fun to come back to it. To my knowledge, no one has come up with some sort of taxonomy of the different degrees of completion of a MOOC†.
But let me rewind for a second. How did we get to the topic of MOOC completion? Well, I've been attempting to make my extended CV more accessible (to me). In the past, I used a WYSIWYG HTML publishing platform to manage my extended CV‡. The idea was that I could easily export it and just push it on the web. In practice, I never did this, and when I changed computers it became a hassle to maintain. So, I moved everything over to google docs for cleanup (and easier updates). In cleaning up my CV sections (I am not done, btw!), I did make a startling self-discovery. In the time-period 2013-2016, I binged on a lot of xMOOCs!😅 The most notable platforms were Coursera, Edx, Udacity, but there were others such as the now-defunct Janux (Oklahoma University) and Open2Study (Australia Open University), as well as overseas platforms like MiriadaX and FutureLearn. In the time period 2011-2012 I didn't have a lot of MOOCs, mostly because during this period a lot were cMOOCs and xMOOCs hadn't really spread like wildfire.
This realization now begs the question: "How many did you complete?" (and you guessed it, Kelvin asked it...). My answer comes in the form of a question "based on whose metrics and measures?". When you sign up for a paid course (e.g., professional development seminar, college course, certification prep course, etc.) I think that there is an unspoken assumption that the goals of the course mirror, to a greater or lesser extent, the goals of the learner♠. Can this assumption be something that transfers over into the world of a free MOOC? I personally don't think so. I've long said that the course completion metric (as measured by completing all assignments with a passing grade) is a poor metric. One very obvious reason to me was that people simply window-shop; and since there is no disincentive to unenroll, people don't take that extra step to leave the course formally, as they would with a paid course where they could receive a refund. I've been saying this since xMOOC completion rates were touted as an issue, but few people listened. Luckily it seems that people are changing their minds about that (or just don't care 😜). I guess George Siemen's advice to Dave Cormier holds true for my own rantings and ravings: publish those thoughts in a peer-reviewed journal or they don't exist 🤪 (paraphrased from a recent podcast interview with Dave).
Assuming that we exclude window-shoppers from our list of completion categories♣, what remains? Well, instead of thinking of distinct categories (which might give us a giant list), let's think of completion in terms of whose perspective we are examining. On the one extreme, we have the learner's perspective. The extreme learner's perspective is characterized by total control by the learner as to what the goals are. In this perspective, the learner can be in a course and complete a certain percentage of what's there and still consider the course as done. Why? The learner might have prior knowledge, and what they are looking for is to supplement what they already know without going through the hoops of any or all assessments in the course. They've probably evaluated the materials in the course, but if they already know something, why spent a lot of time on something already known? Or, an item that should be done to obtain 100% completion is only available in the paid version (some FutureLearn courses are like this), and are inaccessible to learners on the free tier.
On the other extreme, we have the perspective of the course designer. This is the perspective that most research studies on completion seem to adopt. The course designer is working with an abstracted learner population, with abstracted goals. The outcomes of the course might be based on actual research into a learner group, they might be based on the intuition of the course designer, or they might just be whatever the course designer has an interest in preparing (sort of like the Chef's soup of the day, it's there, you can have it, but it doesn't mean that this is what you came into the restaurant for). In a traditional course (the ones you pay and get credentialed for) it makes sense that a learner could simply go along for the (educational) ride because they are paying and (presumably) they've done some research about the course, and it meets their goals. In a free offering, why would a learner conform to the designer's assumptions as to what the learner needs? Especially when a free offering can (and probably does) gather the interest of not just aspiring professionals, but people in the profession (who presumably have some additional or previous knowledge), as well as hobbyists who are free-range learning?
Given those two extremes of the spectrum, I would say that there is a mid-point. The mid-point is where the power dynamic between the learner and the designer is at equilibrium. The educational goals (and what hoops the learner is willing to jump through) 100% coincide with what the designer designed. Both parties are entering the teaching/learning relationship on equal footing. If you lean over a little to one side (learner side), the designer might consider the course incomplete, and if you lean over to the other side (the designer side) the learner might start to feel a bit annoyed because they have to jump through hoops that they feel are not worth their while. Some might begrudgingly do it, others not, it really depends on what the carrot is at the end of that hoop. For me, a free certificate or badge did the trick most times. The threat of being marked as a non-completer (or more recently the threat of losing access to the course altogether ðŸ˜) however does not motivate me to "complete" the course on the designer's terms.
That said, what about my experience? Well... my own behaviors have changed a bit over the years. When xMOOCs first hit the scene I was willing to go through and jump through all the hoops for the official completion mark. I did get a certificate at the end; and even though it didn't really carry much (or any?) weight, it was a nice memento of the learning experience. Badges were custom made (if there were badges), and the certificates were each unique to the MOOC that offered them. Back in the day, Coursera had certificates of completion (you earned the minimum grade to pass), and certificates of completion with distinction (you basically earned an "A"). It was motivating to strive for that, even though it didn't mean much. It was also encouraging when MOOC content was available beyond the course's official end, so you could go back and review, re-experience, or even start a bit late. As we know, things in the MOOC world changed over the years. Certificates became something you had to pay for. Sometimes even the assessment itself was something you had to pay for - you can see it in the MOOC but you can't access it. Peer essay grading on coursera wasn't something that I found particularly useful, but I was willing to jump through the hoops if it meant a free moment at the end of the course (achievement, badge, certificate, whatever). Once things started having definitive start- and end- dates♪ , and content disappeared after that when certificates (which still we're worth much to the broader world) started costing money, the jumping through the same silly hoops (AES, CPR, MCEs, etc.) it just didn't feel worthwhile to go above my own learning goals and jump through someone else's hoops.
So, did I complete all those MOOCs? Yup, but based on my own metrics, needs, and values.
What are your thoughts on MOOC completion? Do you have a different scale? Or perhaps defined categories?
Marginalia:
†There may be some article there somewhere that I've missed, but in my mission to read all of the MOOC literature that I can get access to, I haven't found anything.
‡ What's an extended CV? It's something that contains everything and the kitchen sink. That workshop I did back in 1999 for that defunct software? Yup, that's there...because I did it, and I need a way to remember it. It's not necessarily about the individual workshops, but about the documenting of the learning journey. The regular CV is somewhat cleaner.
â™ Maybe this assumption on my part is wrong, but I can't really picture very many reasons (other than "secret shopper") that someone would pay money to sign-up for a course that doesn't meet their goals.
♣ Window-shoppers I define as people who enroll to have a look around, but either have no specific educational goals they are trying to meet (e.g., lookie-loos), or have goals to meet, but they deem the MOOC to not meet them (e.g., "thanks, but not what I am looking for"). Either way, they don't learn anything from the content or peers in the MOOC, but at the same time, they don't unenroll since there is no incentive to do so (e.g., a refund of the course course).
♪ e.g., module tests deactivating after the week was over and you couldn't take them - AT ALL if you missed that window
But let me rewind for a second. How did we get to the topic of MOOC completion? Well, I've been attempting to make my extended CV more accessible (to me). In the past, I used a WYSIWYG HTML publishing platform to manage my extended CV‡. The idea was that I could easily export it and just push it on the web. In practice, I never did this, and when I changed computers it became a hassle to maintain. So, I moved everything over to google docs for cleanup (and easier updates). In cleaning up my CV sections (I am not done, btw!), I did make a startling self-discovery. In the time-period 2013-2016, I binged on a lot of xMOOCs!😅 The most notable platforms were Coursera, Edx, Udacity, but there were others such as the now-defunct Janux (Oklahoma University) and Open2Study (Australia Open University), as well as overseas platforms like MiriadaX and FutureLearn. In the time period 2011-2012 I didn't have a lot of MOOCs, mostly because during this period a lot were cMOOCs and xMOOCs hadn't really spread like wildfire.
This realization now begs the question: "How many did you complete?" (and you guessed it, Kelvin asked it...). My answer comes in the form of a question "based on whose metrics and measures?". When you sign up for a paid course (e.g., professional development seminar, college course, certification prep course, etc.) I think that there is an unspoken assumption that the goals of the course mirror, to a greater or lesser extent, the goals of the learner♠. Can this assumption be something that transfers over into the world of a free MOOC? I personally don't think so. I've long said that the course completion metric (as measured by completing all assignments with a passing grade) is a poor metric. One very obvious reason to me was that people simply window-shop; and since there is no disincentive to unenroll, people don't take that extra step to leave the course formally, as they would with a paid course where they could receive a refund. I've been saying this since xMOOC completion rates were touted as an issue, but few people listened. Luckily it seems that people are changing their minds about that (or just don't care 😜). I guess George Siemen's advice to Dave Cormier holds true for my own rantings and ravings: publish those thoughts in a peer-reviewed journal or they don't exist 🤪 (paraphrased from a recent podcast interview with Dave).
Assuming that we exclude window-shoppers from our list of completion categories♣, what remains? Well, instead of thinking of distinct categories (which might give us a giant list), let's think of completion in terms of whose perspective we are examining. On the one extreme, we have the learner's perspective. The extreme learner's perspective is characterized by total control by the learner as to what the goals are. In this perspective, the learner can be in a course and complete a certain percentage of what's there and still consider the course as done. Why? The learner might have prior knowledge, and what they are looking for is to supplement what they already know without going through the hoops of any or all assessments in the course. They've probably evaluated the materials in the course, but if they already know something, why spent a lot of time on something already known? Or, an item that should be done to obtain 100% completion is only available in the paid version (some FutureLearn courses are like this), and are inaccessible to learners on the free tier.
On the other extreme, we have the perspective of the course designer. This is the perspective that most research studies on completion seem to adopt. The course designer is working with an abstracted learner population, with abstracted goals. The outcomes of the course might be based on actual research into a learner group, they might be based on the intuition of the course designer, or they might just be whatever the course designer has an interest in preparing (sort of like the Chef's soup of the day, it's there, you can have it, but it doesn't mean that this is what you came into the restaurant for). In a traditional course (the ones you pay and get credentialed for) it makes sense that a learner could simply go along for the (educational) ride because they are paying and (presumably) they've done some research about the course, and it meets their goals. In a free offering, why would a learner conform to the designer's assumptions as to what the learner needs? Especially when a free offering can (and probably does) gather the interest of not just aspiring professionals, but people in the profession (who presumably have some additional or previous knowledge), as well as hobbyists who are free-range learning?
Given those two extremes of the spectrum, I would say that there is a mid-point. The mid-point is where the power dynamic between the learner and the designer is at equilibrium. The educational goals (and what hoops the learner is willing to jump through) 100% coincide with what the designer designed. Both parties are entering the teaching/learning relationship on equal footing. If you lean over a little to one side (learner side), the designer might consider the course incomplete, and if you lean over to the other side (the designer side) the learner might start to feel a bit annoyed because they have to jump through hoops that they feel are not worth their while. Some might begrudgingly do it, others not, it really depends on what the carrot is at the end of that hoop. For me, a free certificate or badge did the trick most times. The threat of being marked as a non-completer (or more recently the threat of losing access to the course altogether ðŸ˜) however does not motivate me to "complete" the course on the designer's terms.
That said, what about my experience? Well... my own behaviors have changed a bit over the years. When xMOOCs first hit the scene I was willing to go through and jump through all the hoops for the official completion mark. I did get a certificate at the end; and even though it didn't really carry much (or any?) weight, it was a nice memento of the learning experience. Badges were custom made (if there were badges), and the certificates were each unique to the MOOC that offered them. Back in the day, Coursera had certificates of completion (you earned the minimum grade to pass), and certificates of completion with distinction (you basically earned an "A"). It was motivating to strive for that, even though it didn't mean much. It was also encouraging when MOOC content was available beyond the course's official end, so you could go back and review, re-experience, or even start a bit late. As we know, things in the MOOC world changed over the years. Certificates became something you had to pay for. Sometimes even the assessment itself was something you had to pay for - you can see it in the MOOC but you can't access it. Peer essay grading on coursera wasn't something that I found particularly useful, but I was willing to jump through the hoops if it meant a free moment at the end of the course (achievement, badge, certificate, whatever). Once things started having definitive start- and end- dates♪ , and content disappeared after that when certificates (which still we're worth much to the broader world) started costing money, the jumping through the same silly hoops (AES, CPR, MCEs, etc.) it just didn't feel worthwhile to go above my own learning goals and jump through someone else's hoops.
So, did I complete all those MOOCs? Yup, but based on my own metrics, needs, and values.
What are your thoughts on MOOC completion? Do you have a different scale? Or perhaps defined categories?
Marginalia:
†There may be some article there somewhere that I've missed, but in my mission to read all of the MOOC literature that I can get access to, I haven't found anything.
‡ What's an extended CV? It's something that contains everything and the kitchen sink. That workshop I did back in 1999 for that defunct software? Yup, that's there...because I did it, and I need a way to remember it. It's not necessarily about the individual workshops, but about the documenting of the learning journey. The regular CV is somewhat cleaner.
â™ Maybe this assumption on my part is wrong, but I can't really picture very many reasons (other than "secret shopper") that someone would pay money to sign-up for a course that doesn't meet their goals.
♣ Window-shoppers I define as people who enroll to have a look around, but either have no specific educational goals they are trying to meet (e.g., lookie-loos), or have goals to meet, but they deem the MOOC to not meet them (e.g., "thanks, but not what I am looking for"). Either way, they don't learn anything from the content or peers in the MOOC, but at the same time, they don't unenroll since there is no incentive to do so (e.g., a refund of the course course).
♪ e.g., module tests deactivating after the week was over and you couldn't take them - AT ALL if you missed that window
Comments
Archive
Sep 2024 (1)
Aug 2024 (5)
Nov 2023 (1)
Aug 2023 (1)
Jul 2023 (1)
May 2023 (1)
Apr 2023 (4)
Mar 2023 (5)
Feb 2023 (2)
Dec 2022 (6)
Nov 2022 (1)
Sep 2022 (1)
Aug 2022 (2)
Jul 2022 (3)
Jun 2022 (1)
May 2022 (1)
Apr 2022 (2)
Feb 2022 (2)
Nov 2021 (2)
Sep 2021 (1)
Aug 2021 (1)
Jul 2021 (2)
Jun 2021 (1)
May 2021 (1)
Oct 2020 (1)
Sep 2020 (1)
Aug 2020 (1)
May 2020 (2)
Apr 2020 (2)
Feb 2020 (1)
Dec 2019 (3)
Oct 2019 (2)
Aug 2019 (1)
Jul 2019 (1)
May 2019 (1)
Apr 2019 (1)
Mar 2019 (1)
Dec 2018 (5)
Nov 2018 (1)
Oct 2018 (2)
Sep 2018 (2)
Jun 2018 (1)
Apr 2018 (1)
Mar 2018 (2)
Feb 2018 (2)
Jan 2018 (1)
Dec 2017 (1)
Nov 2017 (2)
Oct 2017 (1)
Sep 2017 (2)
Aug 2017 (2)
Jul 2017 (2)
Jun 2017 (4)
May 2017 (7)
Apr 2017 (3)
Feb 2017 (4)
Jan 2017 (5)
Dec 2016 (5)
Nov 2016 (9)
Oct 2016 (1)
Sep 2016 (6)
Aug 2016 (4)
Jul 2016 (7)
Jun 2016 (8)
May 2016 (9)
Apr 2016 (10)
Mar 2016 (12)
Feb 2016 (13)
Jan 2016 (7)
Dec 2015 (11)
Nov 2015 (10)
Oct 2015 (7)
Sep 2015 (5)
Aug 2015 (8)
Jul 2015 (9)
Jun 2015 (7)
May 2015 (7)
Apr 2015 (15)
Mar 2015 (2)
Feb 2015 (10)
Jan 2015 (4)
Dec 2014 (7)
Nov 2014 (5)
Oct 2014 (13)
Sep 2014 (10)
Aug 2014 (8)
Jul 2014 (8)
Jun 2014 (5)
May 2014 (5)
Apr 2014 (3)
Mar 2014 (4)
Feb 2014 (8)
Jan 2014 (10)
Dec 2013 (10)
Nov 2013 (4)
Oct 2013 (8)
Sep 2013 (6)
Aug 2013 (10)
Jul 2013 (6)
Jun 2013 (4)
May 2013 (3)
Apr 2013 (2)
Mar 2013 (8)
Feb 2013 (4)
Jan 2013 (10)
Dec 2012 (11)
Nov 2012 (3)
Oct 2012 (8)
Sep 2012 (17)
Aug 2012 (15)
Jul 2012 (16)
Jun 2012 (19)
May 2012 (12)
Apr 2012 (12)
Mar 2012 (12)
Feb 2012 (12)
Jan 2012 (13)
Dec 2011 (14)
Nov 2011 (19)
Oct 2011 (21)
Sep 2011 (31)
Aug 2011 (12)
Jul 2011 (8)
Jun 2011 (7)
May 2011 (3)
Apr 2011 (2)
Mar 2011 (8)
Feb 2011 (5)
Jan 2011 (6)
Dec 2010 (6)
Nov 2010 (3)
Oct 2010 (2)
Sep 2010 (2)
Aug 2010 (4)
Jul 2010 (9)
Jun 2010 (8)
May 2010 (5)
Apr 2010 (4)
Mar 2010 (2)
Feb 2010 (3)
Jan 2010 (7)
Dec 2009 (9)
Nov 2009 (5)
Oct 2009 (9)
Sep 2009 (13)
Aug 2009 (13)
Jul 2009 (13)
Jun 2009 (13)
May 2009 (15)
Apr 2009 (15)
Mar 2009 (14)
Feb 2009 (13)
Jan 2009 (10)
Dec 2008 (12)
Nov 2008 (6)
Oct 2008 (8)
Sep 2008 (2)
Jun 2008 (1)
May 2008 (6)
Apr 2008 (1)
Aug 2024 (5)
Nov 2023 (1)
Aug 2023 (1)
Jul 2023 (1)
May 2023 (1)
Apr 2023 (4)
Mar 2023 (5)
Feb 2023 (2)
Dec 2022 (6)
Nov 2022 (1)
Sep 2022 (1)
Aug 2022 (2)
Jul 2022 (3)
Jun 2022 (1)
May 2022 (1)
Apr 2022 (2)
Feb 2022 (2)
Nov 2021 (2)
Sep 2021 (1)
Aug 2021 (1)
Jul 2021 (2)
Jun 2021 (1)
May 2021 (1)
Oct 2020 (1)
Sep 2020 (1)
Aug 2020 (1)
May 2020 (2)
Apr 2020 (2)
Feb 2020 (1)
Dec 2019 (3)
Oct 2019 (2)
Aug 2019 (1)
Jul 2019 (1)
May 2019 (1)
Apr 2019 (1)
Mar 2019 (1)
Dec 2018 (5)
Nov 2018 (1)
Oct 2018 (2)
Sep 2018 (2)
Jun 2018 (1)
Apr 2018 (1)
Mar 2018 (2)
Feb 2018 (2)
Jan 2018 (1)
Dec 2017 (1)
Nov 2017 (2)
Oct 2017 (1)
Sep 2017 (2)
Aug 2017 (2)
Jul 2017 (2)
Jun 2017 (4)
May 2017 (7)
Apr 2017 (3)
Feb 2017 (4)
Jan 2017 (5)
Dec 2016 (5)
Nov 2016 (9)
Oct 2016 (1)
Sep 2016 (6)
Aug 2016 (4)
Jul 2016 (7)
Jun 2016 (8)
May 2016 (9)
Apr 2016 (10)
Mar 2016 (12)
Feb 2016 (13)
Jan 2016 (7)
Dec 2015 (11)
Nov 2015 (10)
Oct 2015 (7)
Sep 2015 (5)
Aug 2015 (8)
Jul 2015 (9)
Jun 2015 (7)
May 2015 (7)
Apr 2015 (15)
Mar 2015 (2)
Feb 2015 (10)
Jan 2015 (4)
Dec 2014 (7)
Nov 2014 (5)
Oct 2014 (13)
Sep 2014 (10)
Aug 2014 (8)
Jul 2014 (8)
Jun 2014 (5)
May 2014 (5)
Apr 2014 (3)
Mar 2014 (4)
Feb 2014 (8)
Jan 2014 (10)
Dec 2013 (10)
Nov 2013 (4)
Oct 2013 (8)
Sep 2013 (6)
Aug 2013 (10)
Jul 2013 (6)
Jun 2013 (4)
May 2013 (3)
Apr 2013 (2)
Mar 2013 (8)
Feb 2013 (4)
Jan 2013 (10)
Dec 2012 (11)
Nov 2012 (3)
Oct 2012 (8)
Sep 2012 (17)
Aug 2012 (15)
Jul 2012 (16)
Jun 2012 (19)
May 2012 (12)
Apr 2012 (12)
Mar 2012 (12)
Feb 2012 (12)
Jan 2012 (13)
Dec 2011 (14)
Nov 2011 (19)
Oct 2011 (21)
Sep 2011 (31)
Aug 2011 (12)
Jul 2011 (8)
Jun 2011 (7)
May 2011 (3)
Apr 2011 (2)
Mar 2011 (8)
Feb 2011 (5)
Jan 2011 (6)
Dec 2010 (6)
Nov 2010 (3)
Oct 2010 (2)
Sep 2010 (2)
Aug 2010 (4)
Jul 2010 (9)
Jun 2010 (8)
May 2010 (5)
Apr 2010 (4)
Mar 2010 (2)
Feb 2010 (3)
Jan 2010 (7)
Dec 2009 (9)
Nov 2009 (5)
Oct 2009 (9)
Sep 2009 (13)
Aug 2009 (13)
Jul 2009 (13)
Jun 2009 (13)
May 2009 (15)
Apr 2009 (15)
Mar 2009 (14)
Feb 2009 (13)
Jan 2009 (10)
Dec 2008 (12)
Nov 2008 (6)
Oct 2008 (8)
Sep 2008 (2)
Jun 2008 (1)
May 2008 (6)
Apr 2008 (1)