Club Admiralty

v7.0 - moving along, a point increase at a time

Multilitteratus Incognitus

Traversing the path of the doctoral degree

MOOC Completion...according to whom?


The other day I had an interesting (but brief exchange) with Kelvin Bentley on twitter about MOOC completion.  This isn't really a topic that I come back to often, given that completion-rates for MOOCs, as a topic, seems to have kind of died down, but it is fun to come back to it. To my knowledge, no one has come up with some sort of taxonomy of the different degrees of completion of a MOOC†.

But let me rewind for a second.  How did we get to the topic of MOOC completion?  Well, I've been attempting to make my extended CV more accessible (to me).  In the past, I used a WYSIWYG HTML publishing platform to manage my extended CV‡.  The idea was that I could easily export it and just push it on the web.  In practice, I never did this, and when I changed computers it became a hassle to maintain. So, I moved everything over to google docs for cleanup (and easier updates).  In cleaning up my CV sections (I am not done, btw!), I did make a startling self-discovery. In the time-period 2013-2016, I binged on a lot of xMOOCs!😅  The most notable platforms were Coursera, Edx, Udacity, but there were others such as the now-defunct Janux (Oklahoma University) and Open2Study (Australia Open University), as well as overseas platforms like MiriadaX and FutureLearn.  In the time period 2011-2012 I didn't have a lot of MOOCs, mostly because during this period a lot were cMOOCs and xMOOCs hadn't really spread like wildfire.

This realization now begs the question: "How many did you complete?" (and you guessed it, Kelvin asked it...).  My answer comes in the form of a question "based on whose metrics and measures?".  When you sign up for a paid course (e.g., professional development seminar, college course, certification prep course, etc.) I think that there is an unspoken assumption that the goals of the course mirror, to a greater or lesser extent, the goals of the learner♠.  Can this assumption be something that transfers over into the world of a free MOOC?  I personally don't think so.  I've long said that the course completion metric (as measured by completing all assignments with a passing grade) is a poor metric.  One very obvious reason to me was that people simply window-shop; and since there is no disincentive to unenroll, people don't take that extra step to leave the course formally, as they would with a paid course where they could receive a refund. I've been saying this since xMOOC completion rates were touted as an issue, but few people listened. Luckily it seems that people are changing their minds about that (or just don't care 😜). I guess George Siemen's advice to Dave Cormier holds true for my own rantings and ravings: publish those thoughts in a peer-reviewed journal or they don't exist 🤪 (paraphrased from a recent podcast interview with Dave).

Assuming that we exclude window-shoppers from our list of completion categories♣, what remains?  Well, instead of thinking of distinct categories (which might give us a giant list), let's think of completion in terms of whose perspective we are examining.  On the one extreme, we have the learner's perspective.  The extreme learner's perspective is characterized by total control by the learner as to what the goals are. In this perspective, the learner can be in a course and complete a certain percentage of what's there and still consider the course as done. Why?  The learner might have prior knowledge, and what they are looking for is to supplement what they already know without going through the hoops of any or all assessments in the course. They've probably evaluated the materials in the course, but if they already know something, why spent a lot of time on something already known? Or, an item that should be done to obtain 100% completion is only available in the paid version (some FutureLearn courses are like this), and are inaccessible to learners on the free tier.

On the other extreme, we have the perspective of the course designer. This is the perspective that most research studies on completion seem to adopt. The course designer is working with an abstracted learner population, with abstracted goals.  The outcomes of the course might be based on actual research into a learner group, they might be based on the intuition of the course designer, or they might just be whatever the course designer has an interest in preparing (sort of like the Chef's soup of the day, it's there, you can have it, but it doesn't mean that this is what you came into the restaurant for).  In a traditional course (the ones you pay and get credentialed for) it makes sense that a learner could simply go along for the (educational) ride because they are paying and (presumably) they've done some research about the course, and it meets their goals. In a free offering, why would a learner conform to the designer's assumptions as to what the learner needs? Especially when a free offering can (and probably does) gather the interest of not just aspiring professionals, but people in the profession (who presumably have some additional or previous knowledge), as well as hobbyists who are free-range learning?

Given those two extremes of the spectrum, I would say that there is a mid-point.  The mid-point is where the power dynamic between the learner and the designer is at equilibrium.  The educational goals (and what hoops the learner is willing to jump through) 100% coincide with what the designer designed. Both parties are entering the teaching/learning relationship on equal footing.  If you lean over a little to one side (learner side), the designer might consider the course incomplete, and if you lean over to the other side (the designer side) the learner might start to feel a bit annoyed because they have to jump through hoops that they feel are not worth their while. Some might begrudgingly do it, others not, it really depends on what the carrot is at the end of that hoop.  For me, a free certificate or badge did the trick most times. The threat of being marked as a non-completer (or more recently the threat of losing access to the course altogether 😭) however does not motivate me to "complete" the course on the designer's terms.

That said, what about my experience?  Well... my own behaviors have changed a bit over the years.  When xMOOCs first hit the scene I was willing to go through and jump through all the hoops for the official completion mark.  I did get a certificate at the end; and even though it didn't really carry much (or any?) weight, it was a nice memento of the learning experience. Badges were custom made (if there were badges), and the certificates were each unique to the MOOC that offered them.   Back in the day, Coursera had certificates of completion (you earned the minimum grade to pass), and certificates of completion with distinction (you basically earned an "A").  It was motivating to strive for that, even though it didn't mean much. It was also encouraging when MOOC content was available beyond the course's official end, so you could go back and review, re-experience, or even start a bit late.  As we know, things in the MOOC world changed over the years.  Certificates became something you had to pay for.  Sometimes even the assessment itself was something you had to pay for - you can see it in the MOOC but you can't access it.  Peer essay grading on coursera wasn't something that I found particularly useful, but I was willing to jump through the hoops if it meant a free moment at the end of the course (achievement, badge, certificate, whatever). Once things started having definitive start- and end- dates♪ , and content disappeared after that when certificates (which still we're worth much to the broader world) started costing money, the jumping through the same silly hoops (AES, CPR, MCEs, etc.) it just didn't feel worthwhile to go above my own learning goals and jump through someone else's hoops.

So, did I complete all those MOOCs?  Yup, but based on my own metrics, needs, and values.

What are your thoughts on MOOC completion?  Do you have a different scale? Or perhaps defined categories?

† There may be some article there somewhere that I've missed, but in my mission to read all of the MOOC literature that I can get access to, I haven't found anything.

‡ What's an extended CV?  It's something that contains everything and the kitchen sink.  That workshop I did back in 1999 for that defunct software?  Yup, that's there...because I did it, and I need a way to remember it. It's not necessarily about the individual workshops, but about the documenting of the learning journey.  The regular CV is somewhat cleaner.

♠ Maybe this assumption on my part is wrong, but I can't really picture very many reasons (other than "secret shopper") that someone would pay money to sign-up for a course that doesn't meet their goals.

♣ Window-shoppers I define as people who enroll to have a look around, but either have no specific educational goals they are trying to meet (e.g., lookie-loos), or have goals to meet, but they deem the MOOC to not meet them (e.g., "thanks, but not what I am looking for"). Either way, they don't learn anything from the content or peers in the MOOC, but at the same time, they don't unenroll since there is no incentive to do so (e.g., a refund of the course course).

♪ e.g., module tests deactivating after the week was over and you couldn't take them - AT ALL if you missed that window

2019: The year MOOC platforms start to die? Adieu Open2Study

Closure screen on Open2Study

Last night, while browsing through my Reddit subscriptions, I noticed on one of the EdTech Reddits that Open2Study is now closed, and that the site redirects to Open Universities Australia (which was the parent entity).  I was a little in disbelief, but since I had not visited O2S in a while I thought I'd check it out with my own two eyes.  Lo and behold, the site was closed (see screencap above) and it was directing people to OU Australia.

On the one hand this wasn't surprising.  I had completed most of the courses that I was interested in within the first year of operation (2013?).  I did check back periodically to see if they had added anything new, but the course offerings seemed to stagnate. I don't think that the platform added any new courses past that initial batch in 2013.  With this stagnation it does seem normal that the platform would close.  However, it does seem a little weird that no announcement was made. Even as late as December 2018 their Facebook page sent kind reminders for students to finish up work before the next iteration of the course started, and a number of facebook posters indicated that they had signed up for the new round of courses starting in early January (this month). So the question is, what gives?

Homescreen of O2S when it ran

The platform itself seemed interesting.  The courses that I took all seemed a little cookie cutter in their design: They were 4 weeks in length, there were videos (coursera-style) to view, quizzes to take, and some discussion forums for interaction, but not necessarily required as part of a course.  There were forums, that were outside of the scope of the course, where people could interact, and the learning process, at a meta-level, was gamified.  Learners could earn badges for completing courses, for getting perfect scores, and for a few other things.  There were some nice ideas on the platform, but it seemed like a rushed response to the MOOC phenomenon, without much follow-through.

I think this is the second MOOC platform to die.  The first one that I can remember was a homebrew experiment that Stanford had (Stanford Open Courses?) which they killed off after a offering a couple of courses; and opted to run an instance of OpenEdX instead. I don't particularly count this first instance of MOOC-platform death as an actual thing since Stanford went on to use OpenEdx, but to my mind the closure (and quite silent one at that) of Open2Study seems quite significant.  I do wonder how many other MOOC platforms will close in 2019 and 2020.

Sample certificate (I need to find my own!)


MOOCs and the Art Studio

Back for another review of a chapter in the book titled Macro-Level Learning through Massive Open Online Courses (MOOCs): Strategies and Predictions for the Future (an IGI global title).  This time I am reviewing (a little) chapter 4 and jumping off from there.  The chapter title is "PMOOCs and the Art Studio: A Catalyst for Innovation and Change in eLearning Development and Studio Pedagogies", and based on the title this chapter had me intrigued.  I am not very familiar with Studio-Based education, so was looking forward to learning a little more about that as well.  This might be a good chapter for fellow Athabascan (or Athabascian?) from Cohort 6.

Anyway the abstract for this chapter is as follows:

The challenges of MOOCs are currently a significant issue for universities. New contexts of openness, massiveness and collaboration on the Web are challenging traditional forms of university education delivery. These challenges are catalysts for change both generally and in studio pedagogies in particular. This chapter focus on how disruption caused to traditional art studio teaching models occur through intersection with MOOC activity. The provision of studio arts subjects by MOOC providers is also shown to be innovative for MOOC design and delivery. The authors show these challenges by drawing on their participation in two arts based MOOCs, The Art of Photography and Practice Based Research in the Arts. The MOOC pedagogies of openness, massiveness and collaboration, provide opportunities inherent in studio-based arts delivery which contemporary MOOC platforms rarely achieve. The authors draw into question potential frameworks for evaluating choosing and designing contemporary MOOC activity. This chapter falls within the ‘policy issues in MOOCs design' with specific relevance for the topic of ‘technology and change management for the MOOCs environment'.

This article was fairly decent in my view. The article took an autoethnographic approach to research where the two faculty members decided to experiment first-hand with MOOCs (xMOOCs) from a learner's perspective to learn more about them and how specific instantiations of MOOCs worked, what strengths and weaknesses these MOOC approaches had, and how they might be used in studio education.  It's not the first time that I've read of faculty members taking MOOCs as learners in order to see how they work, but I think that this might be one of the few times that fellow faculty come in with an analytic and inquisitive eye, and they put themselves into the learner frame of reference. Other times it seems like faculty have an axe to grind and what they write is total garbage based on a non-analysis, colored by their own perception of what learning should look like.

In any case.  Another benefit here, in this chapter, for me what that the two MOOCs pursued were actually from lesser known providers: Open2Study in Australia and NovoEd (apparently a Stanford University experiment).  I've taken courses in Open2Study, and I have taken one course on NovoEd, so it was interesting to compare their experiences with my own.  The funny thing is that I had taken the Open2Study course they described in this article and I found myself reflecting back to the mechanics of O2S courses, and that course in specific.  I think that is someone has not taken an Open2Study course, or a course on NovoEd, this article give you a little taste of the mechanics of those two platforms, the strengths, and some of the frustrations.

From the description of what studio education is, it sounded to me that the cMOOC (or something like that) might actually make much more sense if you are thinking about a MOOC that has the same (or similar) frame of mind as studio based education (Hmmm.. wonder what Lisa thinks about this).

The one thing that was brought up was the meaning of the world Massive.  O2S courses are basically 4 weeks long, regardless of the course topic. Once the course is over, you can actually take it again in the subsequent month, so I think there are 12 opportunities in a year to take a course.  While this cuts down on the 'massiveness' of one section's registration, I do wonder if it enables learners to take the MOOC when they feel they have the time, as opposed to signing up and either lurking or not participating at all because they don't have time.  If they don't have time, you might ask why do they sign up?  Well, fear of missing out  (FOMO) might be one reason, but there is also the ability to just download the videos (or access them later in the case of Edx), so you sign up when the registration period is open. I wonder if the FOMO is lessened by this, if fewer people sign up for different sections, but the engagement/registrant ratio is higher.

Anyway, interesting chapter.  Worth a read if you are new to MOOCs, and especially these platforms.

Your thoughts?

Errey, H., & McPherson, M. J. (2015). MOOCs and the Art Studio: A Catalyst for Innovation and Change in eLearning Development and Studio Pedagogies. In E. McKay, & J. Lenarcic (Eds.) Macro-Level Learning through Massive Open Online Courses (MOOCs): Strategies and Predictions for the Future (pp. 61-73). Hershey, PA: Information Science Reference. doi:10.4018/978-1-4666-8324-2.ch004

The perils of external rewards

A couple of years ago I was working on hashing out this idea of Academic Check-ins.  Think of it as Foursquare meets informal learning meets campus engagement meets alternative credentialing. A paper came out of that brainstorming with a proposal of what such a system might look like.  While working on hashing out some ideas I wanted to dive deeper into this concept of motivation, both internal and external.  One of the potential issues with extrinsic motivators such as the various "goodies" that you get for checking into places†.

While there wasn't a magic bullet (at least in the background research I did) for a good balance between internal and external motivators there was one huge warning: be very careful of external rewards for doing things. They slowly start to replace internal motivation that was there, and if you remove those external rewards, there is a danger of internal motivation not being there to sustain the learner.  This was somewhere in the back of my mind, but it really hit me yesterday as I was browsing the newly opened Learning to Teach Online course at Coursera. Now, I've been teaching online for the past three (or so) years, so I've gotten some of my own research done on this and I seek out communities of practice in this area to improve my own practice.  I decided to join this course as part of my own PLE.  When I logged in I saw that this course does not offer a Statement of Accomplishment (i.e. I was there, I read the materials and passed the quizzes), but they do offer a Verified Statement if I wish to pay for it.  This is the third Coursera course in two months that I have started that does not award a Statement of Accomplishment because (in my opinion) doing so would cannibalize their Verified Statement monetary opportunities.  While my final grade will be reflected on the course records page of my profile, I won't have a snazzy, but ultimately not very useful, statement to print out.

Now, this, for me anyway, has had an interesting effect on motivation, and how I approach these courses! This morning, during my commute to work, while reflecting on this, the aspect on motivation, extrinsic rewards, and what happens when you remove rewards that were previously given for a certain task, and the ultimate resulting detriment to motivation that this has on the learner. I have to say that now that Coursera has conditioned me to expect a Statement of Accomplishment for completing their courses I'm finding it hard to motivate myself when there is not statement of accomplishment available unless you pay for the Verified Statement.

Don't get me wrong, Udacity, FutureLearn and countless cMOOCs I've been part of in the past don't have statements of accomplishment, but I participated in them (in the forums too!) and really enjoyed the course. However when I entered those MOOCs I had no expectation of a statement of accomplishment.  My motivation was purely internal.  With Coursera (and any other instance where statements are issued) the motivation is internal when it comes to going through the material and browsing through the forums (passive participant in MRT parlance) but to get me to be active in that environment I need the carrot (statement or badge), especially when I am not a complete novice at the course. When that carrot doesn't existed and then it is removed, I am thinking much more judiciously about the amount of time I invest in any given course endeavor (i.e. not burning the midnight oil for graded assignments that won't give me something tangible to hold at the end).

I am simply reflecting on my own motivations here (so I don't expect a certificate), but I do wonder what adverse reactions might there be not only having paid certificates now that we've been conditioned to them. I've already caught a few stray comments on other courses indicating that they wouldn't have signed up if they know that there was no certificate for them.  Interesting.

Your thoughts on the matter?

† examples of this would be stickers in the case of GetGlue/TVTag, stickers in the new swarm check-in app, stamps in the old Gowalla, and some might argue badges would fit into this category too!

Four weeks, Five MOOCs, One Open2Study experience

Last year when I put out the call for the Great Big MOOC Book, one of the submissions came from a colleague in Australia who is going to write a bit about MOOC experiments that they ran on the Australian Open2Study platform, which is sponsored by the Open University of Australia.  I had heard of the platform before, but I never really tried it out since I was testing out other platforms at the time.  Well, since there wasn't much on Coursera to keep me going (too much of the same makes for a dull MOOC), and since rhizo14 is winding down (to some extent) I decided it was time to check out this platform.

I originally signed up for two topics: Teaching Adult Learners, and Becoming and Confident Trainer. The Adult Learner topic was mostly to see what others say about the topic since I've already taken courses on this topic as part of my master's coursework.  The confident trainer was a bit of a repetition, but it was also an interesting look into corporate training, something I am not too much into given that I work in Higher Education.  Once I sampled these two course, I decided to sign up for a few more, Concepts in Game Development, going back to my computer science days; Human Resources, going back to my MBA days for a refresher; and User Experience for the Web, a refresher on my UX days in computer science, but also to see what a self-paced topic looks like.  The other four courses were "paced" courses.

My badges thus far
There are a few things that were head-turners. The Open2Study platform has a certain level of gamification where badges are awarded for certain types of actions. These actions are on aggregate for all courses taken on a platform. So, you are rewarded for participating in the forums, voting on other posts, getting voted up by peers, connecting with others, passings quizzes in the course, getting a perfect score in a class, and completing multiple courses (just to name a few).  By satisfactorily completing five courses, I have earned 29 badges. None of these are Open Badge compliant, so I can't export them to my backpack.  Oh well! The key thing here is that there seems to be a carrot for completing the work in the class, being social, helping others in the course, and actually completing the courses in a satisfactory manner (beyond the certificate that you get at the end).

That said, I did notice that all courses followed the same format.  There are four modules in each course, no more, no less,  Each module has nine or ten mini-modules which include a short video (less than ten minutes) and a multiple choice quiz. Each module has a final, summative, quiz which contains five or ten multiple choice questions.  The final summative quiz can be taken up to four times for a better score, but the thing I noticed was that when I took the quiz, I got a final score. I was never told which questions I got wrong, so when I went back to take it again, I had to second guess myself on some quizzes.  This time around I didn't care about getting the badge for 100% on all quizzes, so I never really tried quizzes again if I got less than 100%.

From a content perspective, I do like some of the video elements, like the glass wall that they write on to illustrate their points (gets us away from horrible horrible powerpoint presentations), and that they limit most videos to 10 minutes. That said, presentations were a bit formulaic.  The formula was:
  1. This is what I am going to tell you
  2. I am telling you what I said I was going to tell you
  3. This is what I told you
  4. This is what we are talking about in the next segment.
It's not THAT bad when you are taking one course, but when you are doing five at a time (and there isn't a ton of of interaction in the forums), it gets kind of dull by week 4.  Speaking of forums, in the self-paced course there was no interaction in the forums since everyone was doing it on their own time. Thus, for me, I blew through the course in 3 days. For the paced courses, due to the fact that there was a barrier (i.e. you can't blow through the content until it's released), there was some discussion in the forums which made for an enjoyable, albeit a little of  discontinuous, experience. While I did get some up-votes for what I posted, and while I did respond to some things that people said, I didn't really feel as "connected" as I do in other MOOC forums, like rhizo.

Finally, I did connect (aka "friend") with some people, but again, feeling disconnected.    On the plus side, I DID end up using some of the knowledge I gained in some of these MOOCs right away, which as refreshingly surprising to me! I'm signed up for five more courses, starting at the end of the week. So more Open2Study thoughts at the end of those.

What you do you think? Have you had any O2S experiences?