Club Admiralty

v7.2 - moving along, a point increase at a time

Multilitteratus Incognitus

Pondering what to learn next 🤔

Badge MOOC Challenge 5: Authentic Assessment and Evidence for a Badge Ecosystem

The real badge?
Alright!  The penultimate week in #OpenBadgesMOOC, brought to us by Mozilla and Coursesites.  Continuing this week is the exploration of how badges can be incorporated into this #ESLMOOC that I've been thinking about designing, implementing, and hopefully collecting some data for some interesting analysis.  Dissertation-wise it seems like a good topic, but considering the University I was considering applying to has suspended operations due to Austerity Measures in Greece...well, I guess I'll keep looking at other programs while Greece sorts its issues out :-)

So, as with previous Badge Challenges, the Prompt comes before my brainstorming.

Challenge Assignment 5: Authentic Assessment and Evidence for a Badge Ecosystem
Badge system design acknowledges that not all learners are the same, not all learning situations are alike, and not all ways of ascertaining learning accomplishments and skills attainment are the same. Badges offer learners cum job seekers not just more flexibility in how they learn but also in how they prove that they have the skills/competencies represented by a badge.

Assessors have a multi-partite role in the badge ecosystem. They must devise strategies that, as much as possible, push assessment activities into the world where actual performance will occur. They must validate both performance and performer. They must provide a robust set of metadata for each assessment that communicates that validation in total.

In a healthy badge ecosystem, learners demonstrate their competencies in authentic learning environments, capture evidence of their achievements, and have valid assessment to back up the earned badge. Just as badges open the field for innovative learning providers, they simultaneously stimulate rethinking how learning opportunities are provided and assessed. Based on the badge system you described in the prior challenge, describe the learning and assessment frameworks that are needed.
  1. How can learners’ needs best be addressed?
  2. Will traditional learning contexts and methodologies suffice, or can/should they be reworked? Are entirely new methodologies needed? Will new/different staff be required?
  3. How do competencies map to the learning activities and assessments?
  4. What types of evidence and assessment are valued and/or required by employers?
  5. Do the methodologies support the validation frameworks the learning provider needs?
  6. Write one or more “after badges” user stories depicting the value of the badge ecosystem for learner personas. What opportunities do badges provide for your personas? What challenges must be overcome in order to optimize the value of badges? Who do they share badges with? What goals do they have for using their badges?

BRAINSTORMING for this week:
It was interesting to see Kyle Bowen at the weekly live session this week in #OpenBadgesMOOC.  I have met him in person at Campus Technology conferences and I have seen some of what Purdue is doing, which is pretty nifty!  After one of his presentations I created an account on Open Passport to mess around with badge creation.  This was a pretty easy system to create badges - if you know what badges you want to create that is!  Just like ePortfolios, the technology is not (generally) the issue, but rather the underlying objectives and learning that you are attempting to assess.

For this week I think that we're back to familiar waters, with fewer assumptions than previous weeks since some decisions in the last couple of weeks were above my "paygrade."  I do think that since I am thinking about Badges in relation to MOOCs it is a little easier. Applying badge considerations to MOOCs is easier, as far as I am concerned, because, in theory, you aren't tied down by the institutional baggage of how assessment has been done for the last decade or century :-)

When thinking of learners, and learner's needs, it's best to do some preliminary needs analysis, however this is hard even when thinking about "traditional" higher education courses, let alone a MOOC where you potentially have learners from all over the world with many, and competing, interests.  That said, when designing the MOOC it's best to think about the target demographic for your MOOC and then make sure that those people who sign up are the learners for which the MOOC is designed. I do believe that in the weeks leading up to the MOOC, when learners sign up, they can potentially fill out a survey that allows the MOOC facilitators to keep certain things in mind as the MOOC is facilitated, and as additional or supplemental material is discovered, evaluated, and rolled into the MOOC. This should help address the needs of specific learners who are signed up for the MOOC.  The big thing to think about is whether or not facilitators and designers of the MOOC keep addressing the needs to non-participating students in the MOOC, in other words, students who signed up, and might be reading along, but are not visibly participating in some way, shape, or form.

As far as staffing and methodologies are concerned, I would say that staff is definitely needed. A MOOC isn't a one-man show, but rather it's a team effort to design, develop, and implement the MOOC.  In my initial thinking, I am thinking of designing and developing the MOOC on my own, simply because it's probably something that is going to be part of a dissertation.  That said, when implementation-time comes along, I would like to recruit some ESL instructors from my institution, maybe on a volunteer basis, or through some sort of grant support to help pay them a small stipend, to help facilitate the course with me and to provide resources for course material evaluation, learner outreach, and in the end, of the trial group - some sort of evaluation support.

As far as methodologies go, as with most situations where instruction changes from one medium to another, there will be some change in methodology because there isn't a 1:1 correspondence between existing face to face language learning courses and the MOOC format.  I suppose you could shoe-horn the face to face approach into a MOOC, but it just won't be successful. The commonality between the MOOC approach to language teaching and existing face to face teaching is that I plan on using a Communicative Language Teaching (CLT) framework for designing, developing and implementing this #ESLMOOC. While CLT can be the underlying method in both approaches, the tools users, and specific methodological approaches will vary between the two mediums. Thus, I foresee that in a MOOC environment we will be using a variety of  Web 2.0 applications like Blogs, Twitter, YouTube/Vimeo videos that learners can use to process and engage with the material, but also to use in the assessment of the learner's acquired or improved skills as the MOOC progresses. The methodology chosen to design and run this MOOC should  be supportive of the frameworks for validating learning.

As far as competencies and their mappings to learning activities and assessments go, that's a subject to be determined - mostly because the competencies themselves need to be determined.  Right now, the CEFR has broad levels of competence.  In order to make those broad levels more meaningful, but also as a way to map badges for those broad competencies, supporting competencies need devising. At the very least there are three supporting competencies, including hearing, speaking and writing. Those in turn would need some supporting competencies of their own.  All of those would have considerations for activities and assessments.  As far as assessments go, I don't plan on having many assessments. There will be two levels of assessment for #ESLMOOC.  The first level would be basic assessment where the assessment is relatively easy and it doesn't need many assessors, or lots of assessor hours, to complete. This is probably where badges fit in.  The second level of assessments is a more in-depth assessment of learners, perhaps in a smaller cohort, that will serve as part of the data gathering process for the PhD. If #ESLMOOC runs again, and if the second stage isn't as time consuming as I am envisioning it now, then those will be "badge-ified" as well.

When thinking about employers, going back to the original use case of this #ESLMOOC to prepare instructors of higher education in non-North American institution to teach their subjects in English, I would say that there are two broad elements of assessment that they need. There may be more, but those are subject to some sort of needs analysis.  The two broad elements I can see at the moment are:

  • Clarity of speaking (speaking with less of an accent, being more comprehensible)
  • Clarity of written feedback to students

Finally, in terms of thinking of some after badges stories let's revisit some of the personas and their colleagues.

We have Professor Tomas.  He is teaching at the University of Milan. Since he is going to be required to soon be teaching in English for his graduate courses, he decides to take part in #ESLMOOC to hone his skills in the English Language.  Also, since he was willing to be a guinea pig for the researchers, he was assessed more than other learners in the MOOC.  At the end of the MOOC he "graduated" the course with a collections of competency based badges, including one that gave him credentials as someone who could communicate orally at the C2 level (CEFR framework).  With this set of badges he is able to pass the requirements at his own institution since his institution signed onto #ESLMOOC as a sponsor and was able to vet some of the outcomes of the MOOC.  Professor Tomas' only issue is that he also moonlights at other institutions, and since they weren't part of the conceptualization process of #ESLMOOC, and they are no familiar with badges, they are not sure what to make of his accomplishments.  Still, he has published his badges from #ESLMOOC on a variety of spaces, including his LinkedIn account, his account, and on his professional website via an embed code that he got from his backpack.  For him, the overall goal for this particular set of badges is to certify his English skills.  Using digital badges he can take care of the immediate needs of his home institution to demonstrate competency in English, but he can also advertise his skills via badges, as a way to also get noticed for any part-time work in institutions abroad, including the US.

That's all for badges for this week.  Thoughts?  One more week to go in #OpenBadgesMOOC
Stacks Image 20


 Jan 2023 (6)
 Dec 2022 (1)
 Oct 2022 (1)
 Sep 2022 (2)
 Aug 2022 (3)
 Jul 2022 (1)
 Jun 2022 (1)
 May 2022 (2)
 Mar 2022 (2)
 Dec 2021 (2)
 Oct 2021 (1)
 Sep 2021 (1)
 Aug 2021 (2)
 Jul 2021 (1)
 Jun 2021 (1)
 Nov 2020 (1)
 Oct 2020 (1)
 Sep 2020 (1)
 Jun 2020 (2)
 May 2020 (2)
 Mar 2020 (1)
 Jan 2020 (3)
 Nov 2019 (2)
 Sep 2019 (1)
 Aug 2019 (1)
 Jun 2019 (1)
 May 2019 (1)
 Apr 2019 (1)
 Jan 2019 (5)
 Dec 2018 (1)
 Nov 2018 (2)
 Oct 2018 (2)
 Jul 2018 (1)
 May 2018 (1)
 Apr 2018 (2)
 Mar 2018 (2)
 Feb 2018 (1)
 Jan 2018 (1)
 Dec 2017 (2)
 Nov 2017 (1)
 Oct 2017 (2)
 Sep 2017 (2)
 Aug 2017 (2)
 Jul 2017 (4)
 Jun 2017 (7)
 May 2017 (3)
 Mar 2017 (4)
 Feb 2017 (5)
 Jan 2017 (5)
 Dec 2016 (9)
 Nov 2016 (1)
 Oct 2016 (6)
 Sep 2016 (4)
 Aug 2016 (7)
 Jul 2016 (8)
 Jun 2016 (9)
 May 2016 (10)
 Apr 2016 (12)
 Mar 2016 (13)
 Feb 2016 (7)
 Jan 2016 (11)
 Dec 2015 (10)
 Nov 2015 (7)
 Oct 2015 (5)
 Sep 2015 (8)
 Aug 2015 (9)
 Jul 2015 (7)
 Jun 2015 (7)
 May 2015 (15)
 Apr 2015 (2)
 Mar 2015 (10)
 Feb 2015 (4)
 Jan 2015 (7)
 Dec 2014 (5)
 Nov 2014 (13)
 Oct 2014 (10)
 Sep 2014 (8)
 Aug 2014 (8)
 Jul 2014 (5)
 Jun 2014 (5)
 May 2014 (3)
 Apr 2014 (4)
 Mar 2014 (8)
 Feb 2014 (10)
 Jan 2014 (10)
 Dec 2013 (4)
 Nov 2013 (8)
 Oct 2013 (6)
 Sep 2013 (10)
 Aug 2013 (6)
 Jul 2013 (4)
 Jun 2013 (3)
 May 2013 (2)
 Apr 2013 (8)
 Mar 2013 (4)
 Feb 2013 (10)
 Jan 2013 (11)
 Dec 2012 (3)
 Nov 2012 (8)
 Oct 2012 (17)
 Sep 2012 (15)
 Aug 2012 (16)
 Jul 2012 (19)
 Jun 2012 (12)
 May 2012 (12)
 Apr 2012 (12)
 Mar 2012 (12)
 Feb 2012 (13)
 Jan 2012 (14)
 Dec 2011 (19)
 Nov 2011 (21)
 Oct 2011 (31)
 Sep 2011 (12)
 Aug 2011 (8)
 Jul 2011 (7)
 Jun 2011 (3)
 May 2011 (2)
 Apr 2011 (8)
 Mar 2011 (5)
 Feb 2011 (6)
 Jan 2011 (6)
 Dec 2010 (3)
 Nov 2010 (2)
 Oct 2010 (2)
 Sep 2010 (4)
 Aug 2010 (9)
 Jul 2010 (8)
 Jun 2010 (5)
 May 2010 (4)
 Apr 2010 (2)
 Mar 2010 (3)
 Feb 2010 (7)
 Jan 2010 (9)
 Dec 2009 (5)
 Nov 2009 (9)
 Oct 2009 (13)
 Sep 2009 (13)
 Aug 2009 (13)
 Jul 2009 (13)
 Jun 2009 (15)
 May 2009 (15)
 Apr 2009 (14)
 Mar 2009 (13)
 Feb 2009 (10)
 Jan 2009 (12)
 Dec 2008 (6)
 Nov 2008 (8)
 Oct 2008 (2)
 Jul 2008 (1)
 Jun 2008 (6)
 May 2008 (1)
Stacks Image 18