Good practice

ByGiuliano Alviani

Good Practice

This section brings together 20 good practices we want to share from the implantation and evaluation of the 3 Pilot MOOCs.

Here is an overview of our good practice collection – just click to get more information:

  • Good Practice Catalogue
    1. Learning Design Principles and Tools for MOOCs
    2. Templates, Questionnaires and Surveys to Identify Your MOOC Target Group
    3. Prototyping in MOOC Design
    4. Time Schedule for MOOC Design
    5. Linguistic Tools for Translating MOOCs
    6. Balancing Out Unequal Experience Levels within MOOC Design Teams
    7. Course Planner Sheet to assign tasks and responsibilities
    8. Quality Dimensions Review
    9. Key Performance Indicator Dashboard
    10. MOOC Evaluation Design
    11. MOOC Platforms
    12. Making MOOC Content Accessible
    13. MOOC Licensing
    14. Flexible Course Tracks
    15. Overview to Deadlines, Certification and Course Tracks
    16. E-tivities by Gilly Salmon
    17. Self-Training Options
    18. MOOC Farewell Party
    19. Co-creation of learners
    20. MOOCs improving the way of teaching/training

Good Practice Catalogue

Learning Design Principles and Tools for MOOCs Develop the structure and content of the MOOC using proven learning design principles and tools. There are existing, tested templates available to be used by anyone. The BizMOOC team used the design templates and methodology of The Open University. A video instruction to the Learning Design process is available here. Further templates and links are available under „Learn more“.
Templates, Questionnaires and Surveys to Identify Your MOOC Target Group Conduct research on the characteristics and needs of the target groups in the early phase of the design process. Using this information to design the MOOC should enable you to engage appropriately with as many learners in target audience(s) as possible. For example, the BizMOOC project started with interviewing both higher education institutions and business representatives to get a broad overview of the gaps and insights to needs of the target groups. It also exposed the level of familiarity of target groups with MOOCs, their presumptions, reservations and expectations. Feel free to apply the templates, surveys and questionnaires used in BizMOOC (open license).
Prototyping in MOOC Design >Involve experts, learners and peers in the evaluation of the design and course prototype well in advance of its launch. In this project, feedback received from 55 experts, 891 learners, 2 external evaluators and an intra-team peer review during design/developments phase allowed the teams to adjust several elements of the MOOC design to fit established standards and existing best practices. All templates are available for use (open license) in the section “Learn more“.
Time Schedule for MOOC Design Take sufficient time for the design of the MOOC. This pays off significantly during the development of the course. In our project, we started one year in advance of the MOOC’s launch by defining course and learning objectives, target groups and developing an initial plan. There was straight-forward time plan (using a GANTT chart), a Project communication plan including a formal Kick-Off Meeting in combination with a workshop on MOOC didactics, regular online meetings, clearly defined milestones and a blue print for the design in place. Check out the “Learn more” section.
Linguistic Tools for Translating MOOCs >Use linguistic tools to overcome language issues. A bilingual course might be a good option to overcome language barriers and engage with a larger number of potential and/or actual users. This was the case for many of our users which enrolled for the creativity MOOC (the second Pilot) provided through a platform with a large offering for Spanish-speakers. The possibility of translating not just written text, but the full content (including the videos) might reduce the dropout rate related to language barriers. There are machine translation options for MOOC, such as the EU-funded project TraMOOC (tramooc.eu), for example. Even YouTube has semi-automatic subtitling options or one could use Google’s subs – but these will need checking and correction manually.
Balancing Out Unequal Experience Levels within MOOC Design Teams Acknowledge and address unequal knowledge and experience with MOOCs within the design team. The BizMOOC project had MOOC teams from various background and sectors, some with hardly any previous knowledge on MOOCs. This was addressed by first producing 14 state-of-the-art papers with contributions from all team members. These papers were then reviewed by MOOC experts. In addition, a dedicated learning design workshop was held next to overall team guidance by work-package leaders experienced in MOOC development and implementation
Course Planner Sheet to assign tasks and responsibilities Assign tasks and responsibilities according to team member’s areas and levels of expertise. In the MOOC development teams, people who were experts on the subject matter produced written content, interviews and other recorded materials, while others worked on the graphics and graphic presentation, layout and technical feasibility of the Course.
Quality Dimensions Review Establish the goals, learning outcomes, characteristics and quality criteria of the MOOC before developing the MOOC. Consequently, MOOC team members should be aware of the main priorities and concerns when developing and implementing a MOOC. In the BizMOOC project, this was not only facilitated by the learning design workshop, but also by discussing the quality dimensions and criteria that apply to the project overall and the specific target group of the MOOCs. Quality dimensions and related criteria discussed include:

  • is it a MOOC?;
  • content & activities;
  • applied IPR and licences;
  • language;
  • accessibility;
  • technical platform and Support;
  • implementation BizMOOC (intervention goal).

The quality dimensions with a more detailed description and corresponding questionnaire can be downloaded with an open license on mooc-book.eu (Learn more).

Key Performance Indicator Dashboard Select and measure key performance indicators that are most appropriate for the MOOC. During the design phase, the BizMOOC project discussed the key criteria that guarantee the quality of each MOOC. As part of this process many pre-existing quality checklists, questionnaires for experts and MOOC participants and other tools were re-used. The majority of these are available openly. In the BizMOOC project, these checklists and questionnaires were jointly designed and it was discussed which were applicable to the project and to each team. As such, some questionnaires have specific questions that reflect the distinct goal(s) and target group(s) of the MOOC, as well as some general items relevant for all MOOCs. All the templates can be downloaded with an open license on http://mooc-book.eu (Learn more). One overall summary online dashboard was used constantly checking the achievement of the pre-defined key performance indicators (e.g. active users > what is the definition and what is the target). A simple, free-to-use cloud service tool is sufficient to start a dashboard.
MOOC Evaluation Design Plan the overall evaluation and quality process, including who will review what elements using which template with applicable criteria, beforehand. In the BizMOOC project, we agreed to have a review of the MOOC design by two external experts and by the two other MOOC teams. In addition, feedback was collected from 55 experts pre-course (open questions in survey) and post-course (focus group sessions) and by MOOC participants by pre- and post-course questionnaire. Here is an overview of the whole MOOC evaluation framework applied to the BizMOOC Project.
MOOC Platforms Review a range of MOOC platforms and carefully select one to ensure that the chosen platform is accessible for the identified target groups (e.g. can be utilised by businesses or Higher Education Institutions), has the functionality you require for the course and is easy for learners to sign-up to and navigate. Some platforms enable learners to track their progress which can have a strong motivational impact. For example, one BizMOOC project MOOC was hosted on a platform enabling full moderation and teamwork activities (mooc.house), as exchange and cooperation were key components of this MOOC.
Making MOOC Content Accessible Ensure the accessibility of content and assets (such as videos) on different devices when developing your MOOC. For example, subtitling videos or providing text that is in a screen reader compatible format or making content accessible on mobile devices.
MOOC Licensing Consider the medium- to long-term life of the MOOC. For example, by openly licensing course content and/or by using open education resources (OER) you enable re-use and further development of the course by both yourself or others. The second iteration of the second Pilot MOOC on Creativity was self-paced and MOOC Pilot 3 on Intrapreneurship was designed in a way to be easily transformed into a self-paced MOOC without a huge additional effort; only the quizzes, certification and forum were deactivated after the course was complete and an explanation provided. By making the course content openly available post-facilitation more than 200 self-paced learners have made use of the MOOC (as at October 2018). Ensuring the longevity of course content was an important factor for all teams.
Flexible Course Tracks Whenever necessary, introduce flexible course options (fast track and full track): This functioned well and was appreciated by the learners of our MOOC Pilot 3 on Intrapreneurship . Learners were motivated after completing the fast track (a selective learning pathway of only essential aspects of the course) to go for the full course experience (the whole course). How these different pathways through course material are referred to is also important: the label “fast track” was appreciated; the label “full track” could be replaced by the term “bonus track” to encourage more learners to consider engaging with more material. To find out more on how this could be done, check out our Intrapreneurship MOOC: http://mooc.house/bizmooc2018/
Overview to Deadlines, Certification and Course Tracks Provide a clear graphic overviews of deadlines (visualisation of course time lines, example: Overview of current position in Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license), certification options (example, Overview of current position in Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license), instructions and a dashboard to monitor one’s own progress (example, Overview of course tracks and certification options in Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license).
E-tivities by Gilly Salmon Introduce e-tivities as good practice examples for ice-breaking activities: Following the e-tivity model of Gilly Salmon, e-tivities need to have a clear encouraging title, a purpose (objective), a brief and straight-forward summary of the task, a spark (direct link with the topic of the week, interesting title etc.), an individual contribution (share), a dialogue encouragement (interact) and e-moderator interventions throughout the discussion (e.g. stimulate certain discussion by asking counter- or follow-up questions, providing short wrap-ups of discussion points etc.). Our Pilot MOOC 3 on Intrapreneurship introduced 7 e-tivities. Examples are provided below:

  1. E-tivity 1: Self-test Intrapreneur or Entrepreneur? This E-tivity took place in the first week of the MOOC and aimed to encourage learners to share information about themelves and find out more about other participants. The task asked learners to consider whether they are more of an entrepreneur or intrapreneur via the help of an external self-test. Subsequently, learners had to share the results in the discussion forum and also describe themselves to other learners. In addition, learners were asked to tell others why they were participating in the MOOC and reflect on the results of their self-assessment.Figure: E-tivity 1 of the Pilot MOOC3 on Intrapreneurship: “Entrepreneur or Intrapreneur?“. Source: openHPI, Thomas Staubitz, CC license.

    As the test was based on only six general short yes/no questions, it was less about learning more about oneself, but more about making the learners curious, encouraging discussion of the test and its result and thereby stimulating a discussion among learners.

  2. E-tivity 2: Intrapreneurship example: In this e-tivity, the learners were asked to find an example of intrapreneurship and share a link to a text, photo, audio or video file. They were asked to explain why this example is special to them – it might be an extremely successful business, or one that almost failed, or one that learners were not intially aware was an intrapreneurship outcome, or was something extraordinary. Learners were asked to look at the examples provided by others and identify examples which surprised them and to share their thoughts via a forum comment. This resulted in possibly the most comprehensive collection of intrapreneurship examples currently available with 255 entries and 507 views by participants.Figure: e-tivity 2 of the Pilot MOOC3 on Intrapreneurship: “Inspire Yourself From Intrapreneurship Examples”. Source: openHPI, Thomas Staubitz, CC license.
Self-Training Options Introduce self-training Options by providing learners with an opportunity to prepare themselves, deepen their learning, be prepared for assessment etc., by offering multiple options (see example: Self-training option for the weekly and final exam of the Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license.) to self-access the learning outcomes of each activity.

Figure 6: Self-training option for the weekly and final exam of the Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license.

MOOC Farewell Party Host an online closing event which recaps the most important take-aways and enables learners to reflect informally on their experiences We experimented with a “farewell party” at the end of one of our MOOCs. See  Example: Invitation to the Farewell party of Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license.

During the ‘Farewell Party” a separate course collaboration room was opened with the option to join a Google Hang-Out or to watch a livestream of the event on YouTube. Learners either participated or contributed by providing comments, feedback or questions in the forum. The developers kept the agenda informal – some content (stories from behind the scenes, video outtakes etc.) was prepared, but there was also room for the learners to contribute.
Screenshot of the Collab Space for the Farewell Party of Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license.
Screenshot of the YouTube-stream of the Farewell Party of Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license

Co-creation of learners Enable learners to become co-creators of the MOOC. Seven course e-tivities and e-modertation (applying the concepts elucidated by Gilly Salmon) strived to turn passive participants into active learners. Consequently, learners started their own discussion threads in the forum (e.g. “How can I encourage my colleagues to be more “innovative”?” with 8 replies and 46 views, or “Intrapreneurship needs change of company culture?” with 19 replies and 62 learners visiting). One highlight was a learner reacting to a forum posts, supported by several learners, with the idea that it would be nice to have transcripts for the videos. This learner then started to provide transcriptions she had created and uploaded them below the videos for the use by all the learners (see example: Learner becomes co-creator and produces transcripts to videos in Pilot MOOC3 on Intrapreneurship. Source: openHPI, Thomas Staubitz, CC license).
Due to the large number of participants and facilitator time constraints the course developers sometimes did not have the time to read new questions in the course forum in a timely fashion. As a result, the community of learners on the course often helped each other out. On some occasions the course developers deliberately waited to respond to queries so that they could enable others to reply first, in other instances they answered almost immediately or raised questions to stimulate further interaction. In total, the community created approximately 2200 entries to almost 300 topics in the Discussion Boards and Collaboration Space of the Intrapreneurship MOOC.
MOOCs improving the way of teaching/training The MOOC and the Cloud has influenced the way of teaching not only in the MOOC and online settings, but also within face-to-face classes and trainings.

This was manifested in different ways:

First, by being filmed and exposed to a global, international, non-traditional, mixed and large community in the MOOCs, the instructors received first-hand feedback to their inputs and didactical approaches – immediately and through multiple channels: course analytics (e.g. video items viewed, observation of learner behaviours, activity levels, drop-out rate development etc.), the forum and the course survey. For many of our instructors, this was the first time (no difference whether university or business expert/instructor) being exposed like that and receiving this kind of feedback. The instructors highlighted this valuable learning experience (although leaving their comfort zone), and everyone agreed that there lies huge potential in significantly improving the own teaching (also in offline settings) by these new feedback channels.

Second, the aforementioned didactical approach of e-tivities by Salmon (2011, 2013 > see also good practices no. 16 & 19) unleashed the “power of the cloud” and resulted in valuable collections of examples directly deriving from actual workplace challenges and topics. The instructors could therefore collect an overwhelming toolbox full of examples, good practices and experiences deriving directly from 21st century industry challenges. This would probably be hard (to impossible) to be collected by oneself. This collection now can enrich other classes and trainings with a huge selection of up-to-date practical real-world examples.

Therefore, our last good practice is to outline the vast potential which a smartly designed MOOC with a critical mass of learners holds to improve general teaching and training approaches in professional lifelong learning AND university & company-training settings.