I’m asked often the difference between studying towards a PhD and an EdD. Whilst many understand a PhD, fewer know of EdD’s and the process. Now I’m not saying this because I’m in my final year of my EdD, but I’m a total EdD convert.
The EdD is more structured and progress reports are similar to drafting sections of chapters, with more PR submissions than a PhD. The EdD is also a professional doctorate with the purpose of the research to contribute to professional practice in education. I’m thankful to say that my doctorate feeds my day job on a daily basis and vice versa.
So what’s the catch? For me there’s only one, the forging of where my day job ends and my doctorate begins. Not in the aspect of time management of undertaking an EdD in three and a half years in parallel to working full time, but in the content writing.
Don’t get me wrong, I sometimes wonder if I should be on a MOOC Eggheads special, however my biggest downfall is actually the mental separation between when I write for business and when I write for academia. Strangely when I write for journals etc. I’m fine (new book chapter with Professor Graham Pike out shortly) but when I write for my doctorate as it’s so close to my day job I sometimes slip between the cracks of my two roles. So what to do?
Well, my supervisor struck upon a genius idea. As my role is partly in learning design and work with academic authors to create pen portraits for MOOCs, then I should have a pen portrait for my thesis.
So everyone meet Jo. Jo this is everyone. Dr Jo is an academic who doesn’t know much about MOOCs but would like to create a MOOC within her role and needs to understand how to create an engaging learning design. I find myself thinking of Jo often, and use her in meetings with my supervisor. I ask myself when writing ‘what would Jo want?’, ‘does Jo need to know this?’ and it’s helping me to frame my writing and forget the divide between doctorate and day job.
My next deadline is the 10 April where I’m due to submit 45,000 word first draft of my thesis. So I best get back to it otherwise Jo and I will be burning the midnight oil. The 31 October final submission isn’t that far away either.
For those that know me, I am the epitome of weird fears. I suffer from acute Potamophobia and Gephyrophobia. To save you reaching for Google and a handy bit of copy and paste that is the fear of (canal) bridges and canals. These fears have plagued me since I was 12 (along with (mostly cured) fear of all things associated with the industrial revolution) and can reduce me to panic attacks, breathlessness and heart palpitations. I. Kid. You. Not.
Well, a while back Twitter aficionado Lawie Phipps took a tour of the canal system on his boat. Scrolling through Twitter became like emersion therapy (no pun intended) and after flinging my phone across the room a few times (thank crunchie it’s an android) I was soon able to view the photos through my veiled fingers. I can’t say I will be hiring a narrow boat anytime soon (though my rescue adventure hound would find it marvellous) it made me think of what else I fear and how I can overcome it.
It is at this point dear readers that I divulge my confession. I am in many ways a shamdemic. I talk the talk, I read academic papers, I walk amongst you, yet before I began working at the OU 12 years ago this June I had never stepped one fairy footstep onto another campus as a student.
Yes, it is true. So how you may ask am I blogging to you as a doctorate student if I never studied at a university prior to employed tax paying life? Well I joined the OU so I could study with the OU as I was never in the position previously to finance my own studies in the traditional manner. So with my 12 years at the OU I will have spent almost 9 of them studying whilst working full time with a U/G, 3 P/Gs and hopefully soon, a doctorate under my belt and after my name.
I’m immensely proud of my achievements whilst climbing the career ladder at the OU as I started out on the lowest grade at the lowest pay when I joined, but I always feel slightly outside of the academic community (lovely though you guys are). To the point that this blog post as been sitting in my drafts now for quite some time, and holding up others, until I could remedy myself to complete it.
Then two pieces of advice came within days of each other that reframed my mindset. Firstly at my final doctorate residential school last weekend we were told that our doctorates make us an expert on our research. That may sound obvious to some, but it’s true. My doctorate is on why learners become engaged with MOOCs and the impact on design. Now there are plenty of people out there that can talk about MOOCs (and we definitely have they really are the marmite of academia) but I have a working knowledge of 58 MOOCs with in excess of 120 presentations over 4 years from conception, through to design, production, presentation, and review. All by the OU. For my doctorate I’m reviewing 19 of them. I can’t say that I know of that many people that have been involved with MOOCs to that volume, to that detail, for that number of presentations, and for that length of time. So does this make me an expert? I guess it does.
The second piece of advice came today, in the form of an amazing female colleague on campus that I’m very sad to see leave, Rachel Cragg. She told me that I should be unapologetic in who I am, because it makes me, me. And that being me is bloody marvellous.
Combined together these are most useful pieces of advice. The biggest battle with my doctorate has been the confidence to know what I am doing is enough. I’ve spent the last two years asking my supervisors is what I’m doing enough fearing I will fall short (it seems my data collated is more than enough). A confidence issue that sits juxta to my usually confident, sassy, outspoken self. I have absolutely no issue in walking into a room filled with hundreds of people and given a presentation on my day job complete with dry wit, humorous anecdotes, and pictures of my dog. I have even delivered presentations with concussion, on crutches, and without power delivering only from memory (thankfully not all at the same time). This is what I love, live, and breathe for.
So what’s the problem I hear you ponder? I freeze up when it comes to presentations or publications on my research. I live in fear of my unique academic record being exposed. Though hundreds of thousands of people study with the OU every year, I can’t say I know of many/or any others like me that have studied for their entire academic career with the OU from undergraduate through to doctorate. Usually at some point they have studied elsewhere even if it’s for a year. I fear that an academic that may know less than me about MOOCs will out me for not having a traditional academic record and therefore void my research.
So sod it, here is me, outing myself. I haven’t studied at any other university than the OU, I haven’t worked at any other university than the OU and if you snap me in half it says OU all the way through me. So what do I have? I have 12 years experience working in the largest university in a variety of roles and grades delivering in OER, social learning, accredited learning and MOOCs. I have 9 years experience of being an OU student from U/G through to doctorate and I have working knowledge both academically and professionally of one of the largest collections of MOOCs by a singular university.
Not too shabby…
From here on out, academic confidence is a go. I repeat, academic confidence is a go.
Following on from Enough Education to Perform, in this blog post I’m reviewing collectively the breakdown of learning design activities of 4 MOOCs that were created in 2014/15 and still in presentation on FutureLearn. The four MOOCs were selected for this blog series due to the variances identified in the learning designs; Forensic Psychology (released on a week-by-week basis due to narrative in the content), Start Writing Fiction (39% productive activities and use of peer review), Childhood in the Digital Age (with the highest percentage of assimilative activities), and The Lottery of Birth (with lower assimilative activities to accommodate for more communication, productive and finding and handling information activities).
Each of the steps in FutureLearn are assigned a step type; Article, Video, Discussion, Exercise, Quiz and Test. The definitions of these step types were then mapped to the Learning Design Activities.
All forms of assessment (summative, formative, and self-assessment)
Write, Present, Report, Demonstrate, Critique
Rienties, Toetenel, and Bryan (2015)
By displaying this activity and taxonomy in a table it is possible to see how on a week-by-week basis the Activity Planners are broken down.
Forensic Psychology had the most number of steps totalling 150 across the 8 weeks of the course and unique in its presentation as it was designed to be released on a week-by-week basis with narrative content that would allow the learners to go behind the scenes of a police investigation to solve a crime. The weekly release was to ensure that learners could not view content in future weeks which would affect their cognition tests in the current week.
Both Start Writing Fiction and The Lottery of Birth have steps for Peer Review, however Start Writing Fiction has a higher number of these steps as an 8 week course, whilst Lottery of Birth is 4 weeks in length. Though both courses have a strong emphasis on assimilative activities, compared to the other courses they both had a prominent communication activities through discussion and peer review, thus both courses being categorised as social constructivist using the cluster analysis categories defined by Rienties, Toetenel, and Bryan (2015).
In comparison for Forensic Psychology and Childhood in the Digital Age, aside from assimilative activities the courses have a much lower focus on the remaining learning design activities, thus being categorised as constructivist using the cluster analysis categories defined by Rienties, Toetenel, and Bryan (2015).
The Lottery of Birth was the only course in which the hours set in the learning design were consistent with the hours stipulated on the course registration page. The other courses were lower in their learning design to allow for repeating cognitive activities in Forensic Psychology or for learners to refine their work for Start Writing Fiction.
Over the next few weeks my blog posts will be reviewing the performance data and learning design engagement survey for each of these courses in detail in relation to their learning designs to bring a conclusion as to whether these MOOCs were successfully designed for the demographic that registered for them.
Rienties, B., Toetenel, L., and Bryan, A. (2015). “Scaling up” learning design: impact of learning design activities on LMS behaviour and performance. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge – LAK ’15, ACM, pp.315-319
When Jane Austen wrote this immortal line she had a different marriage of minds in her sights. However last weekend I had a diversion from my doctorate studies. Well, more like a meander.
Recently my colleague the eminent Professor Graham Pike and myself were approached about writing a chapter for a forthcoming ed tech book Creativity and Critique in Online Learning. The book is set to explore and examine digital innovation developments from the perspective of critical practitioners. Instead of simply focusing on describing particular pedagogical tools, each chapter will also draw on the experiences and action research of those actively engaged in teaching in an online environment.
Given our work together on the MOOC Forensic Psychology: Witness Investigation currently presenting on FutureLearn, it may come as no surprise that our chapter is entitled: The Challenges of Massive Open Online Courses (MOOCs).
The chapter unfolds with a literature review to set the scene as to the challenges of MOOCs (different pedagogical approaches, large heterogeneous learner populations, limited data, large drop out rates, to name but a few) before reflecting on what this means for MOOCs in a critical context. The latter half of the chapter is set to explore data from Forensic Psychology: Witness Investigation in comparison to other MOOCs that I’ve worked on for presentation through FutureLearn.
Last weekend I authored the literature review and the critical reality of MOOCs. Given my work and doctorate studies in MOOCs it seemed an obvious task divide, which is now with Graham for critical reading. This week we progress to how we write up our data findings. We have specifically been reviewing engagement and drop off with MOOCs as this is one of the biggest challenges that MOOC authors and platform providers face.
Forensic Psychology sits separately to the other MOOCs that I’ve worked on for presentation on FutureLearn in that it’s learning design centres around a narrative that is released like a Dickensian serial on a week-by-week basis. Learners are unable to jump ahead into future weeks to find out which police inspector was right and who committed the crime. There are no plot spoilers in this MOOC just a clever narrative of clues, plot twists, and red herrings that learners have to navigate. Due to its learning design this course fares better than most in sustaining engagement.
So where is the challenge?
Well, pedagogically it’s not always possible to replicate this method of narrative and weekly release in every MOOC and though at a slower rate, learners are still disengaging in a similar pattern. In understanding when learners are more likely to disengage may help us to understand when would be best to apply interventions to aid to maintain learners throughout the course.
There will of course be additional challenges as to whether those inventions are suited to such a large population and whether the data demonstrates an increase in engagement. Such interventions may be platform, course or demographic specific, so may be difficult to scale when widening the scope.
The challenges may indeed only create more challenges.
I have a feeling that will be the subject of another chapter or paper in the future.
In my recent blog posts I’ve been unpacking the methods I have been using to conduct my research – but to what purpose?
My hypothesis is that the engagement in and study of MOOCs by learners does not replicate that of students in formal courses. Nor in many ways should it. Controversial I know.
I’ve spent the last 4-5 years looking at MOOCs, and became more involved since taking my first MOOCs #Change12 and #opened12.
Three years ago I became involved in the production of MOOCs by The Open University for FutureLearn and OpenLearn. I’ve now partaken in over 50 MOOC learning design workshops and in all that time the same discussions take place. How do we strip down big courses into smaller ones? What is the learner journey? How does the narrative flow from week to week? What do you mean learners won’t want to learn the whole course? Why is drop out so high when it is free?
I think you can start to see why I decided to tackle these questions through the medium of my doctorate.
In my time I’ve reviewed over 100 dashboards of our course presentations. And from that, wider reading, interviews with learners, conducting my own research and working day to day in MOOCs over the years I’ve come to my own hypothesis through the guise of an analogy. And here it is…
Instead of learners studying MOOCs in an expected linear journey, they are actually reviewing MOOCs as recipe books. Something to dip in and dip out of, take what they need and then move on.
I have a plethora of recipe books – I’m attracted by the title, the cover, the celebrity chef, the promise of learning new recipes and the knowledge of subsequently recreating them in the future without the recipe will bring. I lovingly covet them on a multitude of online sites, read reviews, look at the enticing pictures, and then add them to my cart. I’ve been known to go a little crazy and add more than one to my cart at any one time, purchase them based on the time of year or in conjunction with a promotional activity or media event. I am a publishers delight.
But am I a Michelin starred MasterChef? Sadly no. So why is this? I have all the tools in front of me, I have the celebrity chef to guide me through the pages, mouthwatering photos and even YouTube channels and Instagram videos at my fingertips. Why am I not creating new delicious recipes every day to fatten my neighbours, colleagues and my faithful(ish) adventure rescue hound with? Because it’s a recipe book.
Now I love cooking and baking, anyone who follows me on Instagram knows my love for food and to veganise pretty much everything, My hound loves this even more than me. However a recipe book is designed to be dipped in and out of. More than one can be used in conjunction with another. Recipes can be merged and remixed to create something new. And there are only so many dinner parties that I can throw before I’m exhausted, need to take a break, put the book back on the shelf and then re-energise for the next one. But what do I do in the meantime? I add another recipe book to my cart. Why? In the hope I’ll cook from it in the future.
So what does this mean for MOOCs?
From the interviews and survey I have conducted many MOOC learners like to dip in, learn about something they already have a vague understanding of, learn enough that they are satisfied and then put the MOOC back on the shelf. When questioned about disengagement, quitting, dropping out, etc. they don’t see themselves conducting this type of activity – simply they learnt what they wanted to and that was that. The MOOC is still there for them to take back off the shelf at a later date, maybe learn another step, combine it will knowledge they have gathered from another MOOC, and recreate the knowledge consumed at a later date either in their own reflections or with friends and family.
So has the MOOC failed? No. Is the MOOC not designed for the learner’s needs? Yes.
Going back to the recipe books, I don’t know a soul that takes a recipe book, follows and cooks every recipe in the book in the order laid out. But why not? The books are designed in sequential order – appetisers, starters, mains, puddings, after dinner munchies – so why not cook all the recipes in that way? Because we cook what takes our fancy from the photos, ingredients needed, time to cook, outcome required, and purpose for cooking. Why can’t MOOCs be the same?
My proposal is this – for some instances MOOCs are needed to be sequential linear designed courses, for example for external assessment or accreditation. However the large majority do not. So scrap the learning design process that we have taken formal courses. These are learners not students. Learners want to dip in and out, they want to learn only the bits that interest them and shelve the rest.
It’s time to apply adaptive learning of our MOOC learners.
How do we do this? Make it modular.
Take the recipe book analogy once more. When creating a dinner party, you have in mind what you want from it. You review the recipe books and select the dishes that interest you the most and disregard the rest. So why not apply this to MOOCs?
If learners had the option to fill a template of 3-4 sections, each of which are a week in length and they had 20 one-week options to choose from. Would they then select the one-week options that interested them the most?
By doing away with the concept of a linear course and creating something I like to call ‘Open Online Learning Objects’ (OOLO) – discreet individual pieces of content that could be studied in any order, could be more engaging for the learner. In return the academic community would have a greater understanding of the types of combinations a learner selects, their ordering preference, and which OOLO’s disengage them the most.
Further study could then be conducted to see if this reflects in the formal curriculum. I’m continuing reviewing this concept as I begin to write up my thesis.
So this is my food for thought that I wanted to share with you all.
Following on from my last blog post Design and Data in which I introduced the latest research I have been conducting for my doctorate, in this post I’ll be unpacking how I will be reviewing the learning performance data from our course dashboards on FutureLearn.
For each presentation of an Open Online Course (OOC) on FutureLearn there is an associated performance dashboard. Within the dashboard are the overview statistics giving a snapshot review of the performance of each presentation and a number of datasets for export (comments, enrolments, question response, and step activity). In analysing the data from a combination of these datasets and the overview dashboard in conjunction with the learning design it is possible to determine the success of the learning design in correlation to the learning performance data. For example, to review from the learning performance data whether the learning activities designed are engaging learners to reaching their learning outcomes.
As the OOCs selected have presented on more than one occasion since their production in 2014/15 it is possible to determine whether the engagement or disengagement of the learners in particular steps repeats in subsequent presentations and therefore associated with the learning design, or is isolated to one presentation whereby mitigating factors may be the cause (e.g. anniversary of an associated event).
The dashboards available from FutureLearn represent the learners’ progress through the course and are categorised into the following Course Measures:
Joiners – Number of learners who have enrolled on the course
Leavers – Number of learners who have elected to unenrol from the course
Learners – Number of learners who view at least one step in any week at any time within the course
Active Learners – Number of learners with user interaction with at least one step in the course (marking steps as complete, submitting content for review, or attempting a quiz or test)
Returning Learners – Learners who have interacted with at least one step in two or more weeks of the course
Social Learners – Learners who have commented on at least one step
Fully Participating Learners – Learners who have interacted with at least 50% of the steps and attempted the tests
The dashboards displayed in these progress reports are taken directly from FutureLearn, it is important to note that FutureLearn calculate their percentages for Active Learners, Returning Learners, Social Learners and Fully Participating Learners from the number of learners who have returned to the course (Learners) and not from the number who have enrolled (Joiners).
When edX reviewed their Harvardx and MITx courses from their first presentation year Ho et al (2015) stated that “Participation initially declines in repeated courses, then stabilises” based on the review of data from 11 of their courses. Ho et al found that participation (defined in the paper as enrolment) declined on average by 43% from first to second presentation. Five courses had a third presentation, whereby enrolment remained unchanged from second to third presentation, thus stabilising. This method of declining engagement will be used when reviewing OU OOCs.
It is important to note that engagement is not tangible and measured primarily through enrolment or course completion (frequently seen as successful engagement), learners may engage with elements of the course, but not reach the requirement of a Fully Participating Learner. Therefore it is important to acknowledge all the course measures identified in the dashboard as part of the review process. To my knowledge (readers please correct me if I am wrong), that this will be the first literature to map across multiple course measurements to ascertain engagement and more importantly disengagement.
In my blog posts next week I’ll be discussing the engagement survey that I created for the purpose of this research and also the hypothesis that I have been forming during this process on how I think learners engage with MOOCs and therefore how and why they disengage.
Until then…I’m doctor in waiting
Ho, A. D., Reich, J., Nesterko, S. O., Seaton, D. T., Mullaney, T., Waldo, J., & Chuang, I. (2014). HarvardX and MITx: The first year of open online courses, fall 2012-summer 2013. Ho, AD, Reich, J., Nesterko, S., Seaton, DT, Mullaney, T., Waldo, J., & Chuang, I.(2014). HarvardX and MITx: The first year of open online courses (HarvardX and MITx Working Paper No. 1).
After spending the last few weeks headlong in my studies I thought it high time come up for air and update my blog about the latest research I’ve been conducting for my doctorate…
My research investigates why learners engage with open online courses (OOCs) and what elements of learning design engages learners associated with course completion. The purpose of my research is to identify the elements of these courses that prompt learners to engage or disengage with learning design.
*NB: The M for massive has been purposefully removed as the number required to make a course ‘massive’ is open for debate and once a course has scaled beyond a few hundred learners the impact on learning design (rather than platform design) is in debate.
During the academic year of 2014-15, 13 courses were produced by The Open University for FutureLearn and for further syndication to OpenLearn. The process for learning design for formal courses is derived from the Open University Learning Design Initiative (OULDI) and has been adopted on a condensed timescale. Whereby a module by the OU takes 2 years to produce from a 3 day Learning Design Workshop, an OOC takes 24 weeks to produce from a 3 hour Learning Design Workshop. The learning design taxonomy for the identification of learning activities for formal courses remained the same for OOCs.
During the Learning Design Workshop, the course ideas from the Lead Educator/authors are discussed and recorded by a Learning Design Manager into an Activity Planner using the classification identified by OULDI. This provides future reference when writing the course content resulting in the creation of visualisations of the activities and resources contained in each of the courses.
Research is growing in the field of formal learning design in the form of modules and qualifications (Rienties, Toetenel, and Bryan 2015), however very little research has been conducted to date on the learning design of OOCs in the same way.
Through identifying this gap in the literature on the topic I’ve begun reviewing the learning designs of all 13 courses starting with 4 courses that were selected due to the variances identified in the learning designs; Forensic Psychology (released on a week-by-week basis due to narrative in the content), Start Writing Fiction (39% productive activities and use of peer review), Childhood in the Digital Age (with the highest percentage of assimilative activities), and The Lottery of Birth (with lower assimilative activities to accommodate for more communication, productive and finding and handling information activities).
The learning designs for these courses have been reviewed in conjunction with their associated learning performance data collated from FutureLearn dashboards for all presentations to date. This was then further reviewed with the OOC engagement survey I designed, replicated for each course and distributed to learners for the purpose of defining what elements of learning design activities learners engage and subsequently disengage with.
Over the next few weeks I’ll be blogging about the methods I used to review this data and the findings for each course.
Until then, I’m still Doctor in Waiting
Rienties, B., Toetenel, L., and Bryan, A. (2015). “Scaling up” learning design: impact of learning design activities on LMS behaviour and performance. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge – LAK ’15, ACM, pp.315-319