Category Archives: Year One

Final Report Year One – Conclusions of Methodology and Next Steps

The application of an initial study not only allowed the testing of both research methodologies identified or use in the main study (surveys and interviews) but also through the undertaking of interviews prior to the development of the beta survey resulted in the confirmation of the research questions to be explored within the main study.

The results of the initial study continue to impact on the literature being reviewed moving forward, concentrating on literature relating to engagement in learning, open online courses, and learning design used in the creation of open online courses. Ensuring that the literature remains aligned to the addressing of the research questions.

The results of the pilot study have also impacted on the methodology proposed for the main study, with a stronger focus on analysis techniques in both the quantitative and qualitative data gathered from the surveys, but also with the application of content analysis and grounded theory methods developed from the initial interviews in the analysis of responses in the post-survey interviews in the main study.

Final Report Year One – Proposed Analysis of Main Study

As both quantitative and qualitative data will be collated for the main study via surveys and interview, different approaches to data analysis will be taken.

With regards the data analysis of the quantitative data from the survey, the use of nominal scales for questions relating to demographics, and ordinal scales operating on the Likert scale principles will be utilised, thus considered non-parametric.

In the analysis of the data a one-tailed test may be applied to test the hypothesis that those learners associating engagement with open courses as related to an extrinsic professional or academic goal in comparison to a leisure learner, are more likely to engage with the course until completion, with leisure learners being more succinct and sporadic in their engagement strategy. The analysis of the data should define whether variables, such as academic and professional current positioning and future goals bear any relation to the learners perception of, and engagement with the open courses and what linear or non-linear relationships can be drawn from this.

To ensure reliability in the data analysis SPSS will be used, especially in taking into consideration the large-scale data expecting to be received via the surveys. It is proposed that the split-half technique be applied in conjunction with the use of SPSS. Data will then be tabulated for expression within the text, and data visualisations in the form of graphs, pie charts, etc. only to be displayed where it adds greater value to the analysis beyond the use if frequency and percentage tables.

The secondary element of analysis will be for the qualitative data acquired through data collection from the interviews following the survey. To amalgamate the key issues emerging from the transcriptions a combination of progressive focusing (Parlett and Hamilton, 1976), content analysis (Ezzy, 2002: 83, Anderson and Arsenault, 1998: 102) and grounded theory (Strauss and Corbin, 1994: 273) will be used, initially taking a wide angle approach to gather the data from interviews across the three data sets, to then through sorting, coding, reviewing, and reflection upon the responses given to systematically gather and analyse the data.

Through typological analysis (LeCompte and Preissle, 1993: 257) the three data sets the organising of the data can firstly be ordered in the three platform groups, then reviewed and organised as individuals to ascertain whether any themes or frequencies through the application of secondary coding (Miles and Huberman, 1984) emerge that allow the organisation of the data by issue to analysis plausibilities as whether it can be organised by research question. The following of Brenner et al’s (1985) steps to content analysis in conjunction with the organisation of the data into groups should ensure reliability in its interpretation.

Final Report Year One – Importance and Limitations of Main Study

The conducting of the initial survey is highly important to the undertaking of the research for the main study this is because the initial study allowed for the addressing and testing of the methodology to be used in the main study to ensure a robust survey that will for the foundations of the research, analysis and conclusions to be drawn.

The limitation in using both surveys and interviews is that there is little flexibility in relating the questions directly to the respondent’s personal circumstances and therefore may limit the respondent in the answers given. Through the use of natural language (Kvale, 1996) ‘stimulus equivalence’ (Oppenheim, 1992) may be achieved, whereby each respondent may understand the questions set before them, even if they are unable to relate it then to their personal circumstances.

However, the use of interview questions from the pilot study and then in the main study, can result in unanticipated answers that can lead to further connections in data relationships and addressing or the creation of hypothesis (Cohen et al., 2007). These responses could potentially be categorised in Tuckman’s (1972) seven modes, of which ‘filled in response’ is most likely due to the type of questions to be posed to the respondents.

The final limitation of the main study to consider is the current absence of a survey being conducted within a presentation of a MOOC on the FutureLearn platform. Presently this is due to the formal use of surveys at the start and the end of a MOOC presentation. However, recently the use of a research survey within the presentation of a course has successfully occurred and is being repeated during its second presentation. Discussions with the Open Media Unit will need to continue to ascertain whether the main study survey could follow the same pattern as the previous research survey by being circulated via a mid-week email as an optional activity for the learners and therefore not disrupting the standardised pattern of formal surveying activity.

Final Report Year One – Ethical Considerations in Main Study

Before conducting any research an application to HREC (Human Research Ethics Committee) at The OU outlining the research proposal was made. To ensure full compliancy an enquiry for further submission to SRPP (Student Research Project Panel) was also made to be informed that as OU students wouldn’t be specifically targeted additional SRPP ethical approval wouldn’t be required. Upon HREC approval further ethics application was made and granted from the Open Media Unit to research and release data on The OU’s open educational resources.

In line with guidelines of BERA and the Association of Internet Researchers the moral duty to respect privacy, confidentiality and anonymity is adhered to. Whereby interviews were used, introductory text was read out at the beginning of each of the interviews (Appendix 1), and participants asked whether they would agree to the presence of a recording device. All participants were informed that their responses would remain anonymous as they would be allocated an identification number for future reference within the research. For the surveys in the initial (Appendix 2) and main study, the introductory page of both surveys display the ethical research statement detailing the purpose of the research, how the research will be used, how to exit the survey at any time, contact details for further information, and that by clicking to enter the questionnaire is a confirmation of acceptance of the ethical statement (information on how withdraw is also given).

Most of the data is collected through the use of surveys and interviews however to protect the participants involved the allocation of a number will anonymise them in this process. The list of names to allocated numbers is be kept in a password protected spreadsheet that will form part of the use, storage, and disposal requirements of the other data gathered for the research. Only completed papers, reports and publications will be published whereby participants are anonymised to protect their privacy addressing Bryman’s (2001) ethic principles and Bassey’s (1999) ethical values, whilst adhering to the guidelines set by BERA (1992).

Final Report Year One – Data Collection in Main Study

The data from the survey will be collected via Qualtrics and held securely within the account for analysis via SPSS (Statistical Package for the Social Sciences) for which both have single account holder access. The survey allows participants to remain anonymous unless contact details are submitted in the final question. If contact details are submitted and then used for follow up telephone or Skype interviews then the same process of storage and number allocation is followed from the initial study.

The current blog ‘Doctor in Waiting’ will utilised further as a research journal (Burgess, 1984b) documenting the decision making processes of the content analysis. The use of grounded theory (Strauss, 1987 and Strauss and Corbin, 1990; 1997) using thematic analysis and to fragment and isolate data from qualitative responses from within surveys and interviews will be used. If a higher number of interviews than expected are conducted within the main study, the software package QSR NVivio will be utilised to aid the extrapolation of attitudes and themes from the qualitative data. Following the methodology by Stroh (2000) the text will be chunked, labelled, and coded to organise the data for analysis.

Though the data to be collected from the three locations is separate it will be analysed for prevalent themes and patterns from within the three individual data sets and collectively. The application of different levels of analysis will be conducted to ascertain whether commonalities occur with regards to engagement, disengagement, preference of learning design elements, etc. but also to clarify whether demographical data and technology use have an affect also. Themes will be drawn together to denote whether there is a stronger narrative within the data and reviewed as to the reoccurrence of the themes to denote emerging patterns. This analysis is undertaken throughout the stages of research (Bryman, 2001) to ensure that the research questions are been considered and answered. Questions from the initial study will be refined for the main study.

Final Report Year One – Factors Introduced in Main Study

In creating three samples from the survey it aids in the broadening out of the respondent demographic to address the representation of informal learners interested in open courses that may produce three similar or different results. From these results, themes across all three samples may be determined, or it may be determined that each sample approaches informal learning differently so therefore has different needs for building and sustaining engagement to completion.

It is also possible that each of the three samples may have a very different response rate, meaning that themes may be detected in larger response rates and not in smaller ones, or that it may be difficult to compare a small sample with a much larger one. In this situation it is proposed that a random sample of equal size is taken from each of the three sets and the data from this is analysed also.

As part of the survey there will be an opportunity for participants to enter an email address if they wish to participate in further research. From the data gathered the provision of contact information creates the further exploration of themes or anomalies that will become apparent from the findings either with a secondary survey or in a telephone interview. At present this research is an unknown entity as the three platforms have not been survey in regards to learning engagement before either individually or collectively for comparison. The surveys will be hosted through Qualtrics from which the numerical data will be analysed through SPSS and the narrative data to be analysed using the protocols of content analysis.

In the hosting of three separate (yet identical) surveys on Qualtrics, it will ensure that the three samples are kept separate for comparison studies and each set to be labelled numerically to give the platform surveyed anonymity as well as the participants within. As noted previously it is expected that the goals for engaging with the courses will be extrinsically different, therefore it is hypothesised that the level of engagement with the individual elements of the course and the course as a whole with the view of completion, will be different with the learners on different completion trajectories dependent on external pressures (such as work and study related deadlines).The data groups can then be combined for additional collective analysis to understand as a collective the data isolated to determine if other factors affect levels of engagement such as age or types of technology used to access the courses.

Final Report Year One – Participants and Samples Chosen for Main Study

From the interviews, a common theme emerged from the content analysis by which the reasons the learners were selecting the course (and other open online courses that they mentioned) fell into two main categories; learning for personal interest and learning for work related interest.

To explore further whether those learning for personal interest are more or less likely to continually engage with the material than those learning for work related interest, or whether further reasons to engage and remain engaged in open online courses will become apparent, three identical surveys are to be hosted in three different locations.

  • OpenLearn website
  • Centre for Inclusion and Collaborative Partnerships (CICP) Partner site and mailing list
  • The Student Room website (OU forum page)

The hosting of three identical surveys in the different locations will provide three separate datasets that can be analysed separately or compared collectively. The three samples will provide insight into the level and endurance of the engagement between learners participating in open online courses for personal or work related purposes, with those engaging for work related purposes most likely to engage to completion quicker than those learning for personal interest.

The three sites provide a wide demographic, with OpenLearn having received over 36 million visits since its launch in 2007 by The Open University as its own open educational resource platform. Analytics and survey data show that learners arrive at the site primarily through search engine results in locating answers to their learning queries, or via click-through from broadcast co-productions. Its learners are not always consciously looking for informal learning experiences so may not be initially engaged to complete a whole course. They may just wish to find the answer to their learning question using a ‘strategic student’ ‘dip-in dip-out’ strategy. OpenLearn hosts over 800 open online courses on their site, including MOOCs.

CICP work in conjunction with a number of widening participation organisations and unions to deliver learning. These learners may select and engage differently with courses as it may be linked to a professional outcome so may approach their learning choices, time to study, engagement to completion activities differently to a learner arriving to OpenLearn in search of an answer to their learning question. Open online courses are recommended by the social partners for enrolment for their learners to develop themselves in a professional capacity.

Finally, the survey is to be hosted on The Open University forum on TSR. TSR is the largest student forum in the UK with over 1.1 million members and its visitors are highly engaged with formal study with the view to progress into higher education. It may be possible that these learners may undertake informal learning with the aim to create a strong academic profile whilst either applying to, or succeeding in university studies. So therefore engagement in the informal is to supplement the formal. At present TSR have not undertaken this type of research with their forum users, so this survey will provide new research data as to how university ready students view and engage with open online courses.

Each of these samples are different, and purposefully selected as to date these platforms and partnerships have not been researched in this field. Whilst it could be argued that the targeting of MOOC specific platforms would be beneficial to the study, this wouldn’t be beneficial as the demographic of those learners on those platforms as discussed previously is not ideal for progression into formal study. Both OpenLearn and CICP attract a very high percentage of widening participation learners who do not have much experience with higher education, and TSR attracts a high volume of forum learners looking to access higher education. What makes this research different and new, is the access to learners who are not the current MOOC demographic to understand how they would engage in open online courses. As highlighted previously, this would be seen by The Open University as a stronger JIFL journey and such findings would be of interest to more than one university.