Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources
Robert Farrow, Beatriz de los Arcos, Rebecca Pitt, Martin
Institute of Educational Technology, The Open University, United Kingdom
Open educational resources (OER) have been identified as having the potential to extend opportunities for learning to non-formal learners. However, little research has been conducted into the impact of OER on non-formal learners. This paper presents the results of a systematic survey of more than 3,000 users of open educational resources (OER). Data was collected between 2013 and 2014 on the demographics, attitudes and behaviours of users of three repositories. Questions included a particular focus on the behaviours of non-formal learners and the relationship between formal and non-formal study. Frequency analysis shows that there are marked differences in patterns of use, user profiles, attitudes towards OER, types of materials used and popularity of different subjects. The experience of using OER is fairly consistent across platforms in terms of satisfaction and impact on future behaviour. On the whole, non-formal learners surveyed were highly positive about their use of OER and believe they will continue to use them. With regards to this making formal study more likely some degree of polarization was observed: some believed formal study was now more likely, while others felt it made this less likely. On the whole, while non-formal learners are enthusiastic about using free and online resources, the language and concept of OER does not seem to be well understood in the groups surveyed. A range of findings relating to OER selection and use as well as differences between repositories are explored in the discussion.
Abstract in Spanish
A pesar del potencial de los recursos educativos abiertos (REA) a la hora de ampliar las oportunidades de aprendizaje, existe poca investigación sobre su impacto en el aprendizaje no formal. Este estudio presenta los resultados de una encuesta realizada entre 2013 y 2014 para recoger datos demográficos, de actitud y comportamiento de más de 3.000 usuarios de tres diferentes repositorios de REA, con especial atención a la relación entre el estudio formal y no formal. El análisis de frecuencias muestra marcadas diferencias en las pautas de uso, perfil del usuario, actitudes hacia los REA, tipos de recursos utilizados y popularidad de diferentes materias. Las distintas plataformas registran experiencias de uso de REA similares en lo que se refiere a satisfacción e impacto sobre comportamiento futuro. Aquellas personas que aprenden en un contexto no formal se muestran muy positivas acerca del uso de REA y manifiestan su intención de continuar usando estos recursos. La transición hacia el aprendizaje formal tras el uso no formal de REA se presenta, sin embargo, polarizada entre aquellos que perciben el estudio formal más probable y los que lo perciben menos probable. En general, mientras que las personas que aprenden en un contexto no formal se muestran entusiastas del uso de recursos gratuitos en la red, se observa la falta de entendimiento del concepto de REA entre los diferentes grupos encuestados. La última sección explora una serie de conclusiones entre las que se encuentran, entre otras, diferencias en la selección y uso de recursos en los repositorios estudiados.
Keywords: OER, data, non-formal learning, Saylor, OpenLearn, iTunesU, MOOC
According to the definition provided by Hewlett (n.d.), open educational resources are “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others”. These can include whole courses of open content, textbooks, multimedia, software and any other materials which may be used to teach or support learning such as lesson plans and curricula. There remains some debate about what should qualify as an ‘open’ resource, with some definitions emphasizing open access to resources and others focusing on the affordances for revising and repurposing afforded by open licenses (see Creative Commons, 2013). However, any disagreements tend to be limited to the specific kinds of licenses for educational that are termed ‘open’ and whether they should permit specific forms of re-use (such as only allowing non-commercial re-use).
Evaluating the effects that OER have on learning is problematic for several reasons. Once materials are published as OER they can be adapted and repurposed in accordance with their licensing restrictions. This is often done to improve or update a resource, or to make it more closely aligned to a specific educational need. But this possibility for adaptation can also make it hard to judge the efficacy of a particular resource, and makes longitudinal studies problematic. Furthermore, even when data can be collected comparing similar cohorts who have consistently used OER and non-OER it can be difficult to reliably attribute any change in performance to the openness of the resources used.
Most research exploring the impact of open educational resources (OER) has focused on learners who are registered for a course of study at an educational institution where OER – often in the form of open textbooks – are used. A recent review of efficacy and perception studies (Hilton, 2014) found only twelve instances of peer-reviewed empirical research where the focus was on OER used as primary learning materials. These studies have indicated a high level of student satisfaction with the quality of OER available to them. Students have also consistently reported – particularly in the case of open textbooks – that OER have helped them financially (Pitt, 2015). OER use has also been correlated with higher test scores and lower rates of attrition (Hilton & Laman, 2012) though it is more common to find that use of OER delivers equivalent student satisfaction and performance at massively reduced cost (Feldstein et al., 2012; Wiley et al., 2012).
The use of open educational resources (OER) by learners who are not registered for a formal programme of study has been subject to increased attention in recent years, with a recognition that there is a need for greater understanding of how to leverage non-formal learning in support of formal learning and support the transition from the former to the latter (Latchem, 2014; McGreal et al., 2014).
Sangrà and Wheeler (2013) have suggested that non-formal learning should increasingly be viewed as a viable alternative to formal education, but recognize that more research is needed to establish effective forms of support. Miyazoe and Anderson (2013) argue that:
“the availability of ever-growing amounts of OER and the consequent non-formal learning opportunities fuel this ‘opening’ of the traditional education systems. These free and open opportunities for both interpersonal and student-content interaction create an interaction surplus that can be used to augment and enhance formal educational curricula and systems”.
The emergence of non-formal learning networks, facilitated by social networking and MOOC platforms, are increasingly recognised as a possible catalyst for non-formal learning. Siemens (2005) and Downes (2007) have outlined the ways in which such networks can be understood to support non-formal learning through the prism of Connectivism, which diminishes the distinction between formal and non-formal learning.
“Non-formal learning is a significant aspect of our learning experience. Formal education no longer comprises the majority of our learning. Learning now occurs in a variety of ways – through communities of practice, personal networks, and through completion of work-related tasks… Learning is a continual process, lasting for a lifetime. Learning and work related activities are no longer separate. In many situations, they are the same.” (Siemens, 2005)
The relationship between non-formal learning and formal study is increasingly an area for interest for higher education providers. Most research into non-formal learning concentrates on either the relevance of open materials for educational institutions, or on recognition of non-formal learning for professional development. As a result, little is known about the ‘non-formal’ OER learners; their ways of using these resources; or their reasons for studying in this way.
Miyazoe and Anderson (2013) call for more attention to be paid to the “changing role of formal education in an era of learning opportunity where online educational resources and opportunities are readily accessible and in many cases completely free of cost to the learner”. Advocacy for non-formal learning tends to be grounded in the role that open resources can play in supporting formal learners who are registered for a course of study at an institution. OER can be used in this way to supplement curriculum, brush up on an area of study, for research purposes, or to otherwise complement traditional learning activities (Schmid et al., 2015). Much attention has been paid to possibilities for recognizing and accrediting the learning that takes place through self directed study which makes use of OER (Yang, 2015). OER also comprise a strategy for exposing non-formal learners with access to university level content; affording them opportunities to become more confident with self directed study. Institutionally produced OER can thus be seen to have both pedagogical and business dimensions. Many universities contribute to MOOC on the strategic assumption that in the long run they will be a source of future registrants.
Part of the reason that non-formal learning has risen to greater prominence is a growing consensus that a flexible approach to learning and assessment can contribute to economic development and a richer understanding of how people learn in situ. But finding adequate ways to assess the effectiveness of non-formal learning remains a challenge. Mozilla Foundation (n.d.) launched the ‘Open Badges’ initiative to provide a framework for providing recognition for non-formal learning achievements through non-proprietary digital badges, which once awarded can be presented as part of a portfolio of learning. Badges tend to focus on skill acquisition rather than retention of ‘academic’ knowledge (Goligoski, 2012). As Glover and Latif (2013, p.1398) note, badges have the potential to support both student retention (through increased motivation) and future employability (through a digital record of achievement, or a portfolio of badges which can be shared with others as evidence of learning).
A more traditional approach to the recognition of prior or non-formal learning is to follow private study with an examination equivalent to those taken by formal students. Conrad et al. (2013, p.46) have argued that digital and openly licensed learning materials are well suited to form the basis of a course of private study that can run parallel to institutional presentation and yet “[t]he greatest barriers to participation in open assessment and accreditation practices are identified as the lack of availability of committed staff members to support such activities, and the potential costs of redeveloping courses as OER”. Institutions who wish to promote recognition of prior learning therefore need to adopt a coherent, holistic strategy.
“Within institutions, key factors for the success of open assessment and accreditation implementation appear to be a reliance on a strong base of support within the institution—both in terms of leadership and resources—and an existing culture of openness that includes policies and practices around the creation and use of OER […] Policies that enable either open access or recognition of prior learning via credit transfer or RPL are also important.” (McGreal et al., 2014, p.130)
The advantage of harnessing this kind of non-formal learning could be significant. Cofer (2000) estimates that every hour spent in formal study inspires up to four hours of non-formal learning, but insists that non-formal learning should not be seen as subordinate to formal study but valuable in its own right. Similarly, Latchem (2014) argues that as much as 70-90% of lifelong learning results from non-formal learning and skill acquisition; and yet the majority of research focuses on institutional learning.
Evidently, there remains much to be known about non-formal learning and how best to integrate it with existing systems of assessment and accreditation. This study aims to improve our understanding of this topography by providing information on those who use online repositories of OER for private study. It should be noted that our aim is not to identify or profile distinct learning styles (Kolb, 1984) for non-formal learners. Not only has this approach been criticized as lacking in empirical support (Coffield et al., 2004) but such a categorization would not be possible without prior work on how resources are used by non-formal learners; their learning objectives; and study techniques used.
This research took place within the context of OER Research Hub project which ran from 2013-2015 (OERRH, 2013). OERRH provided a focus for research on OER, and was designed to describe how openness is making a difference to learning and teaching practices. The project operated an open collaboration model and worked with a range of projects, initiatives and organisations across four education sectors (K12, college, higher education and non-formal learning). In addition to the survey based research presented here, the project collected data through interviews, focus groups, institutional visits and through research fellowship. The open collaboration model involved negotiating the parameters and design of the research according to the needs of the collaboration partner. In order to ensure that the areas of focus remained consistent throughout the project the research was guided by eleven key hypotheses about open education (de los Arcos et al., 2014). The main hypotheses investigated across the project focus on the impact of OER on learners and the ways in which open licensing affects sharing and use of resources:
- Use of OER leads to improvement in student performance and satisfaction;
- The open aspect of OER creates different usage and adoption patterns than other online resources.
In addition, several hypotheses pertained directly to non-formal learning:
- Non-formal learners use a variety of indicators when selecting OER;
- Non-formal learners adopt a variety of techniques to compensate for the lack of formal support, which can be supported in open courses;
- Open education acts as a bridge to formal education, and is complementary, not competitive, with it;
- Non-formal means of assessment are motivators to learning with OER.
The survey was designed by the research team to collect evidence relevant to the project hypotheses concerned with non-formal learning from repository users.
Three OER repositories were selected for the study: iTunes U (Apple, 2014), OpenLearn (2015) and Saylor Academy (2015). iTunes U makes use of Apple’s iTunes content delivery systems – primarily used for commercial purposes – to make educational content available for free. This often comes in the form of podcasts and videos, and includes OER from museums and other cultural institutions as well as from higher education. The Open University (UK) runs has the most popular educational channel on iTunes U, though most major OER providers are represented in some way. Institutions can set up their own channel although can’t necessarily upload content to Apple servers. There is a course builder facility which is designed to help users create a programme of study within the platform. iTunes U is restricted to users of proprietary software/hardware made by Apple. Simply put, learners who wish to access these materials must at some point buy into the Apple ecosystem – even if only minimally. The advantage of this for the student is the assurance that all the technology should work adequately together, though there is a case for arguing that this is a significant impediment to openness and accessibility.
OpenLearn (2014) is The Open University (UK) collection of free online learning materials, many of which are taken directly from courses presented to fee-paying students at the University. These include games and interactive media, materials from both undergraduate and postgraduate programmes as well as complete certified short courses. More than 33 million unique visitors have visited the OpenLearn website (which receives about 3 million yearly visitors) (Perryman, Law & Law, 2015).
OpenLearn does more to accommodate non-formal learners who may lack confidence, providing short introductions to subjects as well as “interactive games, videos, blogs, [and] podcasts” (OpenLearn, 2015). A newsletter can also be subscribed to in order to get information about new courses as they come online. Most of the content on OpenLearn is designed to be explored through a web browser, and includes some multimedia content. It is structured so as to encourage the site user to browse their way straight into the content that interests them rather than ‘choosing a course’ from the onset. This ‘magazine’ style may be intended to ameliorate the potentially intimidating nature of the content for learners who are inexperienced or lacking confidence. On the homepage for OpenLearn there are often links made between current events (e.g. local and European elections) and relevant course content. This seems to be designed to make best use of pre-existing user interests, current TV shows and topicality to encourage the personal inquiry of the learner as the start of a learning pathway.
The third repository in this study is Saylor Academy. Founded in 2008, its mission is to use technology to drive the cost of a college education to zero. Working towards this through curating and compiling open content, Saylor acts as an aggregator of existing content, compiling OER into complete courses that emulate or complement existing college courses. Where Saylor is unable to source adequate open content they commission academic consultants to write additional content to fill gaps in the curriculum. In addition to curating college- length courses Saylor also offer a range of featured pathways through several OR collections. These can offer coverage similar to an introductory level liberal arts course, or training in core professional competencies.
Upon completing a course of study with Saylor students can be examined. For successfully completing a Saylor commissioned programme of study students can earn a ‘Saylor Certificate of Completion’ which, while conveying no institutional credit, provides evidence of learning and may be recognised differently in the future. Hilton, Murphy and Ritter (2014) have provided an account of the pedagogical and organizational theories informing the provision of OER as well as describing how these have been applied in the case of Saylor Academy resources. They argue for improved institutional recognition of non-formal learning, noting anecdotal evidence that students have gained college credit after studying non-formally using Saylor materials and subsequently passing the same invigilated exam as formal learners following a non-open curriculum.
One thing to note about the different repositories of OER is that they tend to be characterized by being free at the point of delivery to learners but the different platforms exhibit different potentials for re-use, re-appropriation and modularization of learning materials. Though Weller (2014) has argued that we need to remain open about openness and the possibility that it can mean different things in different contexts, what qualifies as open in different contexts remains contested. For some, an open licence which legitimizes re-use and re-appropriation is a minimal requirement. In addition, we might expect that materials are natively formatted and licensed in such a way as to encourage re-use and re-purposing (iTunes U, for example, would not meet this requirement).
Collecting data from non-formal learners presents particular challenges around sampling and validity because of difficulties with verification. A multi-dimensional analysis (including factor analysis and smallest space analysis) of research literature published in the fields of non-formal and informal learning concluded that “the familiar qualitative and quantitative tools already in the hands of researchers, such as interviewing, observing, surveying, etc. are valuable and appropriate for studying non-formal education” and that no new tools or approaches were required to produce meaningful results” (Cohen, 2007).
OER repositories who acted as OER Research Hub collaborators – iTunes U, OpenLearn, and Saylor – circulated an invitation to complete the survey to non-formal learners who used these sites between 2013 and 2014. 2299 usable responses were received from users of Saylor Academy, 725 from OpenLearn users and 110 from users of iTunes U. The total sample size was 3127 (though not all respondents answered all questions and some questions were omitted from the iTunes U survey). Precise sample sizes for each question are provided in the reporting below. Data was collected through SurveyMonkey, refined and compiled before being analysed in SPSS and Excel.
The survey was answered by OER users from a wide range of countries, as shown in Figure 1. Most responses came from the USA (n = 862) or the UK (n = 473) though India (n = 117) Canada (n = 87) and Brazil (n = 84) also contributed significant amounts of data. Most countries were represented and people from every continent contributed information. Excluding Africa, only Bolivia, French Guyana, Greenland, Kyrgyzstan, Suriname, Turkmenistan, Uzbekistan, and Venezuela were not represented. Most countries recorded between 5 and 40 responses.
A set of demographic questions across the surveys asked about age, gender, prior qualifications and employment status to facilitate profiling OER users. As Figure 2 shows, the repositories exhibit clear differences in the age profile of their users.
Users of iTunes U showed a much lower average age profile with 71.8% of their users aged below 35. By contrast, OpenLearn users tended to be older, with 69% aged 35 or over and relatively few younger users. The pattern of user age profiles was closer to a standard deviation for Saylor users (perhaps reflective of a larger sample size). Similar patterns of difference can be discerned when considering the gender of respondents.
Figure 3 indicates that Saylor users were more evenly split between male and female (and had a greater proportion of users who identified as transgender) while iTunes U users were more slightly more likely to be male. OpenLearn showed the largest difference in the gender of their users, with female users outnumbering male users by approximately half.
Non-formal learners were also asked about their highest academic qualification and their employment status to ascertain differences in patterns of users across repositories. As is perhaps to be expected given the age profiles associated with the different repositories (Figure 3) iTunes U users, generally younger, were most likely to report holding a school leaving qualification as their highest. But as Figure 3 shows, the general trend was that users of these platforms tended to report already holding a degree. 59% of Saylor users held at least an undergraduate degree. The proportion of university graduates for OpenLearn and iTunes U were 46% and 38.2% respectively.
The age of non-formal users of OER also seems important when profiling employment status. Figure 5 illustrates the main differences in pattern. Saylor users were much more likely to be in full time employment (52%, n = 2300) while the majority of iTunes U users (43%, n = 110) were in full time education. iTunes U also had the highest proportion of users who volunteer part time (15%, n = 110). Approximately 40% of OpenLearn users were in full-time employment, and OpenLearn had the highest proportion of retired users (14%, n = 732).
OER repository users were also asked about the ways that they have typically connected to the internet in the previous three months. Figure 6 summarizes this information.
When taken in conjunction with the geographical diversity of the sample these patterns suggest that those accessing these online resources in developing countries are more likely to do so through a smartphone, tablet, or home broadband. A small proportion (less than 10%) of respondents accessed the internet through a dialup connection. iTunes U users were more likely to access materials through a tablet or while at an educational institution or community facility.
The final part of background information collected concerned disability. For the combined sample of all three repositories, 10% of users disclosed a disability (n = 3160). OpenLearn had the highest proportion of users who identified as disabled (15.7%. n = 737) while Saylor reported the lowest proportion of disabled users (7.9%, n = 2298). The relatively high rate of declaration by OpenLearn students may reflect similarities to the approach taken to improving accessibility at The Open University through an inclusive framework which covers human and technical elements from course production through to evaluation (McAndrew, Farrow & Cooper, 2012). Figure 7 provides more information on the types of disability reported, and shows that long term or chronic illnesses and mobility impairments were more commonly reported by OpenLearn users while mental health problems more likely to be reported by iTunes U users. iTunes U users were also much more likely to declare a speech disability. It should be noted that some respondents declared more than one disability.
With respect to reasons for using OER outside of a formal educational context, by far the most common response was a personal interest in a subject. Professional development and study relating to work were also popular. Particularly with users of Saylor. OpenLearn and Saylor users were more likely to report using OER to support the development of study skills or a second language. iTunes users were more likely than others to seek out OER for the purpose of sharing with others. Generally, iTunes users reported fewer reasons for accessing OER. Responses to this question are summarized in Figure 8.
The next part of the survey asked about the subject areas where open resources are typically used by the cohort. The responses are shown in Figure 9, where the most striking pattern is that users of iTunes U reported using OER across a much wider range of subject areas than users of the other repositories. This could be taken to suggest that users of this platform are more inquisitive and diverse in their approach to self-study, or the fact that older casual learners access a broader range of OER as they indulge their curiosity about several subjects. Alternatively, this trend may reflect elements of the podcast model which allows users to download several resources that they are interested in at once to listen to later. In general, there was quite an even spread of interest across all subject areas.
Across the entire sample (n = 2356) the most popular subjects for OER were computing (31.1%, n = 732) and economics (30.4%, n = 716). Table 1 shows the most popular subject areas by platform.
Respondents were also asked about the type and format of the resources they access. Videos, ebooks and open textbooks were the most commonly used resources. On the Saylor Academy platform ebooks and open textbooks were more popular than video. Saylor users reported using less multimedia content, games and quizzes but more lectures. No non-formal learners in any group reported using open data sets or lesson plans in their studies. Figure 10 summarizes the responses to this question.
Users of Saylor and OpenLearn were also asked about their own reasons for accessing OER. (This question was not part of the iTunes U survey.) Figure 11 summarizes their responses. The most popular reason across both platforms was the chance to study at no cost. Saylor users were marginally more likely to cite the flexible and online nature of the resources while OpenLearn users were more likely to state that the chance to try university level content was important to them.
These learners were also asked whether they did more than simply consume OER as it is presented to them through the platform they most often use; i.e. did they involve themselves in the kinds of re-use behaviours that are encouraged and legitimized by open licences? A relatively small number elected to answer this question, which may indicate a lack of confidence with the terminology used. Figure 12 presents their responses. It shows that Saylor users were much more likely to report engaging in processes that support OER production and evaluation. Although the sample size for this question was smaller, it is remarkable that more than 80% of Saylor users who answered claim to have adapted open materials for their own purposes as non-formal learners. (iTunes U users were not asked this question.) This is an interesting and counterintuitive result, possibly affected by different understandings around the concept of adaptation.
Survey respondents were also asked about the other repositories of OER that they have used to assess the extent to which platforms are used exclusively or in conjunction with each other. The results show that YouTube is the most popular place to find open resources, with over 50% of each sample reporting that they used it to find OER. iTunes U and TED talks were also popular across the samples, as was Khan Academy (though less so with users of OpenLearn).
Saylor users were much more likely than the other groups to be studying via MOOC platforms (41.7%, n = 751). Figure 13 presents patterns in OER repository use.
To further understand patterns of use, the survey also asked about the indicators that non-formal learners look for when selecting OER (Figure 14). (Note that iTunes U users were not asked this question.)
We see a striking similarity between the responses given across the two samples. The most important criterion (for approximately 70%) was relevance to a particular need; i.e. OER were sought to plug a specific gap in knowledge or skills. The reputation of the OER creator (whether an individual or repository), clarity around learning outcomes or objectives, and ease of download or access were all cited as important. Less important were reviews of OER or personal recommendations. Open or Creative Commons licensing allowing adaptation was only thought important by a minority of 12-16% which is interesting given the high level of adaptation rates reported elsewhere (see Figure 12). It may be that non-formal learners adapt OER without paying much attention to licensing permissions.
The least important factors appear to be the attractiveness of the presentation of a resource and whether an OER has previously been used in an institutional context. In the case of the latter it should be noted that despite this, more than 50% of the samples thought that the reputation of the resource creator was important and this reputation is often closely associated with institutional affiliation. Both Saylor (20%) and OpenLearn (21%) users identified a resource being required for formal study as important, suggesting that some of these learners may be using OER in lieu of non-open materials required for their formal studies.
It is also noteworthy that between 16% and 25% of each sample believed that lack of support from a teacher or tutor was a challenge to their learning with OER. This could be seen to reflect a general level of confidence around independent, non-formal study among these groups, but still identifies a significant proportion who feel that they would benefit from more support of this kind.
OpenLearn and Saylor users were asked in more detail about the techniques they employ to structure their learning when they have no formal support. Figure 16 shows their responses.
The general pattern suggests that most non-formal learners on these platforms are not using many of the techniques described. Saylor users were more likely to report using a calendar or journal to organize their study as well as using specialized note-taking software; no OpenLearn user reported using either of these. Writing study notes was in general more common among Saylor users (60.2%) while OpenLearn users were more likely to use a blog to record and organize their learning. Significant proportions of both samples said that they discussed their learning with others either face-to-face, via online forums, on social networks, or, less commonly, through micro-blogging platforms like Twitter or Tumblr.
Finally, the survey asked about the likely impact of OER use in terms of the future behaviour of the learners. Figures 17-19 summarise their responses (with values for ‘more likely’ behaviours highlighted).
We can discern here a very similar pattern of satisfaction with the quality of open materials across the three samples. Between 69% and 74% would recommend the repository to others and approximately half would share them directly with others (though the figure was slightly lower for OpenLearn at 44%). High proportions across all samples indicated that they would download further materials from the repository and feel empowered to undertake further study in a related area. But this satisfaction with the learning experience may also translate into a polarization between those who find that non-formal use of OER has made it more likely that they will seek formal study, and those who feel that it has made it less likely; presumably because of satisfaction with the kind of learning materials that are on offer openly. In each case those who answered ‘more likely’ or ‘less likely’ to this were about equal in number. OpenLearn users reported slightly greater likelihood of going on to formal study; this is perhaps reflective of the structured pathways leading from OpenLearn to degree programme course credits.
The vast majority of those surveyed said that they would continue to use OER in the future – 84.8% across the three samples as a whole. A mere 0.4% of those who participated in this study said that using OER had made it less likely that they would use OER in future.
This dataset provides us with a more detailed sense of how non-formal learners using OpenLearn, iTunes U and Saylor Academy materials select and use OER. We can distinguish some broad patterns in the user bases of the different repositories. With a majority of the sample below the age of 30, iTunes U users were much more likely to be younger and were mostly male. They are often in full time education and use OER on an informal basis outside of their formal studies. By contrast, Saylor Academy users are more likely to be in employment and already in possession of a degree. They tended to be middle aged and primarily motivated by professional development. OpenLearn users were more likely to be older, retired, and female, and had a higher proportion of users who were motivated mainly by personal interest (though 40% are in full time employment).
Data about prior qualifications supports the pattern often seen in MOOC research (Laurillard, 2014; Emmanuel, 2013) where users of free resources are often already graduates of higher education institutions. This was particularly evident with users of Saylor materials, more than half of whom (59%) were already in possession of at least an undergraduate degree. This was not the case for iTunes U, which had a markedly younger user base.
One standout finding is that users of OER across all platforms expressed a high degree of satisfaction with the OER that they had accessed. Attitudes toward OER were overwhelmingly positive across the sample as a whole. This strongly supports the idea that OER (at least for the repositories studied) were of sufficient quality and appropriate for the education level of users. However, as Figure 19 indicates, this satisfaction does not always translate to a willingness to study formally. In fact, people who have used OER tend to become somewhat polarized. On the one hand we have those who feel more inspired or more confident after non-formal use of OER; while on the other there appear to be those whose learning needs are entirely met by use of open resources. Table 2 summaries this polarization.
The higher figure for OpenLearn may be explained by the existence of planned pathways between OpenLearn content and degree level content provided by The Open University (UK) which are intended to facilitate the transition from non-formal to formal study.
Even as learners reported a high level of satisfaction with the materials provided by their respective repositories, many identified difficulty locating subject-specific resources of adequate quality to be a significant challenge (Figure 15). In light of this it is noteworthy that there is a certain amount of ‘brand loyalty’ apparent with most learners using a small number of repositories (Figure 13). This can in part be attributed to lack of clear understanding of the nature of OER and what qualifies as a repository. It seems significant that specialist repositories of OER like Merlot, Jorum, and Curriki were hardly visited at all by the non-formal learners who answered the survey. This suggests that learners who use free online digital materials rarely visit specialized OER repositories, perhaps because they are only aware of resources being provided online for free rather than the concept and language of openness and OER. Respondents typically indicated a lack of understanding about the nature of an OER repository: between 9% and 20% of each sample said that they did not use any OER repositories despite the fact that they had only been offered the chance to participate in the survey specifically because of their OER repository use.
One standout finding (Figure 12) was the high reported level of adaptation of OER by non-formal learners (as high as 85% for Saylor Academy users but still over 20% for OpenLearn). This was not anticipated and may indicate an unclear question. The research team intended by adaptation to mean some sort of editing or remix of resources in order to meet a perceived learning need. Given these responses, it may have been interpreted as collecting and curating resources together; or as taking material from one country/level in order to study in a different context. Comments received in relation to this question were not insightful, but did suggest that most informal OER users are not confident enough to remix and revise OER: not least because of the lack of interest in open licensing (Figure 14). This is an area where follow-up qualitative research focused on illuminating understandings of ‘adaptation’ would be beneficial.
A lack of knowledge about the technicalities of OER could be taken to imply that non-formal learners may not have much sense of reflecting on their own activity. Yet at the same time clarity of learning objectives was one of the most important factors affecting OER selection, so learners do seem to care about some aspects of pedagogy. In relation to this, it is noteworthy that no more than 25% of each sample felt that lack of tutor support was a barrier to them learning using OER (Figure 15). This confidence could be reflective of a number of factors, including faith in the quality and presentation of OER; skills gained through prior study; or overconfidence in ones ability to study unsupported. Data received about the impact of OER use on future behaviour would favour the first interpretation. As Figures 17-19 show, most users are overwhelmingly positive about their experience of using OER across all three repositories.
Limitations and future research
It should be noted that those who responded to the survey invitations constitute a self-selecting, self-reporting sample. A further limitation of the data is that respondents were free to skip any question they did not wish to answer. This means that there are some gaps in the data provided. Furthermore, in discerning patterns between the different user groups it should be borne in mind that the sample sizes were quite different across the three platforms. This may be particularly pertinent for the relatively small sample size of iTunes U users. Future studies could apply more rigorous sampling methods in order to extrapolate to more general populations. The lack of prior research in this area means that even with these caveats the results are likely to be of interest to a range of stakeholders. Further analysis of this data set (Farrow et al., 2015) is encouraged. Such work could examine patterns of response according to country; employment status; area of study; prior study; and other variables.
Two areas that would likely make a good focus for qualitative work are to better unpack what non-formal learners understand by phrases like ‘adaptation’ and ‘relevance’, since these appear to be key considerations in the selection and use of OER. This may also facilitate understanding of why some learners ultimately embark on formal study as a result of their use of online resources.
This paper has presented data collected from users of three prominent repositories of OER. It has shown that there are marked differences in patterns of use, user profiles, attitudes towards OER, types of materials used and popularity of different subjects. Data also indicate that the experience of using OER is fairly consistent across platforms in terms of satisfaction and impact on future behaviour. Overwhelmingly, non-formal learners are very positive about their experiences of using OER, with a huge majority stating that they are more likely to use OER in the future. Indeed, such is the level of apparent satisfaction with OER use that it makes a significant minority less likely to (re-)enter formal education at all. While users are enthused about using free and online resources, the language and concept of OER does not seem to be well understood in the groups surveyed.
In relation to the research hypotheses of OER Research Hub, we can offer the following concluding comments in relation to this study.
- Apple (2014). iTunes U. Retrieved from http://www.apple.com/education/ipad/itunes-u/
- Cohen, E.H. (2007). Researching Informal Education. Bulletin of Sociological Methodology, 93(Jan), 70-88. Retrieved from http://bms.revues.org/529
- Colardyn, D., & Bjornavold, J. (2004). Validation of Formal, Non-Formal and Non-formal Learning: policy and practices in EU Member States. European Journal of Education, 39(1), 69-89. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.0141-8211.2004.00167.x/abstract
- Cofer, D. (2000). Non-formal workplace learning: Practice application brief, NO 10. US Department of Education: Clearinghouse on Adult, Career and Vocational Education.
- Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning. A systematic and critical review. London: Learning and Skills Research Centre. Retrieved from http://www.lsda.org.uk/files/PDF/1543.pdf
- Conrad, D., Mackintosh, W., McGreal, R., Murphy, A. & Witthaus, G. (2013). Report on the Assessment and Accreditation of Learners using Open Education Resources (OER). Commonwealth of Learning. Retrieved from http://oasis.col.org/handle/11599/232
- Creative Commons (2013). What is OER? Retrieved from http://wiki.creativecommons.org/What_is_OER%3F
- de los Arcos, B., Farrow, R., Perryman, L.-A., Pitt, R. & Weller, M. (2014). OER Evidence Report 2013-2014. OER Research Hub. Retrieved from http://oerresearchhub.org/about-2/reports/
- Downes, S. (2007, February 3). What Connectivism Is [Blog post]. Half an hour. Retrieved from http://halfanhour.blogspot.co.uk/2007/02/what-connectivism-is.html
- Emmanuel, E.J. (2013). Online Education: MOOCs taken by educated few. Nature, 503, 342. doi:10.1038/503342a
- Farrow, R., Pitt, R. de los Arcos, B., Perryman, L-A., Weller, M., & McAndrew, P.(2015). Impact of OER use on teaching and learning: data from OER Research Hub (2013–2014). British Journal of Educational Technology, 46(5), 972–976.
- Felstead, A., Fuller, A., Unwin, L., Ashton, D., Butler, P., & Lee, T. (2005). Surveying the scene: learning metaphors, survey design and the workplace context. Journal of Education and Work, 18(4), 359–383. Retrieved from http://dx.doi.org/10.1080/13639080500327857
- Feldstein, A., Martin, M., Hudson, A., Warren, K., Hilton, J., & Wiley, D. (2012). Open textbooks and increased student access and outcomes. European Journal of Open, Distance and E-Learning, 15(2). Retrieved from http://www.eurodl.org/index.php?p=archives&year=2012&halfyear=2&article=533
- Friesen, N. (2009). Open Educational Resources: New Possibilities for Change and Sustainability. International Review of Research in Open and Distance Learning, 10(5), 1-13.
- Glover, I., & Latif, F. (2013). Investigating Perceptions and Potential of Open Badges in Formal Higher Education. In J. Herrington, A. Couros & V. Irvine (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2013 (pp. 1398-1402). Association for the Advancement of Computing in Education (AACE).
- Goligoski, E. (2012). Motivating the Learner: Mozilla’s Open Badges Program. Access To Knowledge: A Course Journal, 4(1). Retrieved from http://ojs.stanford.edu/ojs/index.php/a2k/article/view/381
- Hewlett (n.d.). Open Educational Resources. The William and Flora Hewlett Foundation. Retrieved from http://www.hewlett.org/programs/education/open-educational-resources
- Hilton, J. (2014). A Review of Research on the Perceptions, Influence and Cost-Savings of OER: Looking Back and Looking Forward. Paper presented at Open Education 2014. Washington D.C., USA 19th November 2014.
- Hilton, J., & Laman, C. (2012). One college’s use of an open psychology textbook. Open Learning: The Journal of Open and Distance Learning, 27(3), 265–272. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/02680513.2012.716657
- Hilton, J., Murphy, L., & Ritter, D. (2014). Open Educational Resources to College Credit: The Approaches of Saylor Academy. Open Praxis, 6(4), 365-374.
- Illeris, K. (2003). Towards a contemporary and comprehensive theory of learning. International Journal of Lifelong Education, 22(4), 396-406. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/02601370304837#.VmWgiF6pLm4
- Järvelä, S., & Renninger, K. A. (2014). Designing for learning: Interest, motivation, and engagement. In D. Keith Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (2nd ed.), (pp. 668-685). New York: Cambridge University Press.
- Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.
- Latchem, C. (2014). Non-formal Learning and Non-Formal Education for Development. Journal of Learning for Development, 1(1). Retrieved from http://www.jl4d.info/index.php/ejl4d/article/view/6/6
- Laurillard, D. (2014, June 26). What is the problem for which MOOCs are the solution? [Blog post]. #ALTC Blog. Retrieved from https://newsletter.alt.ac.uk/2014/06/what-is-the-problem-for-which-moocs-are-the-solution/
- Law, P., Perryman, L-A., and Law, A. (2013). Open educational resources for all? Comparing user motivations and characteristics across The Open University’s iTunes U channel and OpenLearn platform. Proceedings of the Open and Flexible Higher Education Conference 2013, 23-25 October 2013, Paris, France, European Association of Distance Teaching Universities (EADTU), 204–219. Retrieved from http://oro.open.ac.uk/39102/1/eadtu annual conference 2013 - proceedings.pdf
- Malcolm, J., Hodkinson, P., & Colley, H. (2004) Informality and Formality in Learning: A Summary of the Report to the Learning and Skills Research Centre. London: Learning and Skills Research Centre. Retrieved from https://kar.kent.ac.uk/4647/
- McAndrew, P., Farrow, R., & Cooper, M. (2012). Adapting online learning resources for all: planning for professionalism in accessibility. Research in Learning Technology, 20(4), 345–361.
- McGreal, R., Conrad, D., Murphy, A., Witthaus, G., & Mackintosh, W. (2014). Formalising non-formal learning: Assessment and accreditation challenges within disaggregated systems. Open Praxis, 6(2), 125-133. Retrieved from http://openpraxis.org/index.php/OpenPraxis/article/view/114
- Miyazoe, T., & Anderson, T. (2013). Interaction Equivalency in an OER, MOOCS and Non-formal Learning Era. Journal of Interactive Media in Education 2013(2), 9. doi: http://dx.doi.org/10.5334/2013-09
- Mozilla Foundation (n.d.). OpenBadges. Retrieved from http://openbadges.org
- OERRH (2013). OER Research Hub. Retrieved from http://oerresearchhub.org
- OpenLearn (2014). OpenLearn – Explore.http://www.open.edu/openlearn/about-openlearn/explore
- OpenLearn (2015). OpenLearn. Retrieved from http://www.open.edu/openlearn/
- Perryman, L.-A., Law, P., & Law, A. (2015). Sustainable business models for OER revisited: researching changes in OpenLearn users’ motivations and experiences.
- Pitt, R. (2015). Mainstreaming Open Textbooks: Educator Perspectives on the Impact of OpenStax College open textbooks. The International Review of Research in Open And Distributed Learning, 16(4). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2381
- Saylor Academy (2015). Saylor Academy. Retrieved from http://www.saylor.org
- Sangrà, A., & Wheeler, S. (2013). New Non-formal Ways of Learning: Or Are We Formalising the Non-formal? Universities and Knowledge Society Journal (RUSC), 10(1), 286-293. doi: http://dx.doi.org/10.7238/rusc.v10i1.1689
- Schmid, L., Manturuk, K., Simpkins, I., Goldwasser, M., & Whitfield, K. E. (2015). Fulfilling the promise: do MOOCs reach the educationally underserved? Educational Media International, 52(2), 116-128. doi:10.1080/09523987.2015.1053288
- Sfard, A. (1998). On Two Metaphors for Learning and the Dangers of Choosing Just One. Educational Researcher, 27(2), 4-13.
- Siemens, G. (2005). Connectivism: A Learning Theory for the Digital Age. International Journal of Instructional Technology and Distance Learning, 2(1), 3-10. Retrieved from http://www.itdl.org/journal/jan_05/article01.htm
- Weller, M. (2014). Battle for Open: How openness won and why it doesn't feel like victory. London: Ubiquity Press. doi: http://dx.doi.org/10.5334/bam
- Wiley, D., Hilton, J. Ellington, S., & Hall, T. (2012). A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. International Review of Research in Open and Distance Learning. 13(3), 261-276.
- Yang, J. (2015). Recognition, Validation and Accreditation of Non-formal and Non-formal Learning in UNESCO Member States. Hamburg: UNESCO Institute for Lifelong Learning. Retrieved from http://unesdoc.unesco.org/images/0023/002326/232656e.pdf
This study was funded by The William and Flora Hewlett Foundation as part of OER Research Hub. OpenLearn, Saylor Academy and The Open University iTunes U channel were research collaborators who assisted with data collection, and the authors gratefully acknowledge their support.