OER Synthesis and Evaluation / HEFCE-OER-Review-Interim-Report
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

HEFCE-OER-Review-Interim-Report

Page history last edited by Jay Dempster 11 years, 7 months ago


 

Introduction

 

This report offers interim findings from our review of HEFCE funded UK initiatives that explore and support open educational practices, which includes all phases of JISC/HE Academy's UKOER programme and the Open University's SCORE activities. Both UKOER and SCORE aim to advise on and support development and publication of resources in more ‘open’ form as well as to raise awareness, draw out key issues and deepen understanding with regard to open practices.

 

The HEFCE OER review is being undertaken by the UKOER Synthesis and Evaluation Team led by Glasgow Caledonian University, using the core UKOER methodology developed iteratively across the three phrases of UKOER, modified and expanded as appropriate to the distinct aims and activities of SCORE. Both retrospective evaluation and evaluation of currently active UKOER and SCORE supported work is being undertaken. However, to fit in with reporting deadlines for SCORE, an early interim report on the SCORE supported activities to date, was submitted in May 2012, based on documentary evidence and early thoughts on qualitative outcomes. 

 

The evaluation will focus on gathering new evidence from stakeholders; with respect to SCORE, these are likely to be recipients of SCORE fellowships (both teaching fellows and residential course attendees), advisory board members, institutional representatives, and (as far as possible or indirectly) will include course leaders and students.  A cumulative report and briefing paper synthesising findings across the entire spectrum of HEFCE's OER work will be produced at the end of October 2012.

 

Background

 

HEFCE's 'OER' badged work builds on past sector investment in sharing resources, including JISC Exchange for Learning x4L, Jorum, JISC Digital Repositories Programme, JISC Repositories and Preservation Programme and JISC ReProduce Programme. Some activity has arisen independently of formal reuse or OER projects as an extension of more general learning and teaching activity, e.g. through the HE Academy subject centre or national teaching fellow activity.

 

Initiated in 2009, HEFCE have funded three phases of the JISC/Academy funded UKOER programme and The Open University's national role through SCORE, which  completes this year. Building on the substantial synthesis and evaluation across the UKOER, the purpose of this overarching 'HEFCE OER review' is two-fold: (a) to deepen understanding of open educational practices and resources, and (b) to produce a solid evidence base and enhance the status of the work supported in the UK and in the international OER field.

 

HEFCE funded OER work in the UK is extensive and impacts on strategy, policy, practice (of a wide range of stakeholders, including learners), research, curriculum design, delivery and support. While the intention is to undertake an evaluation of UKOER and SCORE as a collective whole, an initial review of SCORE's documentary evidence enabled early consolidation of the UKOER framework with SCORE's unique aims and activities. Throughout all three phases of the UKOER programme, evaluation findings from projects have been externally synthesised by the team through the development and refinement of an evaluation framework and associated methodologies, which have contributed to a series of reports and background data collated on a wiki platform.

 

Purpose & Scope

 

This HEFCE OER review extends the synthesis framework used for the UKOER programmes to include the aims and activities of SCORE, thus promoting a unified emphasis on the programmes as part of a whole HEFCE investment in OER. The existing synthesis framework has a number of key focus areas, each with a range of evaluation questions that can be addressed by project teams as they carry out their individual project evaluations. Individual project and strand/theme findings are mapped to the questions in the framework, providing an overview of key issues and trends across the programme. This approach highlights both key outcomes and significant outputs that demonstrate evidence of these. 

 

The framework for UKOER has evolved throughout all phases of the UKOER programme. Evaluation & synthesis has been an iterative, two-way process such that projects and support teams contributed to the development of the framework throughout and that each iteration of the framework reflected current work. The OER phase 1 pilot programme enabled the large scale release of OER, the three strands of funding allowed different approaches, benefit cases and technical solutions to be trialled in a genuinely diverse mix of contexts. Phase 2 extended OER release but also supported activity areas around OER use and discovery. The current phase 3 projects are investigating the use of OER and open approaches to work towards particular strategic, policy and societal goals through a thematic approach: Theme A : Extend OER through collaborations beyond H; Theme B: Explore OER publishing models; Theme C: Addressing sector challenges; Theme D: Enhancing the student experience.

 

All three phases have explored individual, institutional and community issues around embedding, sustainable practice and widening engagement with OER. This review of HEFCE's OER work across both the UKOER programme and SCORE initiative both consolidates and potentially expands the representation, themes and findings from previous synthesis across the existing framework. A wide range of answer options (including an 'other' free text option) in the detailed survey and the use of very open questions in interviews allows new and unexpected themes and gaps to surface. However, encouraging respondents of both the wider poll and the OER survey to choose/ state their 'top 3' also enables us to identify more categorically the major influences and priorities across stakeholders.

 

Methods

 

A dual approach to data gathering was taken. Evidence was drawn from the OER-specific communities through a detailed survey and interviews. Wider sector engagement was sought through an online poll designed for this review, as well as some analysis of social media activity. From initial findings, a focus on the "OER journey" emerged as a way into interviews and interim reporting.

 

  Online communities, surveys & crowdvoting

 

A first entry into the OER community and wider sector perspectives has been achieved through design and delivery of two online questionnaires during July 2012. The first was a short 'poll' of five questions distributed widely across professional contacts, HE/FE mailing lists and social networks, which aimed to to snapshot cross-sector awareness of HEFCE's OER initiatives (namely the three phases of UKOER and SCORE). The second was a longer survey circulated specifically to individuals directly participating or indirectly involved in the HEFCE funded UKOER programme and OU's SCORE work.

 

Survey questions explored the way open educational practices are being perceived in terms of who OERs are being produced for, as a key indicator of potential longer term impact. For instance, questions were intended to identify changes in attitudes towards risk that can lead to a greater and more widespread release and use of OER by teaching staff, enhanced student centred approaches, as well as the role of OERs in marketing, leading to reputational gains for the institution. Central to supporting the OER community is the online presence of both UKOER and SCORE participants. Some data concerning activity and usage via the web, blog, social media, has been reviewed, as this offers valuable indicators concerning sector engagement. What is interesting here is the general inter-connectedness of the OER community. Such community structures, growth and cohesiveness are certainly worthy of further analysis. 

 

However, for those not necessarily involved or funded directly, we recognise that different terminologies, contexts and associations may apply. It is important that these are identified and incorporated into our cumulative knowledge and understanding. We therefore have welcomed and encouraged wide engagement with a short, cross-sector 'crowdvoting' type online poll that aims to introduce a broader set of opinions and judgment - "wisdom of the crowd" - against which we can overlay our deeper analysis of open practices from individuals within the OER community.

 

  Interviews with direct participants

 

A set of follow up interviews were elicited from respondents to the detailed OER survey. The selection was made from those who (i) volunteered to participate further, (ii) were directly funded/supported as individuals or led an institutional project, and (iii) collectively represented one or more phases of UKOER and/or SCORE. Interviews aimed to deepen our understanding in the four key areas represented by the OER framework and pick up on any new emerging themes, including unanticipated or unexpected findings.

 

Interview questions were designed to draw out perceived institutional /subject related benchmarks, specifically:

  • awareness of UKOER and SCORE work
  • sector, roles and disciplinary nuances
  • individual and institutional drivers / motivations
  • perceived /experienced benefits of release and use of OERs
  • perceived /experienced barriers to release and use of OERs
  • evidence of impact within different communities of practice
  • evidence /examples of student /learner involvement and benefits
  • how OERs are impacting future directions & the design of education.

 

A selection of 16 survey respondents were invited to participate in interviews and care was taken to ensure the sample (i) incorporated practitioners from both UKOER (8) and SCORE (4) ; (ii) senior staff directly involved in either or both (2 individuals in institutional management or mentoring roles); and (iii) did not favour only those highly engaged in OER (2 individuals unknown to the team): numbers as indicated.

 

Interviews were semi-structured, drawing out personal benefits and examples of impact and sustained outcomes in their organisation (e.g. in terms of culture, practice, skills, support, partnerships, resources, and so forth. Relatively structured initial questions captured their institutional role, skills and experience to baseline, and follow up questions were used to clarify and expand on the areas listed below and any new themes emerging.

  1. the scale of personal & organisational engagement in OER at the start;
  2. their sense of primary spheres of practice (role, communities, disciplinary groupings, use of OERs from different sources);
  3. their perception of their organisation's 'OER journey'
  4. evidence of benefits and sustained outcomes/ impact.

 

Interview questions were deliberately open and used to prompt a fairly free narrative yet draw out specific examples. The data is likely to be 'messy', but this approach enables new and interesting findings to surface and avoids the danger of only exploring those themes already highlighted in existing synthesis work.

 

  Parameters for analysis

 

The wider 'crowdvoting' poll elicited 129 responses, predominantly from the UK (58%) but also a fair spread of international engagement mostly from English speaking countries. 96% were working within an organisation/institution and most were working in the HE sector: US (7%), New Zealand (6%), Australia (4.5%), India (4.5%), Canada (4%), South Africa (3%) and one or more responses from 12 other countries. Some differences in countries represented by respondents working in the HE sector. Those working in school education were largely UK and India; other sectors included further/vocational education (predominantly UK), public sector (UK & US only), private sector (UK, India, Brazil, Netherlands & South Africa), charity/voluntary sector (entirely UK & US).

 

The detailed survey with the OER community elicited 50 full responses, 98% of which were HE, which is unsurprising given the HEFCE funding was HE focused. Respondents represented a spread of roles; the majority were teachers/tutors (52%) yet with a fair percentage indicating their role (solely or additionally) as managers (22%), pedagogic support (26%), researchers (28%) and OER initiate support project (26%). 10% indicated a technical support, 10% librarian/information worker, few or none in marketing or admin, and 2% were students. Responses represented people involved in all initiatives as individuals (UKOER projects and/or SCORE fellowships, residential course or workshop attendees) or as part of an institutional project (as in UKOER projects) or subject community (UKOER pilot and phase 2).

 

The survey attracted a good mix of people involved across both initiatives, as individuals (predominantly SCORE), support services, lead institutions (only UKOER), partner institutions and/or user/recipients. Only UKOER phase 1 (pilot) and 2 indicated subject community involvement. UKOER phase 3 involvement was predominantly as 'lead institution' (50%). These profiles mirror the focus/nature of the UKOER programme phases and SCORE opportunities (personal fellowship projects, residential courses & workshops).

 

Quantitative statistics of responses presents a valuable picture of current perspectives and evidence for awareness, priorities and outcomes in some areas. However, in the timeframe for this interim report, and also since the questionnaires are still on-going over the summer months, we have presented only the highlights as they appear from charting the data, rather than any detailed analysis. For instance, we have not attempted to correlate responses to different communities or initiatives (and indeed the relatively small sample size may preclude the validity of doing so). It is anticipated that qualitative data will yield the deeper understanding arising from 'softer' indicators of benefits and impact expressed through the narratives of personal OER journeys.

 

Interviews, in particular, enabled participants to recount their story, to build up a picture of what has changed since the initial UKOER pilot programme in 2009 (e.g. how they have moved from where they were to where they are now). Notably, interviews were not intended to cover technical details, although examples of technological practices do provide indicators of impact in terms of embedding. This kind of technological detail is usually evidenced through project reporting and from the JISC CETIS synthesis activity. Results concerning discoverability, including choices around granularity and hosting, also provide valuable emergent findings and evidence of organisational outcomes.

 

One focus has been the strategic level within the higher education sector, but analysis has drawn heavily on the perspectives of individuals and communities who have both benefitted from the HEFCE initiatives, contributed to outputs, and influenced open practices both locally within their organisation and collectively across the sector. Here, trends of particular interest are in terms of how OERs are changing the design of education and how OEP is being informed and informing other aspects of academic practice, such as research, student employability and business/community engagement.

 

In particular, interviews were intended to help deepen understanding by expanding on early synthesis findings and analysis has involved identifying examples of how practice is changing, for individuals, communities and organisations. For instance, the apparent shifts in relationships between academics and students (and 'users' more generally), such as their involvement in the development of OERs in terms of "pedagogic co-creation", rather than simply development of OERs for use by other teaching staff. The way open educational practices are perceived in terms of who OERs are being produced for is a key indicator of potential longer term impact. It suggests changes in attitudes towards risk that can lead to a greater and more widespread release and use of OER by teaching staff, enhanced student centred approaches, as well as acting as marketing tools leading to reputational gains for the institution.

 

 

Interim findings

 

Themes & patterns from previous synthesis/ review

 

 Throughout the first two phases of the UKOER programme, evaluation findings from projects have been externally synthesised by a team led by Glasgow Caledonian University. The team has developed and refined an evaluation framework and associated methodologies, which have contributed to the developments of reports and background data collated on a wiki platform.

 

The UKOER programme has led to increased understanding around the complexity of the open educational landscape - discussed in both the Open Practices Briefing Paper, a blog post by one of the JISC Programme managers visualisations of the wider open education landscape and the initial report on SCORE supported work. Notably, the synthesis work to date has highlighted aspects of culture and practice change (at individual, community and institutional levels) which has been enabled and supported by engaging in the release and use of OER. Another significant focus of UKOER work has been in identifying benefits and impact of different models for a wide range of stakeholders across a number of sectors, both within and outside the educational sector.

 

HEFCE's UKOER & SCORE supported work has highlighted legal, technical and pedagogical considerations that may seem insurmountable to individuals, particularly in the absence of strategic institutional support. The current review/synthesis highlights intrinsic pro-open philosophies of those involved in these initiatives and their perceived benefits of releasing and using OERs. However, although educational resources are an essential feature of the digital landscape, and one that students need to engage with, it is not clear that practitioners should focus primarily on producing/releasing open content if the motivation is to enhance access to educational opportunity and public knowledge.

 

An early report on the SCORE supported work was submitted to the OU National Advisory Board for its final meeting on 25th May 2012. The report confirmed that SCORE had achieved a significant contribution in terms of output and knowledge exchange across the sector, with some principal findings around embedding, sustainable practice and widening engagement through OER being highlighted. Aspects of SCORE's staff development aims for OER/OEPs were also commended.

 

Both UKOER and SCORE have enabled many varied and valuable opportunities to develop and test OER led by consumers and development of a network/ community of practice capable of driving sector progress going forward. In sustaining benefits and impact, a key message that emerged (top of page 11 in our early SCORE report) was the need to balance awareness raising, training needs and community engagement, reflected in any marketing/dissemination in terms of target audience and level (individual practice, curriculum and/or institutional strategies and operations). 

 

Findings to date suggested that OER may not, in fact, be the first sign of openness in educational practice. Practices other than releasing educational content may have more immediate pay-offs and a lower adoption threshold, while OER development continues quietly as – for example – materials developed for virtual learning environments become more 'open-ready' through better practices of content design. Conclusions across the synthesis findings therefore strongly suggested that future funding should address open content development and management within the wider landscape of open educational practices.

 

There is evidence of a highly engaged and expert community of practice that networks across institutions, regionally, nationally and in most cases internationally, representing a vibrant UK presence in the field. These shifts in thinking and practice, in sharing and debating, are crucial in supporting change in the culture and acceptance of OER in the HE sector. Broad indicators of success and impact included:

 

  • The degree of sector engagement & high volume of wider awareness raising achieved
  • Changes in awareness and knowledge of the OER field & capacity to find, evaluate, use & create relevant & useful OERs
  • Readiness of individuals and organisations directly funded to implement new working practices & initiate new partnerships
  • Impact of these funded 'champions' on indirect spheres of practice (other colleagues, organisations & subject/research communities impacted)

 

Attention is drawn to the summary of 'meta-trends' (p.15-16 of the report) and the potential (and probably necessary) tension between OER being promoted/ developed as a specialist activity or becoming more embedded within academic practice (research, teaching, curriculum/staff development, external engagement). A sector wide OER network linked by the concept of "professional learning" was favoured, which would tie in well with existing recognition pathways for staff and students, researchers, teachers and developers.

 

Feedback from the SCORE Board members included the need to further support:

 

  • 'leadership' and strategic thinking about OER/learning technology and setting the priorities/sustainable practice. 
  • staff development to sustain the momentum achieved, e.g. there was demand from 'aged' academic staff to keep up with pressure from students. (We had a brief discussion across the board about the assumptions about students' digital literacies here though or age distinctions.) 
  • processes & activities that inform & empower (staff, policy, strategy, practice)
  • dissemination practices (links to WP agendas, digital literacy were mentioned, beyond skills, also access & equality
  • a focus on OER that is not separated from academic practice & digital literacies.

 

Conclusions from this initial review of SCORE supported activities included a need for future OER work and research to focus on:

  • practitioner communities
  • student experience & outcomes
  • institutions and HE system
  • consumer behaviour (use of OERs).

 

Cumulative trends & patterns

 

Awareness, engagement & participation

Themes : UK to international interest, individual champions, impact on others

 

The OER survey, distributed to UKOER/SCORE related communities and networks specifically, indicates the importance of the funded work to raise awareness, link and empower individuals and communities of developers and users (both students and staff as "Open Education Practitioners"), and galvanise the institution (see also the section below on benefits/impact). When asked how far they felt people in their institution/organisation are aware of HEFCE funded OER initiatives (outside the project team members), responses indicate that this is still predominantly limited to pockets within institutions:

  • awareness in pockets (67%)
  • very low awareness (27%)
  • generally wide-scale awareness (6%).

 

Although a difficult question to ask or answer, as it relies heavily on a personal 'sense' of an institution-wide phenomenon, it nevertheless reveals that even with all the funding and the significant institution-wide engagement, the awareness is still fairly limited. Not least this indicates that there is still a long way to go in terms of awareness and maturity of organisation level OER, and open practices more generally. With regards to OER specifically, there is a danger within the community of a sense of moving beyond OER onto other higher profile and 'buzzwordy' initiatives, such as MOOC. However, the majority of institutions are actually only part-way along the OER journey, despite all these activities and funding. As a sector, therefore, there is a need to consider how this work can best be consolidated and how  institutions might be encouraged to continue to move forward with 'open' developments in a climate of financial constraints.

 

However, this level of awareness, engagement and participation remains potentially an inner circle. Respondents (129) in the wider sector poll mostly indicated they had had no or some engagement with either the OER infoKit, UKOER projects, SCORE supported activities or some other international OER related initiatives (namely OERu, University of the People & EdX). (Engagement was defined as involvement in a project, using a service or resources, attending events, supporting roles, collaborative roles.) The relatively high response rate from those engaged in OERu was the result of an email posting on the OERu discussion list encouraging these respondents. Analysis of the 38 "other" responses threw up MOOCs (5), OPAL (3), Coursera (3), DS106 (3), learnhigher (2), OERtest (2), OER Africa (2), OpenLearn (2), Creative Commons (2). A minority indicated directly funded engagement or their intention to engage. 

 

The wider sector poll data shows good awareness and engagement among the international OER community of UK initiatives. As anticipated, a high proportion of the 56 non-UK responses come from those involved in OERu (12% indicated directly funded engagement and 27% some engagement), although there was also a significant engagement with the OER infoKit (20%), UKOER projects (14%) and SCORE fellowships/events (9%). Comparing UK respondents with non-UK for overall responses, we can infer that SCORE's impact is felt more strongly within the UK. 15% of respondents had no engagement with any OER initiatives, either those named or "other". This shows that the survey achieved, to some extent, its aim of reaching beyond the OER community.

 

SCORE and UKOER had different aims and models of engagement. Both appear to have achieved their aims, but the differences in engagement models show up with SCORE showing a much higher ratio of "some engagement" to "direct funding" than UKOER, but UKOER showing greater absolute numbers of both "some engagement" and "direct funding".  The Infokit shows both better ratio of "some engagement" to "direct funding" and greater absolute levels of "some engagement" than either UKOER or SCORE.

 

Inter-connectedness of the OER community

Themes : networking, the role of social media, influence of/on subject communities, collaboration/sharing

 

The majority of OER survey respondents indicated their primary sphere of practice to be as a member of an educational institution (81%), but there was also crossover with both wider and specialised communities and networks. Research and subject disciplinary communities also featured largely (47% & 39% respectively) as well as reasonable representation across international networks (27%), professional/learned communities (29%) and social media networks 36%). Subject disciplines of respondents were predominantly education (52%) and languages (32%), but included both sciences and humanities as well as generic skills. Education and CPD are likely high due to OMAC strands of work, but also a focus of much teacher education on OER production.

 

Many other categories, such as research skills, information literacy etc. might legitimately be included in 'generic skills' resources, which may have further dominated the spread. In previous reports, we have shown that generic OERs (i.e. those that cross disciplines) present attractive options for institutions in terms of efficiencies of sharing generic materials across faculties. Furthermore, their low perceived value may appear low risk and 'an easy win' in terms of the perceived requirement for high quality in subject content. These subjects therefore, unsurprisingly, featured strongly.

 

It is interesting to note that from the whole list of subject areas, not one subject was left blank, suggesting OER was covered in all subjects. A text analysis of responses concerning the perceived impact of respondents own OER work within subject disciplinary community of practice revealed 'awareness' as the most important term (20%), alongside 'practice' (14%), 'raised the profile' (8%) and 'students' (8%). In the free comments, there was some evidence that the UKOER/SCORE activities had introduced new ways of teaching in the disciplines, particular sub-disciplines (e.g. media, photography), with an undercurrent of repurposing ideas or producing bespoke versions. Although there remain concerns and challenges, some felt OER was gaining acceptance and influencing course development processes with greater sharing across traditional boundaries. "Sustainable models for OER as a service" was also mentioned by at least one respondent with several indicating that other OER initiatives and mentoring schemes within their organisation had developed as a result of the HEFCE funded activity.

 

Open educational practices & culture 

Themes : organisational positions, OER release/use/re-use, attitude to risk/competition

 

Open educational practices (OEPs) as users/learners appears more prevalent than OEPs as producers, even among the more OER-community respondents to the wider poll.  The most common practice as a producer is "I consider myself to be an open scholar (making content openly available and collaborating openly to further research)".  It might therefore seem surprising, and says a great deal about prevailing hierarchical culture in HE of teachers vs students/learners, that by far the smallest category (30%) is "I design courses where learners contribute to public knowledge resources". (We note, however, that one of the open comments raised ethical concerns about expecting fee-paying students to contribute their outputs for free.) The responses may stem from how the question is asked (and in a future iteration of the poll, we might include an option for "I encourage learners to use OER" in order to get a better handle on this observation).

 

Comparison with the segment of 19 respondents not engaged in OER initiatives, as might be expected, adoption of OEPs is generally lower than for respondents overall. The profile of practices also differs, showing a much more marked prevalence for OEP as users rather than producers for those not engaged in OER initiatives than for the respondents overall.

 

For those who have received SCORE or UKOER funding (24 respondents), the profile is markedly different.  Interestingly, use of OER ("I use open content freely available on the web for my learning") is lower than for respondents as a whole, as is participation in open courses (slightly). Production of OER appears to be higher than any usage of OER, as might be expected since much of the UKOER funding was for OER release. It occupies first spot along with use of open source software (and we will need to check back whether this is significant or simply that high usage of open source software was associated with the conditions of the JISC funding).  Along with production of OER, facilitation of open courses is also significantly higher for this group than for respondents as a whole. The practice "I share ideas and examples of my teaching practice on the open web" shows a slightly lower rate than for respondents as a whole, in contrast to the greatly increased response for releasing material as OER.  This may indicate an increased awareness among those engaged in this funding stream of the importance of licensing, i.e. that they are formally releasing as OER rather than informally making materials available on the web.

 

Comparison of these findings with that for respondents as a whole, or even more with those not engaged at all, shows the impact that HEFCE funding has had in shifting practice towards staff as open producers and facilitators rather than simply as consumers. However, this shift may be one that actually entrenches existing cultural practice, as implied by one of the free text comment: "There is a major shift in mindset to be overcome if more educators and institutions are to engage in OEP and with OER. The sense that 'we are what we produce' continues to prevail in terms of educational materials and resources, and until we can disassociate 'the material we produce' from notions of our perceived worth as educators there will continue to be fear of OEP and OER and the implications for reputation, job security, and what the role of an educator should be". The implication of such a view is that use of open materials marks a greater shift towards open practices than is releasing open materials.

 

However, design of courses where learners contribute to public knowledge resources remains low and is, indeed, slightly lower than for respondents as a whole. It is debatable, at this stage, whether this finding highlights a facet of UK culture and the perception of learners/students in the UK. Comparison of non-UK and UK does indeed show a marked difference in the practice of designing courses where learners contribute to public knowledge resources.  Rates for those involved in UKOER/SCORE/OERinfoKit are slightly higher than for UK respondents as a whole suggesting that there might be some shift in this practice also towards production. Comparison of non-UK and UK also suggests that rates of adoption of OEP outside the UK are higher than within the UK, at least for respondents to this survey.  The general shape of the profile remains similar, however, and as noted above, is in marked contrast with that of those receiving funding from UKOER/ SCORE/ OERinfoKit.  However, this data may be skewed by the high proportion of the 'non-engaged' who come from the UK (15 out of 19). Excluding those not engaged at any level in OER from the UK sample still leaves UK engagement in OER looking lower generally than that internationally, especially as users of OER.

 

The picture of institutions/organisations' position in relation to OER release, use, open educational practice (OEP) and open licensing reflect more the focus of the funding. What is interesting to see is that OEP seems to be fairly well established in both documentation and institutional commitment, even though we only really started talking about open 'practices' last year. This therefore appears as a real move forward from thinking specifically about content (OER), although it may simply be a term picked up by the community, because of involvement with OER funded programmes using it. However, there is evidence also of growing interest now in open assessment and open courses. This momentum is also evident in the number of institutions currently investigating open courses as a result of high profile MOOCs (Massive Online Open Courses).

 

Overall, the survey responses offer a very positive picture of engagement from the institutions represented. It would be very interesting to see just how institutions that have not been involved in funded initiatives compare. Nevertheless, our sense is that the data shows some real impact of HEFCE's funding in OER, particularly in relation to institutional documentation changes and developments.

 

Legitimate barriers in creating & using OER

Themes : the cycle of implementation, access, co-creation

 

In the wider sector poll, similar issues appear to dominate both individual and institutional barriers, notably lack of awareness of OER and their benefits, lack of coherent overall institutional educational strategy, and (by inference) lack of supportive workload planning. Legal concerns, although high, are not as high as might be expected and are not one of the major barriers, especially at institutional level (which may indicate a lack of institutional awareness of OER issues).  Concerns are relatively higher (compared to lack of institutional strategy and support) for those working in the FE/vocational sector.

 

Similarly, quality issues are not a major concern either for institutions or individuals (less important than legal issues). However, the evidence from poll responses indicates that insufficient digital literacy is a major barrier for individuals, coming just below the institutional factors mentioned above.  It leaps way into first place for those working in the school sector, and is also just first for those not based in the UK. Looking at USA and New Zealand (the other two countries with a significant number of respondents - 9 and 8 respectively), shows interesting differences, especially for individuals, with "does not fit with existing work practices of staff" becoming the major barrier in NZ and equalling lack of institutional support/strategy in USA. This suggests that, as might be expected, the barriers depend on the national character and practice of HE.

 

Perceptions of barriers, both individual and institutional, are similar among respondents who have received HEFCE funding (UKOER & SCORE) as among the overall respondents. Interestingly, perceptions of barriers among those with no engagement with OER initiatives appear similar to perceptions among the overall respondents - the same barriers in top positions, led, as might be expected, by "lack of awareness". However, for this group, technical challenges appears as an institutional barrier that is as important as lack of strategy and institutional awareness of OER. One conclusion would be that in the UK much more work needs to be done with institutional senior managers, and on raising the digital literacy of individuals.

 

In our detailed survey with the direct OER community involved in UKOER & SCORE funded activities, unsurprisingly, barriers that featured highly were:

  • time to adapt and repurpose
  • legal aspects and licensing, and
  • OEP not fitting with current work practices of staff.

 

As with the wider sector poll, we might have expected to see more responses around quality - although in the survey, this was split this into (i) concerns about quality of OERs out there to use and (ii) concerns about their own materials being released as OERs. Quality was a significant issue during pilot phase of UKOER and this may indicate that increased experience with OER allays some of those concerns. Nevertheless, a significant number still indicate that it is difficult to locate relevant OER, indicating either a potential lack of resources or issues around discoverability (due to limited meta-data, time and/or skills). Lack of digital literacies does feature highly as a barrier too, although our first few interviews suggest that finding/evaluating quality OERs is a time issue not a skill issue and likewise many staff avoid releasing OERs due to the time involved in making them sufficiently polished (reputational concern) and fully compliant (legal concerns).

 

There remains a view that lack of institutional support, strategy or investment is a barrier. It is important to note that these respondents may be responding with an overview of how barriers have affected staff in their institutions as many of them have had to expend considerable energy raising awareness, and supporting staff to engage and change practice. UKOER project teams are very aware of the barriers and have been most articulate about this throughout the programme. 

 

When looking at this question in more detail, it is interesting to see that respondents involved in the pilot phase of UKOER identified the same top three barriers equally. Phase two respondents however had legal aspects and time factors as equally high, but awareness of benefits much lower. This perhaps reveals that phase two work built very much on the pilot phase work and some of the work around raising awareness of benefits had started to be successful. SCORE (fellowship, workshop and residential) respondents reflected the same top three as the overall group. UKOER phase three, on the other hand, broadened activities to other sectors and they report 'lack of awareness of benefits' as their top barrier by a significant amount. 'Time to adapt & repurpose' was replaced in the top three by 'lack of institutional investment'. The latter may reflect increasing financial constraints beginning to impact on work outside what is perceived as core work.

 

Involvement of students & impact on learning

Themes: types of student partnership, types of learners

 

The OER survey specifically asked about the capacity in which students had been involved in the OER initiative. The responses revealed the wide range of ways that students had been (and are) participating. This is being investigated at much more depth in UJKOER phase 3, so this is an area we are likely to be able to talk more about this by end of the review in our final report. Currently, however, there is evidence of students' involvement at multiple points: creating OERs (22%), evaluating OERs (41%) or simply as recipients of OERs (32%). A smaller set of responses indicated students were part of the project team (16%), students as researchers (11%) and/or students on open (or partially open) courses (9%). However, nearly 30% of respondents answered N/A, indicating that students had had no involvement in their OER activities.

 

When asked about impact on students/learners, many respondents indicated it was "too soon to tell". However, the long set of open comments provide many early indicators of a positive response to, and interest in, OERs, garnered anecdotally/informally from feedback in teaching/class, student support and discussions with colleagues as well as more formal student focus groups and surveys. Evidence suggests students/learners are gaining confidence through their engagement with OERs, including greater confidence in their learning, higher use of online 'open' resources (e.g. YouTube views/followers), enhanced student projects, collaborations and shared initiatives, including internationally, such as blogging and OER editing/production. Evidence of students' positive engagement with OERs included emails to tutors about their usefulness.

 

This positive impact is equally true of staff as learners on professional development programmes, with impact on staff awareness both of OER licensing and copyright issues and of the benefits of sharing. This has an indirect impact on students, as improving the practices of teachers also indirectly impacts the experiences of their students. In fact, the majority of user evaluations mentioned by OER survey respondents were with staff within their institution/organisation (55%), but also students formally enrolled on a course (40%); some were people outside the institution (36%), informal learners (21%) and/or general users (19%). However, only a low proportion (22%) indicated they had carried out any follow-up evaluations of the use of OER released as a result of funding after the funded period, but comments suggest in many cases that evaluations are just getting underway.

 

Motivation & benefits

Themes : professional development, pedagogy, sharing/cost-benefits


Our detailed survey with the directly supported OER community indicated a range of motivations for being involved in UKOER and/or SCORE initiatives. However, the predominant response (78%) was that of being "experimental' - to investigate the potential of OER". This is not surprising given HEFCE funding was aimed to investigate the potential of OER release. However, coupled with other motivations such as building on previous OER work by individuals, departments & institutions, and availability of funding, the finding is significant since it recognise the value of HEFCE funding in this area.


When asked about their top 3 perceived benefits of releasing and encouraging use of OERs, survey respondents predominantly indicated:
1. increased access (55%)
2. enhanced pedagogy (49%)
3. increased sharing between educators in the same discipline (41%).


There is little comparative data as to whether these potential benefits (in hindsight) were perceived in the same way before the initiatives ran. However, UKOER participants have reported that OERs present an ideal opportunity to enhance pedagogy and these results really confirm that previous finding. This is a very strong argument to support engagement with OER release and use. Early interviews suggests that some benefits may not have been realised in practice. While it is interesting to find increased access for learners and sharing amongst educators standing out as significant as an overarching benefit (and important motivation for releasing OER), there is actually very little evidence that learners are using the OERs or educators are in fact sharing (and using/repurposing) each others' OERs. The first two phases of UKOER had significant involvement of subject communities and this is most likely to have really supported this kind of sharing.


Others indicated motivation around the potential benefits furthering OER infrastructure (24%), cross-sector sharing (24%) and institutional reputation gains (22%); with a lower proportion indicating supporting disadvantaged learners (%), increased efficiencies (%) and student recruitment (%). A very low percentage suggested increased personal reputation as a benefit. However, the majority of respondents reported personal benefits in terms of professional development, with increased understanding of issues around licensing featuring strongly. As we evidenced in phase 2 (and reported in the Open Practices Across Sectors Briefing Paper), many respondents reported as benefits "an increased awareness of wider sectoral approaches and increased collaboration across sectors", although UKOER phase 3 respondents reported this most significantly - which reflects the cross-sector focus of activities.


Impact & sustainability

Themes : sustained institutional policies & new strategies, support/collaboration


When asked about sustainable development, 55% of respondents indicated that their institution/organisation had continued support of the OER initiative and vision since project funding ended, 8.5% had not. For the other 36%, the project had not yet ended.  This is also reflected in phase 1 and 2 reports - although a lot of these have had ongoing funding from the programme. Overall, this does appear to indicate effective sustainability approaches in the majority of cases, with some exceptions where institutions had not (yet) taken OER developments or strategies further. However, examples from respondents included:

 

  • Financial support - e.g. £10k "to help the institution to share understanding of high quality approaches to ensuring that the institution complies with legal good practice".

 

  • Development/technical support - e.g. "to implement recommendations" arising from the project, such as staff time allocated, regular monitoring and evaluation of key sources of OER in our specific areas of interest, "The fact that the repository now exists and has replaced a myriad of smaller schemes and (less open) ways of distributing resources has been welcomed widely."

 

  • Partnership support  e.g. "supporting opportunities to apply fo further funding ... to develop bespoke versions" for specific purposes, work allied to other ongoing initiatives in the institution.

 

  • Embedding support e.g. "work to continue to explore, support and exploit the potential offered by OERs is included within the strategic objectives", "systems for supporting teachers and providing teaching materials for reuse", "Contributions form tutors have been coming in, and new collaborative practices emerge, but the time-factor and worries over copyright are probably the main limiting factor."


With regards to the wider sustainable outcomes, some of the open comments from our wider sector poll suggested a need for better and more strategic coordination of open practice initiatives, beyond the local HE institutional level, if HEFCE's intervention in this area is intended to have a global impact. Key drivers mentioned were the need for business models for institutions, senior management buy-in and development of assessment and accreditation that align with open practices by academic staff.  However, the OER survey and interview data (collected so far) reveal that by far the greatest driver for sustained activity in this area has been the enthusiasm of staff. If this is the key aspect then sustainability may be precarious if staff leave or enthusiasm wanes or shifts focus. This answer, and findings from our earlier SCORE review, tends to imply the notion of 'champions' who will take the ideas and awareness forward. 

 
As expected, respondents also identify senior management buy-in as important, mirroring findings from the pilot UKOER project. UKOER phase 3 had higher responses than other two phases with regard to increased awareness at institutional/organisational level. This is epxected as some institutions had been through one or more iterations of UKOER by then. SCORE respondents show very similar pattern to UKOER 3 choices - increased awareness and enthusiasm of staff being very important across both groups.


The main factors hindering sustainability in OEP/OER appears to be the desire and time to first put in place good compliance with high quality procedures in relation to staff development, clearing copyright and licensing consent. Some technical issues remain, but appear less prominent in terms of slow progress. The use of both local OER repositories and systems and national services, like Jorum is growing, yet there appear to remain challenges with finding relevant OERs.


Evidence for involvement in the HEFCE funded UKOER/SCORE work having an impact on institutions/organisations, overall, was very encouraging. A significant percentage of responses indicated increased institution-wide awareness, changes in culture and practice, collaboration (both internally and with external partners & wider sectors). There were also some excellent reports of new and adapted strategies and policies. While a small percentage reported little or patchy impact across the institution, the findings are a significant indicator for the most part of institution-wide change and commitment (impacting on sustainability of funded activities discussed above).

 

Some more poignant comments around sustainability of OER release from OER survey respondents include:


"I believe that the direction of travel is permanent, we will not go back from here, and the institution isn't anti-OER, just slow to respond. However it is responding as an organisation (not as a cottage industry) and my group has been commissioned for £10K to work with staff and schools (departments) to educate them in using OER from elsewhere and having a high standard of copyright compliance. It is a first step."


 "[over the past few years]... we have sustained a gradually increasing portfolio. In 2012-13 we intend to expand the number of Open classes further within the department and to offer a model for their development [to others]."


"[Sustainability] is both a question of technical digital literacy skills and understanding the specificities/ and philosophy of OER approaches .... [it's the] time input [for staff development and] to develop high quality OERs, lack of recognition/reward for doing so, [as well as] some technical and licensing legal issues." (mix of responses collated)


Indeed, this reflects what projects report across all phases, strongly suggestive that policy change and strategic buy-in is a very important part of the institutional journey (and perhaps 'OER/OEP maturity') with clear evidence of institutions commitment to continuing it at an organisational level.

 

Conclusions - the UK OER journey

 

Evidence from this interim analysis of findings from surveys, initial interviews and our earlier UKOER synthesis and SCORE review clearly supports the impact of HEFCE funding in providing a real impetus to the journey of both individuals, communities and institutions in OER and open educational practices more generally. Due to the scale of cultural, practice and institutional  transformation, findings clearly suggest that the UKOER and/or SCORE funded work was part of a bigger, longer movement. However, a significant majority of direct beneficiaries cite the HEFCE-funded support as a key critical success factor, as it enabled them as individuals, and in many cases institutionally, to move things along much quicker, building on earlier work and/or leading to further policies, strategies and developments in practice and IT/library services/support.

 

As we anticipated, there is significant cross-over between the various UKOER and SCORE findings. Many participants in surveys and interviews constitute both communities; many of the SCORE Fellows have been involved directly and indirectly in UKOER projects across one or more phases. Unsurprisingly, therefore, findings from our earlier SCORE review in May reflected a similar holistic picture to previous synthesis of the UKOER programme. For instance, there was significant evidence of the benefits at individual and organisational levels of engaging with open practice and OERs across several sectors in the UK, including various educational sectors, NHS and government sectors, 3rd sector, public sector, and private sector (including publishers).

 

The professional learning opportunities afforded by participation in UKOER and/or SCORE are particularly relevant in relation to evidence of the impact on individual teacher's expectations of e-learning and learners' approaches to learning (recent JISC e-learning programme work;  Sharpe, 2010; Margaryna & Littlejohn, 2011; McAndrew,2011).  Meta- trends identified so far, and successfully contributed to by HEFCE supported work, include:

 

  • Focus shift from OER production to use of OERs in professional practice (teaching and learning practice);
  • ­Shift in emphasis of OER production  from producer-led to consumer-led;
  • ­Shift in discussion of OERs from production/ practice issues to consideration of meta-level factors such as epistemic approaches and beliefs.

 

This initial review of SCORE, surveys and interview findings reveal a pattern of change in institutional culture and practice and movement towards new or adapted policies. In particular, from the SCORE supported work, we noted changes with regard to the roll out and embedding of OERs which reflect findings from UKOER. For instance: 

 

  • integrating OER as a sub-set of existing educational strategies (such as curriculum innovation, e-learning, digital literacy, professional development 'good practice', quality enhancement initiatives);
  • informing and problem-solving technical developments and library/publishing practices; 
  • increasing the volume and culture of sharing and reuse (with the effect of course teams gaining visibility on the web, collating digital resources as one mechanism for collaboration);
  • blending in assessed work (as a means of promoting quality & trust and countering plagiarism);
  • expanding the boundaries of study skills (e.g. by moving beyond text skills, citing non-text based resources).

Overall synthesis activities so far indicate that HEFCE funding in this area has had significant impact on culture and educational practice, sector awareness and understanding of OER release (and to a lesser extent use), and on defining the benefits of various institutional, community and individual models. Key questions remain as to who is using/reusing OERs (see also JISC funded study in 2010-2011 on the Impact of OER) but both SCORE and UKOER activities are clearly addressing issues around:

 

 

Different journeys are evident across institutions as well as disciplines, mirroring findings with other programmes, particularly JISC's Digital Literacies work. Open publishing/ open data/ open resources appear as a more "naturalised practice" in some subject areas, and completely at the starting line in others.  Some institutions have clearly articulated openness as part of their mission and ethos for informal learning and sharing, while others use OEP/OER as a collaborative tool for external engagement and reputational/marketing benefits. For some, OER/OEP is an 'add-on' rather than integral to institutional culture across traditional boundaries, or as a means of organisational and educational transformation.

 

These differences are likely to stem from specific projects, groups or communities championing initiatives and pockets of innovation within subject centres /departments, which in turn drives change in institutional policies/strategies, usually in keeping with the main ethos of the organisation and vision of its most senior managers. In some institutions/organisations, there is a far longer track record of open practices and partnerships with public and business stakeholders. The general trend is that OER/OEP is "adding on to what is already happening in the institution, rather than necessarily fundamentally changing how they do things. Final comments on our OER survey suggested a gratitude for the HEFCE funding to seed intellectual capital in this area, particularly the legal compliance requirements of releasing/using OER. Major educational themes emerging is the interest in exploring and making more concrete links between informal learning (use of OERs) and formal learning, new models of open delivery and flexible participation.

 

There are strong tie-ins with the JISC's digital literacies programme and business & community engagement (BCE) work, with a view from the OER community of a need to test new HE business models for external engagement, links with open access research and publishing, and provide more recognition and reward for staff working on these initiatives. Future directions, priorities, gaps, research areas is an element of what is being explored in interviews and will be reported in the final review. However, several messages that emerged from interviews focus on inclusion.

 

  • First, that OER/OEP not become a clustered activity but one of much more 'open' dissemination, engagement and participation, even for individuals, groups and organisations not directly funded.
  • Second, that universities look to ways to bridge transitions for learners, between themselves and schools/college, through partnerships with external organisations and businesses, through greater outward facing opportunities for OERs, credit transfer between formal and informal 'adult' learning ('open badging').
  • And third, that OERs are not just about publishing content or using someone else's material, but can be about extending learning design - creatively thinking how to add value to learners, repurposing an OER and sharing and using it in novel formats and new contexts.

 

In this regard, Higher Education must look to sustainable external partnerships, collaborative development with professional bodies and commercial (IT/learning) organisations to "unlock existing materials" and "enrich what they do for learners", to buy into how open practices and OERs add value to a wider public awareness and participation in HE. There is a need to develop strategies and encourage a vision of openness to reflect and build on these. Consumption vs publishing, individual vs collaborative partnership, formal vs informal, local impact vs international reach are all interesting 'maturity' indicators to explore how we move forward, nationally and organisationally, in the growing, global OEP/OER space.

 

 

OER Synthesis & Evaluation Team

31-07-12

 

Sector engagement & dissemination

 

 

 

 

 

 

 

 

 

 

Comments (0)

You don't have permission to comment on this page.