Pilot phase

Phase 2

Phase 3

Demand

 Tracking emerged as a priority area for pilot programme projects and continued to be an area of significant activity during this phase. This links to sustainability and the provision of evidence around use of the OERs. Most of the platforms adopted by projects included some form of tracking mechanisms, but Google Analytics stands out as the key tracking software used by projects. The SCOOTER project invested much effort into Search Engine Optimisation to enhance discoverability and took tracking very seriously. They produced guidance for projects SEO Guidelines and SEO "How to" resources and postulated that returning visitors may be one indication of re-use.

Effect of collaborations

Making clear the original attribution and quality-checked (or not checked) status of OERs in the collection, emerged as a clear user need (C-SAP collections, Triton).  This was particularly so in the Triton project where users saw the Oxford and Cambridge logos on the Politics Inspires blog and tended to assume that the OERs carried the Oxford or Cambridge brand.

...  there was an (inaccurate) assumption that because of the Oxford/Cambridge affiliation that the OER resources generated through the Xpert widget had been reviewed by the institutions (C-SAP user testing of Politics Inspires site)

Negotiating access rights to the blog hosted by one institution but with partners from another also proved difficult (Triton, Delores), as was found previously in the pilot phase projects. The solution adopted by the Triton project was for an access policy generated by an oversight group comprising members from both institutions setting clear standards and guidelines, with access rights managed by Wordpress (Triton).

Issues arose in the static collections in evolution of a common style for describing the resources, solved by the Oerbital project through collaborative development on their project wiki.

 Participatory development of OER through open platforms offers an agile approach encouraging community engagement and early feedback

Collaborative development and sharing can ensure that OER are pedagogically appropriate.

Involving external partners enhanced quality and accessibility of the OER and led to increased understanding across sectors of how OER and OEP can support learning in particular subject disciplines or professional areas

Involving intended user groups in the development and testing of OER can ensure that they are pedagogically and technically accessible

Trust has been an important issue in quality processes, with tensions apparent over the extent to which consortia or institutions are prepared to rely on the professionalism of academics either as developers or reviewers of resources.

In the end you have to rely on the professionalism of colleagues; mutual trust is a delicate commodity and it will not be developed through an emphasis on standards, unnecessary quality processes etc.(OCEP final report)

Commercial publishers were keen to be involved and new collaborative partnerships have begun to investigate some of the tensions between OEP and commercial publishing - work is at very early stages and may need additional investment to work through some of the challenges highlighted during project activities

Several projects worked with commercial publishers (HALSOER, Great Writers, PublishOER, ALTO UK) to establish new collaborative partnerships and to begin to investigate open publishing. This sector was keen to be involved and allowed some investigation into the 'synergies and tensions between open education and publishing'. (HALSOER final report).

Initial discussions by the working group focussed on the practicalities of linking or embedding content from publishers as OER into WikiVet or other teaching material. The consensus was that simply transferring digital text from a textbook into an OER was of limited value. The richer media assets such as images and diagrams along with assessment resources were considered to be of greater importance. In addition the publishers suggested that auxiliary digital resources provided with a textbook such as videos, web pages and interactive media might actually be made open instead of protecting them behind a password. (PublishOER Final Report)

The overall feedback from internal discussions at Elsevier was that decisions were made on a case-by-case basis for resources of the sort considered in the case studies. No global policy could be applied. This obviously made progress slow. The main reason for allowing open access resources to be used and reused was the need to drive sales directly or indirectly, coupled with a consideration of the wider benefits of resultant partnerships. Another major issue was that CC licences were disliked, as there was a perception by some staff at Elsevier that by applying such licences lost control of their content. (PublishOER Final Report)

Making OER available

Many OER are aimed at multiple user groups. At the same time, a number of projects have noted the importance of placing materials near the intended user communities. Syndication of materials through RSS has generally been the chosen solution to this problem, allowing subject and skills communities to harvest a selection of OER material and present it in within or near their preferred systems.

Many projects used multiple channels to publish and publicise OERs, often using syndication to ensure that a single authoritative version existed for updating, any necessary take-down, ongoing refinement etc.

Institutional repositories can be key drivers for open content release, but they are not always the best hosting solution and sometimes can present problems, such as poor management of complex Learning Objects, poor integration with other institutional technologies, resource duplication, branding and unhelpful presentation. Institutional policies are required to clarify and support processes around branding, hosting, quality assurance, and IPR but these can be perceived as barrier to release. Many projects also used social media sites to host and promote content and clear policies on these are helpful. Robust take-down policies are essential to provide some security and reassurance if the status of a resource changes or is challenged. This is clearly linked to issues around IPR and quality.

The increasing development of institutional repositories for learning and teaching resources has had a significant impact on the Pilot Programme. Whether these are new repositories, or expansions of research repositories, institutional strand projects and some subject strand partners chose to utilise institutional repositories as the primary host. This has had an impact on institutional practice and the development of OER support mechanisms and ultimately on sustainability of project activities.

From the beginning we decided to embed our new OER repository, called Open Exeter into this wider infrastructure. The University is now developing another repository for research data. All four repositories are based upon DSpace and so sustainability is factored into them collectively (OpenExeter final report)

In exploring the fitness for purpose of existing repositories, projects are considering issues such as updating, versioning, tracking and management, preservation and archiving. With some repositories, different degrees of openness are available to allow depositors control over how widely they share content. While JORUMOpen remains the repository of choice for the programme, it is recognised that the open content ethos means that specific hosting solutions matter less than the general approach to making content available. Also there are some concerns that the development path of JorumOpen may have been out of sync with project requirements.

Some individual projects expect to rely on social networking and the user community to keep content and indices up to date - normally one of the stated contributions of established repositories.

 Institutional repositories and strategic approaches to content management are often enhanced by engagement with OERs but can also act as motivators to make content more open, both inside and outside the institution. Links to institutional research was noted in the pilot phase and continues to be seen as a motivator.

The RAE used to measure impact by the quantity of research publications in reputable journals; now [with the REF] there is a new impact in the open arena via Google, and it provides a great marketing tool. One of our resources now comes top of a Google search. We are repurposing formerly turgid research into something usable that gives you credibility that other materials could never give. (Ripple)

Sustainability of links to materials was noted as an ongoing problem.

What has been significant, and continuing trends that emerged during the pilot phase, is the number of projects who felt the need to present the resources with surrounding context to cater for different audience needs (other teachers cf learners), such as accompanying information providing metadata, licencing information and pedagocial context. Several projects did anticipate potential users who would want to add their own context - in a sense 'unknown' users. A number of projects produced materials for specific groups of learners - particularly non-traditional remote learners and they sometimes developed materials for one particular target audience, with an eye to potential secondary audiences, including 'unknown audiences.

As I always work very much with the user of resources in mind it has been a challenge to know how to adjust materials and pick what will work best ….. Adapting to EITHER others who will be working to help academic staff to develop as teachers OR writing directly for the teachers themselves so they can work on their own CPD ‐ that has been my dilemma.” Dr. Rosalind Duhs, CPD4HE teacher‐developer

Several OMAC strand projects included student generated OERS. This might be expected due to teachers being their primary audience, but a few Release strand projects also included student generated content.

We soon realised that we had a great opportunity to engage our current postgraduate students in creating the resource. They were keen and so we were able to video many of the sessions in which they discussed the issues considered in each study area. We videoed ourselves discussing issues of pedagogy with them and we videoed colleagues who we felt had important things to add to the study topics.  So a good deal of the material in our OER is made for purpose film. This was a major development of the OER as we realised how interesting it would be for teachers and supporters of learning who often work alone to hear and see their peers discussing the enhancement of practice. (RLT Performing Arts)

In some projects, such as ChemistryFM and OpenExeter, pedagogic approach has been a driver for production of OER, rather than vice versa. There is conflicting evidence on student willingness to release their work as OERs. Some projects have found students very ready to contribute materials, as part of their course, for pay, or entirely voluntarily. The Otter project, however, reports a third of students as saying they would not be willing to turn materials such as lecture notes into OERs and share them with other students. 

'It was felt that OER has a significant part to play in co-created activity, getting away from the model of the University as producers and students as consumers towards students being co-producers.' OpenExeter Project Final Report.

Several project teams put energy into developing a shared understanding within their institutions, subject and sector communities of how learning and teaching might be supported by open educational resources. Some went so far as to consider what an 'open pedagogy' would look like (ADM and C-SAP). 

“I have learned a number of processes from the design workshop, we have done some of it but not all because of time frames. I would change how to write libraries and programming and would start from the beginning again. The way we approach things has changed – in terms of timescales and things – I have definitely learned some things. It has definitely changed the way I work and measure time, and how I structure my work. I think I have changed the process of the way I approach clients too.” (Project team member)

Many OER are aimed at multiple user groups. At the same time, a number of projects have noted the importance of placing materials near the intended user communities. Syndication of materials through RSS has generally been the chosen solution to this problem, allowing subject and skills communities to harvest a selection of OER material and present it in within or near their preferred systems.

Many projects used multiple channels to publish and publicise OERs, often using syndication to ensure that a single authoritative version existed for updating, any necessary take-down, ongoing refinement etc.

Projects took various approaches to hosting and syndication of resources, and variation can be seen between the three strands. Subject strand solutions can tentatively be related to their situation on the two axes identified in the strand as 'integrity = reusability' and 'wild' = 'community-based' (see Subject Strand - Developing Managing And Sharing OERs), whereas Individual strand solutions tended to be based more exclusively on the intended audience (see Individual Strand - Developing Managing And Sharing OERs. Institutional projects focused very largely on institutional repositories or learning/content management systems.

Platform choice emerged as one of the most interesting technical aspects of this phase with many projects depositing in several places to present materials at different levels of granularity and to support varied contexts of use for different users (teachers/students). Whilst most projects utilised repositories and content management systems for management and curation purposes, several also added context and guidance by presenting them in other formats. It is notable that four projects from the OMAC strand chose to use the Open University labspace to present their whole course (RLT for PA, ASSAP, Learning to Teach Inclusively and IPR4EE). Even though RLT for PA included the course on the open labspace and deposited into JORUM they also presented their materials on a website which facilitated different kinds of use...

Although each area of the site is carefully structured to allow for increasing levels of familiarity and ease, it is entirely possible for individuals to mark out very specific pathways and ‘journeys’ in accordance with specific research aims. By switching from one room to another it is possible to access different perspectives on similar material depending on the vantage point of the contributor. In this respect the resource offers every opportunity to develop informed, dynamic and multi-faceted teaching practice that is alive to different conceptions of knowledge – experiential, embodied, theorized, intellectual, emotional, intuitive etc. There is no attempt to order or prioritize the material into specific hierarchical categories; indeed it is the imbrication of all forms of information that makes the site so rich and enabling. David Shirley, Manchester Metropolitan University RLT for PA

The first technical choice for a collection was that of an appropriate platform for the collections.  The majority of projects used a Wordpress blog because of its open ethos (C-SAP), strong developer community (C-SAP, Triton), usability (C-SAP) and large existing user base (Delores). However the Oerbital project used Mediawiki as a catalogue and to prevent duplication, while OF (GEES) developed its own system and map interface.

 OER release in a variety of formats and across multiple platforms improves discoverability and accessibility and allows presentation at different levels of granularity

OER have been made available through Jorum, institutional and community repositories and resource banks, content management systems, Moodle courses, and openly on the web through a variety of interfaces. The use of persistent URLs for OER meant that they could be aggregated through feeds into a variety of services.

Providing alternate pathways through the material increases potential use by different groups

Imaginative use of feeds (particulaly RSS) and sometimes a good understanding of metadata have resulted in some excellent solutions to making the resources available to a wider community of potential users. A few projects deposited separate copies.  A number of projects explored third party hosting solutions such as slideshare, flickr, scribd, youtube, and had variable experiences. Web2.0 sites are at different stages of development and inconsistent in the media they will support (e.g. Flash animations), the terms of hosting, and the openness of access. There is an issue of resource ownership in relation to some or all third party web sites, e.g . in Flickr the person uploading the resource is, by default, the resource owner. Many sharing sites require consent to conditions of use statements that may violate CC and other licences. However, the use of web 2.0 services remains attractive to support flexibility, portability and accessibility, particularly by learners and have been used very imaginatively to publicise and encourage use of resources released by projects.

The most successful dissemination has been the embracing of social networks and OER delivered through mobile technologies... The mobile technologies are adored by a large number of students, if you engage them through something that they use every day the material is more likely to be assimilated as part of their learning experience. (BROME final report)

Feeds out from collections sites can place OERs where stakeholders are and raise awareness. For such push discovery standards are needed for describing OER.  The Delores project developed such a standard xml schema for rss feeds from its static collection and from resources discovered through its sux0r filter.  OF (GEES) recommended standard ways of describing location for fieldwork resources which if adhered to by contributors will aid dynamic collection and learning pathways.  C-SAP collections wrestled with the difficulties of providing consistent descriptions of video resources.

Triton developed Wordpress widgets for feeds into its site to help keep the site up to date, drawing users in by acting as an information portal rather than just a repository

'There is a clear model now for content to be stored locally and surfaced nationally. Institutional repositories first, then surfaced via links in subject portals and national repositories.' OpenSpires Project (University of Oxford)

Many projects have developed a model of depositing once and then utilising a range of mechanisms to support discoverability through more accessible sources. Several projects, across all strands, used institutional or community repositories as their place of 'primary' deposit. Project funding has helped to strengthen the case for using institutional repositories for learning and teaching resources, although several institutional strand teams preferred to use content management systems, which they felt provided better support for complex learning objects. Whilst institutional mechanisms can provide stable and supported spaces to deposit, many institutions have not even started to consider open access for learning and teaching materials. Individual strand projects experienced difficulties around deposit access for non-institutional collaborators and reluctance to support collaborative forums associated with dynamic resources. As institutions integrate support for a range of open resources within their technological infrastructure individuals will be able to take advantage of a reliable and supported place primary deposit whilst continuing to utilise web 2.0 mechanisms and other portals to publicise and encourage use.

 As previously mentioned, most projects deposited OERs in repositories and content management systems (CMS) to facilitate management and curation, but several also adopted a range of other approaches to increase access, including added guidance about how to use OERs for different contexts, review and comment features. Whilst there was a general view that using open platforms and open document formats increased accessibility - the CPD4HE project did highlight that open document formats could be seen as a barrier if it meant downloading new software. Only a few projects reported on activities to ensure that OERs were accessible through mobile technologies, but this did emerge as an area for future consideration.

Triton and C-SAP collections emphasised the importance of a “hook” to draw users to their collection in the first place. Following user surveys, C-SAP promoted the visibility of videos and instituted a “video of the week”. For Triton, the “hook” in the form of topical blog postings from  Oxbridge academics in the field of politics, allied with a news feed, form the most prominent part of the site. These are juxtaposed with related OERs but seldom make explicit links to them.

if the focus is discoverability, a broader conception of OER needs to be pursued in order for people to be introduced to open resources. What initially draws people to the site how this translates to OERs is an essential part of this process. (C-SAP interim report)

The difference in the nature of the hook may largely be attributable to the difference in the intended audience for the OERs.  In the C-SAP case, the audience is teaching staff who value videos, and reviews of videos, for re-use in their own teaching, whereas for Triton, the users envisaged for the Politics Inspires site are learners. 

The key factors influencing  [student] use of the site were frequency of posts, global coverage, and quality of materials (Triton)

In addition to alternative hosting solutions, use of JorumOpen was mandated in the call for funding. Whilst projects have, in most cases, deposited either content or metadata linking to content held elsewhere, there have been significant challenges to deposit due to perceptions around issues relating to version control, ongoing management and how resources are exposed. There is still much discussion required around how a shared national repository, such as JorumOpen, supports the process of OER release and re-use, particularly in relation to the increasing use of institutional repositories or other mechanisms for learning and teaching resources.

 Subject and sector communities identify OERs as having potential to enhance existing communities and building new networks. Increasingly projects have talked about the importance of broader activities around OERs - the community networking, curriculum development and student engagement. Motivations are very diverse ranging from institutional, personal or community led. Projects often start with one or two main motivations but other emerge as individual perceptions and activities develop.

Organising OER

Project sites provided a portal to static and dynamic OER collections and sometimes other information. The majority used a Wordpress blog as the basis of the portal, but took different approaches to linking the resources to the blog posts.  Thus C-SAP used its blog to review resources in its static collection, Delores blog posts described a single OER in the static collection using a standard template, while Triton's "Politics Inspires" blog posts comment on current political topics and are a hook to draw users in - they only occasionally make explicit links to OERs, though relevant podcasts are juxtaposed on the site and OERs are available through a separate tab.  Access to dynamic resources was through search and categorised browse options, to which Triton added "learning paths", ie. collections bookmarked by registered users. 

In contrast, the Oerbital projects used Mediawiki to provide a catalogue of the (static) collection and associated discussion with feeds in/out.  

OF (GEES) provides multiple ways into its collection, via map-based or text search, word cloud, or browsing categories. "[user survey] findings provide empirical support for the development of a map-based interface for discovering fieldwork resources, and for the need to include multiple search/criteria.” “[word-cloud] provides the user with suggestions for possible search terms, together with an indication of whether the search is likely to return few or many resources” user comment, “Map search is great” (OF (GEES) ) Catalogue entries describe individual resources with clear licencing information 

To enable browsing and filtering of results, projects used a combination of classification into categories and author tagging for their static collections. While C-SAP directed their authors to use a recognised subject taxonomy for tagging, Triton allowed free tagging allied to fixed categories.

Delores and OF (GEES) enabled classification of their dynamic collections, OF (GEES) by identification of location-specific coordinates and geoparsing, and Delores through its Waypoint software:

user access to the content [is via Waypoint] software ...which implements an adaptiveconcept-matching algorithm [allowing] browsing of material that has been classified against a set of facetted classification schemes. The user is able to interactively ‘prune’ a classification tree list which responds in real time to the user’s multiple selection of taxonomic labels....: Waypoint search/browse implementation [required] the development of a coherent subject taxonomy (classification scheme) and rules which allow classification of the resources against the scheme (Delores)

While Delores provided separate interfaces to static and dynamic collections, OF (GEES) gave a unified interface through a subject-specific taxonomy represented as a clickable tree -at top level resources were categorised as generic or location-specific

Projects derived their categories by a variety of means: developed by subject librarians and linked to local course structures (Triton); devised by the project's subject experts (Delores); or a recognised social sciences taxonomy (C-SAP) 

Guidance through the resources was an area in which projects conducted extensive user testing. They found particularly, that google and Amazon have set user expectations that searches will be simple, personalised, and produce relevant results.

The importance of Google and other popular commercial sites cannot be underestimated; both in terms of the resources they produce and the expectations that they provide the  user. (C-SAP blog )

Users who have become accustomed to google were frustrated with the search and browse facilities of most OER repositories

 the OER descriptions in RSS feeds are not particularly helpful to repositories or to users (e.g. very often the descriptions are of a course overall rather than the particular resources (lecture notes, examples, etc.) made available through the course). (Delores)

Projects had to make decisions - which they never entirely resolved - over:

• the desirable granularity of classification  

in the static collection, how do we select and provide descriptions at the fine level of granularity that Chris wants while also keeping he valuable information of the original course context of the resource;  (Delores blog post "finding OERs")

These decisions were sometimes driven by the constraints of the project, rather than what seemed best for users. Delores, for example, decided in the end to describe the OER at module level, (because this was more efficient of project time), but believed that a more useful grain size would be lecture level. 

• how to enter multi-site resources on the map interface in the OF (GEES) project, and how close together sites were before being counted as a single site  
• how describe authorship - which proved particularly problematic for videos on the C-SAP project.

Both C-SAP and Triton, in their user testing, found that filtering was important - a few highly relevant results were preferred.  C-SAP users preferred filtering of search results to being forced to choose between categories.

Triton also found that searches for an authors name were very common among students, emphasising the importance of attribution.  They ensured that author information was prominently attached to their blog posts

In addition to making resources available through a variety of formats and platforms some projects offered a range of routes into the content.

Four routes into the content were agreed: through a searchable Library of all resources; through Themes which were mostly related to historical periods in literature reflecting the way the subject is taught at university; through individual Writer pages which would possibly appeal more to schools and the general public; and through a searchable list of eBooks. (Great Writers Inspire Final Report)

Providing OER across a range of platforms allows projects to present the same OER in different ways - either as separate assets in a database, with a linear pathway through VLEs, with supporting pedagogic information or with features to add reviews or comments to augment OER.

Ensuring pedagogic accessibility

Key characteristics that made discipline-based collections usable across institutions were the adaptability of resources to different pedagogic contexts, and clear licensing information attached to all components of the resource.

Adaptability to pedagogic context could be enhanced by reviews and examples of adaptation (C-SAP collections), and by preparing resources with an OER framework in mind for disaggregation into components (Oerbital)

Involving potential user groups in the development and testing of OER has proven to be an important way for most projects to ensure that they are pedagogically and technically accessible.

Some OER are released for a very specific group, and whilst they may be highly accessible to that particular group the resources can be less valuable for others. This relates back to considerations around definitions of OER - around how open and accessible a resource needs to be, and also to some of the challenges reported by those who are trying to encourage others to use OER.

Releasing OER for wide-scale use in a global context is often only one of several motivations for OER development. In fact, most UKOER projects had primary intentions to release for a quite specific audience, either in a particular discipline, course or stakeholder group. Projects often focus on requirements of their intended audiences which has an impact on the OER content, presentation, organisation and hosting arrangements. This approach is particularly valid for release by educational institutions, and ensures senior and academic staff buy-in. This can actually have negative impacts on wider use and does result in some OER being less accessible, which presents a very real tension for funders of OER initiatives.

Examples of the ways on which this can affect development and release include:

OER to support specific courses which reflect course structures and are difficult to re-use in other contexts

OER which include references to specific laws or regulations that  are not relevant in other countries

OER which include specific pedagogic approaches or language that might not be relevant in other context

OER primarily developed for use within one institution but made open more widely may be less usable in other contexts than anticipated

The University of Leicester OTTER project from the pilot phase developed the  CORRE framework.

Two OMAC strand projects utilised and adapted the framework.

We found the CORRE framework useful for setting out the process for turning teaching materials into OERs but, like our partner project, DELILA, we adapted it to suit our very small project. (CPD4HE)

DELILA adapted the CORRE framework to provide a practical check-list for rating materials against OER criteria. The project team also made contact with OTTER/ CORRE project members, and welcome continued developments in this area. http://delilaopen.files.wordpress.com/2010/10/adapted-corre.xls (DELILA)

Three of the Release strand projects utilised and adpated the framework

TIGER Quality Framework

A TIGER pedagogical model that will enable other IPE and health and social care professionals to benefit  -TIGER has developed a quality framework which allows professionals to clearly understand how the OERs have been developed. https://openeducationalresources.pbworks.com/w/page/24838164/Quality-considerations

During the project, TIGER adapted the workflow originally produced by Alejandro Armellini for content and resource management with built-in quality criteria (TIGER)

Several OMAC projects had already mapped their proposed resources to the framework and modified this during the timeframe of the project.

OERs can be mapped in several ways.

• CPD4HE produced a second map relating to two priority areas - Digital and Information Literacies and Discipline-Specific Learning & Teaching. These also might help people aiming for accreditation against the UKPSF. 
• DELILA carried out mapping work to cross tabulate the SCONUL 7 Pillars of Information Literacy (SCONUL, 2007) and the FutureLab (2010) Digital Literacy model with the UK Professional Standards framework (HEA, 2011) revealed that while digital and information literacy was not explicit in the UKPSF, they did map well. In fact the team concluded that digital and information literacy underpinned much of the UKPSF.

Many of the CPD4HE resources ask learners to create outputs (usually written). In the programmes from which the resources are drawn, these outputs contribute towards assessment, and hence UK PSF accreditation. An additional guidance document, showing in more detail how these pieces of writing might be submitted as evidence for accreditation could be developed and would probably be useful to learners. (CPD4HE)

Mapping to the UKPSF allows for ease of integration within accredited programmes of teaching & learning or academic practice in higher education. (Dundee) (O4B)

Changing UKPSF's were noted as a challenge

A number of the frameworks used in DELILA are being / have been revised. These include the UKPSF (HEA, 2011) and the SCONUL 7 pillars (SCONUL, 2011). Whilst the DELILA mappings remain useful as exemplars – changing frameworks may mean that new mappings need to be over-laid. (DELILA)

There was a delay in the mapping exercise due to the fact that the new UKPSF was not ready before the project end.  We chose to defer the final detailed mapping of all available resources until the new framework is in place and this would become part of the planned sustainability process. In the meantime we explored alternative approaches to mapping so we were ready to do so when the new framework was available. (ACTOR)

ACTOR mapped a sample set of resources from each partner against the framework in order to help guide its effective use on both accredited and non-accredited programmes.

Whilst we have not attempted to generate resources to cover all aspects of the UKPSF we have sought to produce materials for areas of the framework where there were felt to be significant gaps.  (ACTOR)

 To encourage discoverability projects tend to focus on appropriate tagging, SEO rankings, effective use of social media, marketing and supporting discovery through community networks

Making OER available through multiple platforms has led to a high dependence on open feeds and metadata  - this needs to be available to many different systems, services and users and also requires appropriate licensing to ensure discovery and use

it seemed counter-productive to try to introduce new QA measures into the course simply because the resources were being made public. In our view, the spirit of OERs is about sharing the work we do, rather than introducing exceptions into our practices.(ChemistryFM final report)

 Quality assurance for the dynamic collections was achieved largely by searching only a limited selection of previously quality-assured repositories such as JorumOpen,

Assuring academic and pedagogic quality was particularly important for the Triton project as resources were associated by users with the Oxford/Cambridge brand. They produced clear policy and guidance documents to help assure the quality of blog postings and the static collections. In addition, the site Administrator and Oversight Team acted as gatekeepers to ensure the reputation of the department and university was upheld, particularly ensuring that sensitive topics were treated with care (Triton).

While C-SAP discarded user ratings as a measure of quality, and put emphasis on user reviews instead, Triton  implemented a “like this resource” function in both static and dynamic collections.

Selection and quality processes depend on questions of

• What is relevant – establishing the scope of the collection
• Processes for selecting what is relevant, especially automatically
• What is meant by “quality”
• Processes to assure quality – which broke down into assuring academic/pedagogic, technical, and legal quality

 Projects aimed to enhance the general quality of resources produced by providing guidance on licensing (OF (GEES), Oerbital). Delores worked with the authors of some non CC-licensed materials to encourage them to license the resources.  Triton required contributors to the Politics Inspires blog to register in order to comply with CC licensing.

Issues around quality seem more pronounced in relation to repurposing of existing resources that in creating new OERs, both in terms of staff perceptions around the quality of their existing 'unshared' materials and also the fact that they were not produced as showcase material.

Projects used their expert groups and user communities to establish the scope of the collection and relevant selection criteria (Delores, OF(GEES), C-SAP collections, Oerbital).

Reviewers have addressed issues related to the granularity and adaptability of research methods as well as issues involved in using generic vs. subject specific methods teaching resources. This has helped the project team to refine the scope of the research methods domain and the scope of potential collection (C-SAP interim report)

Ensuring adaptability

There are tensions relating to the size or granularity of materials with recognition that bite-sized resources may be preferable to encourage repurposing and are easier to develop. However institutions may prefer more sophisticated materials for showcasing and also some academics and institutions prefer to maintain the pedagogic integrity of materials. Drivers and motivation for release do have a significant impact on the types of materials released. 'Pedagogic wrappers' emerged as a way to provide context to granular items and also served as a meta-record which could be deposited in a range of sources pointing to the actual OERs hosted in a primary store.

In terms of the types of resources produced these ranged from small individual assets to whole course modules. Podcasts were found to be particularly cost effective to produce and also received very positive responses from learners. Because projects were not focused on use per se, it is not possible to disentangle the different types of resources favoured by teachers for reuse or repurposing, from those preferred by students to support their learning. Several projects have started to tease these issues out through evaluation activities but this is an area needing further investigation.

'OER can be made or released at various levels of ‘granularity’. There is a distinction between material that has been aimed at the end-learner and developed to be used in its entirety, as a whole ‘module’ or ‘series’ and material that has been released as a ‘bundle’ of resources, which teachers can use with a ‘pick and mix’ approach for creating their own teaching materials.' C-change Project Final report

  See also discussion relating to accessibility above, particularly around issues of granularity/context and making content available in a range of formats often across several platforms. TIGER created the resources not only for local users, but also for users all around the country, Europe and the world. TIGER needed to make the resources as generic, open and transferable as possible. The team needed to think about the use, but also reuse and repurposing, making resources that people can easily translate into their own context.

In relation to granularity the C-SAP Cascade project noted that

A smaller resource is:

• less academically credible/ significant
• supports autonomous learning
• creates less work for the tutor in introducing for autonomous use - flexibility 

A bigger resource on the other hand:

• is more academically credible/ significant
• requires a higher level of self directedness from the learner.
• creates less work for the tutor in introducing for autonomous use

   

•  

Granularity has been discussed in relation to accessibility and re-usability and much of what projects said with regard to this did not appear to be subject specific. However the OMAC strand illustrated a clear and definate preference to present resources within a context or framework. This reflects the nature of the strand requirements which focussed on accredited programmes or schemes of professional development. This meant that projects often presented OERs is two or three ways - in their own repository, in Jorum and on the web. Interestingly 4 of the 11 projects chose to use the OU Labspace moodle environment - OPENSTEM, RLT for PA, Learning to Teach Inclusively, IPR4EE and other projects chose to use institutional moodles - ASSAP developed The Pool.

The Moodle development was not part of the of the early project plan, but offered an opportunity to present the materials in a more structured and holistic way than is possible in an OER repository where they necessarily become more fragmented, though more easily retrievable.  By making the learning materials available through both a VLE and two OERs, we hoped to capture the best of both platforms. (ASSAP)

Adaptability of OER materials was seen as essential. All interviewees said they preferred to tailor materials to suit their own purposes and match the needs of their students. A key recommendation for ORBIT was that resources should be provided in formats that permit easy adaptation:

“They need to be good resources and ‘good’ doesn’t just mean, you know, looks pretty. . . but good in terms of is useful to me, hopefully, because they’re open and, you know, the graphics are vectored so I can take things out rather than embedded bitmaps, and the file size is reasonable and it’s not saved in Adobe Illustrator format that I don’t have, . .using free and open source software as much as possible.”  (secondary teacher) (ORBIT Final report)

However, in many institutions, 'e-learning' or 'online learning' quality processes are still immature, and even in an e-learning context there may be little specialist understanding of what makes resources repurposable or reusable in other contexts.

Institutional QA measures generally relate to whole modules, of which the released resources were only one aspect, and within which the resources would have been used in pedagogically specific ways. So some projects have argued explicitly that it could not be assumed that the materials themselves would 'carry' the quality assurance into a more open context, in which they might be accessed by a wide range of different users with different requirements. This argument is more apparent in subject strand projects, and in institutional strand projects where institutions saw OER release as showcase or 'reputation enhancement' mechanisms and where the individual authors might feel very exposed to the high expectations of their institutions. It was less explicit in individual strand projects, where individuals are very close to their materials and may be trying to enhance their personal reputations but have fewer external expectations to meet. The response, in many cases, across all strands, has been to provide a second line of peer review within the consortium in subject-based projects, or through stakeholder and user review and beta testing for institutional and individual projects.

'CORRE process includes: internal validation by OTTER team (proof-reading, testing links etc.), validation by contributing authors, reality check by students and feedback data gathered from users who fill in questionnaires.' OTTER Project final report

 Some projects attempted to re-use OER developed during earlier phases of the programme and discovered difficulties as not all of that content was accessible, either due to format choices or barriers of the platform used

OER producers often have to make compromises when releasing content, for a variety of reasons, which impacts on long term accessibility and re-usability

Making OER discoverable

 Anticipated benefits of collections in particular, expressed by all projects, were the enhanced visibility and discoverability of a collection of cognate materials.  To gain such enhanced visibility is a motivation that was posited in the call for funding and formed the rationale for this strand of projects.

Equally important was the exploration of effective means of pull (ie search) and push (ie feeds) discovery. The projects found that their users expected searches to be "google-like" in their ease of use, personalisation, and production of relevant results.  Searches for OERs were very difficult to automate, as required for the dynamic collections, because of the lack of standardisation and machine readability of licence information and lack of explicit badging as OER (Delores, Triton, OF (GEES), EALFCO).

API and metadata issues mean dynamic collection does not return  the “google-like” relevance that  users expect " (Triton)

 ...for full realization of the potential of OERs in general, adoption by providers of a more standard way of representing information about a resource would be beneficial. The data encoded should be machine-readable and at least include information that explicitly identifies the material as being an OER (Delores)

Projects emphasised that search tools needed to be appropriate, both in the user interface – where OF(GEES) developed a map-based interface for finding fieldwork OERs and in the relevance of results returned.  To classify OERs, and search results projects either adopted existing classification schemes for their domains (C-SAP), used their expert groups to develop a classification (Oerbital) or adopted the local institutional curriculum (Triton).

During phase three Amber Thomas, JISC Programme Manager wrote a blog post which stated that:

One of the findings that has emerged clearly from the UK OER Programme and from the UK Discovery work is that for a healthy content ecosystem, information about the content needs to be available to many different systems, services and users. Appropriately licensing the metadata and feeds is crucial to downstream discovery and use. http://infteam.jiscinvolve.org/wp/2012/10/09/opendatalicensing/

This illustrates a more nuanced understanding that has emerged during the programme - around appropriate licencing of open data, not just the OER themselves.

Projects tend to focus on appropriate tagging, SEO rankings, effective use of social media, marketing and supporting discovery through community networks. However, making OER available through multiple platforms has led to a high dependence on open feeds and metadata.

External content was more time consuming, with a manual process to identify and upload with correct metadata. Metadata was sometimes hard to find on external resources with a great variety in how things were displayed and published. Finding the licence in particular and exposing this in the metadata took significant effort. (Great Writers Inspire Final Report)

The aim of the collections strand was to enhance the discovery and reuse of OER materials, by building collections of materials around particular thematic areas. The assumption underlying the strand was that such collections would provide a critical mass of materials in a particular area, raising the profile and awareness of OERs among teachers in the discipline. Beyond this, though, projects were concerned to ensure that their collections were usable and quality assured - without usability and quality, a mass of materials has little value. Projects achieved these four characteristics of critical mass, usability, quality and high profile through their technical and practice-based choices.

Addressing gaps

 Many projects were addressing gaps in their subject area through the choices made at the proposal stage. Several collections were addressing subject-discipline needs such as modern languages (FAVOR), renewable energy and built environment (ReACTOR), literature (Great Writers), materials science (CORE-SET), medicine and veterinary studies (HALSOER, PublishOER), open media studies (COMC), art and design (ALTO UK, COMC), and business (Opening up a future in business).

Some projects were focused on generic collections which reflect areas such as digital literacies (Digital Literacy and Creativity, HALSOER, ALTO UK, DEFT, ORBIT, Teeside, Academic Practice in Context), employability (PARIS, Opening up a future in business, COMC) continuing education (SESAME) legacy materials (FAVOR).

Others focused on developing OER for specific types of practitioners or stakeholder groups such as part-time tutors (FAVOR, SESAME), teachers/educators (DEFT, ORBIT, HALSOER, BLOCKED, Teeside, Digital Literacy and Creativity, Academic Practice in Context), 3rd sector (PARIS, CORE-SET), adult learners (SESAME), disabled students (HALSOER, ReACTOR).

Some projects focused on developing and releasing particular types of materials, such as 3D simulations (ReACTOR), podcasts (Great Writers), ebooks and open textbooks (Great Writers, ALTO UK, ORBIT, DEFT, PublishOER), case studies (DEFT, CORE-SET, PublishOER, SESAME), course materials (COMC), mobile device ready OER (ReACTOR, PARIS, HALSOER),

Drawing on existing collections

The lack of consistent metadata for describing OERs, the variety of repository APIs, and the low visibility of licensing information severely restricted projects’ ability to collect OERs dynamically (Delores, Triton, C-SAP collections, OF (GEES), EALFCO). The solution adopted by most projects was to restrict their collections to a limited number of repositories whose resources were known to be CC-licensed; C-SAP collections used Google custom searches on these sites; Delores inspected the repositories manually to discover schema for resource description and then wrote scripts to extract resource data; and Triton used php to query the APIs and then cached the rss feeds.

There are many aggregation services we can use to bring in content to Politics in Spires. Often these aggregation services are without API (Folksemantic, OER Commons, MERLOT) and as such we need to fake requests to generate RSS content which we can then search. Services with APIs (Xpert and Jorum) present a problem by having different approaches to querying. Xpert can be searched for a keyword, whereas JORUM has a mandated list of keywords to be searched against.  Xpert sadly was the slowest API, which did lead to issues with timing problems. This all demonstrates that to query five repositories you would need at minimum three pieces of code. It is likely aggregators in future will increase this language diversity, and so complicate algorithms. At present it is useful to have a service which can alter the metadata returned (such as with the flickr API), but effectively proprietary API formats do not assist (Triton)

OF (GEES) used a combination of searching resource descriptions for decimal degree coordinates, and geoparsing, to identify location-specific resources.

Even with these solutions, further issues arose with broken links returned from aggregation services (Triton) and with duplication of results (Delores, Triton).

Projects found that licensing information is not always attached to the object leading to ambiguity about licensing status,

Often the conditions of use of a resource are not stated in the document itself, but at one remove, perhaps being a catch-all for the source site itself " (Delores)

Having collected resources, making them available relied upon designing an appropriate and usable interface. OF (GEES) developed a map-based interface for discovery of its location-specific resources, and a clear traffic light system that made the licensing status of resources instantly visible. Projects using Wordpress chose their themes for the balance of static and dynamic resources (Triton), and to display clearly the attribution, licensing information, and origin of the resources described (C-SAP collections).

Whilst most projects were collating a range of existing materials from a variety of sources and stakeholders, some made specific efforts to work with existing OER with varying degrees of success

There was a general consensus amongst project teams that had worked on previous phases of UKOER, that using existing OER was a goal to aim for and some offered evidence of success in achieving this.

Feedback also suggests that using third party OER has allowed content creators to focus on the way they wish to best engage learners with the materials, rather than spending time creating materials. ... Content creators have also been supporting each other in the creation of the sustainability resources and sharing work prior to publication. (PARIS Interim Report)

Whilst the contact with other teacher education HEIs was not as productive as had been anticipated, it also revealed unexpected instances of materials developed for EU funded primary/secondary facing projects that had been released under a Creative Commons licence thus enabling ORBIT to reuse those OER materials. In addition, while attending the World OER Congress, the Research Associate became aware of a variety of further non-UK based Teacher Education and/or interactive lesson ideas and OER materials on offer from supplementary HEIs elsewhere, e.g. Vyatus Magnus University, Lithuania; Mauritius OER project, etc. (ORBIT Final report)

While we used resources from OpenSpires11 and other UKOER phase 1 and 2 projects as much as possible, these materials had to be heavily customised, and in many cases we eventually developed materials virtually from scratch for our context. (SESAME Final Report)

Influence of discipline or sector

The wide range of materials released as OERs as a result of this funding is impressive, both in terms of covering a range of subject disciplines and in terms of the variety of resource types that have been released. The range of different approaches to subject coverage reveal specific advantages and benefits:

• Subject/discipline approach:
• cross-institutional collaboration
• the ‘greater good’ outcomes are to the fore
• existing Community of Practice
• open, collegiate way of working suits academic culture
• Thematic approach (where relevant):
• provides a coherence and profile to the OER resource set
• provides a rich set of fully contextualised resources, ready for reuse; supports deep collaboration and detailed reflection
• Generic approach (such as cross disciplinary skills)
• can appeal to managers looking for cost benefits
• can engage a range of faculties

There was a notable appetite for release of generic materials in interdisciplinary subjects such as work-based learning, digital and information literacies, skills and CPD materials which can be seen as supporting sustainability and efficiences within institutions, although strong institutional branding and specificity can make these materials of less value outside the institution.  There was also a demand for generic materials to be made more subject specific and several projects in the Release strand described these as highly valued by their subject communities. On the other hand there was a general openness to adapting subject specific resources for other subject areas.

Subject-specificity is not confined to the development of activities and materials but has changed the culture and methodology of our STEM strand.  We have found that STEM participants respond well to a strongly evidence-based theory and approaches which involve quantitative as well as qualitative measurement and evaluation.  Hence, our approaches to ‘meta-learning’ make this more explicit than in previous cohorts. (OPENSTEM)

Projects working with HE in FE identified OER requirements from stakeholders as being similar to those required generally for learning and teaching materials (ie not open or sector specific).

ACTOR raised dissues around the long term usability of OER in healthcare education due to changes in policy, technology and public opinios

some shared resources containing recordings of people (which complied with good-practice guidelines at the time of collection e.g. CyberAnatomy at Newcastle University, and the Bristol Biomed Image Archive) have since been ‘locked down’ to local virtual learning environments (VLEs) or completely withdrawn due to concerns firstly about the clarity of how the people depicted wanted their recordings to be used, and secondly about the clarity of ownership and licensing of copyright. (ACTOR)

Healthcare is particularly problematic in relation to patient and practitioner consent and due to the nature of the challenges of patient consent ACTOR developed a Consent Commens framework 

to support digital professionalism recognising the rights of people to be treated fairly and with respect.  It balances a desire for sustainable open access with protecting patients' and other peoples’ rights and expectations of how recordings of them, especially if captured in a clinical setting, may be used. Proposing a consent commoms in open education paper 

This is likely to be difficult to apply in a retrospective way to existing resources.

In the OMAC strand there were challenges relating to the boundaries between learning resources and guidance material

For example, we have created audio commentaries on many of the resources; the primary purpose was to give more information about how the resources might be used but it might well be that a conversation about educational values, key skills or academic literacies could be used as a learning resource. This is something we only realised through creating the  recordings. (CPD4HE)

The nature of the subject discipline or theme did impact on the type of resources to some extent (e.g. art and design  - tacit information and rich media; healthcare - patient and practitioner sensitivities and consent issues).

The only type of material that inherently realises the aim of making statistics more appealing and more accessible for social science students is the videos, by very graphically showing how statistics can enhance understanding of real-world phenomena (although this is also true to some extent of the interactive graphs containing worked examples). The other types of material follow on from it by supporting the student in their developing their statistical understanding and skills.  For this reason it seems that the material will be most effective when it is used in an integrated way. DeSTRESS evaluation report

However it is important to note that even within the same broad subject discipline areas that different approaches to using content in teaching meant that materials did not always easily transfer to other departments.

Many existing Art & Design OERs are not seen to be appropriate… we should not rush to create poor quality OER randomly, but should first plan a methodical approach which will take longer, but has the opportunity to enhance the reputation of our Faculty.” (Kingston University) ADM

OMAC strand projects were producing materials for teachers and which were naturally focused on practice. The nature of capturing and recording authentic learning and teaching activies did raise issues around quality of recordings.

We took the view that any loss of sound quality would be more than compensated by the addition of subtitles, that also served to reinforce the inclusive principle of universal design i.e. subtitles help all users.(Learning to work inclusively)

Kinds of OER being released and used

 As expected projects delivered a wide range of OERs ranging from individual assets to whole modules. Projects produced a wide range of different kinds of resource (eg, lectures, assessment banks, interviews, statistics, quizzes, case studies, discussion starters, workshop plans, taster materials, reports, newspaper articles, case-notes, etc.) in a range of formats (video, photographs, pdfs, ODF, ppt, word documents, etc..). There were international materials, commercially published materials, authentic learning and teaching activities, and legacy materials. Some formats presented significant challenges (newspaper articles, photographs, patient data, journal articles).

 Open textbooks/eBooks emerged during this phase as a significant format offering personalisation opportunities for users, publishing  opportunities for authors and taps into the current interest from commercial publishers and students

Designing OER for use on mobile devices is still challenging due to different fluidity, resolution, graphic quality and general working compatibility

Most projects released a mixture of existing (often re-purposed) and newly created resources.  It is difficult to tease out how far the subject area or theme impacts on these choices, apart from projects with an obvious focus on legacy material. Projects with OER experience may already have understood the complexities and time implications of releasing exisiting resources and instead focussed on new materials.  Decisions like this are impacted by several issues, for example, the availability of new technologies can so transform the pedagogic potential that it makes sense to develop new resources rather then repurpose existing ones - simulations are a good example of this. Other aspects that can impact on these decisions relate to funding and resourcing, particularly as changing economic conditions might affect how resources are developed. 

There is a strong compliance culture, with fewer resources for innovative development. However, developing a sharing culture was seen as being possible, thanks to initiatives such as ‘content clubs’ and the national repository (NeLR) and PORSCHE itself. This move is still tentative, with a worry that transition to Foundation Trusts might result in a further monetising of content and the commissioning of more materials from commercial suppliers. PORSCHE Evaluation report

The biggest challenge in evaluation is measuring impact. Surveys, comment boxes and social networking is potentially useful for gaining feedback but is under-used. How do we measure re-use and re-publishing? This is particularly confounded by the fact there is not a clear definition of use or re-use. (HALS OER Interim Report)

It could be argued that all use requires some element of re-purposing for each different context. The problem for projects is that tracking downloads does not actually provide information about how the OER is being used. One solution adopted across all phases has been to integrate mechanisms to collate information about use, for example through comments and survey forms, into the resources themselves. Most projects in phase three devised comprehensive evaluation activities to provide feedback on the OER through various stages of testing or release with controlled groups, through surveys, focus groups, workshops and interviews. These evaluations were usually focused on the intended target user group/s and many of these may have also been involved in scoping or user requirements activities at the beginning of the projects. (NB. Evaluation activities were often broader than simply relating to use of OER and may have been targeted more at measuring changes in attitude or open educational practice in a wider sense).

The question of 'level' was raised by several projects. A number of projects have pointed out that introductory or generalist level materials benefit from a wider number of potential users, but C-Change note that advanced materials can add more value because of their scarcity and specialist nature. ChemistryFM noted that videos relating to theoretical concepts received the highest number of views, but were less highly rated than those showing how to perform calculations.

Two main practice-based choices emerged as important: the inclusion of "grey" or "non" OERs in order to generate critical mass; and the importance of a "hook" to pull users in.

However good the search facilities,a critical mass of potential resources is needed, otherwise search results are sparse and users revert to google. Thus OF (GEES), C-SAP collections, Oerbital and EALFCO argued for the inclusion of "grey" or "non" OERs in their collections. 

There are not many “open” resources currently. It would be remiss to ignore very good resources that are publicly available but not under a cc licence + we’d only have a small number of pins on the map. By broadening the scope we can engage colleagues who are not aware of OER and CC and also provide a collection that might be interesting enough / have enough critical mass for our community to want to look after and keep contributing to (OF (GEES))

These projects all adopted some means of making clear to users which resources were CC-licensed and which were not.

Existing resources containing images, particularly of children, sports people or patients, raised issues around data protection and significantly hampered release.

It is very early to identify which of the OERs developed during this phase of activity are being used - although projects intended to observe and report on this. It seems that most projects underestimated the effort that goes into developing OERs and timescales did not allow much time to evaluate use. However most held focus groups and surveys with staff and students. 

Some projects report on anticipated use based on initial feedback from stakeholders and others have used tracking analytics to generate some information on views and downloads. Most projects have ongoing activities and mechanisms to encourage and promote use, including obtaining feedback from resource users.  

The National Skills Academy will be promoting the resource on its ‘Get into Theatre’ and ‘Get into Live Music’ websites, so that the latest version can be downloaded by its members. The tool will also be promoted within its support of the new FE Artsmark as a current resource for the arts to be shared between students, with tutors and between industry and education. SPACE (simulation 3d environment)

Generally more use is anticipated for some of the disaggregated resources but many projects felt the need to provide additional pedagogic support or intention. There is evidence of use of whole units/modules as well as smaller granular elements. This highlights a focus on activities around the OERs being as important as the OERs themselves and their potential to encourage 'pedagogically infomed use' (SWAP) - ie good learning and teaching practice (open practices).

PORSCHE highlighted that student expectation may also have an impact as their use of open resources continues to increase. PORSCHE also found evidence that demand is there (particularly in this economic climate) but that there is still a lack of critical mass of OERs at the right level of granularity. Cascade strand projects noted that choice of search engine affected which resources were found and used and many projects across strands understood the value of making OERs discoverable and usable for a range of stakeholders.

Participants at the eLearning in Health Conference identified examples of OER that would be of use: interprofessional practice teaching was raised as an area where sharing of resources could be of particular benefit. Other particular needs identified were: taking content straight to a patient’s bedside; and providing students the opportunity to take risks without harming patients. PORSCHE Evaluation Report

Technical and hosting issues

include this in the report at start of tech section as it provides a comment about the whole three year programme

Technical support for projects was provided by JISC CETIS. Ongoing articles about the programme and OER generally are available at http://jisc.cetis.ac.uk/topic/oer. For an overview and detailed discussion of technical issues raised by the three year programme see the publication Technology for Open Educational Resources - Into the wild Edited by Amber Thomas, Lorna M. Campbell, Phil Barker and Martin Hawksey, November 2012

This report concludes that 'it is neither feasible nor desirable for the OER community to develop a separate infrastructure, but rather to enrich existing tools, services and applications to support OER, and to support the signposting of OER-friendly services.'

 Underpinning many of these technical issues and directions is the concept of an open content ecosystem.  By adopting the principles and ethos of openness and exploiting existing open technical approaches and applications, the global OER community has the potential to develop a rich ecosystem of services.  Many of these services and applications may not be designed specifically for OER, but can be regarded as "OER-friendly". Given the fuzzy boundaries between open education, open source, open access, open educational resources, open content and free online content, this pluralistic approach to infrastructure may be more sustainable than centralised approaches. The key characteristics of this OER-friendly ecosystem are: prominence given to licensing, integrated tools and APIs, aggregation and remix solutions, educational authoring platforms, and rich release and export functionality.

In relation to learner produced oer-

Other aspects of OER release, such as metadata and accessibility, are less well covered, though ease of managing these has figured in projects' decisions about what software to use to enable learner contributions.

Technical accessibility

Many aspects of accessibility are inherent in the notion of making resources open and were stipulated as part of project requirements (such as open licensing, deposit in JORUM, etc.)

In addition to the legal requirements of OER preparation, best practice also provides for compliance with accessibility standards. All resources provided by this project underwent a thorough editing and formatting process to ensure that they were as accessible as possible (e.g. proper formatting of headings, sub-headings, body text and hyperlinks and the provision of alternative text for images). Document properties were also completed to ensure that provenance was embedded within the core of the resources themselves. Again, this represents a significant undertaking which could be ameliorated by the inclusion of the practices at the point of resource creation.   LEARNING LEGACIES

Where openness or accessibility was compromised, either due to copyright clearance issues or technical issues, projects responded pragmatically and chose not to release problematic resources. Long term tracking and evaluation to find out if all intended audiences can access OERs may reveal more in the future, but what did emerge was a significant amount of thought about how different audiences might use OERs. This often translated into OERs being made available in a range of formats, at different levels of granularity and through several platforms. This effort to make OERs pedagogically accessible  went beyond the 'pedagogic wrappers' which became so prominent in the pilot phase (although many projects did adopt this approach). Due to these efforts it is likely that the OERs released during this phase will be accessible to, and used by, a wide range of audiences/groups. There is a consensus that less complex granular items are more easily resused and re-purposed but that there is a fine balance between facilitating engagement and retaining discipline related contexts and complexities.

Improved accessibility for disabled students was a significant driver for some projects (SPACE and DHOER).

Some projects attempted to re-use OER developed during earlier phases of the programme and discovered difficulties as not all of that content was accessible, either due to format choices or barriers of the platform used. This reflects an important challenge relating to OER use and re-use generally. Previous projects often had to make compromises when releasing content, for a variety of reasons, which impact on long term accessibility and re-usability.

It was quite notable that many OER produced as part of earlier UKOER projects were not accessible and were constrained behind technical barriers. For example, resources using Articulate Presenter and other propriety software can only be repurposed in the institution has the license to use the software, and only if all the project files and folders are available. Multiple choice questions published using QuestionMark Perception released as a SCORM package for uploading into Blackboard do not make accessible the actual questions, and these could be simply included into a document. (HALS OER Final Project Report)

)