OER Synthesis and Evaluation / Phase2 Development and Release Issues
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

Phase2 Development and Release Issues

Page history last edited by Lou McGill 12 years, 3 months ago

Back to UKOER Phase 2 final report contents page

 

Cross strand evidence to support this section is recorded at: DevRelease-evidence

Strand evidence is recorded at: release | cascade | omac | collections

 

Many issues relating to development and release emerged during pilot phase activities. The questions addressed during this phase naturally overlap with these to some extent, especially for projects new to the programme. This section records notable differences rather than re-iterating lessons that reflect those from the pilot phase. The pilot phase report also reported in detail on different benefits of release for different stakeholder groups. See also briefing paper on models and benefits developed as part of this phase of activities [insert link]

 

We also examined issues around how subject disciplines and themes impacted on development and release.

 

Jump to the appropriate section of this page

 


 

 

 

OER development and release

What technical issues emerged?

Detailed technical aspects are covered by JISC CETIS synthesis

 

Platform choice emerged as one of the most interesting technical aspects of this phase with many projects depositing in several places to present materials at different levels of granularity and to support varied contexts of use for different users (teachers/students). Whilst most projects utilised repositories and content management systems for management and curation purposes, several also added context and guidance by presenting them in other formats. It is notable that four projects from the OMAC strand chose to use the Open University labspace to present their whole course (RLT for PA, ASSAP, Learning to Teach Inclusively and IPR4EE). Even though RLT for PA included the course on the open labspace and deposited into JORUM they also presented their materials on a website which facilitated different kinds of use...

Although each area of the site is carefully structured to allow for increasing levels of familiarity and ease, it is entirely possible for individuals to mark out very specific pathways and ‘journeys’ in accordance with specific research aims. By switching from one room to another it is possible to access different perspectives on similar material depending on the vantage point of the contributor. In this respect the resource offers every opportunity to develop informed, dynamic and multi-faceted teaching practice that is alive to different conceptions of knowledge – experiential, embodied, theorized, intellectual, emotional, intuitive etc. There is no attempt to order or prioritize the material into specific hierarchical categories; indeed it is the imbrication of all forms of information that makes the site so rich and enabling. David Shirley, Manchester Metropolitan University RLT for PA

 

Tracking emeged as a priority area for pilot programme projects and continued to be an area of significant activity during this phase. This links to sustainability and the provision of evidence around use of the OERs. Most of the platforms adiopted by projects included some form of tracking mechanisms, but Google Analytics stands out as the key tracking software used by projects. The SCOOTER project invested much effort into Search Engine Optimisation to enhance discoverability and took tracking very seriously. They produced guidance for projects SEO Guidelines and SEO "How to" resources and postulated that returning visitors may be one indication of re-use.

 

There were less issues raised around JORUM during this phase which probably reflects previous experience of projects, growing expertise in the JORUM team and ongoing technical developments to support ukoer projects. Improved harvesting mechanisms appear to have resulted in more actual OERs being deposited in JOURM (as opposed to just links). Pilot phase projects indicated a preference to deposit on one place (to facilitate updating and management) but, as previously mentioned, this year projects have tended to deposit in several places. It may be that projects have recognised that depositing in more than one place improves google rankings, and ultimately improves accessibility. (See also discussion around JORUM which includes specific comments from projects)

 

Cross strand evidence

Strand evidence: cascade | release | omac

 

What legal and IPR issues emerged?

Projects were supported during this phase by JISC Legal and the Web2Rights OER IPR Support project. The latter produced a range of support materials including   diagnostic tools, a CC compatability wizard and a Risk calculator. The Risk calculator has emerged as the most used tool across a range of sectors (300-400 visitors a week). These tools and the support offered have had significant impact on project teams, particularly those new to OER. Projects were aware at the start that IPR challenges would be an issue but often had not anticpated how they might solve them.

The SCOOTER project aimed to release OERs under the Creative Commons BY SA Licence, although in reality the decision had to be tailored to each individual resource. In order to fully understand the copyright nuances, the MedDev OOER Copyright Checklist was used to provide a framework to work by. Unfortunately the ToolKit resources went off-line at a critical time the start of the project, but Web2Rights were a wonderful support and provided information not just via their website but by personal communication many times. In fact, in many instances, we would not have proceeded with releasing OER without the work of this team SCOOTER

 

One issue emerging during this phase included uncertainties around the meaning of commercial use and the ipact on this relating to CC license choice. John Roberston from JISC CETIS has generated some interesting discussion around this in relation to choice of CC licenses in his technical synthesis blog post) and Amber Thomas, JISC provides an  excellent description of issues realting to licensing for phase 2 of ukoer in the 'Importance of licensing section' of her OER turn blog post

 

Other issues inlcude the challenges of integrating what projects have learned into daily practice and the need for OER to be incorporated into existing IPR policies or for specific OER IPR policies to be developed. In fact strategic positioning of open licensing within HE instiutions has been noted as very challenging and depends of institutional cultures and structures. However open licences can contribute to sustainability of materials as they surface them within an institution. Projects found that legal advice needed to be tailoered for different institutions, making general advice insufficent in most cases and indicating that ongoing specialist support is highly important.

 

OSTRICH faced a particular difficulty supporting the University of Bath, where (very unusually) academics own the IP of their 'scholarly output' including learning materials. A new Deed of Licence was developed by the Intellectual Property Legal Services team to allow academics to give the institution permission to release resources as OERs. However, other IP problems emerged from this situation.

 

JISC Terms and Conditions for funded projects, such as the requirement to produce signed consortium agreements or Memoranda of Understanding early in the project lifetime, can pressurise teams to conclude negotiations that need more time and care. There may be problems releasing withdrawn units of study as OERs. IP may be governed by more than one pre-existing agreement, may require sign-off from a range of external parties, and may become blurred as content is developed and redeveloped. Bath IPR FAQs offer guidance on these and other issues.

 

The notion of degrees of openess emeged during the pilot phase and has carried forward into this phase with the ALTO project developing a UAL comons Licence, based on the Canadian BC Campus licence where staff can choose how open to be. This was intended to build trust between staff from s highly autonomous individual colleges and engage them in first steps to opening resources more widely.

 

During this phase we have seen the outcomes of the Hargreaves Review of IP which is likely to have a significant impact on the UKOER programme. The proposal to establish a Digital Copyright Exchange (DEC) will make it easier to locate rights holders and seek permissions to use their works. This has not impacted on this phase of work but may do on phase 3.

 

John Roberston form JISC CETIS has generated some interesting discussion around choice of CC licenses in his technical synthesis blog post.

See also: IPR_Support

Cross strand evidence

Strand evidence: cascade | release | omac

 

Are the OERs accessible to all intended groups?

(Technically, legally and pedagogically)

Many aspects of accessibility are inherent in the notion of making resources open and were stipulated as part of project requirements (such as open licensing, deposit in JORUM, etc.)

In addition to the legal requirements of OER preparation, best practice also provides for compliance with accessibility standards. All resources provided by this project underwent a thorough editing and formatting process to ensure that they were as accessible as possible (e.g. proper formatting of headings, sub-headings, body text and hyperlinks and the provision of alternative text for images). Document properties were also completed to ensure that provenance was embedded within the core of the resources themselves. Again, this represents a significant undertaking which could be ameliorated by the inclusion of the practices at the point of resource creation.   LEARNING LEGACIES 

 

Where openness or accessibility was compromised, either due to copyright clearance issues or technical issues, projects responded pragmatically and chose not to release problematic resources. Long term tracking and evaluation to find out if all intended audiences can access OERs may reveal more in the future, but what did emerge was a significant amount of thought about how different audiences might use OERs. This often translated into OERs being made available in a range of formats, at different levels of granularity and through several platforms. This effort to make OERs pedagogically accessible  went beyond the 'pedagogic wrappers' which became so prominent in the pilot phase (although many projects did adopt this approach). Due to these efforts it is likely that the OERs released during this phase will be accessible to, and used by, a wide range of audiences/groups. There is a consensus that less complex granular items are more easily resused and re-purposed but that there is a fine balance between facilitating engagement and retaining discipline related contexts and complexities.

 

As previously mentioned, most projects deposited OERs in repositories and content management systems (CMS) to facilitate management and curation, but several also adopted a range of other approaches to increase access, including added guidance about how to use OERs for different contexts, review and comment features. Whilst there was a general view that using open platforms and open document formats increased accessibility - the CPD4HE project did highlight that open document formats could be seen as a barrier if it meant downloading new software. Only a few projects reported on activities to ensure that OERs were accessible through mobile technologies, but this did emerge as an area for future consideration.

 

Sustainability of links to materials was noted as an ongoing problem.

 

Improved accessibility for disabled students was a significant driver for some projects (SPACE and DHOER).

 

Cross strand evidence

Strand evidence: omac | release

 

Are the OERs adaptable for re-use and re-purposing?

(likely to link to impacts/benefits)

There was an intention that materials would be adaptable/repurposable but not much evidence around actual use as yet.  Appropriate licencing has been selected to facilitate reuse and re-purposing, and efforts by projects involved with the NHS addressed issues around patient and practitioner consent. See also discussion relating to accessibility above, particularly around issues of granularity/context and making content available in a range of formats often across several platforms.

TIGER created the resources not only for local users, but also for users all around the country, Europe and the world. TIGER needed to make the resources as generic, open and transferable as possible. The team needed to think about the use, but also reuse and repurposing, making resources that people can easily translate into their own context.

 

In relation to granularity the C-SAP Cascade project noted that

A smaller resource is:

    • less academically credible/ significant
    • supports autonomous learning
    • creates less work for the tutor in introducing for autonomous use - flexibility 

A bigger resource on the other hand:

    • is more academically credible/ significant
    • requires a higher level of self directedness from the learner.
    • creates less work for the tutor in introducing for autonomous use

 

Cross strand evidence

Strand evidence: omac | release 

 

Frameworks to support development and release

The University of Leicester OTTER project from the pilot phase developed the  CORRE framework.

Two OMAC strand projects utilised and adapted the framework.

We found the CORRE framework useful for setting out the process for turning teaching materials into OERs but, like our partner project, DELILA, we adapted it to suit our very small project. (CPD4HE)


DELILA adapted the CORRE framework to provide a practical check-list for rating materials against OER criteria. The project team also made contact with OTTER/ CORRE project members, and welcome continued developments in this area. http://delilaopen.files.wordpress.com/2010/10/adapted-corre.xls (DELILA)

Three of the Release strand projects utilised and adpated the framework

TIGER Quality Framework

A TIGER pedagogical model that will enable other IPE and health and social care professionals to benefit  -TIGER has developed a quality framework which allows professionals to clearly understand how the OERs have been developed. https://openeducationalresources.pbworks.com/w/page/24838164/Quality-considerations

During the project, TIGER adapted the workflow originally produced by Alejandro Armellini for content and resource management with built-in quality criteria (TIGER)


We evolved a workflow based on the CORRE project at Leicester, and used RLO CETL “RLO” specification form as a basis for resource development, in the form of a “quick” specification check list for existing resources and a “full” specification for new resources. These forms served to be a useful tool for staff, and were an essential step in formulating the meta-data to accompany each resource (description, keywords). They were an essential checking point for the provenance of any assets embedded within existing resources. All of this was built up into an Interactive OER Development web page, with links through to relevant forms. (SCOOTER)

 

By and large we let ourselves be led by the tried and tested CORRE framework for transforming teaching materials into OER, developed by the OTTER project and consisting of the consecutive steps of Content, Openness, Re-use & Repurpose and Evidence (DHOER)

 

For the Cascade strand the Leicester CORRE framework proved directly transferable to one of the partner institutions, but in the second was revised to support more ab initio development, becoming the DORRE framework. In both cases, integrating OER development into the workflow of existing academic support teams proved highly successful as a change strategy. 

The depth of detail CORRE gives in the checklists and tracking sheets about the decisions that need to be taken during each stage of the conversion formed a useful starting point for informing and guiding participants in the project. (University of Bath, internal evaluation)

 

Cross strand evidence

Strand evidence: release | omac | cascade


OER collections

 

How are different means of making OERs discoverable within disciplines effective?

The aim of the collections strand was to enhance the discovery and reuse of OER materials, by building collections of materials around particular thematic areas. The assumption underlying the strand was that such collections would provide a critical mass of materials in a particular area, raising the profile and awareness of OERs among teachers in the discipline. Beyond this, though, projects were concerned to ensure that their collections were usable and quality assured - without usability and quality, a mass of materials has little value. Projects achieved these four characteristics of critical mass, usability, quality and high profile through their technical and practice-based choices.

 

Technical choices

The first technical choice for a collection was that of an appropriate platform for the collections.  The majority of projects used a Wordpress blog because of its open ethos (C-SAP), strong developer community (C-SAP, Triton), usability (C-SAP) and large existing user base (Delores). However the Oerbital project used Mediawiki as a catalogue and to prevent duplication, while OF (GEES) developed its own system and map interface.

 

Equally important was the exploration of effective means of pull (ie search) and push (ie feeds) discovery. The projects found that their users expected searches to be "google-like" in their ease of use, personalisation, and production of relevant results.  Searches for OERs were very difficult to automate, as required for the dynamic collections, because of the lack of standardisation and machine readability of licence information and lack of explicit badging as OER (Delores, Triton, OF (GEES), EALFCO).

API and metadata issues mean dynamic collection does not return  the “google-like” relevance that  users expect " (Triton)

 ...for full realization of the potential of OERs in general, adoption by providers of a more standard way of representing information about a resource would be beneficial. The data encoded should be machine-readable and at least include information that explicitly identifies the material as being an OER (Delores)

Projects emphasised that search tools needed to be appropriate, both in the user interface – where OF(GEES) developed a map-based interface for finding fieldwork OERs and in the relevance of results returned.  To classify OERs, and search results projects either adopted existing classification schemes for their domains (C-SAP), used their expert groups to develop a classification (Oerbital) or adopted the local institutional curriculum (Triton).

 

Feeds out from collections sites can place OERs where stakeholders are and raise awareness. For such push discovery standards are needed for describing OER.  The Delores project developed such a standard xml schema for rss feeds from its static collection and from resources discovered through its sux0r filter.  OF (GEES) recommended standard ways of describing location for fieldwork resources which if adhered to by contributors will aid dynamic collection and learning pathways.  C-SAP collections wrestled with the difficulties of providing consistent descriptions of video resources.

 

Triton developed Wordpress widgets for feeds into its site to help keep the site up to date, drawing users in by acting as an information portal rather than just a repository

 

Practice-based choices

Two main practice-based choices emerged as important: the inclusion of "grey" or "non" OERs in order to generate critical mass; and the importance of a "hook" to pull users in.

 

However good the search facilities,a critical mass of potential resources is needed, otherwise search results are sparse and users revert to google. Thus OF (GEES), C-SAP collections, Oerbital and EALFCO argued for the inclusion of "grey" or "non" OERs in their collections. 

There are not many “open” resources currently. It would be remiss to ignore very good resources that are publicly available but not under a cc licence + we’d only have a small number of pins on the map. By broadening the scope we can engage colleagues who are not aware of OER and CC and also provide a collection that might be interesting enough / have enough critical mass for our community to want to look after and keep contributing to (OF (GEES))

These projects all adopted some means of making clear to users which resources were CC-licensed and which were not.

 

Quality assurance for the dynamic collections was achieved largely by searching only a limited selection of previously quality-assured repositories such as JorumOpen,

 

Triton and C-SAP collections emphasised the importance of a “hook” to draw users to their collection in the first place. Following user surveys, C-SAP promoted the visibility of videos and instituted a “video of the week”. For Triton, the “hook” in the form of topical blog postings from  Oxbridge academics in the field of politics, allied with a news feed, form the most prominent part of the site. These are juxtaposed with related OERs but seldom make explicit links to them.

if the focus is discoverability, a broader conception of OER needs to be pursued in order for people to be introduced to open resources. What initially draws people to the site how this translates to OERs is an essential part of this process. (C-SAP interim report)

The difference in the nature of the hook may largely be attributable to the difference in the intended audience for the OERs.  In the C-SAP case, the audience is teaching staff who value videos, and reviews of videos, for re-use in their own teaching, whereas for Triton, the users envisaged for the Politics Inspires site are learners. 

The key factors influencing  [student] use of the site were frequency of posts, global coverage, and quality of materials (Triton)

 

Strand evidence: collections

 

What issues arise in collecting and making OERs available dynamically?

The majority of issues projects encountered in collecting and making OERs available dynamically centred around the lack of consistent metadata in existing resources, provision of consistent metadata about resources on collections sites, the variety of repository APIs, and lack of clear or any licensing information attached to resources.

 

The lack of consistent metadata for describing OERs, the variety of repository APIs, and the low visibility of licensing information severely restricted projects’ ability to collect OERs dynamically (Delores, Triton, C-SAP collections, OF (GEES), EALFCO). The solution adopted by most projects was to restrict their collections to a limited number of repositories whose resources were known to be CC-licensed; C-SAP collections used Google custom searches on these sites; Delores inspected the repositories manually to discover schema for resource description and then wrote scripts to extract resource data; and Triton used php to query the APIs and then cached the rss feeds.

There are many aggregation services we can use to bring in content to Politics in Spires. Often these aggregation services are without API (Folksemantic, OER Commons, MERLOT) and as such we need to fake requests to generate RSS content which we can then search. Services with APIs (Xpert and Jorum) present a problem by having different approaches to querying. Xpert can be searched for a keyword, whereas JORUM has a mandated list of keywords to be searched against.  Xpert sadly was the slowest API, which did lead to issues with timing problems. This all demonstrates that to query five repositories you would need at minimum three pieces of code. It is likely aggregators in future will increase this language diversity, and so complicate algorithms. At present it is useful to have a service which can alter the metadata returned (such as with the flickr API), but effectively proprietary API formats do not assist (Triton)

OF (GEES) used a combination of searching resource descriptions for decimal degree coordinates, and geoparsing, to identify location-specific resources.

 

Even with these solutions, further issues arose with broken links returned from aggregation services (Triton) and with duplication of results (Delores, Triton).

 

Projects found that licensing information is not always attached to the object leading to ambiguity about licensing status,

Often the conditions of use of a resource are not stated in the document itself, but at one remove, perhaps being a catch-all for the source site itself " (Delores)

 

Having collected resources, making them available relied upon designing an appropriate and usable interface. OF (GEES) developed a map-based interface for discovery of its location-specific resources, and a clear traffic light system that made the licensing status of resources instantly visible. Projects using Wordpress chose their themes for the balance of static and dynamic resources (Triton), and to display clearly the attribution, licensing information, and origin of the resources described (C-SAP collections).

 

Those projects that used Wordpress blogs to describe or refer to their static collections, encountered further issues in the metadata attached to the blog postings, which  described the blog post rather than the OER being written about. The solution adopted by Delores was to use WordPress custom fields to record the author, pub date, CC-licence URI, other rights, source and URI of the resource. They adapted the Carrington theme so that this information is displayed in the posts.

 

User testing showed the importance of a critical mass of relevant results (and user frustration with sparse or irrelevant results).  C-SAP collections found that users wanted to be able to filter results, rather than to have a forced choice, and that tag clouds were popular. OF (GEES) found that users were frustrated by having to download resources from a repository rather than scanning them quickly from a web link, which supports C-SAP’s finding that teachers tend to search for OERs to meet a very immediate need and that time is important to them.

 

Strand evidence: collections

 

How are different ways of organising, and guiding users to and through resources effective?

Project sites provided a portal to static and dynamic OER collections and sometimes other information. The majority used a Wordpress blog as the basis of the portal, but took different approaches to linking the resources to the blog posts.  Thus C-SAP used its blog to review resources in its static collection, Delores blog posts described a single OER in the static collection using a standard template, while Triton's "Politics Inspires" blog posts comment on current political topics and are a hook to draw users in - they only occasionally make explicit links to OERs, though relevant podcasts are juxtaposed on the site and OERs are available through a separate tab.  Access to dynamic resources was through search and categorised browse options, to which Triton added "learning paths", ie. collections bookmarked by registered users. 

 

In contrast, the Oerbital projects used Mediawiki to provide a catalogue of the (static) collection and associated discussion with feeds in/out. Wiki catalogue entries describe individual resources and suggest pedagogic uses. They are accessed from the main page by category links beside the names of the expert panel.

 

OF (GEES) provides multiple ways into its collection, via map-based or text search, word cloud, or browsing categories. "[user survey] findings provide empirical support for the development of a map-based interface for discovering fieldwork resources, and for the need to include multiple search/criteria.” “[word-cloud] provides the user with suggestions for possible search terms, together with an indication of whether the search is likely to return few or many resources” user comment, “Map search is great” (OF (GEES) ) Catalogue entries describe individual resources with clear licencing information 

 

 

To enable browsing and filtering of results, projects used a combination of classification into categories and author tagging for their static collections. While C-SAP directed their authors to use a recognised subject taxonomy for tagging, Triton allowed free tagging allied to fixed categories.

 

Delores and OF (GEES) enabled classification of their dynamic collections, OF (GEES) by identification of location-specific coordinates and geoparsing, and Delores through its Waypoint software:

user access to the content [is via Waypoint] software ...which implements an adaptiveconcept-matching algorithm [allowing] browsing of material that has been classified against a set of facetted classification schemes. The user is able to interactively ‘prune’ a classification tree list which responds in real time to the user’s multiple selection of taxonomic labels....Waypoint search/browse implementation [required] the development of a coherent subject taxonomy (classification scheme) and rules which allow classification of the resources against the scheme (Delores)

 

While Delores provided separate interfaces to static and dynamic collections, OF (GEES) gave a unified interface through a subject-specific taxonomy represented as a clickable tree -at top level resources were categorised as generic or location-specific

 

Projects derived their categories by a variety of means: developed by subject librarians and linked to local course structures (Triton); devised by the project's subject experts (Delores); or a recognised social sciences taxonomy (C-SAP) 

 

Guidance through the resources was an area in which projects conducted extensive user testing. They found particularly, that google and Amazon have set user expectations that searches will be simple, personalised, and produce relevant results.

The importance of Google and other popular commercial sites cannot be underestimated; both in terms of the resources they produce and the expectations that they provide the  user. (C-SAP blog )

Users who have become accustomed to google were frustrated with the search and browse facilities of most OER repositories

 the OER descriptions in RSS feeds are not particularly helpful to repositories or to users (e.g. very often the descriptions are of a course overall rather than the particular resources (lecture notes, examples, etc.) made available through the course). (Delores)

 

Projects had to make decisions - which they never entirely resolved - over:

  • the desirable granularity of classification  

in the static collection, how do we select and provide descriptions at the fine level of granularity that Chris wants while also keeping he valuable information of the original course context of the resource;  (Delores blog post "finding OERs")

These decisions were sometimes driven by the constraints of the project, rather than what seemed best for users. Delores, for example, decided in the end to describe the OER at module level, (because this was more efficient of project time), but believed that a more useful grain size would be lecture level. 

  • how to enter multi-site resources on the map interface in the OF (GEES) project, and how close together sites were before being counted as a single site  
  • how describe authorship - which proved particularly problematic for videos on the C-SAP project.

 

Both C-SAP and Triton, in their user testing, found that filtering was important - a few highly relevant results were preferred.  C-SAP users preferred filtering of search results to being forced to choose between categories.

 

Triton also found that searches for an authors name were very common among students, emphasising the importance of attribution.  They ensured that author information was prominently attached to their blog posts 

 

Strand evidence: collections

 

What selection and quality processes are appropriate for dynamic and dynamically collected OERs?

Selection and quality processes depend on questions of

  • What is relevant – establishing the scope of the collection
  • Processes for selecting what is relevant, especially automatically
  • What is meant by “quality”
  • Processes to assure quality – which broke down into assuring academic/pedagogic, technical, and legal quality

 

Projects used their expert groups and user communities to establish the scope of the collection and relevant selection criteria (Delores, OF(GEES), C-SAP collections, Oerbital).

Reviewers have addressed issues related to the granularity and adaptability of research methods as well as issues involved in using generic vs. subject specific methods teaching resources. This has helped the project team to refine the scope of the research methods domain and the scope of potential collection (C-SAP interim report)

 

In defining scope, projects made various assumptions and decisions: 

  • the static collection was, "... based upon the assumption that the core knowledge requirements of engineering design students in the early years of study will be substantially unchanging over an unspecified but extended period of time" (Delores)
  • The OF (GEES) project decided to accept all offered contributions subject to quality control, because if offered then it was assumed that they will be, in some way, useful for fieldwork

 

Criteria included subject relevance, adaptability, level of interest/engagement, accessibility, and ease of use. To what extent a resource needed to be up to date emerged as a matter for debate in OF (GEES).

 

For their static collections, projects relied on the judgement of their expert groups for selection and quality control. Automatic, dynamic, collection, though, presents challenges at present.

At present, entirely reliable automatic judgement-making [of appropriateness, quality, etc] is probably out of reach. However, work in progress may make this more attainable in the short term. Such work includes, for example, the Learning Registry initiative which is seeking to facilitate the collection and provision of the information that would be necessary. Likewise, work on the assessment of information value (including measures of quality) will support this sort of judgement making (Delores)

Delores seeks to increase the relevance of the dynamic results by a combination of bayesian filtering and the Waypoint classification software. User ratings of the relevance of the search results help to train the system.

 

Projects argued that quality needs to encompass pedagogy, technical and licensing criteria.

 

Assuring academic and pedagogic quality was particularly important for the Triton project as resources were associated by users with the Oxford/Cambridge brand. They produced clear policy and guidance documents to help assure the quality of blog postings and the static collections. In addition, the site Administrator and Oversight Team acted as gatekeepers to ensure the reputation of the department and university was upheld, particularly ensuring that sensitive topics were treated with care (Triton).

 

While C-SAP discarded user ratings as a measure of quality, and put emphasis on user reviews instead, Triton  implemented a “like this resource” function in both static and dynamic collections.

 

Assurance of technical and legal quality for both dynamic and static collections entails at least some degree of manual control.  Projects sourced their dynamic collections from a limited number of repositories, manually selected for their assured quality (Delores, C-SAP, OF (GEES), Triton).  The projects took different approaches to the inclusion of material that was not CC-licensed. Oerbital, OF (GEES) and Delores included some that was not explicitly licensed. Triton excluded from its collections aggregation services that returned non-CC licensed material or broken links.

 

Projects aimed to enhance the general quality of resources produced by providing guidance on licensing (OF (GEES), Oerbital). Delores worked with the authors of some non CC-licensed materials to encourage them to license the resources.  Triton required contributors to the Politics Inspires blog to register in order to comply with CC licensing.

 

Strand evidence: collections

 

What issues arise in linking social technology-based marketing and community portals to resources from a number of institutions?

Making clear the original attribution and quality-checked (or not checked) status of OERs in the collection, emerged as a clear user need (C-SAP collections, Triton).  This was particularly so in the Triton project where users saw the Oxford and Cambridge logos on the Politics Inspires blog and tended to assume that the OERs carried the Oxford or Cambridge brand.

...  there was an (inaccurate) assumption that because of the Oxford/Cambridge affiliation that the OER resources generated through the Xpert widget had been reviewed by the institutions (C-SAP user testing of Politics Inspires site)

Negotiating access rights to the blog hosted by one institution but with partners from another also proved difficult (Triton, Delores), as was found previously in the pilot phase projects. The solution adopted by the Triton project was for an access policy generated by an oversight group comprising members from both institutions setting clear standards and guidelines, with access rights managed by Wordpress (Triton).

 

Issues arose in the static collections in evolution of a common style for describing the resources, solved by the Oerbital project through collaborative development on their project wiki.

 

Strand evidence: collections


 

Comments (0)

You don't have permission to comment on this page.