Learner Use of Online Educational Resources for Learning (LUOER)

Paul Bacsich, Barry Phillips and Sara Frank Bristow

  Sero Consulting Ltd was asked by the Higher Education Academy to undertake a literature review “to provide a greater understanding of the ways in which learners, whether or not in formal education, use online resources to aid their learning experiences and the factors which influence the selection of resources.” It was anticipated that “collectively this work will enable practitioners, policy makers and researchers to adopt more effective evidence-informed or research-informed approaches to their decision-making, research and practice on matters relating to the use of open-educational resources in learning and teaching.”

 Twelve areas of interest for the research were proposed by the Academy:

• Learners’ rationale for searching for online resources
• Types of online resources being sought
• Complexity/granularity of resources being sought
• How resources found are used; Whether learners in some subject areas appear to conduct more searches for online resources than others
• Educational level of resources being sought
• Location of resources
• Extent to which resources are the principal or a supplementary source of learning materials
• Whether or not learners are in formal education
• Enablers and barriers to use of online resources
• How learners retain access to the resources
• Provenance information and copyright status of resources being used.

This report does address issues raised by and investigated by the UKOER projects although learner use of OERs was not a specific programme focus. The review included a consideration of UKOER project reports, the Evaluation and Synthesis final report and this wiki, and other experts and resources.

Summary findings

•  The literature on learner use of online educational resources is very immature, with a lack of meta-reviews. The overwhelming majority of published studies do not generalise beyond their particular contexts of study. There is no consistent methodology. 
• There are significant gaps in the literature: there are almost no meso-level studies, no international comparisons, and very little on learners other than university undergraduates.
• The JISC OER Programme has so far produced relatively little data on learner use (some partial exceptions are noted). This is to a lesser extent true for all OER literature – but the non-OER literature is much richer.

The following are the findings from the literature review that seem most capable of generalisation: 

• In formal learning, the rationale for searching online is dominated by assessment requirements, explicit or implicit.
• There is evidence for the following: 
•  learner need for structure in or above the resources 
• the importance of a task-based pedagogy which guides learner use 
•  student use of multiple methods for discovery (browsing, search engine, tutor and peer guidance) but a particular approach is more shaped by pedagogical task context than by subject area differences (or other contextual variables 
•   student preference for audio over video 
•  student preference for tools that are previously familiar to them 
•  positive student attitudes to sharing.
•  Only one key study could be found that demonstrated OER having an impact on student attainment.
• Only one key study addressed how learners retain access to resources (around half of the sample used bookmarks). 
• Students are found to be generally lacking in their understanding of provenance and quality.
• A more nuanced approach to digital literacy than the ‘digital natives/digital immigrants’ discourse is now gaining traction in the literature. 
• Some evidence exists that the challenge of designing resources for users with unknown characteristics (including their level of prior understanding) acts as a barrier to OER development
• There are few UK or EU universities with institutional policies on OER.

Worthy of note:  

• "It is clear that the JISC/HEA OER Programme and the presence of the Open University/OpenLearn have had a significant impact, unequalled in any other English-speaking country. " nice soundbite
  

• "An emphasis on assessing the experiences of learners situated in postsecondary educational environments is clear, leaving major gaps in our understanding of learner use of OER in primary, secondary, postgraduate, adult (except for university-level at open universities) and informal education."

•  "Generally speaking, we do find a richer set of articles, conference proceedings, and other media when we allow our research to stray beyond the confines of OER alone. Decades of well-cited research undertaken by the library science community in particular thoroughly address the search for, and discovery of, digital library resources in particular; these seem especially applicable to “learner selection and use of OER”." Highlights that focusing on oers rather than learning resources in general may not be that helpful. Maybe the better question is - what do we already know about how learners use learning resources and what is the impact of OERs on that?
• " Types of online resources being sought: JISC OER projects encompassed a wide range of formats and noted the student preference for audio over video confirmed by non-OER work. The project work still seems dominated by supply-side aspects. Non-OER work confirms the commonly held view that today’s learners utilise numerous types of media. They hint at the primacy of Wikipedia and journal material but quantitative information is scarce. Project domination by supply side aspects is reflected in the research study aswell - this is not surprising though as projects have been funded to release OERs rather than focus on use. Phase 2 did intend to examine use more than the pilot phase but the short funding period has hampered this somewhat. This is reflected in many project final reports and in their recommendations relating to project timescales.

• "Possibly the topic needs to be refined to distinguish between “How resources found are used” and “How services providing resources (and other things) are used”.  "

• "There seemed to be no substantial studies of disciplinary difference with regards specifically to OERs. On the non-OER side, select subject-specific studies have been undertaken, but rarely in a manner which allows comparison; the issue also gets confused with that of the “digital native”. It is tempting to suggest that the bigger differences may lie in “how subjects are taught” rather than “what subjects are taught. It is not clear that further research work in this area is fruitful – but clearly the issue needs to be considered in course redesign."

• "In OER most of the more substantial studies have concentrated on undergraduates. On the non-OER side, there is evidence that users of all levels are (on a basic level) engaged in nearly identical search methods despite their differing requirements as learners. There are also many studies asserting that high-level postsecondary students do spend a good deal of time in search of journal articles, despite gloomy prognostications to the contrary from some academics. There is need for more research work on postgraduate student needs, especially since this is an area where the UK excels, yet needs to keep its market lead."

• " On the OER side, the vast majority of resources that we have reviewed (via the papers on them) have been created and then hosted by universities in the USA and the UK – but used by learners from many countries. So there are some interesting “cultural leadership” and marketing issues here. On the non-OER side, many of the items used are Wikipedia (somehow rarely thought of as OER) and full-text journals, where the issues of “where” they are “hosted” are not so clear." Correlates with the TALL study. (Noted by Amber Thomas)
  

• "Extent to which resources are the principal or a supplementary source of learning materials: The answer seems to depend a lot on “what did you design them for?”. There is plenty of evidence from outside this study, over many years, that resources designed for principal use will be used in such a way (and also for supplementary use by others if they are OER). On the other hand, if certain universities take good care not to produce coherent sets of OER for principal use (because of fears of erosion of their customer base) then one cannot be surprised if the only use is supplementary."
  Correlates with TALL’s “delivering” distinction, definitely something to explore (Noted by Amber Thomas)
• "Whether or not learners are in formal education: One of the driving forces for many institutions involved in OER creation and release is to broaden participation and to entice users not in formal education to sample the materials – and then perhaps “enter” formal education (even if online). Thus the literature is bound to be skewed towards those not in formal education – or seemingly not: there is the issue that the learners may be both in formal education and using OER in an informal way, sourcing OER from either their home institution or elsewhere. So the question needs more careful phrasing in future studies. In particular, it is not often realised in the UK how common it is in some countries with well-developed credit transfer systems and per-credit student support loans (US and Sweden in particular) for students to be taking courses from several institutions simultaneously. Very useful: highlights the need to unpick the causation/”conversion” path between content and course-sign ups - comment from Amber Thomas
• "There still seem to be few UK or EU universities with university-wide policies and strategies on OER, not only in terms of their student focus." Some of the phase 2 projects have developed institutional oer strategies - however several have argues that OER does not need separate strategies but should be incorporated into appropriate existing strategies to acheive embedding and sustainability - so teaching, learning and assessment strategies, or policies on qualit 

• " Guide to good practice: A study team should be assembled from a number of institutions and involving external examiners to produce a practical guide for students, staff and external examiners to cover the issues raised by study – and citation – of OER materials including, but not only, Wikipedia and podcasting" This would imply that OERs should be treated differently to learning resources. Institutions already provide guides and teaching sessions and learning resources/activities around resources for studying - is the suggestion that an open article be cited differently? Using and citing from Wikipedia, for example,  is often already included in many information literacy teaching sessions and  teaching resources. How is an open podcast different from a podcast stored in a closed VLE in terms of either use or citation? Do they mean that citations should include licensing information?
• In the non-OER area one of the biggest issues was that very few papers dealt with the learner use of content. Researchers seem far more comfortable dealing with learner interactions with systems, other students or in some cases tutors. Content seems rather passé to researchers, as if constructivism had won when it clearly has not – and even less so at pre-tertiary level. - Interesting

• "The OTTER project at Leicester University undertook probably the most systematic collation of learner feedback with regards to OER (71 students responded) and discovered that the favoured places to access the OER were the institutional VLE and OER repositories" Contradicts TALL study but should highlight that these were distance learning students (may need to check this - their oers were produced for DL students) - based a way from the institution and often in different countries - interacting with institutional systems can be an important connection to the university (as indicated by some of the JISC CDD delivery projects re remote students).

• "Whilst the student responses were not explicit that it was the “re-usability” of the resources which was so highly valued, it would seem fair to comment that resources such as the Virtual Patients may only be available as a consequence of the OER ethos. Bespoke commercial offers would be economically prohibitive and restrictive." This is interesting but it seems that the students were responding to the pedagogic value of the learning resources not the fact that they were open. In fact the follow on project for JISC CDD (G4) did not make the cases open, but put out some sample cases.

These two items in section 5.2.2 are cited as anon but are from the evaluation and synthesis wiki - strand synthesis pages and include a concept map which was part of the sysnthesis mechanism for the institutional strand by McGill.

1. “Institutional Strand – Pedagogy And End Use Issues” (anon 2010) offers a theoretical and future-looking summary of some JISC OER projects’ approach to learner use issues. (One project carried out surveys with learners, but – as analysed above – most felt that more work was needed to understand learner needs and use patterns). [UK]
2. “Institutional Strand – Learners And Other Stakeholders (anon 2010) touches briefly on the subject of students as co-creators of OER. [UK]

    

Research Study

HB to do