UKOER 3 - Themes Evaluation Toolkit Development
Note by Isobel (16 March). The top table below shows evaluation questions aimed at projects, organised by what I think are the most relevant themes and our focus areas. It can be used to work out what links to google forms we need to make, and how project-specific evaluation questions will map back into our evaluation framework.
The lower table below turns our evaluation questions from the evaluation framework into questions phrased to address individual projects. It also asks specifically related questions about evidence, sometimes with examples of the sort of evidence that might be available.
Not all the questions are relevant to all the projects. I'm wondering whether, in putting something like this out to the projects, we might be better organising the questions by the themes they are addressing, rather than by our focus area. The table below, then, would help us map their evaluation back to our framework.
Suggestions and edits welcome- I am making some notes in green (Lou)
1. we could be thinking about this in relation to the cross phase review. are some of these questions making up a set of survey/interview questions.
2. there are two sets of questions? - programme level questions under the heading "Evaluation Questions" or the framework and and project level questions under the heading toolkit questions - should we present these as two different things - framework and toolkit - we probably did say this in a round about way but just thinking through it out loud...
3. think we do need to have a principle of not overwhelming them with questions but that we hope to stimulate them to better prhase their own questions and see where they fit into the bigger picture, and of course identify gaps.
4. Can we make the toolkit seem less table based (like a framework) and present it more visually - expanding mind map type thing??
is a suggestion - see what Inspitre research did for jisc curriculum delivery programme, not used with projects though
http://www.inspire-research.co.uk/EvaluationMapDelivery.html
Interestingly way back I used concept maps to synthresise institutional strand...
one big map Pilot Phase Institutional Strand Synthesis
example of smaller map for a particular section Institutional Strand OER Release
IJF 16/3/12
|
|
|
Practice Change in HE |
Development/release |
Organisation/institutional stakeholders |
Stakeholder/other sector |
Impacts/benefits |
|
Theme A : Extend OER through collaborations beyond HE
|
|
|
|
|
|
|
|
Element i You will work in partnership with organisations from another sector in order to release and/or collect OER materials that meet their identified needs. Potential sectors for involvement in the partnership are:
another sector of education/training
public sector agencies
private sector companies
3rd sector organisations ie charities and voluntary groups.
|
ReACTOR HALS ALTO PARiS DEFT CORE-SET OFB |
How has engaging in open partnerships across sectors affected the practices of the HE staff involved in your project?
|
In what ways has collaboration with your partners supported development, collection and release of OERs?
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What measures have you taken to ensure that your OER are accessible to all intended groups (pedagogically and technically)? What legal and IPR issues emerged during your project, and how did you overcome them?
In what ways do you think your discipline, or sector (HE, FE, NHS) impacted on the development and release of OER?
|
What institutional/organisational issues and challenges have you met in working with another sector to develop, release and re-use OERs?
How has your institution ensured embedding and sustainability of open practices?
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What institutional enablers and barriers to adoption of open practices and OER have you found among your stakeholders and how have you addressed them?
|
How have you attempted to meet the priorities of your sector?
What has been the impact of OER development, release, and re-use on the practices of your partners or stakeholders in other sectors?
What enablers and barriers to adoption of open practices and OER have you found among your partners/collaborators in other sectors?
What kinds of OERs are being adopted and re-used by your partners/stakeholders in other sectors, and how?
What issues have you found in developing and sharing collections of OERs across institutional/sectoral boundaries?
What issues over curation of OER collections have you found when working across institutions or stakeholder groups?
How are the cultures of your stakeholders/partners in other sectors being challenged, strengthened, contested, or changed through the availability of OERs?
How have your partners/collaborators in other sectors integrated OERs sustainably into the curriculum processes of their organisations?
|
In what ways have you found that OERs or OER release process benefits your stakeholders?
In what ways has your approach to OER release and re-use enhanced the profile and reputation of your institution?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Theme B: Explore OER publishing models
|
|
|
|
|
|
|
|
Element i Collaboration with commercial publishers. You will work with one or more commercial publishers in collecting or releasing OER. This might include university presses, digital publishers, textbook providers or other forms of commercial publisher.
|
HALS ALTO PublishOER
|
How has engaging in open partnerships across sectors affected the practices of the HE staff involved in your project?
|
How has working with commercial publishers supported the development, collection and release of OER?
What means have you used to make OERs discoverable?
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What legal and IPR issues emerged during your project and how did you overcome them?
|
What institutional/organisational issues and challenges have you met in working with another sector to develop, release and re-use OERs?
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What institutional enablers and barriers to adoption of open practices and OER have you found among your stakeholders, and how have you addressed them?
|
What issues have you found in developing and sharing collections of OERs across institutional/sectoral boundaries?
What has been the impact of OER development, release, and re-use on the practices of your stakeholders in other sectors?
What enablers and barriers to adoption of open practices and OER have you found among your stakeholders in other sectors?
What issues over curation of OER collections have you found when working across institutions or stakeholder groups?
|
In what ways have you found that the OERs or OER release process benefits your stakeholders?
In what ways has your approach to OER release and re-use enhanced the profile and reputation of your institution?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Element ii Using a range of openly-licensed collections of materials as the basis for new resources. You will release predominately materials that have been developed from other collections of open materials. For example, you may wish to draw on openly licensed resources from JISC Digitisation collections, or from other collections of OER and other suitable material.
|
CORE-SET Great Writers ORBIT |
|
How have you drawn on existing OER collections in developing and releasing OER?
What technical issues have you encountered in collecting and making OER available?
What means have you used to make OERs discoverable?
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What legal and IPR issues emerged during your project and how did you overcome them?
What range of OER exist in your area, and what gaps are you addressing?
|
|
What kinds of OERs are being adopted and re-used by your partners/ stakeholders in other sectors, and how?
What issues have you found in developing and sharing collections of OERs across institutional/sectoral boundaries?
What issues over curation of OER collections have you found when working across institutions or stakeholder groups? observation?
|
In what ways has your approach to OER release and re-use enhanced the profile and reputation of your institution?
What other benefits (apart from reputation) of OER release and re-use have you found?
|
|
Theme C: Addressing sector challenges |
|
|
|
|
|
|
|
Element i Supporting emerging forms of learning and accreditation. You will release and/or collect together material in ways that support an identified non-traditional mode of study and accreditation: for example, the accreditation of prior (experiential) learning, personal learning, independent online learning communities, mentorships, collective learning, new approaches to online learning or other emerging initiatives. The materials will need to be used within these activities within the timeframe of the funded project.
|
COMC ORBIT DEFT FAVOR
|
What methods did you use to engage your stakeholders?
What changes have you observed to the practices of teaching staff as a result of the project?
|
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What measures have you taken to ensure that your OER are accessible to all intended groups (pedagogically and technically)?
|
How has your institution ensured embedding and sustainability of open practices?
What has been the effect on sustainability of involving students in the OER development, release, re-use lifecycle?
How has your institution supported staff to change practice and develop skills/knowledge?
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What institutional enablers and barriers to adoption of open practices and OER have you found among your stakeholders, and how have you addressed them?
What changes in the culture or policies of your institutional stakeholders (including teaching staff) have you observed?
|
How have you attempted to meet the priorities of your sector?
What kinds of OERs are being adopted and re-used by your stakeholders, and how?
How are the cultures of your stakeholders/partners in other sectors being challenged, strengthened, contested, or changed through the availability of OERs?
|
What impact have you found of OER development and release on the student experience?
What have you found is the impact on staff of OER release and use?
In what ways have you found that the OERs or OER release process benefits your stakeholders?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Element ii Involving academics on part-time, hourly-paid contracts. You will work directly with academics on part-time, hourly-paid contracts – staff in this situation deliver a sizeable percentage of the undergraduate curriculum in England. You will work directly with them, supporting them in releasing the materials they have created as OER. We would hope to see evidence of OER release and sharing amongst academics on hourly-paid contracts, and evidence that they have been drawn more closely into institutional teaching culture via their involvement in this project.
|
Sesame FAVOR |
What methods did you use to engage your stakeholders?
How did you support practice change among your stakeholders?
|
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What legal and IPR issues emerged during your project and how did you overcome them?
|
How has your institution ensured embedding and sustainability of open practices?
How has your institution supported staff to change practice/develop skills/knowledge?
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What enablers and barriers to adoption of open practices and OER have you found among your stakeholders and how have you addressed them?
What changes in the culture or policies of your institutional stakeholders (including teaching staff) have you observed?
|
What kinds of OERs are being adopted and re-used by your stakeholders/partners in other sectors, and how?
How are the cultures of your stakeholders in other sectors being challenged, strengthened, contested, or changed through the availability of OERs?
|
What have you found is the impact on staff of OER release and use?
In what ways have you found that the OERs or OER release process benefits your stakeholders?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Element iii Enabling Sustainable practice. You will release or collect materials supporting the development of sustainable activity within higher education, either in terms of the environmental impact of higher education practice and/or in terms of supporting higher education during a time of limited resource availability.
|
PARiS PublishOER ALTO |
What methods did you use to engage stakeholders?
What changes have you observed to the practices of teaching staff as a result of the project?
How did you support practice change among your stakeholders?
|
How have you ensured that your OER are adaptable for re-use and re-purposing?
What legal and IPR issues emerged during your project and how have you overcome them?
|
How has your institution ensured embedding and sustainability of open practices?
What has been the effect on sustainability of involving students in the OER development, release, re-use lifecycle
How has your institution supported staff to change practice/develop skills/knowledge? )
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What institutional enablers and barriers to adoption of open practices and OER have you found among your stakeholders and how have you addressed them?
|
How have you attempted to meet the priorities of your sector?
What kinds of OERs are being adopted and re-used by your stakeholders/partners in other sectors, and how?
How are the cultures of your stakeholders/partners in other sectors being challenged, strengthened, contested, or changed through the availability of OERs?
How have your partners/collaborators in other sectors integrated OERs sustainably into the curriculum processes of their organisations?
|
What have you found is the impact on staff of OER release and use?
In what ways have you found that the OERs or OER release process benefits your stakeholders?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Element iv Preserving the breadth of UK higher education in a time of consolidation. You will release or collect OER material that preserves subject areas and teaching approaches that may otherwise be lost to UK Higher Education.
|
|
|
What range of OER exist in your area, and what gaps are you addressing?
What measures have you taken to ensure that your OER are accessible to all intended groups (pedagogically and technically)?
|
How has your institution ensured embedding and sustainability of open practices?
|
How have you attempted to meet the priorities of your sector?
What kinds of OERs are being adopted and re-used by your stakeholders/partners in other sectors, and how?
What issues have you found in developing and sharing collections of OERs across institutional/sectoral boundaries?
What issues over curation of OER collections have you found when working across institutions or stakeholder groups?
|
What have you found is the impact on staff of OER release and use?
In what ways have you found that the OERs or OER release process benefits your stakeholders?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
Theme D: Enhancing the student experience
|
|
|
|
|
|
|
|
Element i Resources to support university applicants. You will release or collect resources supporting the decision making of university applicants (specifically but not limited to the 16-19 age group). These resources will aim to give applicants a flavour of study at a particular course or level, and may also serve to prepare them for higher-level study in this area. One area of interest would be in resources that would introduce prospective students to the idea of online study.
|
Great Writers OFB |
|
What means have you used to make OERs discoverable?
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs?
What measures have you taken to ensure that your OER are accessible to all intended groups (pedagogically and technically)?
|
|
What kinds of OERs are being adopted and re-used by your stakeholders/partners in other sectors, and how?
|
What impact have you found of OER development and release on the student experience?
In what ways has your approach to OER release and re-use enhanced the profile and reputation of your institution?
What other benefits (apart from reputation) of OER release and re-use have you found?
|
|
Element ii Drawing on student-produced materials. You will release or collect student produced work (student notes, assessed work, student-developed learning materials) under an open license.
|
ReACTOR COMC FAVOR |
What changes have you observed to the practices of teaching staff as a result of the project?
How did you support practice change among your stakeholders?
|
What means have you used to make OERs discoverable?
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
How have you organised, and guided users through, your OERs? What measures have you taken to ensure that your OER are accessible to all intended groups (pedagogically and technically)?
What legal and IPR issues emerged during your project and how did you overcome them?
|
How has your institution ensured embedding and sustainability of open practices?
What has been the impact of OER development, release, and re-use on the practices of your institutional stakeholders?
What institutional enablers and barriers to adoption of open practices and OER have you found among your stakeholders and how have you addressed them?
What has been the effect on sustainability of involving students in the OER development, release, re-use lifecycle
What changes in the culture or policies of your institutional stakeholders (including teaching staff) have you observed?
|
What kinds of OERs are being adopted and re-used by your stakeholders/partners in other sectors, and how?
How are the cultures of your stakeholders/collaborators in other sectors being challenged, strengthened, contested, or changed through the availability of OERs?
How have your partners/collaborators in other swectors integrated OERs sustainably into the curriculum processes of their organisations?
|
What has been the effect on sustainability of involving students in the OER development, release, re-use lifecycle? What impact have you found of OER development and release on the student experience?
What other benefits (apart from reputation) of OER release and re-use have you found?
Has your stakeholders’/partners/ appreciation of the benefits of OER changed through your project?
|
|
|
|
|
|
|
|
|
|
Focus area |
Evaluation question |
Toolkit questions |
Evidence questions |
Projects & their current questions |
|
Practice change
|
What are the main enablers and barriers to adoption of open practices and open content in FE and HE? immediate thought is - are we assuming they know what we mean by open practice, are we assuming that they do it. should we ask them if they have observed any, what it constitutes for them - this might be very revealing for future papers.... are we going to distinguish between different types of practice - academic practice, learner practice, institutional practices
this means we'd have to add questions |
What have you found enables adoption of open practices?
What barriers have you found to adoption of open practices?
|
|
(Overarching) |
|
How are different approaches overcoming barriers to adoption of open practices effective?
|
How did you overcome barriers to adoption of open practices?
|
How do you know you were successful? (eg. number of staff adopting open practice; institutional policy changed) |
Which aspects of the new approaches in the project were successful? (COMC) |
|
|
How are different means of engaging a variety of stakeholders and embedding open practices effective?
|
What methods did you use to engage your stakeholders?
Which methods were successful?
|
How do you know they were successful? (eg. range of stakeholders active in community) |
Have these been successful in engaging community? (CORE-SET)
Have these been successful in engaging community? (core-set)
Are staff, students and other participants contributing to the classes. (COMC)
Have students actively participated in the classes and engaged with both the ‘open’ content and this approach? (COMC)
Has each class enlisted external contributors as speakers, experts and/or practitioners? (COMC)
What level of external visitors and contributors has each class received? (COMC)
Has the English Faculty embraced the project and has it been embedded as a resource? Has the project provided an effective case study for impact and collaboration? (great writers)
Has the community engaged with the project? (Favor) |
|
|
What changes to individual academic practice are fostered by OER development and use?
|
What changes have you observed to the practices of academics as a result of the project?
|
What evidence do you have to support your observation? (eg. number of staff sharing OER increased by x%) |
Has the Nottingham Advantage Award team embraced delivery of materials through the U-Now site? (Paris)
Are staff increasing their sharing of materials? (Alto)
Are staff using the OERs to reuse materials and adapt? How has use of OER influenced academic activity? Are there any indicators of efficiency savings (Alto)
Has the project initiated a community of sharing amongst part-time language tutors? (Favor) |
|
|
What changes to stakeholder culture and policies are fostered by OER development and use?
|
What changes in the culture or policies of your stakeholders have you observed?
|
What evidence do you have to support your observation? (eg. policies changed) |
|
|
|
How are different means of supporting practice change by projects effective |
How did you support practice change among your stakeholders?
Was your support for practice change successful? |
What evidence do you have that your methods were successful? (eg. positive feedback from workshop participants) |
How effective is the support / training offered to staff to develop their knowledge and understanding of OERs and the ALTO UK system (Alto)
Are guidance and support materials useful? Can any improvements be made? Are they being used? (Sesame) |
|
|
How has OER release affected academic practice of staff running HEA accredited courses or schemes of professional development that meet the UK Professional Standards Framework for Teaching and Supporting Learning in Higher education (UK PSF)?
|
|
|
|
|
|
Development/release issues
|
How are different approaches to making OERs discoverable effective?
|
What means have you used to make OERs discoverable?
Which means have been successful? |
What evidence do you have that you have succeeded in making your OERs discoverable? (eg. hits on your collection) |
What are the pros and cons for different stakeholders of the different models investigated? (publishoer)
Are our OER discoverable? Do they impact on the user? (Hals)
Can students find OER? Can they critically evaluate it? (Hals)
Are our OER discoverable nationally and internationally? Have they made a difference? What granularities and types of resource are most popular? (Hals)
What means of social networking are most effective in making OER found? How effective are APPs in promoting OER? (Hals)
How easily can appropriate content be discovered and re-used (given a particular set of content and technology choices)? (Orbit) |
|
|
How do different means/formats for making OER available suit the needs of various stakeholder groups? |
What means (eg. platform, format, interface) have you used to make OER available to your stakeholder group?
|
How do you know this meets the needs of your stakeholder group? (eg. focus group discussion) |
Are all stakeholders happy with the outputs of the project? (COMC)
Are eBooks useful in teaching and learning? (great writers)
Can an effective method of delivery be found? (great writers)
What types of resources would be most useful to end users – looking at subject, format, type and size? (Sesame)
Are the resources created as part of this project available in open formats that support use and re-use? (Paris)
What qualities should resources possess? What kinds of resources are considered most useful? (orbit)
|
|
|
How are different ways of organising, and guiding users to and through, resources effective for different stakeholder groups?
|
How have you organised, and guided users through, your OERs?
Have your methods been effective in meeting the needs of your stakeholders? |
What evidence do you have that your methods have been effective in meeting the needs of your stakeholders? (eg. usage tracking) |
What information about resources would make them more useable? Are our release platform and other proposed release routes appropriate? (Sesame)
Are the user interfaces appropriate for learners? (CORE-SET)
Does our Discovery point and other release routes facilitate uptake? (Sesame)
Does the information provided about them aid their use? (Sesame)
|
|
|
How does cross sector collaboration support the collection and release of OERs?
|
In what ways has collaboration with your partners supported development, collection and release of OERs? |
What evidence do you have to support your views about the collaboration? (eg. decisions made as a result of stakeholder input) |
Collaborative/reflexive elements, engagement of partners, transferability and scalability (Deft)
Feasibility of OER in non-HE contexts, transferability and scalability of approach (Deft) |
|
|
How does working with commercial publishers support the collection and release of OER?
|
How has working with commercial publishers supported the development, collection and release of OER? |
What evidence do you have that working with commercial publishers has supported the development, collection and release of OER? (eg. decisions made as a result of publisher input) |
|
|
|
How does development and release of OER draw on existing OER collections?
|
How have you drawn on existing OER collections in developing and releasing OER? |
What is the evidence that you have drawn on existing collections? (eg. number of existing OER drawn on) |
|
|
|
What technical issues arise in collecting and making OERs available?
|
What technical issues have you encountered in collecting and making OER available?
How have you overcome the technical issues? |
What evidence do you have of overcoming technical issues (eg. technical decisions made, widgets developed) |
Has the project team adapted the technology? (core-set)
Can ebooks be enhanced by the project? (great writers) |
|
|
Are the OER accessible to all intended groups (pedagogically and technically)?
|
Are your OER accessible to all intended groups (pedagogically and technically)?
|
What evidence do you have that OER are accessible to all intended groups? (eg. user needs analysis, focus groups) |
Early questions about appropriateness of the resources, aligned to curriculum, accessible, etc? (reactor)
Is the material easy to utilise and appropriate to target group? (OFB)
Is material understandable and easy to utilise? (OFB)
Have disability needs been met? (OFB)
Is the structure & material identified the most relevant to include? (OFB)
Do the resources created by the Ear Foundation meet the identified needs of the target audience (as identified through the needs analysis at the start of the project) (Paris)
Are the collections useful in teaching and learning? (Great writers)
Fitness for purpose, accessibility, uptake (Deft)
Is newly-created material appealing and pedagogically sound? (Favor)
Are tools and interfaces easy to use? Can processes be easily followed? Can any improvements be made? (Sesame)
Are resources accessible? (Sesame)
|
|
|
How are different means of making OER adaptable for re-use and re-purposing effective?
|
How have you ensured that your OER are adaptable for re-use and re-purposing?
|
What evidence do you have that your methods are effective? (eg. evidence of re-use or re-purposing; user needs analysis) |
|
|
|
What legal and IPR issues emerged?
|
What legal and IPR issues emerged during your project?
How did you overcome the legal and IPR issues?
|
What evidence do you have of having overcome the legal and IPR issues? (eg. new institutional IPR policy, author agreements) |
Do all resources comply with copyright regulation and risk levels set by each partner (Paris)
Are the OER collected available as discrete items, with licence embedded (if available)? (great writers)
Adherence to QA indicators, accessibility; IPR and copyright (Deft) |
|
|
How easily can OERs be mapped to the HEA professional standards framework and what are the gaps?
|
|
|
|
|
|
How far does the subject discipline or partner sector impact on the development and release of OER?
|
In what ways do you think your discipline, or sector (HE, FE, NHS) impacted on the development and release of OER |
What evidence do you have to support your conclusion? |
|
|
|
What range of OER are available?
|
What range of OER exist in your area?
Have you identified any gaps in OER in your area?
|
What evidence do you have to support your observations? |
Are all the Open Class sites being populated with resources and are these being well curated (COMC)
What subject OER already exist internally and externally (Hals)
Have gaps been identified? (great writers)
Are the collections of appropriate quality and breadth? (great writers)
Has the number of resources in the Language Box repository increased significantly? (Favor)
Are there sufficient resources in terms of number and variety to be useful? (Alto) |
|
|
What support is needed for OER release by projects, OER practitioners and senior managers in HEI?
|
What support for OER release has your project, and its participants and stakeholders required? |
|
|
|
Organisational/institutional issues
|
What are the cross sector issues and challenges in developing, releasing and re-using OERs?
|
What issues and challenges have you met in working with another sector to develop, release and re-use OERs?
|
What evidence have you got to support your observations? |
What were the benefits or challenges arising from using OER in a commercial venture? (Hals) |
|
How do different types of institution or stakeholder ensure embedding and sustainability of OER and open practices?
|
How has your institution or stakeholders ensured embedding and sustainability of open practices? |
What evidence do you have? (eg. changed curriculum development process) |
|
|
|
How do institutions support staff to change practice/develop skills/knowledge?
|
How has your institution supported staff to change practice/develop skills/knowledge?
|
What evidence do you have (eg. recognition scheme set up; workshop programme) |
Has the project work of part-time language tutors been recognised in their institution? (Favor)
Is the Department able to sustain supporting the technical infrastructure? (Sesame) |
|
|
What is the impact of OER development, release, and re-use on the practices of different stakeholders or sectors?
|
What has been the impact of OER development, release, and re-use on the practices of your stakeholders?
|
What evidence do you have of this impact? (eg. number of stakeholders continuing open practices) |
Are part-time tutors prepared to continue to produce OER? Is the weekly class programme able to support the work on an ongoing basis? (Sesame)
Can students find OER? Can they critically evaluate it? (Hals) |
|
|
What are the main enablers and barriers to adoption of open practices and OER in different stakeholder groups?
maybe need to clarify that this means within the institution so eg, teaching staff, librarians, learning technology staff, ed developers, etc...
|
What enablers and barriers to adoption of open practices and OER have you found among your stakeholders?
|
What evidence do you have of these enablers and barriers (eg.focus group discussion) |
Do the materials meet needs of teaching staff, students or external stakeholders? (Alto) |
|
|
Stakeholder/Cross-sector issues interesting to have a separate section - I think we called the above section institutional/organisational issues because not all partners were ed institutions.
I think there is a value in highlighting stakeholders separately. - to get them to think about partners and other stakeholders. may mean we can change institutional/organisational issues to just educational institution issues... |
What ways of supporting sector priorities are effective for different sectors?
|
How have you attempted to meet the priorities of your sector?
Which ways of supporting your sector priorities were effective?
|
How do you know your methods were effective? (eg. user survey) |
What resources (subject and type) are identified as being most needed by Ear Foundation target audience? (Paris) |
|
What kinds of OERs are being adopted and re-used by different stakeholders, and how? |
What kinds of OERs are being adopted and re-used by your stakeholders, and how? |
How do you know? (eg. analytics, user feedback) |
How effective are the OERs? (Orbit) |
|
|
What issues arise in developing and sharing collections of OERs across institutional/sectoral boundaries
|
What issues have you found in developing and sharing collections of OERs across institutional/sectoral boundaries?
|
|
|
|
|
What issues arise in curation of OER collections across institutions or stakeholder groups?
|
What issues over curation of OER collections have you found when working across institutions or stakeholder groups?
|
What evidence do you have to support your observation? |
Are all the Open Class sites being populated with resources and are these being well curated (COMC) |
|
|
How are the benefits of OER usage and open practices appreciated? How does this change through engagement with a UKOER project?
|
Do your stakeholders appreciate the benefits of OER usage and open practices? Has their appreciation changed through your project?
|
What evidence do you have for your observations (eg. reward and recognition scheme instituted) |
Does the Ear Foundation see benefit in continuing an OER initiative? (Paris)
Have external partners found being involved in OER a useful experience? (Hals)
Has the project met or addressed the goals and objectives? Has the project articulated the benefits of OER to the teacher education sector? What other outcomes have been noted (tangible and intangible)? (Deft)
How did part-time tutors find the experience of generating OER to support their courses? What are the main motivations for, and barriers to, use and creation of OER? Are part-time tutors prepared to continue to use and produce OER beyond the end of the project? |
|
|
How are existing stakeholder cultures being challenged, strengthened, contested, changed etc through the availability of OERs?
|
How are the cultures of your stakeholders being challenged, strengthened, contested, or changed through the availability of OERs?
|
What evidence do you have to support your observations? (eg. participant survey) |
|
|
|
|
How can OERs be integrated sustainably into different curriculum processes of the various stakeholder institutions?
|
How have you integrated OERs sustainably into the curriculum processes of your stakeholder institutions?
How effective is the integration in ensuring sustainability?
|
What evidence do you have that the integration is effective (eg. number of OER integrated into curriculum) |
Have we put in place robust plans for sustainability? (core-set)
Early questions about appropriateness of the resources, aligned to curriculum, accessible, etc? (reactor)
Are sustainability resources embedded in taught curriculum at Nottingham? (Paris)
Are Ear Foundation resources embedded within taught courses? (Paris)
|
|
Impact/benefits issues
|
How does student involvement in OER development, release and re-use affect sustainability?
|
Have you involved students in the OER development, release, re-use lifecycle?
Has student involvement aided sustainability? |
What evidence do you have that student involvement has affected sustainability (eg. student-developed OERs released) |
|
|
What is the impact on the student experience?
|
What impact have you found of OER development and release on the student experience?
|
What evidence do you have to support your observations? (eg. number of students engaging with resources) |
Has the student experience been enhanced - re skills, opportunities, particpation? (reactor)
Can disabled students particpate in things previously problematic? (reactor)
Have work experience and placement opportunities increased? (reactor)
Have students actively utilised the networks, additional contributors and linked resources? (COMC)
Are students enrolled on Nottingham Advantage Award materials happy with the materials? (Paris)
Are students willing/able to participate in the project? (great writers)
How far does an open textbook support student learning in comparison with using closed resources? How can the benefit of open academic practice be measured? (Orbit)
Methods of engaging students with OER and digital literacy issues (Deft)
Are students using the OER provided for their courses? Do students value the availability of these additional resources?Can anything be done to make the resources more relevant to students? |
|
|
What is the impact on staff of release and use?
|
What have you found is the impact on staff of OER release and use? |
What evidence do you have to support your observations? |
|
|
|
How do the OERs or OER release process benefit various stakeholders?
|
In what ways have you found that the OERs or OER release process benefits your stakeholders?
|
What evidence do you have to support your observations? |
Have the benefits been met for different stakeholders? (reactor)
How do the business models impact on user behaviour (Kirkpatrick’s scale) and what are the intended/unintended consequences of embedding third party published content within OER? (publishOER) |
|
|
In what ways do different approaches to OER release and re-use enhance the profile and reputation of UK HEIs?
|
In what ways has your approach to OER release and re-use enhanced the profile and reputation of your institution?
|
What evidence do you have to support your observation (eg. enquiries from potential students increased) |
Have the small test cases of new modes of ‘freemium’ engagement elicited new, and new kinds of, Open Class participant? (COMC)
Is newly-created material appealing and useful to prospective students? Do other universities intend to use them? (Favor)
Are the OER produced being used beyond the Department? Are teachers and learners finding the resources useful? (Sesame) |
|
|
What other benefits are projects finding? |
What other benefits (apart from reputation) of OER release and re-use have you found? |
What evidence do you have of these benefits? |
Have there been cost savings? (reactor)
Have there been health & safety benefits? (reactor)
Are staff using the OERs to reuse materials and adapt? How has use of OER influenced academic activity? Are there any indicators of efficiency savings (Alto) |
|
|
Evidencing and Programme issues:
|
How can we demonstrate that different OER publishing models are effective in aiding discoverability?
|
|
|
What metrics need to be collected/analysed in order to interpret the business model success? (publishOER)
What further research is required to develop the most successful concepts? (publishOER) |
|
How do projects evidence that OERs enhance the student experience?
|
|
|
How far does an open textbook support student learning in comparison with using closed resources? How can the benefit of open academic practice be measured? (Orbit) |
|
|
How do projects claim they will support the relevant sector priority?
|
|
|
|