Evaluation buddy groups

Evaluation questions

NB does not include project or event specific questions

Significant Stakeholders

all projects have institutional staff as stakeholders and both educators and students as ultimate recipients

CORE-SET

University of Liverpool

http://coreset.liv.ac.uk

Are the user interfaces appropriate for learners?

 
Have these been successful in engaging community?

Has the project team adapted the technology?

What lessons have been learned?

Have we put in place robust plans for sustainability?

students - as producers of content

3rd sector

private sector

public sector

REACTOR

Doncaster College

(Renewable, Environmental

and Construction technology

Open Resources)

http://reactor.fusedworks.com

Early questions about appropriateness of the resources, aligned to curriculum, accessible, etc.

Have the benefits been met for different stakeholders?

Has the student experience been enhanced - re skills, opportunities, participation?

Have there been cost savings?

Have there been health & safety benefits?

Can disabled students particpate in things previously problematic?

Have work experience and placement opportunities increased?

students - as evaluators of content

students as partners/workers on project (interns)

private sector (commercial vendors, industry)

public sector (skills councils)

HE in FE (colleges)

Opening up a Future in Business

Southampton Solent University

http://solentoer.wordpress.com/

Have we identified the right topics? Is the structure clear? What may others add to the framework?

Relevance of materials and structure to check IPR issues/consent & accessibility is working?

What reactions do employers/students/ colleagues have to OERs and the concept of creative commons

Are these resources of appropriate quality? Are they accessible and useable with the target group?

Are there lessons from production to share with our FE colleagues?

How will tutors in FE/6^th form engage with the resource?

students

HE in FE (colleges)

private sector (SMEs)

COMC

Coventry University

Coventry Open Media Classes

http://openmediaclasses.covmedia.co.uk  

Have students actively participated in the classes and engaged with both the ‘open’ content and this approach? 

Are all the Open Class sites being populated with resources and are these being well curated 

Are staff, students  and other participants contributing to the classes.

Have students actively utilised the networks, additional contributors and linked resources?

Has each class enlisted external contributors as speakers, experts and/or practitioners? 

What level of external visitors and contributors has each class received?  

Have the small test cases of new modes of ‘freemium’ engagement elicited new, and new kinds of, Open Class participant?

Which aspects of the new approaches in the project were successful? 

PARiS

University of Nottingham

Promoting Academic Resources

in Society

www.nottingham.ac.uk/open/parisproject

What resources (subject and type) are identified as being most needed by Ear Foundation target audience?

Do the resources created by the Ear Foundation meet the identified needs of the target audience (as identified through the needs analysis at the start of the project)

Are students enrolled on Nottingham Advantage Award materials happy with the materials?

Are sustainability resources embedded in taught curriculum at Nottingham?

Are Ear Foundation resources embedded within taught courses?

Are all stakeholders happy with the outputs of the project?

Are the resources created as part of this project available in open formats that support use and re-use?

Has the Nottingham Advantage Award team embraced delivery of materials through the U-Now site? 

Does the Ear Foundation see benefit in continuing an OER initiative?

Do all resources comply with copyright regulation and risk levels set by each partner

3rd sector (charities)

PublishOER

University of Newcastle

http://www.medev.ac.uk/ourwork/oer/publishOER

What are the pros and cons for different stakeholders of the different models investigated?

What metrics need to be collected/analysed in order to interpret the business model success?

How do the business models impact on user behaviour (Kirkpatrick’s scale) and what are the intended/unintended consequences of embedding third party published content within OER?

What further research is required to develop the most successful concepts?

commercial sector (publishers)

HALS OER

De Montfort University

Health And Life Science

Open Educational Resources

http://vrolfe00.our.dmu.ac.uk/

What subject OER already exist internally and externally

Are our OER discoverable? Do they impact on the user?

Can students find OER? Can they critically evaluate it?

Have external partners found being involved in OER a useful experience?

What were the benefits or challenges arising from using OER in a commercial venture?

Are our OER discoverable nationally and internationally? Have they made a difference? What granularities and types of resource are most popular?

What means of social networking are most effective in making OER found? How effective are APPs in promoting OER?

commercial sector (publishers)

public sector (NHS)

HE in FE

Schools sector

Great Writers

University of Oxford

http://openspires.oucs.ox.ac.uk/greatwriters/

Are the collections useful in teaching and learning?

Have gaps been identified?

Are eBooks useful in teaching and learning?

Can an effective method of delivery be found?

Can ebooks be enhanced by the project?

Are students willing/able to participate in the project?

Are the collections of appropriate quality and breadth?

Are the OER collected available as discrete items, with licence embedded (if available)?

Has the English Faculty embraced the project and has it been embedded as a resource?

Has the project provided an effective case study for impact and collaboration?

commercial sector (publishers)

ALTO UK

University of the Arts London

Arts Learning and Teaching Online UK

http://blogs.arts.ac.uk/alto/

Are staff increasing their sharing of materials

Are staff using the OERs to reuse materials and adapt?

How has use of OER influenced academic activity?

Are there any indicators of efficiency savings

How effective is the support / training offered to staff to develop their knowledge and understanding of OERs and the ALTO UK system

commercial sector (publishers)

HE in FE (colleges)

ORBIT

University of Cambridge

An Open programme and

Resource Bank on Interactive

Teaching for teacher education and development

http://www.educ.cam.ac.uk/research/projects/orbit/

How easily can appropriate content be discovered and re-used (given a particular set of content and technology choices)?

How far does an open textbook support student learning in comparison with using closed resources? How can the benefit of open academic practice be measured?

How effective are the  OERs?

schools sector (primary, secondary)

students

(DeFT)

Sheffield Hallam University

Digital Futures in Teacher

Education

http://deftoer3.wordpress.com/about/

Collaborative/reflexive elements, engagement of partners, transferability and scalability

Feasibility of OER in non-HE contexts, transferability and scalability of approach

Methods of engaging students with OER and digital literacy issues

Adherence to QA indicators, accessibility; IPR and copyright

Fitness for purpose, accessibility, uptake

Has the project met or addressed the goals and objectives? Has the project articulated the benefits of OER to the teacher education sector? What other outcomes have been noted (tangible and intangible)?

schools sector

FE

Commercial sector (industry, publishers)

students

FAVOR

University of Southampton

http://www.thefavorproject.wordpress.com

Has the community engaged with the project?

Has the number of resources in the Language Box repository increased significantly?

Is newly-created material appealing and pedagogically sound?

Is newly-created material appealing and useful to prospective students? Do other universities intend to use them?

Has the project initiated a community of sharing amongst part-time language tutors?

Has the project work of part-time language tutors been recognised in their institution?

part time teachers

students

schools sector

Sesame

University of Oxford

http://www.tall.ox.ac.uk/research/current/sesame.php

What types of resources would be most useful to end users – looking at subject, format, type and size? What information about resources would make them more useable? Are our release platform and other proposed release routes appropriate?

Are tools and interfaces easy to use? Can processes be easily followed? Can any improvements be made?

Are guidance and support materials useful? Can any improvements be made? Are they being used?

How did part-time tutors find the experience of generating OER to support their courses? What are the main motivations for, and barriers to, use and creation of OER? Are part-time tutors prepared to continue to use and produce OER beyond the end of the project?

Are students using the OER provided for their courses? Do students value the availability of these additional resources? Can anything be done to make the resources more relevant to students?

Are the OER produced being used beyond the Department? Are teachers and learners finding the resources useful?

Are part-time tutors prepared to continue to produce OER? Is the weekly class programme able to support the work on an ongoing basis? Is the Department able to sustain supporting the technical infrastructure?

Are our resources useful in terms of subject, file format, type and size? Does the information provided about them aid their use? Are resources accessible? Does our Discovery point and other release routes facilitate uptake?

part time teachers

students (adult learners, part time learners)

Teesside Open Learning Units

Teesside University

Academic Practice in Context:

OERs for Exploring Discipline-based Learning & Teaching

University of Bath

How well is the project progressing against its plan?

To what extent are the consultants engaged in the overall project process?

To what extent are potential end users involved in the project?

What value is placed on the outcomes by the potential end users?

Digital Literacy and Creativity

for University Teachers (PG Cert Module)

University of Bedfordshire

Building Learning Objects for

Collaborative Knowledge Development (BLOCKeD)

London South Bank University