EMA Programme - One Year On

Friday, 23 February 2018

One year ago the Electronic Management of Assessment (EMA) Programme secured full agreement to proceed. This three-year programme aims to deliver the University’s long-term vision for online assessment, including the development of underlying processes and supporting systems, to further improve the assessment experience for both students and staff.

Specifically, the core objectives are:

1. To enable a consistently good student assessment experience
2. To provide an improved and supported assessment experience for staff
3. To reduce the administrative burden of assessment for the University

Significant progress has been made over the last 12 months and this update outlines what has been achieved and what colleagues can expect over the coming year. You may also like to view the new screencast on the EMA website, which provides some visual examples of the new capabilities that are being introduced.

Enhancing Core Systems – Sub Modular Marks within RISIS

In a significant step forward for the University, the component, submodular assessment marks for each undergraduate student will now be available within RISIS as marks are entered and released.

New functionality has been built in RISIS providing the ability to maintain assessment information, enter and manage marks and see progress at a tutee level. New information screens are now available for Programme Administrators, Module Conveners, Markers and Personal Tutors. A new Student screen will be released shortly, which collates an overview of assessment and marks in one location.

The new functionality also includes the ability to track and calculate turnaround time for feedback and marks, as well as online submission and recording of extenuating circumstances.  

This marks a major improvement in our reporting capability with a range of new School-wide reports available to analyse assessment patterns, timing and student performance.

To achieve the above, a complex series of system developments and testing took place and Programme Administration Teams worked tirelessly over the summer to collate assessment information previously held on over 3000 spreadsheets into newly created fields in RISIS. Marks started being entered on November 17, achieving a major milestone on December 17 with a first module to have all the submodular marks entered within each type of assessment.

Training for Programme Administrators is nearly complete, and from January 18 the EMA Programme will provide further briefings and dissemination of supporting material to enable full use of all new features in RISIS.

The focus now moves to providing sub modular marks information in RISIS for PGT and Foundation modules in readiness for the 18/19 academic year rollout, along with the intention to create better integration between Blackboard/Turnitin and RISIS with as much automation as possible by the end of 2018, with further integration potentially in 2019.

Improving Process - Electronic Submission Feedback and Grading (eSFG)

At a School level, a series of workshops were held with the School of Archaeology, Geography and Environmental Science (SAGES) and School of Agriculture, Policy and Development (SAPD) to explore the most effective processes for electronic submission, feedback and grading for their schools. This involved mapping out detailed processes, roles and responsibilities and establishing the best methods of submission for different types of assessment to optimise the use of Blackboard and Turnitin.  Full engagement and training has been provided to these two early adopter schools, led by the CQSD TEL team, to support colleagues in their use of these online assessment processes and take up of online marking.

This is supported by a number of special sub-projects, led by seconded Academic Partners, focused on managing mathematical and scientific notation, offline working and use of IPads, large files, anonymisation, external moderation as well as highly innovative work on quickmarks, rubrics and criteria level marking.

The EMA Programme are now commencing engagement sessions with all T&L Deans and senior School colleagues to understand discipline-specific needs ahead of a series of workshops designed to explore school specific processes for electronic submission, feedback and grading. A variety of training approaches, using multiple routes to suit different needs, will support colleagues to move towards greater engagement with electronic assessment in the 18/19 academic year.

Supporting Change for EMA

More broadly, the Programme continues with collaborative change management to ensure that all stakeholders are engaged, trained and supported. At the beginning of this academic year, 3 graduates and 7 academic partners joined the EMA programme, along with team members from CQSD, Support Centres, IT and the SIS team. This has proved invaluable in ensuring appropriate engagement and that solutions are meeting a variety of needs.

A launch event was held in March 17, with a further university-wide update in May. Throughout the year, significant engagement has taken place to raise awareness, keep all informed, seek feedback on approaches and better understand concerns and support that may be needed. A broad variety of well-attended engagement events have been held with colleagues including SDTLs, Programme Administrators, Communities of Practice, school and department meetings and RUSU.

A full review of existing T&L policy has been conducted to ensure that the solutions and new ways of working are compliant. A focus group was also held, led by an external expert, to ensure inclusivity and accessibility, in line with the new University inclusivity guidance.

A wide range of training, briefing, engagement and communication events with all stakeholders will continue throughout 2018, as well as continued development of supporting materials including a case studies series and supporting screencast suite, along with additional resources. The Programme will develop this understanding further through a one day EMA symposium in June, an event designed to facilitate opportunities for colleagues to share good practice.

Further Analytical Capabilities

With the increased amount of data that will become available comes the opportunity to create further reporting and analytical capabilities to measure progress, inform pedagogical practice and enhance attainment and pastoral care, and in turn increase the profile of University of Reading amongst other sector leading universities.

Towards the end of 2017, a set of initial requirements were established for the creation of a dashboard, for students and relevant staff, to provide enhanced views of the summative assessment data that will now be available, as well as supporting tools. Options are now being assessed and development will commence in spring with the intention of providing an initial dashboard solution by the end of 2018.

Alongside this, research was conducted on the potential use of learning analytics, to draw on a number of data sources to provide early warning of disengagement or under-performance and enable intervention where needed. It was agreed in October to progress to identify the technical options and produce a business case. The business case will be ready early summer, and (if approved) planning will commence for a separate Learning Analytics Programme.

Where to find out more

The EMA website has further information surrounding how the introduction of electronic management of assessment will impact colleagues. There is a monthly newsletter and the site is regularly updated with progress and events: http://www.reading.ac.uk/ema/

Hopefully, you will be able to attend some of the face to face or online briefings and share any ideas or concerns that you may have. 

You can also contact the programme by emailing ema@reading.ac.uk, or by contacting Emma Mayhew (EMA Academic Director) on e.a.mayhew@reading.ac.uk