Durham Distance Learning Tutor’s Programme

The Durham Distance Learning Tutor’s programme is taking place at the Business School in Durham on Saturday 24th and 25th March.

Click Here to download the Tutor’s weekend programme-2007

“Evaluating electronic feedback in formative assessment: evidence from a first year economics course”

I’ve been involved in evaluating the feedback from electronic assessment tools for some time now. As class sizes become larger there are greater demands on staff and electronic assessment is often thought as a means by which we can improve the efficiency of feedback. The downside is that it can be impersonal and thus detract from the richness of the learning experience. Three tools were examined during the last year: multiple choice questions on DUO (Durham University’s eLearning platform), feedback and marking software, and an audience response system (as in ‘Ask the Audience’ on the ‘Who wants to be a millionaire?’ television programme). A questionnaire was distributed to those studying on the ‘Elements of Economics’ module, a first year course that attracts almost 300 hundred students from a variety of disciplines across the University.

Unfortunately the returns were lower than anticipated (around 50 questionnaires), making the results from cross tabulation less robust. Nevertheless, a number of interesting findings are revealed. Students were generally very positive about the feedback from multiple choice questions on DUO. They thought it was appropriate to their learning needs, was a good indicator of their current knowledge, (in many instances) built their confidence, motivated them to study further and helped them prepare for the final examination (which did have a multiple choice component). Overall, it did enhance their learning experience. However, there were concerns about the brevity of feedback in some instances and the need to have appropriate feedback where the wrong answer is entered. There were also some technical issues reported in the questionnaire; students felt frustrated when they could not complete the multiple choice tests.
The audience response system also registered very positive replies in the main from students. This tool is used in lectures to encourage students to think about a particular problem or as a diagnostic tool to check their understanding (and my explanation!). Students are asked to respond to questions and activities using a zapper (much like a TV remote control). The collated responses are then displayed on the projector screen. There are (deliberately) not enough zappers to go around, so encouraging discussion and the sharing of ideas and responses. Not surprisingly this came through in the results. Students often declared that this tool was ‘fun’, that it made lectures ‘interesting’ and ‘interactive’ and that they liked to see how they had fared relative to the rest of the group. However, a few thought that it was ‘pointless’ and a lot of effort for little return. It also came below multiple choice tests on DUO when students were asked to rank the various feedback systems, so care needs to be taken in understanding the role of such an innovative technology in the big scheme of things.

The evaluation of the marking and feedback software (developed at Liverpool John Moores University) is hampered by small sample size. Students are given a formative test in the Elements of Economics module and seminar tutors mark the papers. Only 4 out of the 20 seminar groups receive feedback using this software (mine). The questionnaire responses revealed that the feedback given to all seminar groups varied enormously in terms of the quality and the presentation of the feedback given. Those responses relating to electronic feedback software were all very positive but too few in number to be statistically representative. Where responses were given about the software, students liked the depth of the feedback, the statistics generated and the fact that it was emailed to them directly. What the software does well is generate a range of comments via a series of mouse clicks as well as allowing personal comments about the assessment from the tutor. The disadvantages are that there are set up costs and it takes time to learn how to use the software.