by Tony Wheeler

For the past 3 years I’ve been working on the e-scape (e-solutions for creative assessment in portfolio environments) project at TERU (Technology Education Research Unit)

The team

Based at Goldsmiths University, the project is funded by DfES and QCA in association with Edexcel and AQA awarding bodies.

Teams, partners and schools credited below

most teachers I meet agree that the present assessment system for d&t is seriously flawed

Despite the fact that learning activities in d&t and other areas of the curriculum are increasingly influenced by digital technology, the final presentation of work for GCSE assessment is almost entirely paper-based, relying on print-outs of any digital work. Most teachers that I meet agree that the present assessment system at GCSE for d&t is seriously flawed and often rewards the wrong students, notably the ones that meticulously follow the rules rather than break them and come up with really exciting ideas.

Project e-scape is a new approach to the assessment of creativity and design innovation at GCSE level where students use digital devices to “hoover-up” evidence of their on-going work during carefully structured design tasks. Their photos, sketches, notes and audio comments are instantly uploaded into a multimedia web-based portfolio that teachers and examiners can then use to assess individual performance. An extension of the project has been to trial a new approach to assessment based on holistic pared judgements instead of traditional marking…

Background

The e-scape project originated from ‘Innovating Assessment’, another QCA funded TERU project, which began work in 2003. A key focus of this earlier research was the development of a range of techniques to both promote and collect visually rich evidence trails of students’ innovative performance as they worked on practical design tasks. These techniques included:

• handling collections (to get pupils contextualised and thinking about the task)
• working in teams of three (to get pupils started and help critique their developing ideas)
• modelling kits (to resource 3D thinking/rapid ideas modelling)
• carefully structured tasks and sub tasks (to keep pupils iterating between thinking and doing)
• photo storylines (to record evidence of each stage of modelling)

Innovating Assessment resulted in a paper based assessment system which took place over six hours on two consecutive mornings. The tightly structured format was designed to stimulate pupil creativity and group work in studio-based GCSE d&t activities. This system is currently being piloted by OCR as part of their new Product design GCSE innovation challenge. A report of this project can be downloaded here.

Digital photos of each student’s 3D models were taken every hour, as their designing progressed. These were immediately printed on a mini printer provided an instant visual story-line of each student’s developmental work. This had significant effects on both performance and behaviour, motivating students to work more purposefully and to take more risks. It also helped the assessors to see a more complete picture of each project. This lead the e-scape team to consider how the collection and recording of other aspects of performance, such as discussion in the form of short audio clips, might help to present an even more complete picture of d&t capability.

e-scape Phase 1 (Proof of concept)

The DfES and QCA were also keen to explore multimedia evidence trails and agreed to support the development of a digital version of the TERU paper based system. In January 2005 we started work on a proof of concept. We were determined to avoid the twin traps of forcing pupils out of workshops and studios into second-hand, virtual experiences in computer suits, or overburdening teachers with data capture and scanning tasks. As technology changes so rapidly we also wanted to make sure we had a broad perspective of what was possible and explored a wide range of web based portfolios and small scale, handheld systems that could be linked to allow students to collect their own evidence. We trialled a series of single media systems including:

• digital cameras (still and video)
• digital memo/voice recorders
• tablets
• digital pens.

Of these the digital pens seemed to offer the most effective route to capturing students’ notes and sketches. However, when we interviewed the students they were not so impressed, complaining that the pen was “too chunky”, “only wrote in biro”, “kept wobbling”. This was clearly not the response we had been looking for, supporting both the activity and the assessment of it. The students went further and suggested the digital pen would be better if it had a camera and microphone built in. They were essentially describing a PDA and when we subsequently trialled with Palm Zire 72 devices we began to see significant motivational and inclusive benefits during, as well as after the activities. You can download a report on the first phase of e-scape below:

Download e-scape phase 1 (final report).pdf

e-scape Phase 2 (Working prototype)

We generated sufficient evidence from the proof of concept trials to move on to develop the e-scape system into a working prototype. We did this in association with Handheld Learning, who created the PDA application, and TAG Learning, who developed the web-based portfolio system. Together the two teams devised a system that managed the dynamic dataflow from a web server to and from the PDAs and the portfolio.

During the sessions, activity instructions were delivered to the PDAs at timed intervals, students’ work was beamed around their PDAs to facilitate group support and review. Throughout, all the media created and collected by the students was uploaded to individual secure web portfolios as the activity progressed. In this way students recorded the progress of their designing on their own PDA using photos, sketches, notes and audio comments. This provided valuable procedural scaffolding for their designing as well as sufficient evidence of their performance for us subsequently to make confident assessments of their work despite not being present while they completed it.

We overcame the problem of inconsistent/unreliable wi-fi network availability in some schools (particularly in d&t departments which are often situated on the far reaches of the school site) by creating our own local wi-fi network using a portable computer and hub.

By May we had a sufficiently robust system and completed a national trial in 14 schools across the country through June and July 2006. As a result we now have 250 e-portfolios of year 10 learner’s performance in the e-scape website, together with a class set of year five work (who had no problems with the technology and produced some really exciting work).

The combination of the activity structure, supported by the handheld technology had a profound effect on all the students we worked with. Just like designing itself, it is difficult to capture the e-scape experience in words: fortunately Teachers TV came and videoed one of the early trial sessions, together with a discussion panel and a broader look at e-assessment. You can view each of these 3 programmes at their web site: