Part 1 of this pair of posts covered questions 1-4 in the Ofqual consultation. This post details our response to question 5, an ‘open’ question, where we discuss both the non-examined and the examined aspects of D&T assessment at GCSE.
Question 5: Do you have any further comments relating to the assessment of this subject?
5.1 The need for minimally invasive assessment of procedural competence
There is nothing in the proposals about how pupils will provide evidence for non-examined assessments. We fear that, without external push, the AOs will default to what they know; death by portfolio.
Nick Givens (writing as long ago as 1998 in response to a consultation on Creativity and Cultural Education) clarifies the problem:
Our [the teacher’s] problem always has been, and remains, that of finding efficient painless ways of generating EVIDENCE that don’t stifle the creativity. So the ritualisation of designing, the conversion of the design folio into a product and the inflexible narrow interpretation of what constitutes design, represent a major problem. There needs to be scope for pupils to model and record their thinking in a variety of ways AND orders. We can’t carry on letting a narrow view of what constitutes EVIDENCE-of-design dictate the NATURE of design.
The work carried out by Richard Kimbell and his colleagues at TERU in the e-scape project provides an interesting approach to assessing capability in D&T. It involves using structured testing materials over two half-day blocks under examination conditions. This is a huge step forward compared to the stranglehold that the coursework portfolio has over pupil assessment in design & technology. However despite the undoubted achievement of this project we have significant reservations about this one-size fits all timed test as the best way for pupils to reveal their capability. It does not meet Nick Givens requirement for pupils to have the scope to record their thinking in a variety of ways AND orders.
We think it is important that any NEA should be minimally invasive, i.e. the way the pupil’s work is assessed will be such that it will not distort their educative experience in tackling their designing and making task. We suggest that the way pupils can be given appropriate freedom to decide on their own designing and making pathway is for the teacher to support them in the use of job bags.
The criteria for the contents of job bags are simple. The pupil’s work, be it in the form of written notes, annotated sketches, 3D models, working drawings, patterns, recipes, plans, schedules, still photos, video recordings, audio recordings, questionnaire data or calculations, must have utility. It must be present only because at the time it was produced it was done to help move the designing and making task forward. Such a miscellany would be personal to the pupil and it is likely that there would be considerable variation in content of such job bags even amongst pupils tackling identical design and make tasks. However, the job bag would not be the primary source of assessment evidence. It would be the evidence that the pupil called upon to reveal and justify their decisions. And it is the revelation and justification of the design decisions demonstrated at three points during a designing and making task that provides the bulk of the assessment evidence for the pupil’s designerly activity.
The first point is reached when a pupil has explored the context and developed his or her first ideas for a product in response to the context. A pupil will be asked to consider whether the developed proposals meet the revelations of the context and requirements of the brief and to clarify and justify the design decisions made so far. The pupil will also be required to review these decisions and consider whether what s/he is proposing is likely to be achievable in relation to resources of time, materials, equipment and personal skills.
The second point is reached when most of a pupil’s design decisions have been made through sketching, 3D modelling, and experimenting. This will be at the point where making is imminent or has just started. Again, the pupil will be asked to clarify and justify the design decisions made so far and then review these decisions and consider whether the design arrived at fully meets the requirements of the brief and whether the plans for making are achievable.
The third point will be reached when the product is complete and will include an evaluation against the brief and the specification.
The emphasis at these points of reflection will be on revealing a pupil’s response to the emerging demands of the task in terms of the decisions made and the extent to which they are realistic.
Note that it is the contents of the pupil’s job bag that is the source of the information the pupil needs to make the necessary reflection and it is the contents of this reflection that provides the assessment evidence.
This approach to assessment is discussed in more detail here but we think this approach is worth exploring further by the Awarding Organisations as an alternative to current practices.
5.2 The need for a rigorous written paper to assess the Knowledge Base of Design & Technology
Rigorous assessment of the knowledge base of design & technology in a written paper will be new territory for the Awarding Organisations as this has not been well explored in previous GCSE written examinations..
Here is an initial consideration about possible questions.
- Some questions will need to probe specifics, others should be synoptic, requiring candidates to draw on knowledge and understanding from a range of specifics.
- Most, if not all, questions should require the use of knowledge to show understanding as opposed to simply being able to recall particular pieces of knowledge.
- We see many questions as requiring candidates to respond to/resolve different sorts of designerly dilemmas.
- Some questions can require explanatory writing; in some cases quite extended writing. How such answers can be marked will be a challenge. However English and History examiners have that expertise and the AOs should look to them for advice and guidance.
- The majority of questions should be structured into parts that scaffold the candidate in developing a solution to a problem.
- Some questions will require quantitative as well as qualitative reasoning.
- Some questions can be multiple-choice.
One type of multiple-choice question that might be interesting to explore is the ‘assertion reason’ question. The overall form is
“Statement A” because “Statement B”
The two statements can each be either true or false and, in any case, Statement B might or might not be an explanation of Statement A.
An example assertion might be “Cows eat grass because the sky is blue”. “Cows eat grass” is true, “The sky is blue” is true, but the sky being blue is NOT an explanation of why cows eat grass. The question offers various possible answers that candidates can select from, these are permutations of the truth or falsity of the initial statements and the assertion.
These questions are demanding to write but really probing in terms of knowledge and understanding.
- Some questions will be like none of the above as we work out different sorts of questions for our purposes.
Developing such questions will be a challenge but it is one to which the subject can and must rise.