Tracking pupil progress in design & technology


The demise of levels and the government position that Ofsted will expect to see evidence of good quality pupil tracking data, but will not expect schools to keep records of pupil attainment in a specific format puts a considerable onus on schools to develop new approaches. The Report of the NAHT Commission on Assessment notes on page 7 that schools should be able to use “suitably modified National Curriculum levels as an interim measure in 2014 but that any use of levels in relation to the new curriculum can only be a temporary arrangement to enable them to develop, implement and embed a robust new framework for assessment. Schools need to be conscious that the new curriculum is not in alignment with the old National Curriculum levels.” So no pressure then although it is worth noting that the Commission did recommend “schools should work in collaboration, for example in clusters, to ensure a consistent approach to assessment” (page 6).I’ve always found the whole business of assessment in design & technology tricky territory so I was really pleased when Dylan Wiliam sent me a draft of his pamphlet Principled assessment design soon to be published by the Specialist Schools and Academies Trust. I’ve used this, as a reference piece to ensure my thinking on assessment is sound. Whatever we choose to assess or how we choose to assess it we need to have general agreement about the nature and content of the subject, what Dylan refers to as the ‘construct’ of the subject. This can be seen as the BIG ideas that comprise the subject – an approach used in science education by Wynne Harlen et al in their publication Principles and big ideas of science education and one which I’ve discussed with regard to design & technology in a previous post. I think we perhaps need to step back and look to the purpose of teaching such BIG ideas and in the case of design & technology I’d argue for at least two purposes; giving young people a) technological capability (usually seen as teaching them to design and make products of various sorts that they consider to be worthwhile) and b) technological perspective (usually seen as teaching them to have insight into ‘how technology works’ such that they develop a constructively critical view of technology, and are able to consider how technology might be used to provide products and systems that help create the sort of society in which they wish to live). So we can see pupils’ learning journeys in design & technology needing the teaching of BIG ideas which enable pupils to acquire knowledge, understanding and skills in the context of developing capability and perspective. To my mind there is little point in, or indeed possibility of, tracking pupil progress if the teaching team involved don’t agree on some such overall purpose for the endeavour.

So if we are at the point where there is consensus on the overall purpose of the teaching the next step for the team is to devise an agreed learning journey that pursues this purpose. The exact detail of the journey I want to leave on one side for the moment and paint a broad brush description of the learning activities that such a journey will include. Clearly there will be some designing and making activities. I suggest that there should be a place for designing without making and for making without designing plus some activities devised to enable pupils to explore the relationship between technology and society. The learning journey would be a sequence of such learning activities and within each activity there would be different degrees of direction and choice for the pupils made according to the teachers’ professional judgement. Now each teacher is in a position to build up a record of the learning journey of each pupil he or she teaches and include in that record data which can be used to make inferences about the progress of that pupil. Dylan is adamant that it is important that any data collected is useful, not overwhelming, should be collected in a way that supports immediate use, and that we should not become complacent in its use. Assuming we keep these warnings in mind the record of the learning journey in terms of the broad activities it involves is starting to look like a decent way to track progress. The key question now is what should be recorded about the response of a pupil to each of the learning activities in the context of this tracking with a view to keeping the whole thing a) manageable and b) useful to those concerned with pupil progress?

For a designing and making assignment it seems to me that it is important for teachers to be very clear about both the learning intention of the task in terms of the BIG ideas that are needed by the pupils to tackle the designing and making AND the capability that might be achieved through this learning in designing, making and evaluating. Hence for any designing and making assignment it will be important for teachers to write criteria describing different levels of achievement in any response to the assignment. I feel that three different levels for each of designing, making and evaluating would provide enough discrimination between pupils differing achievement so a table summarizing this might look as follows:

Table

This could be represented as a tripod with pupil’s performance across the criteria could be presented by drawing a line between the variously coloured discs as shown:

Tripodwithlines

It would be essential to have a description of the assignment plus an image of the item the pupil had produced and the table listing the criteria all on the same sheet so that anyone reading the sheet had all the information they needed to think about the pupils achievement in this particular designing and making assignment. I can also imagine at the bottom of such a record of achievement sheet a sequence of such tripods each with the appropriate lines showing how the pupil had performed in previous designing and making assignments thus giving a record of how the pupil was progressing with this aspect of design and technology across time across time – say five or six designing and making assignments across Key Stage 3.

in a row jpeg

It’s important to remember that the criteria on which this performance is assessed are assignment specific and it will be part of the teaching team’s remit to ensure that the assignments become progressively more demanding across time and this increasing demand is reflected in the criteria they write. This won’t be an easy task but one that will focus the team’s efforts into devising a learning journey that is progressive with regard to designing and making. I can see very useful discussions taking place with pupils and parents around such a progress tracking sheet. During Key Stage 3 the on going data provided could be used to help the pupil see what they need to do to improve. Towards the end of Key Stage 3 the overview provided should be a great help to parents and pupils in thinking about GCSE prospects in design & technology should the pupil choose the subject. I see that this approach to progress tracking won’t give a test score and that this is often seen as a desirable feature of assessment. However I am heartened by these recommendations made in the Assessment Commission Report (page 6):

  • Pupils should be assessed against objective and agreed criteria rather than ranked against each other.
  • Pupil progress and achievement should be communicated in terms of descriptive profiles rather than condensed to numerical summaries (although schools may wish to use numerical data for internal purposes).

In response to the last point I’m sure it is possible to give the red, yellow and green discs a number value and then calculate scores for individual designing and making assignments and perhaps aggregate scores across time but I think the resulting numbers would be pretty meaningless. And it is worth noting that the summary information on the design & making tracking sheets can be supplemented at any time during a discussion with stakeholders by reference to the pupil’s class work – available in a traditional ‘design sketch book’ perhaps or in digital form as an on line design diary. So if this approach to tracking progress works for the designing and making assignment activities then can it be used with the other suggested activities?

Designing without making stems from the Young Foresight Project and was recommended in the National Strategy for design & technology in 2004. Pupils are required to design but NOT make a product and associated services using a new and emerging technology as a starting point. The design has to be justified from four perspectives – technical feasibility, meeting peoples needs and wants, acceptability in society and marketability. It seems to me that the SOLO (structure of observed learning outcomes) approach might be useful for assessing such a complex set of responses. Thanks again to Dylan for information on SOLO taxonomy which identifies five different levels of structure that we might observe in students’ work:

  • Pre-structural:          the response does not address the requirements of the task;
  • Unistructural:          the response addresses a single aspect of the task;
  • Multistructural:          the response addresses multiple aspects of the task, but these multiple aspects are treated independently;
  • Relational:          the response addresses different aspects of the task that are related to each other and therefore become an integrated whole;
  • Extended abstract:          the ‘integrated whole’ is conceptualized as a higher level, for example as an element of a more abstract structure, or is applied to new relevant areas.

Pupils usually produce a sketch of their design idea with ‘justification annotations’ and this could be scrutinized using the SOLO taxonomy. I’m not sure any would reach the ‘extended abstract’ category but the other categories would be appropriate. So in terms of tracking I envisage a sheet with an image of the pupil’s idea plus a teacher comment using SOLO. And note David Didau is tweeting about SOLO!

Making without designing is particularly useful for teaching making skills and it is not difficult to create a set of criteria by which the results of such making might be assessed: the level of precision in producing the parts, the correctness of the assembly and the care taken over finishing for example. So in terms of tracking I envisage a record sheet with an image of the pupil’s made item plus a teacher comment using the agreed criteria.

Exploring technology and society activities are likely to cover a wide range of considerations such as the possibility of developing a circular economy , the possible impacts of disruptive technologies, the influences of iconic designers etc. Here again it seems to me that a SOLO approach to assessing this work would be appropriate. So in terms of tracking I envisage a summary sheet describing the exploring technology and society tasks plus teacher comments using SOLO.

So taken together what would I have as my tracking record for a pupil? There would be a collection of sheets assessing progress in designing and making plus a single sheet showing assessment of making, designing and exploring technology and society, one sheet for each year of the key stage describing two making and two designing activities plus a range of the exploring technology and society activities. This seems to be very rich data that would be extremely useful in any conversations with individual pupils and parents and also provide the teaching team with an easily accessible feedback on the effectiveness of the various task types.

The methods for tracking that I am suggesting combine tracking with a clear statement of the teaching intentions and learning experience. This I think is particularly useful for design & technology as it is a much-misunderstood subject. It confused the Expert Panel in 2011 who found it so difficult to comprehend that they questioned its epistemological roots. With the BIG ideas identified and the way in which they are taught through particular approaches made clear this is no longer the case. I believe the tracking I’ve described here takes this into account and will inform pupils, parents and SLT as to the nature and worth of the subject and the way pupils make progress. Note I have deliberately concentrated on the learning experience and pupils response to that experience as the method of tracking. I have deliberately avoided setting ‘tests’ but used an approach that is minimally invasive in that it uses the work produced by pupils as part of their everyday learning in the subject as the basis for tracking. There may be a case for setting tests particularly with regard to finding out about pupils learning of so called ‘troublesome knowledge’ – concepts that are known to be, for example conceptually difficult, as in the relationship between force applied and movement achieved in mechanisms or alien in that they require pupils to adopt different perspectives in order to understand the views of different stakeholders. Such tests could reflect the nature of questions used on GCSE written paper questions – more of this in a future blog.

I must now return to the question, ”Would this be useful to those concerned with tracking pupil progress?” To my mind it provides very rich data that would be extremely useful in any conversations with individual pupils and parents, provide the teaching team with an easily accessible feedback on the effectiveness of the various task types. It doesn’t easily yield any sort of quantitative data so an SLT wedded to figures might not see it as that useful but I think it would paint a good picture of the messy business of what it means to make progress in design & technology and the sort of progress being made by individual pupils. But would collecting all this data be manageable? Here I have my doubts but would welcome comments from teachers and heads of department. Lots of effort would be required to set up such a system but once set up would it be that demanding to maintain? Could it respond flexibly to a changing curriculum? Could it be generated and stored digitally? Questions which only those working in schools can answer.

 

 

 

 

 

Advertisements

6 thoughts on “Tracking pupil progress in design & technology

  1. I think the ‘star chart’ assessemnt graph is a fantastic idea.. and I would worry ablout adding too many strands Stewart, which then need explaining to the various users of the data.. KISS!
    I also think developing the rubric(s) for each specific assessement activity is a really important part of developing a good purposeful project. These can be assessed through digital or paper based systems..
    I don’t totally subscribe to the SOLO information.. not because I dont believe it works and is self evident. But that it is a bit of an intellectual construct and as such can muddy the waters for your jobbing teachers trying to make sense of teir day to day practice.. Perhaps if teachers had more time to reflect and act then the whole ‘taxonomy’ thing would have more impact/relevance to their planning. Please don’t read that as disagreement Dylan.. the structure is meaningful ,relevant and workable. Time to reflect is a key stumbling block.

    Thank you David, Lots of good stuff to think about as usual.

    Like

    • Really glad you find this stuff useful
      I’ve never seen the SOLO instrument as something for pupils. More an approach to give teachers a way of looking at the complexity of answers that pupils might provide. Asking pupils to design but NOT make a product (and associated services) for the future is a reasonably common task within design & technology classes at KS3. In some cases the pupils are asked to justify their design ideas from four perspectives: technical feasibility, meeting peoples needs and wants, acceptability in society and marketability. Using the SOLO categories gives the teacher a way of looking at and interpreting pupil responses and it is this interpretation that then informs the AfL conversation that the teacher has with the pupil. No need to be explicit with the pupils about SOLO as such and of course the extent to which they meet the criteria will depend on how much they know and understand in relation to the particular design without make task. This can feature in the conversation, which would include discussing with the pupil their knowledge base and perhaps that might reveal that he or she needed to know more in which case there was extra knowledge to acquire or that in fact he or she knew more than it appeared from the design justification in which case the lesson would be about making better use of what you know.

      Like

  2. I would be interested to know why Stewart Dunn says that SOLO is not useful for chalkface use given that (a) over the last thirty years, we have seen a vast number of examples of its use by practicing teachers, and (b) right now in England, a lot of teachers are finding SOLO useful in thinking about a “life after levels”.

    Like

  3. Excellent debate for academics but SOLO method given not suitable for chalkface use.
    Progress reporting needs to be student staff parent school friendly otherwise it will just be a chore and worse than useless just like the phoney levels were.

    The SOLO taxonomy is off the scale and in La La Land so i will ignore that.

    But
    The CHART method shown is very practical
    DESIGN
    MAKING
    EVALUATING

    But would prefer it to also include ‘Designing Without Making’ and ‘Knowledge’
    so it would become
    RESEARCH out of 10*
    DESIGN out of 10
    MAKING out of 10
    KNOWLEDGE out of 10
    EVALUATING out of 10

    OR even
    Design and Research out of 10
    Making / inventing out of 10
    Knowledge and evaluating out of 10

    *Out of 10 is the most practical way as it fits in with percentages especially useful when there is a need to compare groups and use normal distribution curves to ensure consistency between subjects.

    When staff work as a team they need numbers so they can easily work out levels etc

    When SCHOOL REPORT WRITING time comes around there is no time for niceties
    X Teacher may be allocated to write reports for ‘A Group’ with inputs from other teachers and maybe staff who have left a term earlier or off sick. It is easier to pass info on spreadsheets

    Keep up the good work

    Like

    • I’ve yet to look at the SOLO information, but will be doing so in preparation for work on assessment in my department over the coming academic year. Interesting that you have highlighted a need for a numerical system. On one hand I agree that schools, teachers, pupils, parents, OFSTED, et al are so entrenched with a system attached to metrics that it is difficult to see any other way that would be practicable or workable,BUT that’s the point of this debate. It will be hard work, but do we actually need a metric system? Also, the NAHT Guidance calls for pupils not to be compared, but to be assessed subjectively but I guess this will be something that we have to develop for public accountability, unless school league tables are scrapped…
      I do also like the idea of a pupil ‘map’ so that stakeholders can ‘see’ progress over time.

      Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s