Providing written feedback to learners within Higher Education can be a time-consuming and frequently ineffective process, with comments often undervalued or not acted upon. The same concerns exist within the primary sector; marking systems do not always effectively support the principles of Assessment for Learning. This frustration resulted in the development of the 'Quality Marking' model for primary schools (Clarke 1998) which follows an explicit process within which the type of comments, or prompts, used are key. This small scale study has investigated whether 'quality marking' – and in particular the use of scaffold prompts – can be applied to the Higher Education context. A group of final year student teachers within initial teacher training received and acted upon the feedback, as well as completing a series of questionnaires to record their perceptions of the process and its formative value. It is clear from the data gathered that 'quality marking' was viewed as an effective approach. Key identified reasons for this were the likelihood of the feedback to feed forward into future learning, the level of engagement from the learner, the clarity of how to bridge the learning gap, and the beginnings of self-monitoring.
The publication of Black and Wiliam's seminal paper on assessment for learning in 1998 had a profound influence on understanding, policy and practice within the primary sector. However, the impact within Higher Education is less clear, with the key aim of empowering learners through a constructive and discursive approach appearing to be at odds with the often modular bound, summative based systems.
Within assessment for learning, feedback has been recognised (Black and Wiliam 1998, Gibbs and Simpson 2004, Cree 2000, Ramsden 2003) as a key formative strategy likely to lead to an increase in standards. Nevertheless, studies have repeatedly shown that within Higher Education, written comments are not perceived to 'close the gap' in learning to allow this to happen, or are not acted upon (Brown and Glover, 2005, Maclellan 2001). Given the hours devoted to providing written feedback, its apparent lack of use by students is frustrating and an opportunity lost. It seems pertinent to look to the successful models of feedback developed within primary schools over the last ten years in an attempt to identify possible areas of development. This small scale pilot project examines whether one such strategy can be modified for use in Higher Education and whether it provides an alternative and more effective form of feedback or feed forward.
What makes written feedback either effective or ineffective is complex to unravel. Not all written feedback informs future learning positively; in fact, poor feedback can cause further confusion and disengagement from the learner. Students are often puzzled about the value of the written comments they receive, perceiving that they are not necessarily helpful to their learning and understanding (Defeyter and McPartlin 2007, Gibbs and Simpson 2003, Maclellan 2001).
It is when we seek to define feedback that its core purpose is revealed; the fulfilment of this purpose can be used to determine whether it is effective or not. Ramaprasad (1983) states: '...feedback is information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way' (1983:4). As such, written feedback needs to close the gap between present and potential learning. This may appear obvious, and it could be assumed that this concept underpins any written feedback, but in fact this is not the case. Merely stating that a gap exists does not necessarily enable learning to move forward. It is only if there is a resulting positive outcome in learning/understanding that the 'feedback loop' (Sadler 1989) is completed and the underlying purpose is met.
Several studies have explored both the features and processes inherent in effective and ineffective written feedback. Chanock (2000) found that the inclusion of comments by the marker was not necessarily an indicator of developed understanding, as learners often misunderstood or misinterpreted their meaning, or were unsure how to rectify the problem. They did not enable 'students to make connections with their own work and so able to close the gap between their current and desired performance' (Brown and Glover 2006 p. 89). This is supported by Nicol and MacFarlane-Dick (2006) who, when identifying seven key principles of effective feedback, included the clarification of good performance and the ability to close the gap between current and desired performance (Nicol and MacFarlane-Dick 2006). In order to close the gap, students need to re-construct their own understanding, and as such feedback should be based upon a constructivist (or dialogic) approach (Gibbs and Simpson 2004). Feedback must be accessible to the learner, implying the need for some form of scaffolding or explanation. Stating what needs to be improved may not be enough; explaining how they can improve is much more likely to inform future performance (Walker 2009). Indeed without this detailing of how as well as what, most intended written feedback will in fact remain at a summative level (Glover and Brown 2006) i.e. it will provide learners with a measure of their current ability but will not inform future learning. The nature of the written comments received is therefore critical. Indeed Ramsden states: '...it is impossible to overstate the role of effective comments on students' progress in any discussion of effective teaching and assessment' (Ramsden 2003 p. 187). In an attempt to explore the nature of comments further, Brown and Glover (2004) analysed and then classified written feedback comments according to whether they: indicated a problem, offered a correction to the problem, or offered an explanation of the problem alongside a correction. It was the third of these that was perceived to be the most effective in that it did not merely identify the difficulty but explored how the difficulty could be remedied. This notion of comments enabling understanding of how to close the gap correlates closely with the long term aim of formative assessment to develop the learner's own ability to monitor, track and direct their own learning (Sadler 1989). The ownership of the process, according to Sadler (1989) relies on three things: an understanding of the goal, an ability to compare against current learning, and the ability to action the closure of this gap. Written feedback needs to encourage these self-regulatory abilities by modelling all three of these.
If feedback describes not only what is successful, what needs improving and crucially how it can be improved, learners also need to use this information to make improvements soon after if they are to become active in their own construction of meaning (Gibbs and Simpson 2004). The immediacy of the improvement is a feature particularly pertinent to Higher Education when modules tend to be organised around a final assessment that is often marked once the module is complete. Sadler identifies this when he states: '...the student must engage in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121). Effective feedback needs to work as dialogue between learner and teacher, and students need to engage with the feedback. In Higher Education, the reality of the nature and timing of written feedback means that it can often act as an obstacle to this engagement rather than facilitate it.
In the primary school setting, the work of Black and Wiliam and the Assessment Reform Group had a huge influence on assessment policy and practice due to its potential to 'produce significant, and often substantial learning gains' (Black, 2000 p. 408). In fact Hargreaves (2001, cited by Kirton et al. 2007) refers to assessment for learning as 'revolutionary' in that it has impacted not only on the way primary school teachers view assessment but its ongoing cyclical relationship with teaching and learning. Key strategies have been developed and are now embedded within teaching and learning, including the use of peer/self evaluation, effective questioning and feedback. However in some settings, practice has become so focused on strategies that assessment for learning has become rather tokenistic, operating merely at a surface/procedural level rather than underlying pedagogy (Ward 2008).
Black and Wiliam emphasised the key role of feedback within assessment for learning: '...when anyone is trying to learn, feedback about the effort has three elements: recognition of the desired goal, evidence about present position, and some understanding of a way to close the gap between the two' (Black and Wiliam 1998 p.6). This emphasis on constructive feedback caused many primary practitioners to reconsider written feedback. Clarke (1998) made direct links between the work of Sadler (1989) discussed earlier and written feedback in schools, noting that it is the third of his key aspects, 'the ability to action the closure of this gap' that was not always evident in schools. As a result, she and her learning teams developed an approach called 'quality marking', a practical written feedback strategy focused on closing the learning gap for primary aged children. The approach consists of four stages:
As the prompts either illustrate or scaffold towards the desired next stage, learners are more likely to understand and construct future learning, effectively bridging the learner's zone of proximal development (Vygotsky 1978). This notion of demonstrating how the gap can be closed correlates closely with Nicol and MacFarlane-Dick's (2006) aforementioned principles of good feedback practice i.e. it can 'clarify what good performance is, delivers high quality information to the students about their learning and provides opportunities to close the gap between current and desired performance' (Nicol and MacFarlane - Dick 2006 p.7). Indeed, these prompts could also be applied to Brown and Glover's (2004) work on categorising written feedback comments; there are clear links between scaffold and example prompts, and the proposed effective category of 'explanation alongside a correction to the problem' (Brown and Glover 2006). The intention of prompts is to explain and engage the learner on how the gap can be closed, rather than correcting or indicating where the gap is. Their value is further supported by Cowie's 2005 research which found that pupils preferred feedback that included suggestions: '...suggestions supported their active engagement with ideas, both their own and those proposed by the teacher.' (Cowie 2005 p. 143) It is also worth noting that an immediacy of response is expected as part of the quality marking process, thereby further encouraging a more dialogic process and ensuring that the learner engages 'in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121).
Although 'quality marking' is a strategy aimed at primary aged children, it reflects the generic principles of effective feedback and also correlates closely with some of the associated difficulties of feedback within the HE sector. These identified parallels led to the question and subsequent project: is it possible to apply quality marking prompts to a different context?
The project took place within the final year of a BA degree (carrying QTS status) as part of a module equipping trainees with the skills required to be an English subject leader in school. There were 20 trainee teachers within the group, the majority of whom were young students. All had recently completed their final school experience and were in their final semester of the programme. The module had two assessments, both assignments of 2,000 words. Quality marking was used for the first of these, a piece of work focused on planning staff development on reading comprehension.
Alongside the standard University feedback for this assignment, three additional improvement prompts (either scaffold or example) were identified for each student to action. A timetabled session was arranged for students to both receive and act upon these prompts. The students worked with a partner to read through the prompt and then use the suggestions to re-draft the original work. The tutor was also available for support. All of the prompts given avoided simply stating the improvement needed but either gave a direct example or model of what could be improved or scaffold the students' understanding towards the next steps. It is worth noting that the researcher was experienced in writing feedback prompts, albeit with young children, and was able to construct similar comments with relative ease. Academics without this experience could find the art of writing feedback prompts initially more challenging.
|Original text||Customary feedback comment||Rephrased prompt|
|This approach was taken because staff should be involved in devisiting strategies to suit all.||Further depth required here.||Why is it important that they are all involved in this? What would be the benefits? Would there be any possible disadvantages? (Scaffold prompt)|
|The subject leader would need to ensure another meeting was arranged to monitor progress made and gain necessary staff feedback?||Why? Justify further and consider how you would do this.||Add an additional sentence. |
This informal discussion would provide an initial starting point but the subject leader would need to ensure specific strategies were in place to monitor the impact on learning. These would include....... (Use the above prompt or your own) (example prompt)
|Chamber suggests the Tell Me approach.||Engage with the text further.||Complete the following |
Chamber suggests the Tell Me approach which is based around........
The main advantage of this kind of approach is........... although....... would also have to be considered. (scaffold prompt)
|Within the CPD day, teachers are advised to refer to these and use where appropriate as well as incorporating the use of question fans. 'If children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical and interactive activates (Reid and Bentley 1996 p85) 'Children need a clear purpose and a structured opportunity to extract and absorb information and to use it in a purposefully known outcome (PAGE-OCC 1994)||Integrate sources/references more effectively||You need to integrate these references more effectively. Choose one of the following or your own (or a combination): 'If children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical, interactive activities (Reid and Bentley...) Not only is this true for children but also their teachers who through the CPD activities would develop their understanding of comprehension. Furthermore they would need a 'clear purpose and a structured opportunity to extract and absorb information in a purposefully know outcome.' (Reid and Bently) |
It was clear that the CPD would need to extend the teachers knowledge of suitable activities because 'if children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical, interactive activities (Reid and Bentley...) or as Reid and Bentley state 'a structured opportunity to extract and absorb information.'
|Upon evaluation of the training day....||Further depth required here.||You need to explore the nature of evaluation further. Complete the following sentence. |
Evaluation is a necessary part of any CPD because.......For this event the evaluation would take several forms including........ These would correspond with the core role of a subject leader to.....(scaffold prompt)
|Tasks were planned to clarify the teachers own knowledge but also allow them to see how inference questions could be represented.||Exemplify||For example, questions such as................... were asked within the worksheet which encouraged the reader to .................... and ..................... (scaffold prompt)|
|The activity asked adults to watch the clip 'The Piano' and talk about how the characters feel and why they behave in certain ways. These questions go beyond the literal which is what questions at an differential level require.||Restructure this sentence||Replace this section with one of the following or your own (or a combination): Visual literacy also provides a good opportunity for thinking beyond the literal. For this reason staff watched 'The Piano' and the trainer asked a number of inferred questions e.g. What is the relationship between the two characters? As the visual text is wordless, all information had to be inferred from the evidence provided. |
Information can be inferred from both written and visual texts using the clues the author or director gives to the reader. The Piano was shown as part of the training to illustrate this and questions such as 'Who is the young boy?' were used to model the process used with children. (example prompt)
|A teacher buddy system can also be used as a guide of support to share successful strategies.||Analyse further||Expand this further by completing the following sentence. |
Working collaboratively in this way is likely to be successful because......
(think about links to theory here e.g. scaffolding)
|Through providing colleagues with an activity to identify and explain the authorial intent, this supports their understanding of the activities that could be completed with children to improve their understanding and aid in developing reader response levels.||Sentence structure||This sentence is quite complicated in structure and detracts from the meaning. Replace it with one of the following or you own (or a combination): |
In order for the teachers to be confident in developing the children's understanding of authorial intent, it was important that they too experienced and discussed appropriate classroom activities.
Colleagues would be asked to complete and discuss a related activity in order to develop not only their understanding of authorial content but also how it could be developed in the classroom. (example prompt)
|.......open ended questions [that] should lead to open-ended answers....the children should be encouraged to have confidence in their own feelings and ideas about the text.'||Extend your analysis/criticality||Read the following quote from Ted Wragg 'Questions are only as good as the answers they get.' Can you add this extend your argument further. |
Questionnaires were used at three points during the project to elicit comparable quantitative and qualitative data. Each used open questioning alongside fixed alternative questions (including Likert scales). The intention of each questionnaire was to gain an overview of the group's attitudes and perceptions of both standard written feedback and quality marking prompts, their ease of understanding, usefulness and formative nature.
The first of these was administered at the start of the module, the second after the first assignment had been completed, marked and the feedback prompts received and acted upon, and then the third after the second assignment had been submitted. All responses were confidential and trainees were able to opt out at any time if they so wished.
In response to the question 'Do you read written feedback?' 75 per cent of respondents indicated that they were more likely to read it than not, but only 20 per cent indicated that they read it thoroughly. 25 per cent indicated that they did not read written feedback at all. When asked whether they found written feedback useful, a very varied response was received, with 55 per cent of responses indicating that it was of some use, but 10 per cent found written feedback 'not at all useful', seemingly reflecting the research of others (Brown and Glover, 2005, Crisp 2007, Maclellan 2001).
The third question sought to find out which type of feedback was the more useful. Students had to rank in order of preference: a. overall numerical score, b. overall summary comments at the front of the work, c. written comments made within or alongside the assignment itself.
80 per cent of trainees stated that despite its summative nature, the overall numerical score was the feedback they were most interested in. Feedback made alongside/within the assignment was seen as the least notable with 80 per cent placing it in third place although it is within this ongoing feedback that improvement prompts would be used. Students felt these written comments were '...sometime too specific to that essay', and '...do not help me with future work'. The point of specificity is an interesting one. Crooks (2001) stated that '...feedback should be specific and related to need' and that benefits come from focusing feedback on 'specific ways in which the student's work could be improved', but if comments are too specific to the context they are less likely to be transferable and formative. Specificity should therefore be in relation to the learner's needs.
The next questions explored the formative nature of the written comments received. Results were very varied, with only 10 per cent of those asked indicating that written feedback was always used formatively. Even when action was taken based on feedback, it was never immediate and often only within the year. As stated earlier, if the learning gap is to be closed, feedback needs to be received and acted upon by the learners (Gibbs and Simpson 2003, Sadler 1989). The data indicates that despite the intention of the marker, the pre-project written feedback received was not generally feeding forward as it was neither engaged with nor acted upon.
Further questions provided qualitative information on the students' perceptions of effective feedback. Identified features included 'give specific ways to improve', 'organised and constructive', 'highlights areas within the assignment itself so it is clear to identify and rectify', 'to not only say what needs improving but how to improve it' and 'identify improvements that can be applied to other pieces of work'. These remarks allude to the earlier comment on specificity of improvement but not necessarily to context. It is also worth noting that the learners were seeking further advice on how to action the improvements, an aim of the scaffold/example prompts used within the project.
Each student worked with peers to respond to the prompts supplied, redrafting their original work. The clear expectations of the session ensured immediacy to the response and it was also apparent from their re-workings that the learning gap had, in the majority of cases, been closed to some extent.
|Original Text||Tutor prompt||Redrafted text|
|In order for the staff to fully benefit from this exercise, they should be encouraged to sit in mix key stage groups.||Why would this enable the staff to fully benefit? Would there be any difficulties with this method or organisation? Rewrite the sentence to include the specific advantages of this approach.||Sitting in mixed key stage groups would enable the staff to fully benefit as a range of texts and abilities could be shared. It would encourage the teachers to familiarise themselves with and learn about suitable and effective texts for other key stages/abilities.|
|At this juncture staff could also discuss whether literacy resources presently cater for these varying needs or whether recommendations can be made to fill any gaps.|| |
Explore this idea further taking into account both the advantages and disadvantages and the implications for the subject leader managing the training. Complete then add the following sentence.
Although such a discussion could have negative consequences such as … and it maybe that this information could be gained in another forum, for example, …
|Although such a discussion could have negative consequences such as apportioning blame towards staff if there is a perceived lack of resources or allowing certain staff to dominate the discussion of this subjective area. It maybe that this information could be gained in another forum, for example, an individual and possible anonymous questionnaire .|
|Upon evaluation of the training day…|| |
Explore the purpose and practicalities of evaluating training further by completing the following sentence.
Evaluation is a necessary part of any Continued Professional Development (CPD) because ……. For this event, the evaluation would take several forms including ………. These would correspond with the core role of a subject leader to…
|Evaluation is a necessary part of any Continued Professional Development (CPD) because the deliverer needs to know if it has been useful in order to prepare for the next CPD session. For this event, the evaluation would take several forms including a questionnaire, verbal feedback and written feedback. These would correspond with the core role of a subject leader to continually evolve their role making it appropriate and relevant to the school.|
|Within the CPD day teachers are advised to refer to these [key questions] and use where appropriate as well as incorporating the use of question fans. 'If children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical interactive activities.' (Reid and Bentley 1996 p.85) 'Children need a clear purpose and a structured opportunity to extract and absorb information and to use it in a purposefully know outcome '( PAGE-OCC, 1994 cited in Reid and Bentley 1996 p.131|| |
You need to integrate the references more effectively within your discussion. Choose one of the following or your own (or a combination):
*'If children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical, interactive activities (Reid and Bentley...) Not only is this true for children but also their teachers who through the CPD activities would develop their understanding of comprehension. Furthermore they would 'need a clear purpose and a structured opportunity to extract and absorb information and to use it in a purposefully known outcome.' (Reid and Bentley)
*It was clear that the CPD would need to extend the teachers knowledge of suitable activities because 'if children are to develop sophisticated and creative responses to texts, they need opportunities to explore these processes through practical, interactive activities (Reid and Bentley….) or as Reid and Bentley go on to state a 'structured opportunity to extract and absorb information.'
|Indicates that the first example is selected.|
|To provide teachers with a range of examples on how to support the teaching and learning of reading, some useful activities and games were provided to show them how to encourage children's critical and insightful reading through texts.|| |
This needs to make stronger links to the key aspects of the scenario. Replace with one of the suggestions below or your own (or a combination).
*Games, activities and strategies are important tools in developing children's ability to infer and evaluate text and identify authorial techniques. Drama activities such as freeze framing, hot seating or conscience alley are all key strategies that could be applied across texts and year groups and games such as 'yes no' also provide opportunities for paired work.
*An important part of CPD is providing the audience with the tools to use in the classroom and as such considerable time was spent looking at activities, games and strategies that could be used to develop inference, evaluation and understanding of authorial intent. These ranged from the simple 'yes no game' to visualisation, drama and use of visual texts.)
|Games, activities and strategies are important tools in developing children's ability to infer and evaluate text and identify authorial techniques. Drama activities such as freeze framing, hot seating or conscience alley are all key strategies that could be applied across texts and year groups and games such as 'yes no' also provide opportunities for paired work. Feedback and group discussion during the whole class time about the activities and reasoning behind specific choices during the activities could also help provide the teacher with evidence of the children's thinking and evaluation skills in relation to the authorial intent within the text.|
|A particularly important piece of information should be strongly highlighted; these are the APP materials highlighted as one of the periodic assessments on slide 14.|| |
The structure of the sentence makes the central meaning and relevance of it unclear. Replace it with one of the following or your own (or a combination):
*It would also be appropriate to make links at this point to the Assessing Pupil Progress (APP) material which is used periodically to assess, amongst other things, several assessment foci related to comprehension.
*Assessment forms a key part of teaching and learning and as such the trainer would also need to ensure that the staff were confident assessing comprehension using the specific foci within the Assessing Pupil Progress (APP) material.
|Indicates that the second example is selected.|
Once all the prompts had been acted upon, trainees were given a copy of their improvements for future reference.
This second questionnaire sought to ascertain whether the students felt the prompts were more effective than traditional feedback. In response to the question 'Do you prefer this form of feedback to the traditional form?' 95 per cent of students gave a positive response with the same number indicating that they were useful.
The level of usefulness was explored further in the next questions which were concerned with improvements made to both quality and understanding. All students felt that the prompts had resulted in an improvement in both. The quality of their redrafted work was perceived to be higher by all students with 59 per cent indicating i was considerably higher. Certainly in the short-term, the prompts appeared to have had had a consequence in learning. It would be interesting to further explore the longer term-benefits of this. Within the primary setting, learners receive prompts to action on a daily/weekly basis but in Higher Education learners submit far fewer pieces of work although they are longer and more significant. To ensure the actioned feedback endures, shorter formative tasks within sessions, possibly working with peers, may provide another context for their use.
When asked to identify the effective features of this feedback a number of common points were made and these were categorised into six broad areas (see table 8.) The specific nature of the feedback and the enhanced understanding were the most common of these. These link to the themes already discussed i.e. the specificity but also transferrable nature of an improvement, the need for a focus on how to improve rather than what and also the value of models (examples).
It was also interesting how formative the trainees considered the feedback to be. When asked, 94 per cent indicated that the feedback would inform future work and when identifying how it would do so, three broad areas were acknowledged: aiding clarity of explanation, developing individual specific weaknesses and increasing overall depth and criticality.
The third part of the project took place at the end of the module after the second and final assignment had been completed. The purpose of this questionnaire was to establish whether or not the feedback had in fact been formative. 100 per cent of respondents felt that the additional feedback had assisted them with their second assignment. They were also able to specify how the feedback had informed the work. These responses can be categorised into four broad areas: clarity of writing, use of sources, criticality and self-study skills.
The final question asked trainees to compare this and previous forms of feedback in terms of its ability to feed forward into future work. Responding on a scale, all trainees indicated that the prompts were more successful.
What is not clear from the results is whether the apparently positive response was due solely to the prompts themselves and not the way in which they were acted upon. It is worth asking, could the process of working alongside a peer actually be more influential than the prompts alone? If so, would peer discussion ensure the usefulness and effectiveness of any written feedback in that learners themselves would provide the scaffolding and examples?
Providing constructive feedback to any learner is recognised as hugely influential in the learning/assessment cycle, but written feedback has limitations which can in fact render the process worthless. Difficulties associated with the immediacy, scaffolding and response to the feedback apply to both primary and Higher Education contexts, so it seems pertinent to use the practice of one to enhance the other. In this study it has become apparent that the use of scaffold and example prompts do go some way to ensure that feedback feeds forward into future learning as they, and the associated process, demand a more immediate engagement from the learner, bridge the learning gap by illustrating not only the next step but how to achieve it, and in some cases appear to develop the beginnings of self-monitoring. However, this was a small scale pilot study that only involved a very small group within a very specific context. In order to assess the potential further a larger group would be needed, ideally from across disciplines. The long-term benefits of this approach would also merit exploration as well as further consideration of the influence of peer work within the process, and how quality marking could be adapted to suit the specific constraints of the HE context.