By Caroline  Elbra-Ramsay  &  Anita Backhouse,  Faculty  of  Education  and Theology, York St John University

Contacts:c.elbraramsay@yorksj.ac.uk/ A.Backhouse@yorksj.ac.uk

Abstract

The aim of this small scale research study was to investigate whether a primary school formative assessment strategy could be transferred to Higher Education. Visualisers are an increasingly used teaching and learning resource in primary classrooms. Using examples, work is analysed and improved allowing for explicit scaffolding of feedback strategies as well as the unpicking of success criteria. This study sought to use a similar approach with part-time PGCE students, many of which were returning to education after a break. Using two curriculum modules (science and English) example assignments were analysed and improved using the visualiser to prompt discussion as a group and with partners. This then fed into peer and self- marking/ evaluation with the opportunity to apply some of the strategies explored earlier.

Data (questionnaires, interviews and assignment feedback) was collected before, during and after these scaffolded sessions and indicated that the students found it to be a valuable strategy in terms of exploring some of the tacit aspects of academic writing. The immediacy of the feedback, dialogic nature of the sessions and common shared understanding that the visualiser allowed all seemed to have impacted positively.

Key words: feedback, assessment, academic writing, visualiser, student teacher, modelling, analysis, science, English

Background to the study

The part-time primary postgraduate programme at York St John University spans an 18 month period and includes academic study focussing on professional skills and attributes as well as curriculum-focussed modules. Students are encouraged throughout to make links between school-based experience and there is a strong underpinning philosophy of reflective practice and enquiry-based learning.

This small-scale study focussed on a cohort of 80 student teachers in the third month of their programme. The cohort comprised of many mature students who had been out of an academic environment for an extended period. Based on evidence and feedback from previous cohorts, the consequence of this was that confidence, in terms of reflective and  critical thinking and subsequent transfer of these skills to academic writing, was likely to be low.

Two non-Masters level curriculum modules were chosen for the study based on helping to develop the students' skills in analysing evidence and linking theory and practice. The aim was to help students to engage in a formative process using peer review and feedback in order to facilitate assessment as learning (Black and Wiliam 1998) in order to become critical reflectors of their own work.

The investigation made use of visualisers which are essentially document cameras that connect to a computer. Images of objects or documents can be displayed, in real-time, via the screen to large audiences. Clarke (2013) suggests that the use of visualisers can be used effectively in the classroom to help 'signpost' those aspects of written text that learners need to consider when comparing and contrasting the quality of each other's work. Visualisers were an important tool in this project to help the students to develop a shared understanding of high quality academic writing through discussion and peer review, mediated and scaffolded by the tutors.

The project consisted of:

• a base line questionnaire to establish students' confidence in academic writing;
• two hour-long workshops focused on specific assessments;
• an analysis of completed written work;
• recordings of student discussions;
• post project questionnaire.

Supporting Literature

Research indicates (Brown and Glover, 2006, Crisp, 2007, Maclellen, 2001) that students are more interested in summative grades than formative commentaries and it is often the case that feedback is unread or not fully embraced and acted upon. Even with the best intentions, feedback does not effectively close the 'learning gap' (Sadler, 1989, p.121). This difficulty is further compounded within Higher Education when learning can be quite specific to modules, leaving little opportunity for feedback to inform future submissions.

A feature of effective formative feedback is that it is based on a socio- constructivist (or dialogic) approach (Gibbs and Simpson, 2004). This assessment dialogue (Clarke, 2007) enables learners to construct their own meaning by translating a judgement into a 'next step' by scaffolding or explaining both the gap and how to close it. Peer feedback strategies result in learners acting as co-responders (Van den Berg, Admirall and Pilot 2006) in collaborative tasks and are a useful mechanism for encouraging a more dialogic approach to the receiving, understanding of and acting upon feedback. Through it, learners articulate their understanding to others, assisting both the giver and receiver in actively constructing knowledge (Liu and Carless 2007.) Higher Education includes aspects of complex tacit knowledge that can  be difficult for tutors to define and students to decipher; working alongside a peer allows this knowledge to be actively engaged with using a common understanding and language. Indeed, Nicol (2010) identifies that feedback should be expressed in a language that students understand if it is to be successful.

Actively engaging with the work of others has a consequential impact on the ability to monitor self-performance which Sadler (1989) recognises  as the ultimate goal of feedback. Giving effective feedback to others is dependent on making critical connections between prior knowledge (Liu and Carless 2007) and desired understanding/performance, encouraging the development towards self-regulated learning (Butler and Winne 1995) or 'learnacy.' As such, it moves from 'assessment for learning to assessment as learning' in that it is 'an active process of cognitive restructuring that occurs when individuals interact with new ideas' (Western and Northern Canadian Protocol for Collaboration in Education, 2006 p.41).

However, working with a peer does not in itself develop learner self-regulation. Nicol and Macfarlane- Dick (2006) identify several necessary elements including the need to clarify good performance or the desired standard. Seeing an example of the desired goal goes some way towards this but relies on the learner having the necessary skills of evaluation and analysis to identify why and how it is the target. Learners need opportunities to develop these skills through modelling, supporting and scaffolding. Increasingly primary school settings do this by making use of a visualiser or document viewer enabling learners to not only see a desired model but, in collaboration, scrutinise its successful features and possible improvements (Clarke, 2013).

An additional benefit of peer feedback is its immediacy. The context of a workshop also means that there is time to not only receive but act upon feedback, which Sadler identifies when he states 'the student must engage in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121). Receiving feedback from tutors can frustratingly be several weeks after the work was completed which compromises the perceived need to act upon it. It could be argued that tutor feedback is worth waiting for but, as Gibbs and Simpson (2004 p.19) state, 'there may be a trade-off between the rapidity and quality of feedback so that, for example, imperfect feedback from a fellow student provided almost immediately may have much more impact than more perfect feedback from a tutor four weeks later.'

Focus of the enquiry

Clarke (2008) advocates the use of a visualiser as a key tool in the modelling, identification and explanation of success criteria. In this way, learners are explicitly taught 'learnacy' or 'learn to learn' an underlying aim of assessment for learning (Sadler, 1989; Black et al, 2006). This project investigated the effectiveness of this approach in an HE setting.

Research methods

According to Cohen, Manion and Morrison (2010, p.289) 'a case study provides a unique example of real people in real situations'. This case study focussed on two cohorts of part-time student teachers and two course tutors working within two curriculum-focussed modules.

Data was collected in three ways: through surveys; audio recordings and document analysis. Informed consent was given by the participants for the surveys, audio recording of sessions and scrutiny and analysis of their written work.

The first set of data was collected via a survey given to an outgoing cohort of students to establish what issues they felt there were in terms of their academic writing (27 responses received). Outcomes from this survey were used to set the focus for subsequent work with the incoming cohort. A second survey was given to the incoming cohort to establish their levels of confidence in terms of the technicalities of academic writing (punctuation and referencing, for example) and their level of understanding in terms of linking theory and practice and in the analysis of writing. This second survey was administered prior to the workshops and before any written assignments were embarked upon (53 responses received).

Two workshops were delivered and attended by all the students in the cohort. They were designed to follow a similar theme with one focussing on the link between theory and practice and the other focussing on analysis of children's work.

Prior to each session, the student teachers were given an initial writing task. Using the model of the assessment itself, each was given a selection of reading for the science task  and evidence relating to an individual child's reading for the English task.

Each workshop began by displaying a sample report on the visualiser as a prompt for the group. This was followed by a second sample of a higher quality. Students were asked to compare the two, identifying which was the most successful and why. The tutor highlighted each sample accordingly and scaffolded the discussion by focusing the analysis on key academic areas and by modelling alternative improvements. This comparison and exploration enabled examples of tacit academic knowledge to be explored, for example, depth of analysis.

Students then applied this process to their own work. Working in pairs, the students identified strengths and opportunities for improvement using some of the ideas modelled by the tutor earlier. Mini-plenaries were held periodically during which the tutor displayed randomly selected samples of this work on the visualiser and encouraged discussion about the quality of improvements and other potential possibilities. Students were then encouraged to revisit their own work and apply similar improvements, if appropriate. The annotated drafts were copied and returned to the students at the end of the session in order to inform their future written submission.

Observations were carried out during the workshop sessions and audio recordings taken of the student discussions. Further data was collected by scrutinising the copies of the students' annotated written work from the workshops and subsequent written assignments (85 participants). A final survey was given to the second cohort at the end of their programme, one year later, to establish how their feelings about academic writing had changed over time.

Theoretical frameworks

Through in-depth qualitative analysis of this wide range of data a rich and informative picture was gained in terms of the students' levels of confidence and understanding of the demands of academic writing. Their construction and re-construction of ideas through peer review and dialogue was monitored. Qualitative analysis provided an insight into social constructivist and cognitive apprenticeship models (Dennen and Burner, 2008).

The overall findings of the research were:

• a high level of student engagement in the tutor-led visualiser sessions;
• increased levels of student perception and awareness with regard to assessment criteria and expectations in terms of academic writing;
• increased student understanding of academic writing convention and tacit knowledge.

Despite the students' perceived levels of confidence in academic writing there were clearly some students for whom there were problems. This then provided the focus for the modelling, discussion and peer review during the subsequent workshop session using the visualiser.

Could draw on some of the success criteria I had a clearer understanding of 'the rules'.

I knew what aspects of essay I was expected to cover.

They allowed me to identify links between theory and practice.

The workshops helped me to understand what style of academic writing was required and gave me the opportunity to discuss my writing with the tutor.

It helped because I was able to utilise the advice I got from the tutor in terms of strengths and weaknesses, especially with regards to writing in a constructive and analytical way.

3. Overall, at the end of your programme, how do you feel the quality of your academic writing has improved in terms of your ability to link theory with practice and analyse critically?

Achieving  a  74  in  my  research  project  was  confirmation  for  me  that  my  writing  had improved.

Definite improvement, but could still improve further

Not much. I still have a long way to go in terms of explaining my ideas critically.

I think that I was competent in this regard, but the programme has definitely enabled me to nurture my writing skills.

It has vastly improved. I hadn't previously done any academic writing and the help and guidance I received enabled me to pass all my assignments. Thank you for your valuable input!

Amazingly well I could not have achieved had it not been for tutor support.

M level writing stayed very much in the C boundary so I don't think that it helped me to improve a great deal.

Discussion and conclusion

Evidence seems to suggest that the workshops developed both the student teachers' perceptions of their abilities to construct an academic and also their actual capabilities. The data collected from the Cohort 2 assignment implies that students were generally more capable in the three areas considered than they had been during the pre-workshop task. The overall level of competence was also better than that of Cohort 1. Feedback during the workshops indicated that the specific focus on those aspects of academic writing that the students felt less confident about, was effective. Tutor modelling using the visualiser, discussion and peer review in small groups / whole class all contributed to a raising of awareness about what constitutes quality academic writing. The high level of collaborative engagement and subsequent impact on understanding indicates that the learners were engaged as co-responders (Van den Berg, Admirall  and Pilot 2006) within an assessment dialogue (Clarke 2007).

The structure and position of the workshops meant that feedback was both timely and purposeful (given the similar summative assessment due later in the term) and was immediate, a key indicator of effective feedback. The overall approach was felt by the tutors to be effective for this reason and would support Gibbs and Simpson's suggestions about the timeliness of feedback (2004).

The need for a shared understanding of the vocabulary for academic writing between students and tutors was also highlighted; a sharing of the criteria for success. As indicated by some of the student responses, it is important that students know what the tutors are looking for and to this extent dedicated time spent on this appeared to be productive. Students needed to experience the explicit unpicking of some aspects of (tacit) academic knowledge and associated conventions to help them construct their own meaning and engage fully with it. Bloxham (2010) identifies the crucial role dialogue plays in this process; the use of the visualiser encouraged this.

There is also some indication of the workshops as a form of 'feed forward' in that students appear to have recognised how they were useful with their later submissions. It appears this consequence is related to their developed understanding of tacit academic knowledge rather than their ability to self-regulate or  self-efficacy which is understandable given  the small number of workshops that took place. It would be worth exploring this feature in more depth in future longitudinal studies.

There are some limitations with regard to any firm conclusions that can be drawn, however, and it is worth bearing in mind that questionnaires and assignments were collected from different cohorts. As such, the groups studied were distinct in size, experience and understanding. The assignment data collected and analysed was very much dependant on the specific learning context. For example, the English assignment did not allow opportunity to both effectively measure a student's ability to link theory and practice and explore this aspect during the workshop. Consequently it is clear that such workshops need to be carefully constructed around specific assessment criteria relevant to the module.

The tutor modelling and visualiser approach has now been deployed across a number of modules in order to help students and tutors share the vocabulary and develop and construct a shared understanding of what quality academic writing looks like.

References

Black P and Wiliam D. (1998) Inside the Black Box: Raising Standards through Classroom Assessment. London, King's College

Black P, McCormick R, James M and Pedder D (2006) Learning How to Learn and Assessment for Learning: a theoretical inquiry Research Papers in Education Volume 21, Issue 2, 2006