Evaluation is For Learning

“Tell us about a time when you enabled a learner to achieve beyond their own expectations and explain how you met their needs.”

Making a judgement about the effectiveness of an individual is probably nowhere more focused than in the interview situation. For it is in this highly charged environment that an interviewing panel draw conclusions about the likelihood of a person to meet the future needs of their organisation.  And how do they do this? They ask the person being interviewed to tell them the “story” about how they have behaved in the past in response to a variety of challenges and circumstances.

This interviewing technique is known as Behavioural Questioning and is based upon the assumption that past behaviour is the best predictor of future performance in similar situations. Of course there is some validation of this narrative in the form of references from employers, etc.

The power of “storytelling” is also recognised in the realm of higher education where narrative methodologies are used with great effect in postgraduate studies up to doctoral level. Story collecting as a form of narrative inquiry allows the participants to put the data into their own words and reflect upon practice rather than merely relying upon the collection and processing of data.

It’s against this background that I want to explore the dominant method of evaluating the effectiveness of teachers, schools and educational systems – and the unintended consequences that such a model has generated. My argument being that we have a measuring (quantitative) weighted system, with qualitative (storytelling) being of secondary import, whereas I would turn that relationship on its head.

For surely the ultimate test for any education evaluation system is the improvement it leads to in outcomes for children and young people – and it is generally accepted that the factor which makes the biggest impact upon the effectiveness of that system is the quality of classroom teaching and learning.

Yet despite this knowledge it is an implicit fact that most school improvement systems are based upon the external collection and interpretation of student outcomes – with little reference to the quality of the teaching and learning process.  The assumption being is that it is possible to improve school performance through external challenge.  The problem with this system of school improvement is that it is based upon the premise that self-improvement cannot be relied upon in isolation.

Such external challenge has the unintended consequence of disempowering staff within the system to the extent that they feel pressurised to improve as opposed to tapping into their professional commitment to improve.

So if the dominance of the counting and measuring (quantitative) model has proven ineffective what might be the alternative? I think the answer lies in a parallel methodology that has had a transformational impact upon many of our classrooms over the last ten years.  I am referring here to the notion of the Assessment is For Learning programme (AiFL).

The logic of Assessment is for Learning is based upon a realisation that simply giving a learner a mark or grade at the end of a course of study (summative assessment) does not enhance the learning, nor the teaching, process. In contrast where a teacher (and learner) use Assessment is for Learning to provide and reflect upon on-going feedback to revise and develop further the learning and teaching process – it actually enhances the final outcome and the effectiveness of the learner and the teacher in the future.

Now it seems to me that that our school evaluation models seem to comply with this simplistic paradigm. We use summative assessments – class, year group, school, authority, results – as the driver for change and make only passing reference to underlying stories which underpin the outcomes.

So what might a system look like that modeled itself upon AiFL? Let’s start by giving it a name – Evaluation is for Learning (EiFL).

I actually think we are beginning to see EiFL manifest itself in an incredibly exciting and organic manner within the Scottish education system in the form of pedagoo.org. Pedagoo represent a group of Scottish educators who are determined to describe and tell stories about their own practice in an open and transparent manner with the view to improving the quality of education they provide.

By tapping into what it is these educators are attempting to do in their own classrooms we begin to see an alternative to the dominant quantitative methodology, whereby teachers take the lead by sharing, reflecting upon and improving their practice.   Imagine a school where every practitioner was “fired up” to the same extent and enabled and encouraged to participate at such a level, where they could share their stories with confidence and a passion for learning and professional inquiry – I’d put my money on that school any time!

School evaluation could be conducted in a similar manner with external evaluation focusing upon the narrative stories of managers and teachers as they describe how they are attempting to improve the quality of the education in their school.

The relationship between the stories (qualitative) and the counting and measuring (quantitative) in EiFL is reversed to the extent that the numbers are used for validation – not judgemental – purposes.

And before any of you think I’ve gone soft – if any teacher couldn’t answer the question posed at the top of this article I’d have extreme reservations about their competence – regardless of the outcomes of the students in their class.

Educational Leadership Metaphors

I’ve started a small piece of research using Twitter  – the hashtag is #leadershipmetaphors

I’m asking people to identify the predominant metaphor they would use to describe effective educational leadership. I’ll run this through until the 27th April.

The score so far is:

Ship’s captain 2

@Mary10478: Captain of the ship: an eye to the horizon; navigate dangerous rocks; steady in storms; motivate crew; be brave far from harbour

Architect 1

Marathon Runner 1

“@johnoneill13: Marathon runner not sprinter.

Diplomat 1

Farmer 1

Theatre Director – 1  Has a clear idea of the overall structure & works as an ensemble with the actors to create the best end product

Master baker 1

Conducter 1

Jazz band leader 1

@realdcameron: It has to be the band leader -like Prince, brings the vision and imagination, lets the band take it on and can still pull it together if it goes off

Air traffic controller 1

“@atstewart: Air traffic controller. (Don’t like word controller) but allowing take off providing safe space as well as safe landing fits.”

Master Builder 1

Tour Guide 1

Submariner 1

“@KarDoh: I like the concept of the submariner – sometimes you can cruise along + enjoy the sunshine but mostly you need to explore deep down”

Virus 1

Gardener 1

Lens 1

Giving permission for partnership working

It’s been four weeks since I took up my position as Director of Education and Children’s Services at Midlothian Council, in addition to my new role as Executive Director of Services for People in East Lothian.

Aside from being able to meet and work with great people the most fascinating – and potentially most significant – aspect of my dual roles has been the opportunity to give permission for people to work together across two authority boundaries. People often talk about the “bottom up” approach as being the most effective change strategy and I’ve been an advocate for that mindset throughout my career. Yet over the last few weeks I’ve come to realise that in the realm of partnership working such “bottom-up” approaches are often stymied by the managers and leaders who are further up the ladder on either side of the partnership boundaries.

From a simplistic point of view let’s take a person who is appointed to a joint post between two authorities but who has a number of layers of management above them. To be a change agent in such circumstances is very difficult – especially if the partnership process may eventually compromise the position of the managers above them in each organisation.

Now contrast that to a situation where the change agent occupies a more senior post than those same managers in either organisation.

It is this latter situation that I now find myself in, whereby I can now give permission, authorisation and encouragement for colleagues to work together for mutual benefit and benefit to the organisation. In the past I’ve often found one of the most common obstacles to joint working, articulated by staff, is that “they won’t let me do that” – with a symbolic upwards finger point, alluding to the fact that management in their organisation would not approve (of course, there may be other more significant and personal reasons behind these assertions but I’ll leave those to your imagination). However, any such assertion now falters on the fact that such an allusion would mean that I (as the Director in both organisations) must have some form of leadership schizophrenia, where I can occupy two points of view on either side of the organisational divide.

Four weeks into the process it’s been very rewarding to see people actively and without direction begin to explore this new world, and from a personal perspective I’ve come to realise that one of the most important things I can do in my unique position is to continue to give permission for innovation and joint working – with a clear and absolute focus upon the quality of service we provide.