Category Archives: Using data

Aspiring to be “good” – would this provide space for improvement?

Posted on by

Amongst a number of  other duties the Standards in Scottish Schools Act 2000 sets out two key responsibilities for Scottish Local Authorities in respect to school education, namely Raising Standards and Reviewing School Performance.

Yet I was wondering if it’s time to reconsider these duties in light of the impact – or otherwise – that Local Authorities have had upon schools in their charge? As a headteacher, and in my ten years a member of school senior management teams in a variety of schools,  I would have to question what impact Local Authorities had upon raising standards in the school and whether or not the School Review process made a positive contribution to the raising of said standards. I want to make it clear that I am not denigrating in any way the efforts and support given to schools by Local Authority colleagues but that the very assumption that an external force can drive improvement within a school is perhaps founded upon a false premise.

For the reality is often that the standards in a school are directly related to the quality of leadership and commitment from staff in that same school. However, by giving responsibility for raising standards to the Local Authority it creates an expectation – from all – that the authority can make an impact from an external position.  This is turn gives rise to what I’ve previously described as the “Dae Sumthin”  mentality where Local Authority managers are under pressure to be seen to be taking action – even if this action doesn’t necessarily result in any observable consequence.  The important thing is that action is taken. 

In a similar fashion Authorities have gone to considerable trouble to create a range of means of “Reviewing School Performance”  . These mechanisms have taken many different forms all with the intention that we can “know our schools”.

What I want to question is the assumption that there is a direct causal relationship between how well we know our schools and how well we can raise standards? ( in the case the “we” are those outwith the school).

It would be my contention that the responsibility – and much more importantly the capacity – to raise standards lies with those who work in a school.  That was always my belief as a headteacher, a principal teacher, or even as a teacher and I’ve seen nothing in the last five years as an educational administrator to change that opinion. I’m not saying here that all of our efforts in Local Authorities are wasted but that there is an unintended consequence of our adherence to the notion that the more we do from “outside” the school the better things will be “within” the school.

So if the responsibility for raising standards should lie with the school does that mean that the Authority can abdicate from it’s responsibilities for school education?  I would argue that the quality of school education should still lie with the Local Authority – yet the responsibility to raise standards should lie with the school.  Now if this seems like “having one’s cake and eating it” I can understand how such an assertion might appear peculiar.  Yet what I have in mind is much more of a commissioning approach, whereby the Authority commissions the school to deliver education on its behalf.  Just as Children’s Services currently commissions a charity to deliver an aspect of its service, the overall responsibility still lies with the  commissioning body. It is the role of the commissioner to ensure that those who are commissioned are delivering the service to the agreed standards – it is not the commissioner’s responsibility to raise standards, simply to ensure that the standards set out in the agreement is achieved.

This actually chimes with something which Pasi Sahlberg said recently at a  conference when describing the success that is Finnish Education. For Pasi said that in Finland to be “good”  is “good enough”.  They do not aspire to excellence as a system but focus on ensuring that everyone is at least “good enough”.  I know this seems to lack the aspiration of our Journey to Excellence – but I actually think that this provides exactly  the kind of space in which teachers and schools can flourish. 

So in such an environment what should happen to the Authority’s responsibility to “Review School Performance”? Perhaps the clue lies with the last couple of sentences in that particular section of the Act when it describes how where the Authority concludes that following a review that where:

” the school is not performing satisfactorily they shall take such steps as appear to them to be requisite to remedy the matter.”

It’s here that I would want to refer to the model of practice which is emerging from many directions, namely Risk Assessment.  What I’m wondering is whether or not a Risk Assessment approach might provide schools with much more space to innovate and develop local solutions to raising standards?  Would it be possible for an Authority to assess the “Risk” relating to the quality of education provided by a school.  Rather than stating that a school’s performance is somewhere on the six point scale we instead provide a simple statement to parents and others that the risk that the school is not providing a “good” education is low, medium or high.  Schools would aspire to be in the “low” risk category.   I would reckon that only around 5% of schools would fall into the high risk category and that the Authorities’ resources could be targetted on those same schools – with others being given ever more freedom to innovate and create local solotions without external interference.

Perhaps we do sometimes need to weigh the pig?

Posted on by

“You don’t fatten the pig by weighing it” An evocative phrase used by those who would rightly challenge the concept of over-assessment or too frequent external assessment or inspection. A Head Teacher’s Union leader even described the English Ofsted as the “Office of Pig weighing”. The use of the phrase has taken on a global currency as the following examples demonstrate: Australia; England; USA or see google.

Let me say at the outset that I am uncomfortable with this analogy – children are not pigs – anyway Chris Thorn does a much better critique of the concept than I ever could in his blog post from 2006.

But for the sake of argument let’s just accept the pig weighing analogy and use it to make a point.  The question I’m interested in is whether or not we need external assessment, or testing regimes, earlier than the certificated courses which young people will encounter in S4 and beyond. In the current regime we have National Tests for 5-14. These have undoubtedly had an effect on how, and what, teachers – particularly primary teachers – have taught over the last 20 years.

With the introduction of A Curriculum for Excellence there is a possibility that there will be no nationally recognised testing regime to take its place for children below S4.  Now I know many people see this as a good thing and at first glance it does seem appealing but I really wonder if such a situation provides sufficient leverage in the system to change the way in which we structure and deliver learning and teaching?

In my post on reverse engineering I pondered on the “trickle down” influence or leverage on the curriculum provided by examination requirements. Secondary teachers in Scotland have been encultured into a system which takes account of the “examinable syllabus”. What is it that makes us so confident that we can make literacy and numeracy the responsibility of all in S1 – S3 simply by appealing to the professionalism of teachers? 

My point here is that I feel we do need to introduce some form of summative assessment of literacy and numeracy at the end of S3.  I would suggest that the internal judgements of teachers are complemented by a external test which when combined with the internal assessment provides an accurate judgement about the a young person’s abilities at that time.  I believe the external assessment would fulfil a number of functions:

  1. Validate the judgement of the teachers
  2. Where there is a discrepancy between the internal and extrnal assessment it provides a means of providing an external baseline with which to provide a comparison.
  3. Provides a purpose and motivation for young people to improve their levels of literacy and numeracy.
  4. Provide a useful benchmark for schools to measure their progress.
  5. Provide a useful and validated measure of a young person’s abilities which can be used by parents and employers.
  6. Appeals to what secondary teachers “know”  – i.e. teaching to the test.

Before you leap up and down at that last sentence I believe that many great teachers do teach to the test but they do so in such a way that benefits their pupils. The challenge facing us would be to create a test for numeracy and literacy which made schools teach these core skills across all areas of the curriculum and sought to test them in these self-same contexts.

We certainly don’t want to see an “Office of Pig Weighing” in Scotland but I think I could confidently predict a positive change in the way in which we teach literacy and numeracy in our secondary schools if we grasped the opportunity to create an imaginative testing system which complemented and validated our internal assessments and for which every teacher in the school was accountable – not just the Maths and English teachers. 

Making decisions

Posted on by

 

Having to make difficult decisions is a key part of my job. Some of these decisions can often be unpopular – but I suppose that’s what I get paid for.

Every decision is usually associated with a variety of options which will usually have a number of distinct features, namely:

  1. Consequences – each option will have positive and negative consequences directly associated with that course of action;
  2. Emotional attachment – there are usually people who will have an positive emotional response to one of the options and a negative emotional response to another.
  3. Evidence – most options will have associated evidence which can be used to either support or counter their effectiveness
  4. Familiarity – options which have proved successful in the past.

My own decision making process tries to take account of the above but there is one other question which I ask myself whenever I have to make a decision: How does this choice of option relate to other features of our practice?

In this regard I was deeply influenced in the mid 90’s by systems thinking as described by Peter Senge which transformed my personal practice.

Prior to that time I tended to make decisions based on a rough amalgam of the four factors mentioned earlier but where I looked at individual decisions as discrete entities. The lesson I learned from Senge was to see “things” as being part of a system, or part of a whole and that no one decision is ever disconnected from another – particularly if you are trying to achieve an overall goal.

Lastly, there needs to be a moral/ethical filter associated with the decision making process and reference to my own personal integrity and honesty.

However, all other things being equal it’s the connectedness to other factors and their relationship to the overall goal which will have decisive effect on which option will be selected.

School Based Management 1

Posted on by

I’m attending the Association of Directors of Social Work conference in Crieff.

One the key themes emerging is that of personalisation of services to users. The social work field is light years ahead of education in terms of using a mixed economy system for delivering services, by commissioning others from the private and voluntary sector to provide a wide range of short and laong term requirements.

As I was listening to the presentations my mind turned to how education might develop such a model.  It’s been something I’ve been considering for a while but the cogs seemed to click together this morning.

The starting point for this is how do we really devolve services to our communities?

What follows is definitely “blue sky” and might be disconcerting for some but I’ve found that sometimes we need to start from the extreme perspective if we are to shift our ground.

The local authority would set the local outcomes which schools would have to work towards.

Each child would carry an educational value credit which directly related to money which would go to the school. All other current budgets would be rolled together and added to the educational value credit.

If a child left the school the money would follow them – even part way through a year.

The school would deliver – though a contract – the educational service for the local authority in that community.  If the outcomes were not achieved in a given period of time then another service deliverer would have to be employed.

The school would purchase services from the local authority – or other providers e.g. finance support, personnel, staff development and even quality improvement and assurance.

The authority would maintain responsibility for strategic estate planning, such a new school buildings but all other items would be devolved.

Schools in a community could combine their resources to purchase a service from elsewhere.

The pupil support function could also be delivered by a independent unit commissioned by the authority and underpinned by a contract arrangement.

Parents would have a significant role in the strategic direction and monitoring of the school and would be involved in the review of outcomes at the end of a contract period. 

I know one of the major concerns would be the fragmentation of the current system which is building very vibrant learning communities where schools work together. However, if we believe that partnership working improves outcomes – and outcomes will be used to judge the effectiveness of a school – then the leverage for it to happen will be even greater than it currently is. In a similar way the need to engage with other agencies would be built into the outcome agreement.

Using outcomes to focus the planning process

Posted on by

 

We had a meeting on Friday where we looked further at how we could use outcomes as drivers of our new service improvement plan.

It was good to give this topic a significant amount of time and it looks like we are making progress.

We have agreed that each part of our plan will have:-

– an overall outcome, e.g. Every school will achieve a very good level of performance in Learning and Teaching. This replaces the aim or objective section.

– a desired impact, e.g. Children will experience a consistently high level of education. This is the “why are we doing this”

– measurable outcomes, e.g. we will identify a range of outcomes which will relate to the overall outcome. It will be important that these outcomes are well balanced and possible to gather.

– the actions, e.g. develop learning teams in every school.  In the school’s version of this we will be less interested in the actions and maintain our focus upon the outcomes.  Hopefully this will free up some of the bureaucratic demands from which the planning process often suffers.

In the past the success criteria (OUTCOMES) came tagged on at the end of the planning process.  What we are proposing is that the process is actually driven by the outcome and desired impact.  I hope to have completed a draft version of our plan for general consideration by the middle of February.

We are replacing the National Priorities as the strcutural framework of the plan with the UN Conventions rights of the child: Safe and Nurtured; Achieving; Included; Healthy and Active; and Respected and Responsible. The example given above would fit within the Achieving dimension.

So much of this relates back to something I encountered last summer relating to Social Return on Investment.

League Table approach and too much Testing remains Harmful to Education, says EIS

Posted on by

 

The Educational Institute for Scotland (EIS) – the  biggest teaching union in the Scotland have issued a number of press releases over the holiday period.

The last of these was entitled League Table approach and too much Testing remains Harmful to Education, say EIS

“The Educational Institute of Scotland (EIS) has called for a radical rethink on the over-use of testing in schools and the damaging construction of ‘league tables’ with the data collected. The EIS believes that too many local authorities continue to place too much emphasis on narrow testing and the collation of associated data which brings little or no benefit to schools, teachers and pupils.

Commenting, EIS General Secretary Ronnie Smith said,

“Despite the end of National Tests some five years ago, many authorities seem unable to cure their addiction to excessive testing in schools and continue to favour the flawed ‘league-table’ approach to measuring school success. This is in direct contradiction to current national educational priorities and has a negative impact on learning and teaching in schools. The use of such widespread testing places additional pressure on pupils and teachers to perform well in these tests – this has the inevitable result of narrowing the scope for teachers to use their professional judgement in what they teach, with considerable pressure to ‘teach to the test’ to avoid criticism of the school when league tables are constructed. This tick-box approach to measuring school success is of little value, and serves only to provide figures for education authority statisticians to crunch while simultaneously demoralising pupils and teachers.”

I found this an interesting perspective, particularly given the direction we are taking in respect to outcome agreements. Certainly from an East Lothian point of view we have never presented school assessment data in a league table format – and I can’t think of any other authority which adopts such a “league-table” approach.

I agree with Ronnie when he warns of the danger of solely focusing upon attainment as the only means of judging the success of a school but the new HGIOS3 makes it clear that pupil achievement is just as important.

However, I have to challenge his assertion that testing and the collation of associated data brings little or no benefit to schools, teachers and pupils. Firstly, schools need to have some way of judging their progress against an external benchmark.  Testing provides that benchmark.  I recently wrote about the “King o’ the midden” complex whereby it’s possible for a school, an authority, and even a country to delude itself about it’s progress, unless it collected data,  compared itself with its peers, and then interpreted how that that information can be used to shape its practice. For example, from the PIRLS data it is apparent that children’s reading in Scotland is not making the same rate of progress as in other countries.  Such knowledge initiates a question about how we currently teach reading and might have a direct impact upon schools, teachers and pupils.

A school can only objectively reflect upon how children are making progress throughout their school careers if they have access to  valid and reliable summative test data.  At an authority level such data helps to provide a means of judging a school’s performance in a particular area. For example, if a school’s attainment in maths is significantly below maths attainment in neighbouring schools, of a similar pupil composition, then it is legitimate to ask questions about the teaching of maths in that school.  Once again summative data leads directly back to the learning and teaching process.

I actually think the key point which Ronnie Smith is making is about how such data is used and the culture which underpins its collection, interpretation and use. My hope is that the culture we aspire to in East Lothian actually helps us to collect and use summative data where our ultimate focus is always upon the learning and teaching process, where formative assesment plays a crucial part.  The trick will be to ensure that such a balance is always achieved.

Last thought, Ronnie Smith refers to the needs of schools, teachers and pupils, but makes no mention of parents…..mmm?

PISA – systems CAN make a difference

Posted on by

The PISA results for 15 year olds were released on the 4th December.

If we are really serious about improving the performance of the lowest attaining 20% then we have to take account of some of the findings of this research.

Consider this:

Streaming at an early age tends to increase the impact of socio-economic background on student performance, PISA 2006 indicates. The earlier students were stratified into separate institutions or programmes, the stronger was the impact which the school’s average socio-economic background had on performance.Streaming at an early age tends to increase the impact of socio-economic background on student performance, PISA 2006 indicates. The earlier students were stratified into separate institutions or programmes, the stronger was the impact which the school’s average socio-economic background had on performance.

What emerges from the research is that systems can change and when they do they can have a dramatic effect upon children’s knowledge and skills. So do we have the courage and conviction to really change to meet the needs of all children – or are the forces of inertia which exist within our system too resistant to such change?

I’ll be looking at this research in more detail over the weekend.

A focus on outcomes – and leave the process up to schools and teachers

Posted on by

I’ve been doing some more work on how we might make use of outcome agreements with schools.  I’ve looked at the logic of this in some earlier posts but it’s only by experimenting with actual outcomes that we can start to see whether or not they would be a good idea.

Sometimes it’s only by looking at such possibilities that we can identify weakenesses and opportunities.  Don’t freak out too much as you read these outcomes but the whole idea is that these outcomes would actually free up clusters, schools and teachers to work out how they might go about achieving these outcomes – as opposed to spending all their time filling in forms and plans.

Some of this links back to something I came across in the summer about social return on investment.

The following are possible examples:

Within three years every child (without severe or complex needs) will reach the international literacy, maths and science benchmark levels for 10 year olds and 14 year olds

Every school will be able to demonstrate how they have developed their curriculum to take account of co-creation, personalisation and flexibility.

Every teacher will be able to demonstrate how they have developed their teaching in response to a curriculum for excellence

Every teacher will be able to demonstrate how children received their entitlement to digital access as set out in our Learning and Teaching policy.

Children will report that they experience a smooth transition in terms of learning and teaching from one stage to another throught their school careers.

Within two years every child will have a personal on-line space in which they can keep a progressive record of their achievments and attainment

All schools will be able to accurately forecast pupil attainment on an annual basis.

All schools will be able to measure value added and use this data to formulate future action.

Within three years all pupils will match or exceed their predicted progression levels

90% of children will report that school has a positive impact upon their health.

The number of children with a body mass index above the norm is reduced by 30% over three years.

The number of children participating in regular physical exercise outside school increases by 30%

All children can run continuously for 12 minutes at the age of 10, 12 and 14 years of age.

All children will be able to identify examples of how they make a contribution to their school or community.

All children will be able to provide an example of how they work successfully with others .

All children will be able to provide an example of how they have demonstrated confidence to work independently of others

PIRLS – some observations

Posted on by

It must be just the time of year but a plethora of international acheivement data is being made available in matter of a few weeks.

The most recent of these is the Progress in International Literacy Study 2006 PIRLS you can download the entire report from here but beware it’s 63mb and takes along time to download.

I’ve taken a look specifically at Scotland’s data and here are some of my observations. 

The PIRLS assessment tests children’s reading ability typically at the end of their fourth year of primary schooling, in  Scotland it’s the fifth as we tend to start one year earlier than most other countries.  The average age of Scottish pupils taking the test was 9.9 years of age which was actually younger than the age of children taking the test in most other countries.

The PIRLS average was 500 points and the Scottish score was 527 – 26th place out of 40.  The top three countries were Russia (565), Singapore (564) and Canada, Alberta (560). Scotland’s score was virtually identical to the last time the test was done in 2001 (528) – down 1 point, whereas some countries had changed their systems in response to the 2001 results and made significant progress, e.g  Russia + 37 and Singapore + 30, England had dropped -13 points.

Gender is a significant issue in terms of reading attainment across the world with girls scoring on average 17 points higher than boys, in Scotland girls scored 22 points higher than boys. Yet in Luxembourg the difference is only 3%, whilst a very large country such as the U.S it’s only 10%.  Scotland is also moving in the wrong diraction with the difference increasing by 3% in girls favour, whilst globally the gap has closed by 5%. 

The average class size was 24, Scotland’s was 26, Singapore was 38 and in Russia the average was 22.

There was some very fascinating data presented about the classroom organisation of students. Whole class reading was a feature of 35% of world classrooms whilst only a feature of 6% of Scottish classrooms; whereas only 8% of global classrooms used same- ability groupings, whilst it was a feature of 54% of Scottish classrooms, a figure only surpassed by New Zealand. In the top five countries only 5% of classrooms had same ability groupings.

Parents reading for enjoyment showed Scotland to be in the top 3 countries in the world at 63%.

Pupil’s positive attitude to reading Scotland has fallen  (- 5%) with only 42 % of children being very positive about reading against a global average of 49%.

Scottish pupils reading stories or novels outside school has also dropped – 5% to 35% just above the global average of 32%.

Only 33% of Scottish pupils said they read for fun outside school, the global average being 40% – there appeared quite a significant correlation between those countries who scored high for reading for fun outside school and teading attainmant.

25% of international classrooms have reading taught for more than 6 hours per week.  In Scotland the figure was 12% – a drop of -2% from 2001, whereas most other countries have seen an increase.

56% of international classrooms have reading featuring as a daily activity, in Scotland it’s 44%

Whole class teaching takes up 57% of the week in intenational classrooms – in Scotland it’s 44%

 13% of Scottish children were reported to need “remedial” (sic) instruction in reading, the international  figure was 17%.

Only 6% of Scottish teachers reported that gave the pupils a quiz about reading on a weekly basis, wheras the international figure was 26%.

I’ve picked out the main discrepancies between Scotland and international comparators – I’m convinced there are some important lessons to be taken from this research.

Taking the PISA

Posted on by

 

The Programme for International Student Assessment (PISA) is an internationally standardised assessment that was jointly developed by participating countries and administered to15-year-olds in schools.

The survey was implemented in 43 countries in the 1st assessment in 2000, in 41 countries in the 2nd assessment in 2003, in 57 countries in the 3rd assessment in 2006 and 62 countries have signed up to participate in the 4th assessment in 2009.

Tests are typically administered to between 4,500 and 10,000 students in each country.

Information about PISA 2006 

Sample test questions Maths, Reading and Science 

The full results will be made available on the 4th December.

My question is whether or not we could be using this information to help us shape our Curriculum for Excellence in Scotland?