Scaled Scores and Setting Targets

Hello,

A brief blog because it’s late, but many people have been asking about this.

A lot of schools wish to be able set end of KS3 and KS4 targets for their new Y7s, based upon scaled scores.

The problem is… the desire to transpose scaled scores on top of existing target setting processes that involve levels, sub-levels or fine-levels. A lot of these attempts seem to say something like well 100 is a 4a so that makes a grade B on our old target-setting system, which on the new spec is a grade 6, so we’ll set a grade 6. And this moves incrementally up and down the spectrum.

I don’t recommend that approach, but who am I to say it is wrong. Generally I do not advocate saying things like a 4a is 100 because it has been made quite clear that the old levels system and the new scaled scores do not correlate in that way.

However I guess it is obvious to say the children achieving the highest scaled scores, would have achieved the highest NC levels and vice versa.

Anyway – enough of what I do not recommend and a word about what I think.

Firstly you have to understand your scaled score distribution and how it compares to national.

  1. What does my scaled score distribution tell me about the nature of the cohorts attainment distribution compared to national.
  2. Does the distribution show something I’d expect? Is it typical?

You can download your scaled scores from ncatools (which I imagine many of you have done) then you can map your scores against national. I suggest starting with Reading and maths, and I provide a spreadsheet you can use to do this here: LINK

Then answer the questions above.

So perhaps your distribution in maths looks like this:

scaled-scores-maths

So you would say, OK this school has a range of scaled scores that are loaded towards the lower range, with a higher distribution there and in the middle but fewer of the higher scores.

If you agree that this is not unusual for your school, then you can move forward with confidence.

Next I would think about what KS4 outcomes you would expect for  such a cohort, and if it appears not unusual to what you are used to then you can fairly confidently benchmark certain areas.

Grade 7 – you can benchmark your existing proportion of A grades achieved to Grade 7, and you can benchmark the national distribution of A grades to grade 7.

So to run through an example, we know that roughly 22% of GCSE entries result in an A grade or above. We also know that Ofqual have said that roughly the same proportions will achieve grade 7s or above as achieved A grades or above.

So we can take a small leap of faith and say that the top 20-25% of scaled scores will be the ones achieving the grade 7s and above. Then you can look at the scaled score distribution and see that this lies somewhere around a scaled score of 107 / 108.

Great, then you can repeat the process with grade 4s and grade Cs as this is the other grade point that Ofqual have said has been ‘pegged’ (roughly) between the two specificati0ns.

A*-C grades nationally is around 70% so reasonable to assume that 9-4 grade proportions will be the same, mapped onto scaled score distribution… that gives you something around a scaled score of 99.

The final thing you know from Ofqual is that grades will be distributed equally. i.e. in theory it is the same ‘distance’ from 4 to 5 as it is grade 6 to 7. Hmmm we’ll wait and see but that’s what they say. This means you can statistically fill in the gaps, sort of anyway.

Then you’ve got to think about what YOU know about YOUR school and the students in it, and what is an appropriate level of challenge.

i.e. if you are a high attaining school and usually achieve 40% A*-As then you are dealing with a different situation to a school that regularly achieves 5% A*-A grades.

You need to marry these factors with your thoughts about your scaled scored distribution. The national data will give you a clue.

Basically you are estimating a whole bunch of things and as such you are producing estimates. Then you need to make a prediction, and then you can end up with a target. But of course everyone wants aspirational targets, so you’ll need to add a bit.

I do feel that the whole affair this year is much more of a personal process and not simply the case of plugging data into a system and trusting the numbers it’s spits out.

Estimation based upon historical evidence is impossible, we have new starting points and new end points.

Then again you will probably want to do something…. so….

This is what we’ve done:

Scaled Score / Target

109+ / Grade 8

106-109 / Grade 7

102 – 106 / Grade 6

97 – 102 / Grade 5

92 – 97 / Grade 4

<92 / Grade 3

I should stress that this targets are not currently shared with students, and possibly won’t be until year 9, that’s a whole other blog about life after levels. Instead students are placed in discreet ‘starting profiles’ for monitoring and support purposes.

Going to stop here, could ramble all night.

 

Advertisements

19 thoughts on “Scaled Scores and Setting Targets

  1. David Bishop September 14, 2016 / 6:45 am

    I have enjoyed reading this – thank you. I like your reasoning with best wishes David

  2. Lewis Sewter September 18, 2016 / 7:08 pm

    Hi Peter,

    I wondered where this would fit with trying to allocate low, middle and high prior attainment bands.

    Would you use set thresholds or would you alter them each year to try and have a certain percentage, e.g. only 20% would be high prior attainers?

    Regards, Lewis

  3. John Simkin September 22, 2016 / 11:08 am

    An excellent piece of work. I’ll be building on this for our Academy

  4. johntyjohno September 26, 2016 / 8:13 pm

    Hi,

    Interest post. I have a quick question – If the scaled score of 106/107 comes out at around a grade 7 then should the target for the ‘group’ by higher? If you target a 7 doesn’t that mean you’re targeting a P8 score of zero.

    I know there’s a lot of if buts and maybes here but mainly asking to test my own understanding of what you’ve written.

    Thanks,

    • dataeducator September 26, 2016 / 8:31 pm

      We don’t know the relationship between Scaled Scores and Attainment 8, this is a different approach using the national proportions of A grades and applying some knowledge of your school context and adding some challenge.

  5. Cristina September 27, 2016 / 8:38 am

    Thank you for this post, really useful! We are looking at how this approach can be adapted to our school.
    I was wondering, where did you get the national scaled scores distribution?

    Thanks

  6. Ian Davenport September 27, 2016 / 10:27 am

    Is the scaled score used just Maths or is it an aggregate of all 3 scores ? Would they be on the same target for all subjects.

    • dataeducator September 27, 2016 / 12:01 pm

      We used the average of Reading and Maths, as per currently secondary accountability and how they generate fine-levels. This may change but who knows.

  7. Jack September 28, 2016 / 2:59 pm

    Hello, I am slightly confused how progress will be measured between KS2 and KS4 based on scaled scores. Will GCSE scores equally be converted to scaled scores?

    • dataeducator September 29, 2016 / 7:28 pm

      No, like now, the government provide Attainment 8 estimates for KS2 fine-levels to GCSE points scores, they will do the same but for KS2 scaled scores

  8. Alan Brooks September 28, 2016 / 9:35 pm

    Hi, thank you for this really useful information. I was wondering, if a score of 99 works out at 70% and therefore this should be the percentage of students achieving a 4 and above, is there a specific reason you chose a score of 92 to 97 as a target of a 4? Nationally, approximately 91% of students scored a 92 or more. Just did not know if there was a specific reason you chose a score of 92 when it is so far above 70%?

    • dataeducator September 29, 2016 / 7:30 pm

      Because our existing target setting system had a similar level of aspiration/challenge. i.e we were setting 3A students a grade C target.

  9. Jacky Clinch November 11, 2016 / 11:51 am

    I like the nested if statement for scaled score to target. Do you adjust your scaled score target depending on P8 bucket?

  10. Drew Lawmaker November 18, 2016 / 1:29 pm

    Another excellent post. We have decided to follow a similar system. I wondered do you use FFT at all in your data setting process? If not why not?
    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s