Can’t Data, Won’t Data

There’s a new trend in town.

It’s the refusal to take into account useful information for very arbitrary reasons. Case in point, this week I was on the periphery of a debate about transition matrices and the school of thought went that because transition matrices can be used to determine levels of progress, and levels of progress are old hat then transition matrices are absolutely redundant and they should no longer be produced.

But actually transition matrices are incredibly useful. Like everything else, they deserve to be freed from the shackles of levels and thought about in a better light. What they actually tell us, can be useful progress information when set against other known information of the student, the class and the subject and there needs to be no mention of levels of progress or any other such concoction at all.

However, on another note, some people still like to check back to levels of progress, and whilst this is not the future of progress measures it can be a reassurance at this ‘transitional’ time as we step into the unknown. I liken this to being on holiday, “that’s costs 87 in the local currency, how many pounds is that?” People like to relate back to things they are familiar with. This is why I won’t publish LoP progress figures as standard but I won’t refuse to generate them if someone asks for them. Why? Because they may want them for a particular purpose (i.e. to enhance their knowledge) and I’m a data manager, a data enabler, not a data refuser.

The new ‘official’ way of measuring progress of course is progress 8, which is all very well and good for thinking about how the whole cohort has progressed, but not very good for telling an individual teacher how the students they taught, progressed in their subject.

The nature of progress 8, where the door has been shut on providing estimates in advance, means that teachers are much more in the dark as to whether the grades their students are achieving are favourable compared to students of similar prior attainment nationally, or not. The manta, every student, every grade which is lovely in theory, until you realise that points gained at the top of the GCSE grade scale are worth more points than those at the bottom. Thus rendering higher attaining students more important for the next couple of years until every subject is on the 9-1 grade scale.

I digress slightly, but measuring progress 8 does raise an pertinent point for this blog, which is – yes progress 8 is impossible to predict accurately – but this doesn’t mean that it automatically becomes ‘can’t data, won’t data’ for schools in current year groups. Internally it can be a very relevant way of measuring student progress compared to other students, and subjects, classes, groups compared to other student’s classes and groups. Now it doesn’t matter (too much) if your estimates you are using for attainment 8/progress 8 are later going to become incorrect, because the basic premise is more than likely to be the same… students with higher prior attainment will attain higher point scores on average, and it is also likely that the pattern will increment.

So in terms of internal planning / reconciliation / discussions / support it is fine for schools to use this methodology, it’s only when the figures are shouted about as being the truth that we have a problem.

As worrying for me, is the refusal of some professional data software companies to even contemplate providing this information to their customers. The reason? we can’t guarantee it’s accuracy, it won’t be valid, therefore we won’t give you the option to look at your data in this way. Basically, we can’t trust you not to interpret the data as fact instead of part-fiction and therefore you can’t have it, we won’t provide it. Now pardon me here, but I believe it is for schools to monitor and develop their own data strategy and not have it policed for them.

However, one professional company has come to my attention for seemingly taking the bull by the horns. 4 matrix is making the step to provide progress 8 estimates, also reduced down to subject level, and probably beyond. With also the ability to adjust estimates up or down by  a certain percentage. Empowering schools. I should add that my own little spreadsheet tool (APA Pro) allows this too, but this is not really my point, I have no affiliation whatsoever to 4 matrix but I do applaud their decision to put data decisions into the hands of their customers and not guard them from making misconceptions.

Educate and empower schools and drop the data donkey’s who saddle schools with a stymying myriad of can’t do and won’t do attitudes.


I may be an ass, but I am an ass of my own making.




3 thoughts on “Can’t Data, Won’t Data

  1. Deborah Harvey June 22, 2016 / 9:30 pm

    We had a trial from the lovely people at 4M and their P8 value matched what I had generated within SIMS. Fairly chuffed at that..

  2. Alex June 28, 2016 / 11:00 am

    SISRA Analytics provides Progress 8 Estimates too, I’m not affiliated with them in any way other than being a customer doing your job in a school. Just FYI.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s