the brilliantly leaping gazelle

I do like a good survey. Not as much as I like a bad survey though. What helps to differentiate the two? Well a large amount of people surveyed certainly helps. So to does the manner in which they were surveyed; which questions they were asked, how they were asked, that sort of thing. Above all, how easy is it for me to check the findings. To see if the claims made because of the survey hold true. 

What first piqued my interest in the notion of there being good and bad surveys were make-up adverts on television. Those big, expensive production numbers, usually featuring an of the moment film star or ‘celebrity’, and a voiceover making all manner of preposterous claims. Preposterous because of the small print, the very small print that appeared in a ‘blink and you’ll miss it’ moment right at the end of the advert. The incredible sounding 92% of women who thought something, or the 87% of people who thought something else was indeed incredible. For the simple reason that those percentages, whilst true in and of themselves, were based on an incredibly low number. 

Most times, less than a couple of hundred people tried something and were then asked what they thought. Essentially it was the make-up company’s marketing department, all of their friends and some of their friends friends. It always baffled me how this could happen, how it was possible for such expensive adverts could get away with making such bold claims based upon so little actual evidence. My favourite one was for a female beauty product. Over four million of them had been sold the advert claimed. A huge percentage agreed with some claim I can’t remember. But I can remember how many actual people that huge percentage turned out to be. 87.

But it was only make-up. So did I care? Not really. It seemed fitting that the claims were based on nonsense, Did anyone really think that ‘healthy looking hair’ was the same thing as healthy hair? That ‘reversing the signs of ageing’ was actually doing anything? That ‘leaves you feeling with fresher, younger looking skin’ was anything other than meaningless when the model in adverts was no older than 25. I just quietly snarked.

Then I began to notice how political option polls were doing the same thing. Making big claims to generate even bigger headlines. One of the themes of this blog has always been the way in which polls and surveys flatter to deceive; to give the impression that a majority of people think something, when actually its what the people who commissioned it the first place want it to show.  In politics this matters. Its called ‘the bandwagon effect’; everyone wants to back a winner and if all the polls are predicting the same person is going to win..

This is all a long-winded way of writing that I don’t trust polls. Surveys are no better. They both depend not just on on the questions asked but equally, how they are asked  This clip from ‘Yes Minister’ illustrates this much better than I ever could.

So a survey can easily be skewed to give the desired outcome. If a pressure group wants to prevent this, for example. Or maybe charity wants to that to be allowed. Most times I don’t care. But then I saw this headline in ‘The Daily Mirror’ the other day, ‘’I came to the UK to help the NHS — but I instead face racism at work’

It went on, ‘Staff have seen a disturbing rise in racism from patients over the last 18 months, which emerged in a poll by theSociety of Radiographers (SoR). Workers who moved to the UK to help the NHS are forced to deal with daily abuse, which they blame on the rise of the far-right movement.’ Unfortunately, aside from a couple of testimonials from two radiographers, there was no other evidence, no other details relating to the survey. So once again, I saddled up my hobby-horse and headed over to the SoR website to try and find out more. As you’ll have no doubt guessed I was disappointed. A page mentioned the survey, but provided no other details. It was little more than a press release. 

Until this morning, when magically all was revealed. Or not so ‘all’ as it turned out. ‘The Society of Radiographers has garnered national attention from national news media following its report about a rise in racist incidents within hospitals. Following the release of results from the SoR’s ‘Racism in the Workplace’ survey earlier this week, media organisations including ITV, The Guardian, Sky News, The Independent and The Telegraph have reported on the findings. The survey, which ran from January to February this year, collated personal experiences of racism from SoR members and received 382 total responses.’

Hang on? 382? But you have over 33,000 members, most of whom work in the NHS? If anything, the survey proved the opposite, that the overwhelming majority of its members had not suffered racism at work. Or more plausibly, put whatever abuse they had suffered down to misdirected yet understandable frustration at long delays, both in getting the appointment itself, and then on the day.  

The link to the survey contained in todays story only took me back to the press release I mentioned earlier. But now that didn’t matter.  The SoR got the story out there. Well done. It wanted to prove that racism was on the increase and the survey did. Bravo. The SoR wanted the blame to be levelled at the increasing support for ‘right-wing’ populist parties in the UK and it did. Hats off. The SoR also wanted to raise its profile, to reassure its members that it was fighting on their behalf, and it did. Pay rises all round.

Just one question though. Exactly how many of your senior staff previously worked for make-up, shampoo or face cream  companies?