Soon after the General Election last year, there was much soul searching in the media as to why the polls had failed to guess the outcome of the general election correctly. Back then we pointed out that we believed that the major problem with the opinion polls was the sampling and quota control.
It would appear from the report published on 19th January by Prof Patrick Sturgis of Southampton University that – we were right!
The strong reliance by the Pollsters on new technologies for market research means a huge emphasis on using on-line panels. These are panels which individuals can sign up to and get paid for completing a range of surveys. Back in May we suggested that these panels cannot be relied on to provide a proper representative sample because there is no real check as to how people describe themselves – or even whether they have more than one identity in order to get paid more money. And respondents select themselves based on their interest and the incentive payment offered rather than being ‘screened’ by an independent researcher.
It seems to be something of a surprise to the Pollsters that the large core of the population who are over 70, who are most likely to vote, don’t answer the phone to unrecognised numbers and don’t actively participate in on-line panels. The Pollsters answer to this conundrum was to ignore them, and then express surprise that they didn’t act like the (younger) rest of the population and vote Conservative – who would have thought it?
The answer, of course, is to make sure that this group of (easily identified) people are actually interviewed rather than assuming that they will vote the same way that a 25 yr old panel member will vote
Perhaps the most appalling admission made by the Pollsters was that they had not set proper quota for their surveys, and even worse had not controlled the quota. Failure to do that meant that they really were guessing at the results, rather than using established and proven market research methods to do the job properly.
Market research works, it gives you the answers you need to plan what you want to do. It helps organisations thrive and prosper. It lets you know what your customers want – and means you can then give them what they want.
But this only works if the research is done properly. It seems that finding a cheap method or data collection (on-line panels) has produced nasty results.
(The British Polling Council, supported by the Market Research Society, is therefore setting up an independent inquiry to look into the possible causes of this apparent bias, and to make recommendations for future polling.)