Can telephone surveys deliver demographic quota?
The challenge set by a large local authority was to carry out a survey of its residents. It had to deliver a truly representative sample of residents. We needed to find out residents’ views on where they lived and how it could be improved. The key aspect was to make sure that there was an equal representation from every Ward. The original specification was to carry out the survey using telephone interviewing.
A reliable set of data was needed for each Ward. This meant that we needed to interview 100 residents from each Ward so that data could be analysed at this level. This sample size of 100 gave a statistical confidence interval of ±10%. It was then possible to compare the views and opinions from these different communities. It means the analysis can then be used by the Council to develop plans for different neighbourhoods. In addition we set an Authority wide quota for age and gender.
Our recommendation, accepted by the Council, was to do the interviewing face to face. An option considered originally was to carry out the interviews by phone. We also highlighted some key concerns we have about telephone interviewing.
The challenges of phone interviewing of the general public have increased over the past few years. The increasing preference for use of mobile phones has led to a corresponding decrease in the use of fixed land line phones.
It means that there has been a sharp decline in telephone numbers which can be identified as belonging to a clear geographical location. This is a key requirement when the Ward in which respondents live is such an important question.
Increasingly, landlines are becoming the preserve of older people and there is well documented evidence that older people use the caller display facility to screen incoming calls, refusing to answer calls from numbers they do not recognise.
Younger people, who tend to rely solely on mobile phones, cannot be identified via phone directories as living within a specific geographic location. And the number of people who can be relied upon to know which Ward they live in is limited.
Because of this there is a clear danger of some sections of the community being excluded from the total sample. From experience we know that this can include older and younger people. It can also exclude ‘hard to reach’ groups.
This problem has been highlighted recently by the errors in public and opinion polls. This has been widely reported in the media. Polls around the last two general elections and the Referendum have emphasised the difficulty of properly identifying respondents over the phone.
So, we took a three stage approach to the project.
The first stage was to interview in the main town centre High Streets. This is where where the footfall was high and which draws in residents from a wide area. We collected full Postcodes to verify participants Wards and working from tablets we were able to closely monitor the progress of the interviewing on a Ward by Ward basis.
The second stage was to interview in neighbourhood shopping centres. So we were able to target those Wards were we could see a shortfall in the numbers needed.
We produced a reliable dataset which can be robustly analysed down to Ward level so it will allow the Local Authority to confidently draw up realistic plans for the future of all their communities. It can also be used to explain to local neighbourhoods how their Council has arrived at their decision by showing that they are based on the current views of local residents.