RIGHT OF REPLY: Let the facts, and our performance, show that our predictions are correct
Ipsos responds to Jonathan Moakes’ article published in Daily Maverick, 10 July, regarding Ipsos South Africa’s polling history.
Ipsos would like to clarify a number of incorrect statements and flawed commentary produced by Jonathan Moakes (JM) in his article: “Pollsters, the DA has a surprise in store for you”, regarding Ipsos South Africa’s polling history and relates to the objectivity, reliability and reputation of Ipsos and our political research, which we have been conducting for over 40 years in South Africa.
Firstly, the way in which data are analysed will be discussed, followed by how polling research is typically conducted, a chronological timeline of Ipsos election predictions and, finally, the methodology employed in the 2018 research cited.
How the data are analysed:
JM: “At the end of the day, the only poll that matters is the one that takes place on Election Day. Elections are about the voters. Not political parties. Not the media. And not political pollsters. The voters will decide who has the most compelling offer and vision for the country. They will decide who will best stop corruption, create fairer access to jobs, protect our women and children and secure our borders.”
This is an accurate representation of politics in a democratic society and polling is used as a means to understand the sentiment in the country. Elections are about the voters and the only way to gauge that sentiment is through research.
Social and Opinion Research is about analysing the data in an unbiased manner, regardless of one’s personal views. Objectivity is vital to understanding the data. We neither fabricate data nor place sentiment on it, we merely convey the voice of the people even if this does not match the results we had imagined. The overall message of Moakes’ article points to the fact that political polling can never entirely predict what will happen on the actual day of the elections, this is true of any social research, as human beings are not static and thoughts and emotions fluctuate. We are dedicated social scientists employing research methods which have been validated over years of both academic and corporate research.
Analysing polling data
Ipsos uses advanced modelling to arrive at the preferences of voters, particularly those who are undecided. This modelling calculates how voters may potentially vote and how this would influence the outcome of an election. South Africans who are registered to vote are asked a range of questions which includes which political party they would vote for should there be an election. This is asked for both national and provincial preferences. The results of various questions concerning politics and those of the ballot paper were combined to arrive at a possible prediction of the election results. All sample surveys are subject to a certain margin of error, as opinion polling is not an exact science.
Polls will always face criticism, with analysts questioning their accuracy (particularly around sampling and margin of error). Ipsos was criticised in the 2016 elections, with many saying that the DA’s strength was being overstated, while the ANC’s support base was being underestimated. In 2016, it was mostly ANC officials who took issue with these numbers and research methodology. However, ultimately these critics were proved wrong, with the final vote tally remarkably close to the predictions in the metros (and well within the margin of error), and on the head nationally. Unfortunately, these results have the potential to upset political parties when they show that support has waned for whatever reason.
Case in point is that this author was not one of our critics in 2016 when it was believed that we were overstating the DA support leading up to the election.
Great news from latest polls from Ipsos. DA ahead in both Tshwane and NMB. And it's neck-and-neck in Joburg! #VoteForChange— Jonathan Moakes (@Jonathan_Moakes) June 16, 2016
History of Ipsos election results
JM: “It is essential to interpret Ipsos South Africa’s political polling in the context of their record, which has not been accurate. Their work is almost consistently wrong and their accuracy appears unrelated to how long in advance of the election their poll is taken.”
The below timeline illustrates the accuracy of our political predictions.
2009: National Election
JM: “They’re also often no good on the eve of an election. In their press release one day before the 2009 general election on 21 April 2009, (“2009 National Election: Possible results”), Ipsos predicted an average 13% support for the DA, with results ranging between 11.34% and 14.66%. The DA received 16.66% of the vote.”
Although the accusation that we did not correctly predict the results for the 2009 election for neither the DA nor Cope is correct – this does not mean that we were far off the mark, with all results within 4 percentage points of the actual outcome.
2010: Country snapshot
JM: “In a 27 July 2010 press release, Ipsos published the results of a poll conducted in May 2010, approximately a year out from the 2011 local government election. In this release, they predicted 13% support for the DA nationally. One year later, the DA received 23.94% of the national vote, almost double what Ipsos predicted.”
As mentioned by Moakes himself, trends are an important consideration when examining polling. Using results one year before the 2011 election and right after the 2009 election as a measure of accurate polling is not good research practice – rather this data provides the trends in social opinion. Optimism about the ruling party straight after an election has become a trend. The press release cited by Moakes indicated that support for the ANC declined from 70.5% in December 2009 to 66% in May 2010, and that support for the DA increased from 12% (December 2009) to 13% (May 2010), and Cope support also increased from 2.2% (December 2009) to 3.6% (May 2010). The trends uncovered from the research were, therefore, in line with expected political attitudes and trends of euphoria after an election. Perceptions about the ruling party become more negative as time goes on.
JM: “It is also important to acknowledge that a poll, no matter how accurate it may be, can only be a snapshot in time. Trends are what count in the lead-up to elections and, as of yet, there is no credible public political polling that can show trends in respect of political parties ahead of the general election in 2019.”
Ipsos absolutely agree with this statement. Social research is a snapshot in time and many things can change the outcome on the day, such as voter turnout (influenced by many unexpected things, such as the weather) and undecideds swinging the results. However, if you ask the same questions leading up to the elections, trends start to emerge – which facilitate the forecasting - and are often reliable indicators of party support leading up to the elections, even if these results are not right on the mark.
2014 – National election
Although Moakes did not refer to the 2014 results in his article, we have included them as an indication of our prediction success over time. Ipsos was right on the mark with these results. Not only do the results fall well within the margin of error, but also our predictions were within 1.5 percentage points throughout.
2016: Local government election
JM: “Their performance in the run-up to an election is no better. This can be seen in the weekly polling conducted for eNCA in major metros in the lead-up to the 2016 local government elections. For example, they had the ANC on 26% two weeks before election day in Tshwane (their Week 8 results). The ANC ended up getting 41.25%. They were equally inaccurate in Johannesburg and Nelson Mandela Bay.”
While these results are not the final predictions, they do illustrate trends in the political landscape and the way that opinions change over time. Violence erupted in the City of Tshwane shortly before the 2016 elections following the announcement of Thoko Didiza as the mayoral candidate for the city. Tensions cleared closer to the election and the ANC recovered, by receiving 41.25% of the votes.
The final predictions by Ipsos were again right on the mark. The ANC results for the City of Tshwane resulted in a 5.75 percentage point difference, but this still ran very close to the margin of error.
2018 – Country snapshot
JM: “It’s not clear exactly when the fieldwork was conducted but it appears to have been done in May. These results have subsequently been cited by many media publications without questioning methodology or even interrogating past performance.”
Ipsos releases the technical detail with every press release and we are completely transparent about how and when the study was conducted. Please see our website: www.ipsos.com/en-za
Fieldwork for this study was conducted on 22 May 2018. GeoPoll provided Ipsos with a fast, high quality means of data collection through mobile phones. Ipsos designed the questionnaire and was involved in every step of the research process and data weighting.
Respondents were randomly chosen from a panel and were representative of the South African voters in terms of gender, age and race. The results of the 750 voters were weighted within this sample to correct for small discrepancies.
In conclusion, Ipsos welcomes any discussion about its record as a political pollster in South Africa, and any other jurisdiction where we do polls. Our record in South Africa as an accurate predictor of political outcomes based on our polling is very strong. We hope that our response above has set the record straight, and readers will be able to judge our performance for themselves based on the facts.
Update 17 July – Release of the eNCA polls
It is worth noting that the latest release from the Ipsos Pulse of the People details results from our “regular” study and methodology (detail below) and has yielded identical results to those indicated by the study cited by Moakes above.
Fieldwork for this study was conducted from 20 April to 7 June 2018. A total of 3,738 South Africans, 15 years and older, were interviewed. They were randomly selected and interviewed face-to-face in their homes and home languages. Interviews were conducted all over the country, from metropolitan areas to deep rural areas. This methodology ensured that the results are representative of the views of the universe and that findings can be weighted and projected to the universe – ie South Africans 15 years and older.
Trained quantitative fieldworkers from all population groups were responsible for the interviewing and CAPI (Computer-Assisted Personal Interviewing) was used. All results were collated and analysed in an aggregate format to protect the identity and confidentiality of respondents.
All sample surveys are subject to a margin of error, determined by sample size, sampling methodology and response rate. The sample error for this sample at a 95% confidence level is a maximum of 1.60%.
In conclusion, Ipsos welcomes any discussion about its record as a political pollster in South Africa, and any other jurisdiction where we do polls. Our record in South Africa as an accurate predictor of political outcomes based on our polling is very strong. We hope that our response above has set the record straight, and readers will be able to judge our performance for themselves based on the facts. DM