Since announcing his candidacy for the Republican nomination, Donald Trump has defied many predictions and held a commanding lead in the polls, both nationally and in the crucial states of Iowa and New Hampshire.
One seemingly odd exception was our poll, conducted in collaboration with WHO-HD, which surveyed likely Iowa caucusgoers between Nov. 2-15. In our poll, Trump polled only 15 percent — behind Ben Carson (27 percent), Marco Rubio (17 percent), and even “undecided” (16 percent).
That 15-percent figure differs from the results of other polls. In our poll, Trump registers 10-15 points lower than in the three polls of Iowa Republicans who were in the field around the same time. Our poll recorded the lowest level of support for Trump in Iowa since July.
So what happened? Was our poll just wrong? Or did we do something different than other pollsters that led to such a different result?
People have already noticed that Trump does better in Internet and automated phone polls than live person-to-person phone polls. But that’s not the explanation here. Similar live-interviewer phone polls conducted just before our poll was in the field (CNN) and just after (Morning Consult) gave Trump 25 percent support, still 10 points higher than our number.
We believe that the explanation has to do with our poll’s sample. Our sampling technique was different from all other polls of Iowans in this cycle, and it may provide an important insight into the Iowa caucus and other state primary elections.
To build a sample, we began with the list of registered voters in Iowa and stratified our sample by factors like age and sex. (More details are here.) Other pollsters have done something similar.
However, what we did — but other have not — is stratify the sample based on another factor: Whether or not people had voted in at least one primary election since 2006.
This is what we think makes our poll different. We assume that voters who have voted in primaries are more likely to attend the caucuses. Of the 486 Republican voters in our poll, 78 percent had previously participated in the primaries, compared with only 48 percent of all registered voters.
Why didn’t we use previous caucus participation, since we are ultimately trying to figure out what will happen in the Feb. 1 caucus? The answer is simple: Iowa doesn’t record that information, and it isn’t part of information in the list of registered voters. Instead, we assume that voters who have voted in primaries are more likely to attend the caucuses.
One theory about Trump’s success is that he is energizing people who have traditionally avoided politics and political participation. If he succeeds, then our sample will understate his support. But if he doesn’t, and participation in the February caucus does depend on whether voters have voted in earlier primaries, then other polls to date have likely overstated Trump’s support.
Here is some supporting evidence for this story. Although our sample contains a disproportionate number of past primary voters, it does include some who have not voted in a primary since 2006. There is a significant difference between these groups in their support for Trump:
Among respondents who have not participated in a primary, 21 percent support Trump, compared with only 13 percent of those who have voted in at least one primary.
Why doesn’t this pattern show up in other polls? As in most other polls, our poll asked respondents how likely they were to attend the caucus. Based on this question, Trump supporters are not less likely to say they will attend the caucus:
Similarly, Trump does equally well among those who say they have voted in the caucus before (16 percent) and those who say they haven’t (15 percent).
So which voters are more likely to say they will attend? Those who have participated in the primaries in the past:
Past behavior is a strong predictor of expected future behavior. This is what motivated our sampling methodology in the first place.
Put simply, Trump’s support in Iowa depends on voters who say they will go to the caucus, but have less of a track record of voting in primary elections over the past 10 years.
So, is our poll more accurate than other polls? We do not know, and that is not the point we are trying to make. Poll results can depend on the assumptions that pollsters make and the questions they seek to answer. Many current polls measure the preferences of people who say that they are likely to caucus in February. We chose to refine our sample further and use past behavior to predict who is more likely to show up to caucus.
We have no way to determine which approach is more accurate. The caucuses are still two months away. Between now and then, opinions could very likely change, making all current polls less relevant and less accurate.
But our poll raises a valuable question nonetheless: how predictive are the current polls, most of which rely only on respondents to report whether they are likely to vote in a caucus or primary?
If our assumption is right — that is, past primary voters in Iowa are more likely to vote in the February caucus — then Trump is in worse shape than many believe. His supporters will be less likely to show up at the caucus.
If, however, Trump is able to get these voters to the caucus, then our assumption will be wrong.
Again, we are not arguing that our poll is more accurate or predictive than other polls. But we hope that our poll’s very different result, which was generated by a relatively minor change in our sampling procedure, informs the conversation about this year’s polls and what we can learn from Trump’s present dominance.
The status of “outsider” may help a candidate like Trump win support in the polls, but it may not be enough to translate that support into actual votes.
