It’s More Than Math
“These polls can’t be right, no pollster has ever called me,” a comment I heard yesterday from a friend who immediately wanted to discount the results of the latest political poll. Independent of my friend’s misguided sentiment, we do, in fact, have a poll problem; or at least, there are reasonable questions about how pollsters conduct polling, present their work, and how the results are reported and/or manipulated by the media and the impacted political parties. Today if you turn on the tv, you will be bombarded with the latest presidential polling numbers, and whether we like it or not, the polls are influencing how we think and what we do; given the impact of polls, we should pay attention to how they generate their numbers.
Unpacking how polling firms determine results is interesting for those who enjoy math and statistics, probably not for the average person; but there is more here than meets the eye. For a polling firm, their work begins with identifying what answers they are pursuing, and defining the specific questions intended to illicit a view; they also must identify what group they are seeking to interview, and what is the optimal size of the random sample of the population they are targeting; to be sure, the sample size of the population they ultimately engage with will be tiny relative to the overall population of interest. For many, this is confounding, and evidence they can’t be trusted. I think that conclusion is wrong, the size of the sample is rarely the source of polling problems. Statistical analysis is such that it is easy to draw a high level of confidence from a well-crafted sample that allows for projections that apply to the larger population. However, as is generally the case, the devil is in the details. Having the optimal number of respondents only works if the sample participants reflect the larger group; this is typically based on demographics. You also have the bias inherent in how they contact poll respondents (not many people have landlines in their home, and if they do are they a good cross-section of the population?) Further, if you are trying to understand the breakdown of expected voters in a state that is known to be purple politically, and you end up with a sample group that identifies as 75% democrat, then your numbers regarding a state-wide election will not capture the truth.
Identifying the appropriate sample size and the relative breakdown of reflective demographics is the easy part and they have a fallback that provides additional comfort; every poll has an identified margin of error; essentially, they are saying we think our conclusion is right plus or minus this stated number.
I rarely question the quality of polls sample size and their demographic allocation within the sample; it may be silly to say, but I trust they are doing the work. What I wonder about is the questions they use and the corresponding conclusions that are drawn from the data. Here’s an example, I have seen many polls that indicate the economy is the most important consideration for people when they vote, and the same polls site Trump as the favored candidate for the economy; I have no reason to believe those results are inaccurate. The question is, can you assume that because the majority of people polled believe the economy is the number one issue, and they also believe Trump is best for the economy that Trump would naturally get the majority of those polled to vote for him? I think not. Consider this, some people are precise at answering the specific question asked of them and not assigning a greater meaning to the question than what was asked. It is entirely possible that people might believe the economy is critical and Trump is the best for the economy, but also harbor a view of Trump that would prevent them from ever voting for him.
The point isn’t whether my specific example is valid, the point is that narrow questions which are best for collecting data, may miss what a person might do. I know plenty of republicans who will admit to the importance of the economy and would even go so far as to say Trump would be better for their pocketbook but would never vote for him.
Another misguided, and often conducted poll that is frequently cited, focuses exclusively on the choice for president. Far too often, someone will conduct a national poll suggesting who is favored to win; the problem is a national view is irrelevant. The president is elected by electoral college votes, not the popular vote; democrats tend to win the popular vote and they will likely win again in November; a fact that won’t ensure they win the election. That doesn’t stop them from celebrating the poll that looks good for their side, the party bends themselves into a pretzel to overstate the meaning of the poll.
The point is, polls have power; admittedly, they don’t decide elections, but they influence opinion and create momentum. I want to trust polls and value their role, but I am not sure I understand how big their influence has become. Take for instance the arcane primary process; I have complained about this before, surprisingly the powers that be haven’t changed the process despite my deep concerns (I guess they are not subscribers). My problem with polls and their influence on the primary system is the ability of aggressive marketers, for a campaign, to boost a pre-ordained leading candidate to the point that they effectively eliminate a viable competitor; you see this occur every four years in Iowa and New Hampshire. Proclaiming a winner, on the backs of a few states and polls effectively prevents most Americans from having the opportunity to vote.
After they impact the primary season, what measurable value do polls bring to the process. Do they assist voters in their evaluation of candidates? Do they allow politicians to change their message to track the polls (independent of what they really believe); maybe they make elections more transparent? Let me guess, polls allow the media to have something to talk about and support their business model. Nothing wrong with a company making money, but I wish they were more transparent: polls are so prominent on news outlets because it gives them something to debate.
I’m not trying to be harsh about the use of polls and the pollsters who produce the results; they tend to be right some of the time. They also get it wrong too often for us to rely on them to conclude what will happen. It reminds me of projecting the financial future for a business; the math necessary to create projections isn’t a tall task and is easily prepared. The art of creating forward looking financials is in the assumptions used; if you are off in your basic assumptions, no matter how good your math is, the output will be wrong.
I am not a conspiracy theorist, in fact I believe virtually all the deep state, looking around corners, gnashing of teeth histrionics surrounding the conspiracy of the month, is ludicrous. Therefore, I don’t subscribe to the notion that the polling environment is fraught with some deep, dark, underbelly of malfeasance. What I do believe is polling is way more imperfect than is recognized, we pay too much attention to polls, and we are too impacted by what they say; polls are supposed to be snapshots of a moment in time, they are billed as being statistically sound and based on good math, and they likely are; but polls are about more than just math.
