Why I really don’t believe polls much

Recently, I got into a brief exchange with a public liberal, who in quite short order, verbally handed my butt to me with very little effort on his part.

I withdrew from the exchange with my tail between my legs not wishing to embarrass myself (I really hate that) with the short text message limits of twitter.  What I found was that you can publicly ridicule someone quite effectively with short text messages, but that mounting any kind of meaningful defense is not quite as simple.

The exchange, although somewhat bruising to my ego did help me to become more convinced of something I’ve long felt, but never really wanted to completely believe….that for the most part polling of public opinion really does not mean that much to me.  By that I mean I don’t believe them a very accurate indicator of public opinion.  Oh don’t get me wrong, I think polls are extremely useful tools when asking questions like do you plan on voting for candidate A or candidate B in forming conclusions about the effectiveness of a campaign.  No one was more thrilled to see Brown polling surge in the days before the election.  But in matters of issues and public opinion, I don’t put much stock in  them.

I don’t claim to have any background in public polling, diving down into cross-tabs, the polling methods, sampling, deviations and all of the other neat little terms mathematicians use.  However I do have a fairly decent exposure to statistics…and statistics used for things like quality control in manufacturing.  When testing the quality of something like say widgets, it’s fairly straightforward.  The tester can set metrics of weight, diameter, thickness and decide allowable tolerances of those metrics to be deemed acceptable.  Failing any single metric is cause to reject a widget using a simple pass/fail criteria.  Inferences are then based on how many widgets pass or fail for a sufficiently large sample size about the quality of the manufacturing method.  

The beauty of such methods is we really don’t have to be concerned about the widget’s feelings, which church it attends, which news channel it watches most often or if the widget had a particularity bad day.  The widget either passes or fails based on specific measure.  And it really doesn’t matter whether the widget is made in the Vermont or Texas plant…all things being equal widgets made in two different locations using the same process and material should exhibit the same pass fail rates over any time period.

This is not such a simple matter when measuring public opinion and beliefs.  Two different polls trying to measure the same exact issue can give different results based on any numbers of factors.  Political people naturally use polls that support their position as justification for that position and question polls that are counter to their position, or require them to further define/explain their position.  

Opinion is not something truly measurable, it is fluid, it changes over time, it is extremely susceptible to outside influence and previous history.  Opinion often also has a strength quality.  One can believe something so strongly that they would not change their opinion with a gun to their head, against all evidence to the contrary, until the last breath.  One can believe something most of the the time, except under specific condition, have no strong feeling either way, disagree most of the time except under certain conditions, or disbelief something to their grave even in the face of evidence to the contrary.  Opinions have degrees and can change with experience.

and the wording of a question can have an enormous effect on the outcome.  The was beautifully illustrated in a recent opinion poll regarding dont’ ask don’t tell in the military.


When the question is framed as supporting homosexuals in the military  44% of sample expresses support.  However when the term “Gay Men and Lesbian Woman” is used in place of the word homosexual 58% express support.  Huh?

That’s right, a sizable difference in the results is shown by changing the question in a very subtle way.  How is this possible and what does it mean?  I suppose one could reasonable infer that somehow the word homosexual has a more negative connotation than the phrase “Gay men and Lesbian Women”.  Someone else could just easily reach a different conclusion that the inclusion of the words men and woman with the adjectives gay and lesbian makes the terminology more human than the term homosexual, and that people are more accepting of people than some descriptive word about (presumably) people.

There are other examples of this phenomenon is virtually every area of political and public discussion…whether the issue is abortion, death penalty, climate change…you name it, the list could go one ad naseum.

So, why have I had this discussion with myself over a twitter discussion.  Well the original question arose over how much credence mainstream GOPers gave the birther movement.  My original stated opinion was (and still is) that what I consider mainstream political beliefs of the GOP do not take the birthers seriously.  The other participant pointed to a recent Research 2000 poll that included the following question in a sample of self identified republicans.

Do you believe Barack Obama was born in the United States, or not?

Yes 42

No 36

Not Sure 22

Wow only 42% of self identified republicans are  certain the president was born in the United States…how is that possible?  Does mean 58% of the GOP can be labeled birthers?  And does a belief by an apparent majority make that a mainstream GOP belief? Well, I’m not so sure.

At the outset I had several questions about what constitutes mainstream GOP beliefs and how  representative a sample of self-identified republicans is of mainstream GOP beliefs.  Does simply saying “I am a republican” during a phone interview mean I am in the mainstream?  Does it mean I am registered in the GOP and typically vote GOP? Does it mean I’m unenrolled and just really pissed at Democrats? Does it mean I’m a devoted Democrat and want to make the GOP look silly?  Does it mean that I live in the hills of Kentucky and spend my weekends attending drills with the local militia in the woods preparing for the end of the world?

And what puzzled me even more in that question is the segment of non-yes responses?  Does “I don’t know” mean I have serious doubts about where he was born?  Does it mean I don’t really know and don’t care?  Does it mean that I know he was born in Hawaii, but I really don’t know when Hawaii became a state so I say ‘I don’t know’?  Or does it mean that I know he was born in Hawaii and this is a test because I’m not sure when Hawaii became a state (so it must be a trick question) so I’ll say ‘NO’…because I want to get the answer right 🙂  Or does it mean the respondent is firmly convinced the president was born in Kenya (or elsewhere)?

For me, when I think of mainstream beliefs I think of registered, politically aware people, who at minimum vote, watch the news, and look for answers when they have a question.  These are the people you see holding signs, belonging to RTC’s, raising money, running for local offices and participating in forums like this.  Maybe that’s not what some would consider mainstream, but rather politically active.  I don’t know.

All I know is that one simple question with simple answers raised more questions for me than I could write in this wordy exercise.  And I have still have serious skepticism about any conclusion one could reach about birthers being in the GOP mainstream.

I really love simple answers…they are the best…they present no ambiguity, and rarely prompt another question.  If this poll is taken at face value and it’s natural inferences are assumed to be true, we have some serious issues that need to be addressed.  At minimum it gives opponents one too many extra free-throws.  It has been my observation that when liberals engage in discussion and a difference of opinion arises, one or more participants may express a claim they know know how most/many people feel.  That assertion is immediately challenged and various polls are cited to either support or refute the claim.  Unflattering polls like this can easily be used and there is little one can do dispute the claim except offer an opinion or anecdotal evidence….which will meet with ridicule and condescension.


About garyrlake