Even though pollsters remind us that their polls are accurate to about 3 percentage points, 19 out of 20 times, we tend to view a poll printed in the newspaper as automatically being a true representation of what people are thinking.
But polling is inexact and even asking the simplest of questions is complicated, as I learned from experience.
Let's say you want to find out whether people are going to vote for Candidate 'A' or Candidate 'B.' It seems that it would be as simple as phoning people within the voting district and asking them for whom they'll vote.
Simple right? Not really.
There's no use in asking someone about their preference, if their intention is not to vote. So you've got to qualify voters before. What about all those who refuse to answer? Do those people have a decided preference?
What about people who don't have a home phone and are impossible to contact? They tend to be younger and more mobile and would likely vote strongly for one candidate over the other, but their voices are not counted.
What about the cross section of the sample? (those polled.) Did the pollsters get an accurate cross section that accurately reflects the district?
Modern pollsters face these problems and work to to make sure their polls are as accurate as possible, but sometimes they fail miserably.
Even the largest of polling firms can fall victim to these problems which can result in a flawed pool. Here's an example of a much ballyhooed poll that has problems on every level. So many problems that the whole thing should never have been published. It appeared in the Journal de Montreal last week and was conducted by Leger Marketing which actually used a very large sample, supposedly making it even more accurate.
First problem is the question;
"Do you wish Quebec to become a sovereign state within ten years"If someone wants to see Quebec sovereign within ten years, does it mean that he or she may not want Quebec to become sovereign in the eleventh year? They should have put a period after the word 'state' and chuck the rest. Reminds me of the famously flawed referendum questions.
The poll purports to show the opinion of francophones towards a sovereign Quebec and separately the opinion of the entire population, including anglophones and allophones.
Too bad it doesn't have a separate bar to show how the allophones and anglophones polled as a group, it would have been interesting.
Here's the problem.
Francophones make up between 80%-82% of the Quebec population. For argument's sake, let's take the lower number, it makes the math easy. (The higher number makes the poll's accuracy even worse)
If one believes the poll and applies the 52% figure to that 80% group of Quebec Francophones, (80 x 0.52) the answer comes out to just under 42% of the total sample and this before any of the allophone and anglophone numbers in favour of sovereignty are added in. (if there are any.)
In fact, for the poll to make sense, 115% of the allophones and anglophones would have to have answered negatively! Hmm.
We all know that the anglophone/allophone group is massively against sovereignty, but 115%?
What likely happened is that the pollsters over sampled the anglo/allo side, a fatal mistake that can be easily corrected through 'weighting,' a process whereby the sample of anglos is reduced down to a level that represents their true demographic weight.
The sample should have included anglos and allos in the 20% proportion that they represent and not the actual poll sample which works out to about 25%.
It's the only answer that makes sense.
It's an amateur's mistake and I'm a bit surprised at the flaw (but not overly)
As far as I know, not one member of the the media pointed out this fatal flaw in the poll.
These errors are a lot more prevalent than you think. Here's a funny one from FOX News;
This above error is comical, the numbers add up to 110%.
But how the numbers got to where they are is a concern. It looks like somebody massaged the numbers and then made an adding mistake. Not so funny.
By the way, I haven't touched on how certain polling questions are so leading that it's obviously going to affect the response.
51% of the polled group agree with those who object to Muslims building a mosque near the World Trade Center, yet when the question of rights is phrased differently, as in the second question, the answer don't coincide!
As for polls, I view them all cynically. A dishonest pollster can easily manipulate data or ask a skewed question.
Then there's the methodology problems as illustrated above.
My least favourite type is those voluntary Internet polls that are completely unscientific.I tend to agree with the late Prime Minister Diefenbaker who commented rather famously that "Polls are for Dogs!"
