Thursday, November 5, 2009

Faulty Poll Leads to Faulty Prediction

When one makes a prediction which falls flat on it's face, it's incumbent on the predictor to own up to his mistake and explain it if he can. I shall endeavour to do so now and tell you how I reached the faulty conclusion that I did concerning my prediction that Richard Bergeron would win the race for the Montreal mayoralty.

As we all know there were many unexpected events that shook both the Harel and Tremblay camps over the course of the election. It seemed that Richard Bergeron, the kooky third place candidate was emerging from also-ran status to a legitimate contender.

He had been running at between 10% and 15% popularity at the beginning of the campaign when things seemed to catch fire and with two weeks to go in the campaign his popularity shot up to 25% with the mayor and Madame Harel neck and neck in the low 30%'s.

Days before the election Angus-Reid published a poll in Montreal's La Presse newspaper indicating that Bergeron had made a stunning breakthrough.
In one short week, he had pushed his numbers to 32% and trailed Louise Harel (34%) by just 2%. Tremblay was behind both candidates at 30% and seemed to be sinking fast.

Based on these polling numbers, it was easy to see that an unstoppable momentum was building for Bergeron and should the trend have continued over the last weekend, it was likey that he'd cruise to victory.

But he didn't, so what happened?

Very simply, one of the world's leading research agencies had completely blown the last poll published in La Presse on the eve of the election.

Every polling organization tells you that their polls are accurate to 4% (the margin of error) 19 out of 20 times

WELCOME TO THE TWENTIETH TIME.

Tremblay
Actual Vote 38%            Angus Reid 30%            Difference 8%

Harel
Actual Vote 32%            Angus Reid 34%            Difference 2%

Bergeron
Actual Vote 25%            Angus Reid 32%           Difference 7%

The errors in the  numbers for Mr. Tremblay and Mr Bergeron are massive and in the polling business represent a humiliating failure.

Perhaps I should have been suspicious of a poll that showed such a monumental shift in opinion over such a short time, but Quebec voters have shown historically that they can jump on a bandwagon quickly. However I take responsibility for making a faulty judgment.

By the way, there's no comment from Angus-Reid or La Presse about the poll. In fact both organizations have purged any reference to the poll from their web sites. It's as if it never existed according to them. The only place I could get confirmation of their numbers were on sites that quoted them as in Branchez-vous. It's as if both organizations are determined to cover up the fiasco.
If you don't believe me, check out both the La Presse site and the Angus Reid site and do you own search.
Poof, the poll has disappeared.

Owning up to mistakes is an important step in reporting. I'm doing so today.

While La Presse rails on and on about the honesty of the Tremblay administration, perhaps they should take their own advice and come clean about the botched poll.

What dishonesty! Shame on La Presse and Angus Reid.

3 comments:

  1. I believe that the difference between those poll numbers and the final results had a great deal to do with two elements you don't mention here:
    Firstly, the smear campaign against Richard Bergeron's person that peaked in the days between the telephone poll and the vote put enough doubt in people’s mind that they were less likely to go out and place their vote. (A smear campaign, I might add, that you perpetuate here in this article by referring to him as "kooky"). A good example of the peaking of this smear campaign is the doctored interview of Mr Bergeron by Ms Jean that was sent to journalist Patrick Lagacé, which, to his credit, he removed once he had seen the original, and much longer clip of the interview that was then posted on Radio-Canada. M Lagacé removed the offending article and clip from his webpage within 3 hours of having posted it, however, the damage had been done and people are still talking about it today without mentioning that it was a doctored tape of 3 minutes of short clips taken out of context spliced down from an interview that was over 20 minutes long. Another example is with regard to the joke Bergeron made about his lung capacity and his marathon running. The joke was turned upside down and sideways into a claim on the part of his detractors and the media that he believed that smoking was good for one's health.
    In these cases, and many others, I would suggest that the real 'kooks' are those who manipulate, pervert the truth and repeat lies for political ends or shock value, especially those who go on to claim that they are unbiased, professional journalists. It is their credibility that should be questioned as much, if not more, than Mr Bergeron's.
    Secondly, the other issue that could well account for the difference between the poll numbers and those of the election result, is the fact that so few people voted. If everyone there had been a greater turnout then we might well have seen higher numbers for Bergeron. However, with such a low turnout, it is important to look at what ‘vote’ is the most motivated to go out and cast a ballot. People who usually vote are more likely to vote, and people are more likely to vote the way they have done so in the past. It is known that changing someone's voting pattern is particularly difficult, they might express a wish to punish those politicians whom they feel betrayed their trust, but it is easier for them to punish them in a telephone poll then it is for them to actually go out and physically vote against them. We are creatures of habit, and voting habits are hard to break! Just look at what happened to the Liberals in Outremont after Adscam: Liberal voters stayed home on Election Day, it was traditional anti-Liberal and/or progressive voters (i.e. those who would have likely voted Bloc in the past), who, seeing the opportunity to beat the Liberals in their stronghold by supporting the NDP, switched their votes.
    There are places like the Plateau, where it is proven that people are willing to break with old habits in order to vote based on their conscience and values. Communities such as those voted overwhelmingly for Projet Montreal.

    ReplyDelete
  2. Article about the poll is still on the site : http://www.cyberpresse.ca/actualites/elections-municipales/200910/18/01-912511-harel-et-tremblay-a-egalite.php

    ReplyDelete
  3. Your suggestion of a coverup is ludicrous. La Presse's story on the poll is still online and isn't going anywhere.

    Angus Reid never had the poll on its website. Polling companies don't publish polls they do for news organizations on their websites, because part of the deal is that the news outlets get the exclusive.

    As for your conclusion that the poll itself was faulty, I don't think that conclusion can be so easily reached. Bergeron's popularity soared during the campaign, which meant almost by definition that much of that new support was soft. It's very likely that many voters said they supported Bergeron but changed their minds at the last minute. Or they might have switched because of some of the negative news Bergeron got in his last week.

    Or it could be that the poll included people among Bergeron supporters who just didn't bother voting.

    There are plenty of reasons a pre-election opinion poll could differ from the results of an election. It happens all the time. This is why there are exit polls, and why those exit polls are much more accurate.

    The polls are designed to gauge vote intentions at the time they are done. They are not meant to predict the winner of an election. The fact that the results differ from the poll numbers does not mean that the poll was faulty.

    ReplyDelete