( – promoted by Rob “EaBo Clipper” Eno)
O'Leary won't disclose, Perry's momentum killer poll
In the race to succeed William Delahunt in the Massachusetts 10th congressional district three of the four major candidates have released polling data of questionable validity and none have complied with minimal disclosure standards established by the polling industry to verify the legitimacy of the data.
So could these polls be more press release than valid survey research project?
If so, the victims of that unethical behavior are the very voters the candidates wish to court.
Why do media outlets fail to request the most basic information to support statistical validity? Multiple resources exist for media outlets, pollsters and candidates including a disclosure checklist developed by American Association of Public Opinion Research (AAPOR).
The latest polling data out last week from State Senator Robert O’Leary suggests he is six points in front of Norfolk County District Attorney William Keating, 44%-38%, in the September Democratic Primary. However O’Leary’s pollster, Gerstein | Agne Strategic Communications, did not return calls last week seeking minimal disclosure details as established by the AAPOR. Rebeca Davis, O’Leary’s campaign spokesperson, did state that the campaign would forward a “polling memo” with further details, but never did, even after an additional call to confirm our contact information. Keating has not released any internal polling numbers.
“If reporters or readers are in doubt about whether a poll is genuine, they can tell a lot from the pollster's willingness and ability to disclose this level of detail. Conversely, if a pollster is not willing to disclose information on such items as the sample frame, the composition of the sample, the way they defined likely voters, the text of screening questions or those preceding the questions of interest, reporters and readers should be highly skeptical,” writes from Mark Blumenthal, editor of pollster.com, in his post: Deciding Whether a Leaked Poll Can be Trusted.
Last fall, AAPOR launched, The Transparency Initiative, a “program designed to encourage routine disclosure of methodological information from polls and surveys whose findings are released to the public.” AAPOR has established minimal disclosure standards to guide professionals, the public and media outlets, “Good professional practice imposes the obligation upon all public opinion researchers to include, in any report of research results, or to make available when that report is released, certain essential information about how the research was conducted.”
Consequently, O’Leary's data is in the form of a press release only, with no creditability and no effort at transparency.
On the Republican side, State Representative Perry issued a poll on June 25th indicating he was 16 points ahead of former State Treasurer Joe Malone and included a remarkable assertion that Perry, who had just undergone several weeks of bruising press coverage, had a heavenly 44% favorable to 1% unfavorable rating amongst likely Republican primary voters. Motherhood and apple pie are known to have higher unfavorable ratings.
Full disclosure: I am a supporter of Jeff Perry in this campaign, however I have never meet Mr. Perry and I have not contributed to his campaign. In the 1990’s while active in Cape Cod politics, I knew Robert O’Leary and supported him in his races for County Commissioner. I have no relationship past or present with Mr. Malone or Mr. Keating.
Unlike O'Leary, Perry did release an edited seven page “interview schedule” after a request by jackgately.com which places into serious question the validity of Perry's polling methodology and question architecture. For instance, the sample frame consisted of only registered voters not the random digit dialing method widely considered more accurate and representative of the potential voter pool. The original public statement from Perry was a press release very similar to the technique used by O'Leary.
Several questions appear to drive and suggest a particular outcome designed to solicit stronger conservative responses later in the survey, the technique is know as “a context effect”. For instance after identifying as Republican the respondent is asked: “Would you call yourself a STRONG Republican or a NOT-SO-STRONG Republican”. No emphasis added.
The strength of the respondents self identification as a Republican holds no relevance to the outcome, unless of course a researcher is seeking hardcore Republican voters. In Massachusetts, unenrolled voters are eligible to vote in party primaries. Perry is considered a pure conservative and Malone a moderate Republican.
The favorable/unfavorable question is highly suspect: “For each one, please tell me, first whether you’ve heard of the person; then, if so, please tell me whether you have a favorable or unfavorable…” The “first whether you’ve heard of the person” directive allows the interviewer the opportunity to subtly suggest or remind the respondent who the candidate is, for instance, “Mitt Romney, the former Governor of Massachusetts” sounds innocent, but could dramatically drive down his “no opinion/never heard of” numbers. Rather, the respondent should self-identify as having no opinion with no prompting or suggestion.
The poll was conducted by Opinion Strategies, a well know Republican polling firm.
The practice of releasing a poll showing your candidate with a substantial lead, especially while conventional opinion would indicate that your opponent is gaining is called: a momentum killer poll. Specifically designed to thwart the progress an opponent may be making, especially amongst opinion leaders and prospective donors. In the weeks prior to the release of Perry’s momentum killer poll Malone and media outlets had been making a significant issue of an incident that occurred during Perry’s tenure as a police officer.
Perry’s self released numbers were tweeted out at 7:38AM by David S. Bernstein promoting his 6:56AM Boston Phoenix story adding an appearance of credibility. By 10:29AM Bernstein asked Perry for more details on the poll, something that should have been done before posting a story: “Some of my Tweeps are having trouble believing your 44/1 fav — Care to provide more deets on the poll” Four hours later Perry, again by tweet, directed Bernstein to contact the pollster directly, but by that time the “news” filled the #ma10 and #mapoli twitter streams, the press release was a success.
Earlier this year, Malone released a poll suggesting he would be the strongest GOP contender against retiring incumbent Delahunt. In February, Malone declined to release any supporting documentation for his poll.
For media: Questions to ask when writing about polls
For candidates and pollsters: Disclosure checklist