- Bearing Drift - https://bearingdrift.com -

August Polling Round-up & Observations

As we head into the final two months of the campaign, let’s pause and take a look at where the race stands right now.

There have been 7 polls released in the past month, ranging from a narrow McAuliffe lead to a wide one. However, I find it valuable to not only look at the toplines, but also look at the polls themselves, and their track record – preferably, in Virginia with an off-cycle election.

I also look at the methodology. Not just how the poll was conducted, but specific decisions that were made in weighting and how to contact respondents. A necessary step here also requires evaluating the level of transparency (or lack thereof) with various polling releases.

In chronological order:

[1]

Roanoke College (link here [2]), conducted 8/3 – 8/17

Results: Terry McAuliffe has a substantial 8-point lead over Glenn Youngkin, 46% to 38%, with 16% undecided.

Methodology: Phone survey of 558 Likely Voters. I’m appreciative that Roanoke released their poll’s details (most college polls do), but there are three big red flags here. First, their phone sample is heavily reliant on landline phones – 55% to 45% cells. Second is the two-week fielding window. Third, the poll doesn’t weight for education. To be clear: they claim to adjust for education, but they weight the results based on 2017 exit polls. This is nominally a sound strategy for re-creating the likely voter electorate, but exit polls themselves aren’t weighted for education.

As a result, 58% of respondents have a college education, which is way too high and is likely inflating McAuliffe’s totals here.

Track Record: The week before the 2017 election, Roanoke showed a tied race (47% – 47%) between Northam and Gillespie. Northam would win by 9 points.

Overview: The landline-heavy phone survey with an overly educated sample is enough for me to view these results skeptically, along with not having a great track record four years ago.

[3]

VCU Poll (link here [4]), conducted 8/4 – 8/15

Results: Terry McAuliffe has a narrow 40% to 37% lead over Glenn Youngkin, with a whopping 23% undecided.

Methodology: Phone survey of 770 Registered Voters. Their poll was roughly split between landline calls and cell phone calls. They released their data, methodology, and crosstabs in full; however, they didn’t release their demographic results. This means that while we know they weighted on important factors like college education, they don’t release what they weighted towards.

Track Record: I can’t find a VCU poll from the 2017 race. In October 2020, they had Biden winning Virginia 51 to 39 two weeks before the election. While Biden would win by a similar margin (54% to 44%), VCU had a staggeringly high amount of unallocated voters that close to the election, similar to this poll.

Overview: The high amount of undecideds isn’t unexpected for a RV poll, but their reliance on landline calls and lack of weighting details give me pause. This poll is a good indication that the Virginia Governor’s race isn’t the talk of the town in early August, but it’s not as useful for much beyond that.

[5]

co/efficient (Link here [6]), conducted 8/8 – 8/9

Results: Terry McAuliffe has a slim, 2-point lead, that is within the margin of error, 47% to 45% for Youngkin.

Methodology: co/efficient contacted 1,200 Likely Voters between August 8th and 9th. We don’t have any other particulars to go off of for this poll. However, co/efficient released a different Virginia poll in July (link here [7]). While it only contacted 762 voters, if we assume the same methodology, it appears they use text and IVR (robo polls to landlines). In that poll, they weighted by education, and ended up with a >50% college educated audience.

Track Record: No previous Virginia polling found.

Overview: I’ve been a long-time skeptic of IVR polling, and that has only increased as landlines become obsolete. Text and IVR combined are the cheapest options on the table, so this is a budget-minded poll. In addition, the only details are what’s in the Hill.com article, so its tough to know what to make of this given the lack of details.

[8]

Change Research (x2) (link here [9]), conducted 8/14 – 8/18, and (link here [10]) conducted 8/17 – 8/21

Note: It is unusual for an outfit to conduct two polls in the same state with overlapping fielding dates.

Results: Their first survey (chronologically) showed Terry McAuliffe with a slim 3-point lead, 47% to Youngkin’s 44%. Their second survey showed McAuliffe with a larger 6-point lead, 49% to Youngkin’s 43.

Methodology: The polling memo for the first survey doesn’t discuss methodology, only that they contacted 1,334 Likely Voters as part of a broader swing state/district project for Future Majority, a Democratic PAC.

The second survey contacted 1,653 Likely Voters for Crooked Media, a liberal media outlet. Their polling memo includes a detailed methodology statement, noting a combination of online and text. They sourced their online invitations through targeted advertisements and social media engagement. This is a budding source of voter contact, as other traditional methods become less reliable. It’s methodology should be considered “unproven” but not necessarily dismissed, especially given the transparency in their release.

Overview: It’s unusual for one firm to release two polls in the same state with overlapping fielding windows. However, it’s not usual for such results to be different.

If we accept polls have a margin of error (this is debatable with certain methodologies), we should naturally expect a variance in the results, even if they were conducted the exact same way at the exact same time. The differences in methodologies, weighting practices, and turnout modeling, means we should expect to see perhaps more variance than we typically do.

[11]

CNU/Wason Center (link here [12]), conducted 8/24 – 8/29

Results: Terry McAuliffe has a substantial 9-point lead and is at 50%, leading Glenn Youngkin’s 41%.

Methodology: Phone survey of 800 Likely Voters, with most calls completed on cell phones. I will point out that college education is split 50/50; you could make a case for higher education attainment among an off-cycle electorate, but its bit high for my tastes.

Track Record: Two weeks before the 2017 election, CNU released a poll showing Northam leading Gillespie 50% – 43%. This is the closest any of these polls to the actual result – and it’s worth noting that CNU’s poll came at the same time as a mini-boomlet of polls showing Gillespie tied (and a few that showed him ahead).

Overview: Despite the 50/50 college educated sample, the Wason Center has a great track record in Virginia and their methodology is transparent and sound.

[13]

Monmouth (link here [14]), conducted 8/24 – 8/29

Results: Terry McAuliffe has a solid 5-point lead over Glenn Youngkin, 47% to 42%.

Methodology: Phone survey of 802 Registered Voters, with most calls completed on cell phones. This poll was properly weighted by education, with 41% of the electorate as college educated. All of their demographic details are released in full.

Track Record: The week before the 2017 election, Monmouth had Ralph Northam leading by only 2 points, 47% to 45%, dramatically underestimated the votes Northam would get.

Overview: While the topline results of the poll are among registered (not likely) voters, I like that Monmouth presents two turnout scenarios based on the vote propensity of its respondents. In higher turnout models (with less frequent voters showing up), Youngkin closes the gap . In a lower turnout model (with only the most frequent voters showing up), McAuliffe has a bigger lead.

[15]

Trafalgar (link here [16]), conducted 8/26 – 8/29

Results: The race is tied, with Terry McAuliffe earning 46.6% and Glenn Youngkin earning 46.3% of the vote.

Methodology: ?????

Trafalgar claims to have contacted 1,074 Likely Voters. Trafalgar doesn’t release any details of this poll’s methodology. They also don’t release any crosstabs or any demographic data.

Their website says [17] that they use a mix of phone, IVR, text, and email methodology; in addition, they use “two other proprietary digital methods [they] don’t share publicly.” Seriously, it says that.

They keep it a secret what those methods are. They keep it a secret which methods were used for this Virginia poll. Was it all IVR? Was it all one of their secret methodologies? They won’t say.

Trafalgar also claims to have unique access to those who don’t like taking polls or answering honestly. One way they claim to address this is by weighting to counteract what they call “Social Desirability Bias.” How they measure or observe this bias is kept secret. How they correct this bias in weighting their results is kept secret. Whether or not they weight the data in any other ways is kept secret. If they did weight, how they weighted the data is kept secret.

In short, anything that might help give these results credibility is kept hidden.

Track Record: In 2017, a week before the election Trafalgar “polled” (guessed?) that the race was tied, 49% – 48%. Northam would win 53% – 44%. In a different off-cycle election, the 2019 Kentucky Governor’s race, Trafalgar “polled” (guessed?) that Republican Matt Bevin was ahead by 6 points the week before the election. Bevin lost.

Overview: It’s difficult to seriously compare these results alongside legitimate polls that release basic details about their methodology. Even releasing basic information, like the methods they claim to use to contact respondents, would go a long way in building credibility.