Polling error: How one survey changed the Newsom recall campaign

Lea este artículo en español.

The panic started to set in for California Democrats in the last week of July.

First, there was a UC Berkeley Institute of Governmental Studies poll on July 27, which showed that likely voters were just about evenly split on whether or not Gov. Gavin Newsom deserves to keep his job. 

A few days later came an even more alarming set of figures from SurveyUSA: A majority of likely voters, 51%, wanted to fire Newsom, compared to a mere 40% who did not. It came at a pivotal moment in the chaotic recall, just hours before the first televised debate. And it showed the power of polls — not just to offer a snapshot of the race, but to shape it.    

This one poll upended the conventional wisdom that pundits and political consultants had been spouting for months: The Democratic governor of the reliably Democratic state had little reason to fear the Sept. 14 recall. “Shock poll shows Gavin Newsom losing recall vote by double digits,” read one headline. The widely monitored poll average at FiveThirtyEight snapped from the “no” on recall campaign leading by 7 percentage points to a tie. With a relative scarcity of large statewide polls, that single survey “kind of swung the numbers a lot,” polling guru Nate Silver said in a podcast last week.

For California Democrats, the prospect of Newsom succumbing to a conservative-led recall campaign was the stuff of nightmares.   

But it was also exactly what Newsom and the “no” campaign needed to rouse liberal-leaning voters. They immediately blasted out the bad news like a statistical bat signal for apathetic Democrats.   

“This recall is close,” read the fundraising email to supporters on Aug. 5, the day after the SurveyUSA poll was released. “Close enough to start thinking about what it’d be like if we had a Republican Governor in California. Sorry to put the thought in your head, but it’s true.”

Only maybe it wasn’t.

In its next poll, released at the end of August, SurveyUSA issued a mea culpa in a memo addendum that said the prior poll may have misworded a question and inadvertently inflated the “yes” on recall numbers. 

The Democratic panic of early August has since subsided. A UC Berkeley poll out Friday shows the recall failing by 22 percentage points. The latest from SurveyUSA gave Newsom a solid 13-point lead. Now a new conventional wisdom has taken hold: The recall faces long odds because Newsom’s Democratic base of voters has woken up. 

SurveyUSA CEO Jay Leve admits that their earlier poll could have acted as an alarm clock. “It was not our intention to cause Democrats to suddenly sit bolt upright in their chairs,” he told CalMatters. And yet, “that, I think, happened.”

There are other factors that likely snapped complacent anti-recall voters to attention: A $70 million get-out-the-vote campaign from Team Newsom, the decision by every major GOP hopeful to vocally oppose popular vaccine mandates, a surprise U.S. Supreme Court decision restricting abortion rights, the arrival of ballots in the mail to every active registered voter and an increase in media coverage as Tuesday’s election approached.

And then there was the emergence of right-wing provocateur Larry Elder as the leading replacement candidate, at 38% in the latest poll. The conservative radio host secured a place on the ballot at the last minute, and only thanks to a judge’s ruling that wiped out a requirement that candidates disclose tax returns. He became a useful foil for Newsom, especially after Elder’s ex-fiancée alleged that he waved a gun at her while high on marijuana — which Elder denied — and reporters began digging through his long history of controversial comments on women, race and other issues.    

“There are two ways in which the polls, potentially, not only reflected but shaped the campaign: One was through this creation of crisis, which motivated Gavin Newsom’s donors, his political allies and his base,” said UC San Diego political scientist Thad Kousser.

“The second and perhaps more important thing was the polls over the past month have made it clear that Larry Elder would be the next governor, if the recall is successful. Since then Gavin Newsom seems to be gaining ground. Those two facts are not unrelated.”

‘A contrarian story’

UCLA political scientist and public opinion researcher Matt Barreto is happy to knock other pollsters for their work on the recall (“It was conducted by drunk clowns,” he said of an August Emerson College survey that showed a close race). 

But he also said the media deserves some blame for mischaracterizing the state of the race earlier in the summer. 

“People in your industry don’t put any effort into it,” he told CalMatters. “A really poor quality poll comes out…and you think, ‘This is a contrarian story, let’s start repeating this!’ Instead, people should have looked at it and laughed it out of their news feed.”

But in early August, there were plenty of data points to support the contrarian narrative that Newsom could be in trouble. And that trouble had a name: the “enthusiasm gap.”

The July Berkeley poll showed that though Newsom maintained a healthy lead among all registered voters, “likely voters” — those the pollster assumed would actually return their ballots — were evenly divided. 

To find likely voters for its early August poll, SurveyUSA asked each respondent if they planned to cast a ballot “in the recall election to remove the Governor.” That phrasing, the pollster’s later memo noted, could have been interpreted not as an inquiry about the respondent’s general voting plans, but about whether they intended to specifically vote “yes.” That would have weeded recall opponents out of the pool of likely voters.

“It was not our intention to cause Democrats to suddenly sit bolt upright in their chairs.”

SurveyUSA CEO Jay Leve

The poll put its thumb on the scales in another way. Rather than list all 46 replacement candidates, the survey included only seven, along with “other” and “undecided” as choices. Only one of the listed candidates, YouTube real estate advice personality Kevin Paffrath, was a Democrat. That led some Democratic respondents to choose him, whether or not they knew who he was, making Paffrath the top replacement candidate with 27% support. 

“The balloon that SurveyUSA had initially shown for Paffrath was just — what’s the right word?” Leve said, pausing. “Indefensible.”

Still, that surprise number earned the candidate a series of national media profiles — and more coverage in California, including in CalMatters. He later qualified for a televised debate, and still hit 10% in the latest poll.

Contrary to popular belief, polling has not gotten more inaccurate over the years. But as Trump-supporting Republicans in particular have grown more distrustful of the news media, they are often undercounted in surveys.

Sorting out who is, and isn’t, likely to vote is among the messiest tasks facing pollsters. They generally start by asking respondents about their voting plans. The Berkeley poll also takes into account a voter’s reported interest in the race and their past history of participation. The Public Policy Institute of California, another prominent pollster, also factors in whether someone has been following the news, their education, their interest in politics and when they last moved.

Anticipating who is actually going to cast a ballot is especially difficult for the Newsom recall. California typically doesn’t hold a statewide election in an odd-numbered year, and certainly not in September. The last time we had a statewide recall was 2003. 

“People are just less reliable in assessing their own likelihood of voting” in such an odd election, said Will Jordan, director of research at Global Strategy Group, a national Democratic pollster. 

What enthusiasm gap?

The focus on “enthusiasm” as an indicator of a likely voter may have been misguided, said Dean Bonner, associate survey director at the Public Policy Institute of California. 

He pointed to the institute’s survey released Sept. 1, which found that only 40% of Democrats reported being more “enthusiastic” than usual about voting in the Newsom recall, compared to 54% of Republicans. But Democrats were more likely to say the outcome of the race was “very important.” 

Given that many Democrats see the entire recall as an unjustifiable waste of time and money, it’s no surprise even committed voters don’t report feeling enthusiastic, he said.

But “with all the discussion about how Larry Elder is going to replace (Sen. Dianne) Feinstein with a Republican…you might look at the outcome as very important.” 

“The polls over the past month have made it clear that Larry Elder would be the next governor, if the recall is successful. Since then Gavin Newsom seems to be gaining ground. Those two facts are not unrelated.”

UC San Diego political scientist Thad Kousser

Garry South, a Democratic political consultant who managed Gov. Gray Davis’ unsuccessful effort to avoid a recall in 2003, said it was always predictable that Democrats would show up, no matter how checked out some may have seemed in July.

“You get a ballot mailed to your house, you fill out ‘no,’ you put it back in the envelope and you put it in the mail — postage paid. This doesn’t require enthusiasm, folks.”

True or not, the narrative of listless Democrats may have acted like a self-reversing prophecy. As Democratic pundits and elected leaders began freaking out, Newsom’s defenders kicked into gear. 

In late July, Massachusetts Sen. Elizabeth Warren hit the airwaves with an ad not only imploring Californians to vote no, but giving detailed instructions how to do so. That same week, the California Teachers Association, the governor’s biggest financial backer in his 2018 election victory, launched a get-out-the-vote campaign and invited the governor to give the keynote address at its summer convention.

The two days after the release of the Berkeley poll in July were among the top five biggest fundraising days for the “Stop the Republican Recall” committee. Between Aug. 1 and 28, the campaign pulled in another $19 million.

Progressive advocacy groups also began a full-court press. Ludovic Blain, the California Donor Roundtable’s executive director, went on a media tour urging Democrats to take the mobilization of voters of color seriously. And on Aug. 12, a constellation of progressive groups, including Courage California, the California League of Conservation Voters and NARAL Pro-Choice held an online get-out-the-vote rally. Some of the speakers were enthusiastically pro-Newsom. But the prevailing sentiment was anti-recall.

“Whatever problem you have with Gavin Newsom, unfortunately this recall is a binary choice,” said Assemblymember Alex Lee, a San Jose Democrat and a frequent critic from Newsom’s left. “Do you want Gavin or do you want something objectively worse than Gavin?”

The new UC Berkeley poll, which found the recall trailing by double digits, looked like a return to “normal” California politics, pollster Mark DiCamillo said in a panel discussion on Friday. 

As of Friday, 33% of registered voters had already turned in their ballots, according to Political Data Inc., which is tracking the voting. Of those 7.3 million ballots, 53% came from registered Democrats. 

Though Republicans, and therefore “yes” voters, are more likely to vote in person on Tuesday, as they did in November 2020, “the recall needs a big red shift,” said Political Data vice president Paul Mitchell. “At some point this becomes a math problem….A high-turnout election ensures that Newsom beats this.”

Not that the Newsom campaign is taking anything for granted. The California Democratic Party tweeted Friday: “Ignore the polls.” After all, it knows as well as anyone what happens when voters begin to take the polls seriously.