Last week the polling website FiftyPlusOne contacted Wedgewood Polls with questions about the authenticity of its surveys. In response, the team behind the supposed pollster acknowledged that its work was fake. The whole thing, Wedgewood claimed, was an “experimental project.”
The project had an impact on election campaigns, with polls it did since June on many races, “shifting from one midterm battleground to another,” the New York Times reported, including Michigan, Maine, Kansas and Wisconsin.
...
The media has reported that Wedgewood began releasing polls in June. On July 7, a story by Urban Milwaukee raised questions about a Wedgewood poll on the Democratic primary for governor in Wisconsin. Up to then, the most recent Marquette poll had shown Hong with 14% of the vote, and a more recent internal poll released by the campaign of Sara Rodriguez showed Hong at 22%. But the Wedgewood poll had Hong at a whopping 30%, which could “help convince Democrats who doubt her electability that she is running away with the race,” the story reported, noting that Wisconsin Republican Chairman Brian Schimming had said that for Republicans Hong would be the best opponent.
...
But the Wedgewood “experiment” clearly sought to push Democratic voters in a way that would help Republican candidates. Just as it tried to elevate the candidacy of Francesca Hong, the most liberal candidate in the Wisconsin Democratic primary, it released a fake poll showing Mallory McMorrow, the moderate candidate in the Michigan primary for U.S. Senate, “plummeting into single digits,” as Time reported. “McMorrow said donations ‘dried up’ and she was ‘hounded’ with calls to exit the race after the circulation of the infamous’ Wedgewood poll numbers.”
Funny how the party that always screams the loudest about election fraud is the party that keeps getting caught (but somehow not really punished) committing election fraud.
Fake polls are not election fraud, although they certainly are a dirty trick.
Fake polls are definitely a “both sides” thing. A notorious example in the 2024 election was the Selzer poll, which was ludicrous when it was released and confirmed to be such during the actual election. 16 point misses by seasoned pollsters are not accidental. Polls generally speaking have long been used to shape public opinion and policy more than they have to actually understand the public .
Nowadays both sides are also doing what they can to manipulate the elections via prediction markets.
The selzer poll was not a fake poll. It was an outlier. She was famously one of the best most reliable pollster in the country. Sometimes polls are just wrong just from pure bad luck and you still publish them if you trust your methodology.
> 16 point misses by seasoned pollsters are not accidental.
Not a pollster, but Nate Silver publicly challenged someone on Twitter to a $100k bet over how red Florida would go. IIRC Silver was predicting 3-5% margin, Twitterer said minimum 8%. Trump won by 14%. Even Silver, who lived and breathed analyzing polls and their reliability, made a mistake of that magnitude with his own money on the line. Sometimes the votes don't line up with polling, and sometimes the professionals genuinely don't see it coming.
I think that that is more of an illustration that polls are always about a statistical likelihood and never about an absolute. With a 3-5% margin 14% is not as unlikely as you might think. (It's ~ 1:3).
You need a measure of variance to perform that kind of estimate. There's scenarios where 98% of results land within 3-5% of a value, and the 14% is extremely unlikely. And scenarios where the variance is high, and it's much more likely. (you can also be wrong about what you think the variance is)
As far as I'm aware these poll results have (roughly) a normal distribution, right now the spread between the two extremes in polls with sufficiently high 'N' to be taken into account in most reports differ by 19%.
The other issue is that in an election like the US, the results in different states are not independent, decoupled rolls of the dice. i.e. If the results of the first few states lean a few % one way off the polling (within the margin of error) then it's most likely that most of the other states will doe the same.
Ann Selzer used a model that had little to no weighting in it and only used phone polls. This worked reasonably well in previous elections where the combination of chance and similar levels of voter enthusiasm.
It failed in 2024 when it didn't weight its model to handle differences in demographics vs the sampled population. She accounted for age, sex, and district - but not education, party identification, or past voting behavior. (The specifics of why the model failed is up for debate, but it was the same model that had been used for many years and was reasonably accurate).
From 2023, here's a description of her polling methodology:
But it's difficult, even I know where to look and a lot of the methodology statements get hidden or are inadequate to make that assumption. The main thing I'm going to tell you that distinguishes us from most other polling, not all, there's some very good polls out there that do what I do, which is we don't make assumptions about what that future electorate is going to look like. We let the data -- we try to get out of the way of the data showing us. So, you'll look and they'll say how they weighted the data, this that and the other, and then they include in that who the likely demographics of the 2024 caucus are likely to be. And I go, really? How do you know what that is likely to be? Or they're weighting it based on who they voted for in the 2020 election or other factors like that, or maybe they're only talking to people who have caucused before. And this approach I call polling backward, that if you're creating a model of the future, what are you going to base it on? You're going to base it on what happened in the past. And I know that things can change. So, I want to have my fingers off the data, my dirty fingers I try to keep them off the data, so it will reveal to me how that's going to go.
... And that approach failed with her 2024 poll. She weighted the overall population, but not the likely voter and it's possible that the approach was too strict.
But for everybody else we keep them on the phone to get age, sex, what county they live in. That is going to give us some geography so that we can weight all of those people, I think there were 1,100 and some in our final poll, to look like the state of Iowa because that we know. And then we extract from that those people who made our likely voter screen and interviewed them. So, if they are proportionately more women who pass that screen, they are going to show up here but they are weighted at the statewide level. It's a little geeky. But one of the things I've been thinking about is with caucuses we take you if you definitely or probably will participate and maybe our screen was too tight. I don't know, there's no way to undo it and sort of check, well what would the probably people have said, because we very politely terminated the people who just said they would probably vote. But that has been our system since 2008.
Yes, that’s her public defense. Did she publish the raw data and methodology showing this? Or are these unprovable statements that cover for her reputation?
I mean this defense is effectively saying that polls are actually worthless and should be ignored anyway - that’s what “the best polling methodology we have can sometimes be off by 15 points” would actually mean.
If you really believe “good” pollsters don’t do this, you should question how the Harris campaign’s internal polling outperformed them. How did a political campaign correctly predict with polling all along that she was behind Trump when public polling consistently showed her ahead until tightening immediately before? Public polling, again, is generally more of a tool to shape opinion than it is to reflect reality. Private campaign polling that’s limited to leadership only is much more likely to be accurate since they actually want the truth.
What it's actually saying is that polling is useful in the aggregate and useless in the individual. One can have a 66% hit rate in the aggregate, well above flipping a coin, and still not be able to predict the future for any individual outcome. Unfortunately, 90% of people are statistically illiterate, so we get takes like "getting a single poll wrong can only be the outcome of a malicious attempt to deceive the public".
Polling aggregators also have a miserable record. You get closer because by aggregating you partly eliminate individual biases, methodological errors, sampling issues, and ulterior motives, but you also have problems with the pool of polls itself and aggregators are ineffective in adjusting for these issues by weighting or eliminating pollsters accordingly.
Another comment even mentioned the notorious Nate Silver (bronze perhaps), the most famous of these types who lost 100k by being completely wrong about Florida by a huge margin. (Did he actually commit and payout? Not sure.)
I was the one who mentioned Nate Silver as well. That case does not undermine his skill. The 538 model's track record over hundreds of thousands of forecasts was, statistically, extremely good.
Holding a forecaster to a standard of "Silver must get 100% of predictions correct or else he's a fucking moron" is, in itself, a sign of low critical thinking skills.
Well, I do it the same way I've been doing it. I've made no change in that, to be clear. And this is going to get a little wonky. What I want to do is to collect data that will reveal to me what the future electorate is going to look like. I call it polling forward. I call people who are weighting their data looking at past elections polling backward.
And there are biases with internal polling. I'm more likely to answer a poll by (A) whom I'm more likely to vote for than by (B) where I may be concerned that they're trying to collect my data for nefarious uses.
If you believe polls are worthless, then don't look at them. However, a lot of the enthusiasm and donations for a political candidate are driven by polling.
The people that got fucked over by Wedgewood were the party-preferred candidates. Wedgewood artificially inflated the polling of progressive candidates that were running against the party to a degree.
On top of that it's not election fraud, though it is potentially some kind of fraud if they were involved in betting markets.
