Did polling manage to correct previous oversights? Are there new gaps that making polling even more challenging! Or did polling do a damn fine job this time around? "Only time. Will tell."
So here is 538 final projection https://projects.fivethirtyeight.com/2022-election-forecast/house/?cid=rrpromo
So, based on that graph, if the Democrats hold on to either the House or the Senate, we can say that the polls were wrong.
Before the next election, you might want to find a better way to poll anyone under the age of 30 since they would rather pick up a pinless grenade than a call from an unknown number.— Ben Collins (@oneunderscore__) November 9, 2022
Actually the pollsters without a shamelessly partisan agenda did alright from the looks of it. Even picked up on some regional variations that didn't fit the national environment.
If the House and Senate odds are independent, then the chance that the Democrats would control at least one of those two bodies, per 538's analysis, is .58 *. 85 = 49.3%. It's kinda hard to say that the polls were "wrong" if a 49.3% event occurs.
My non expert opinion is that traditional polling was pretty ok. partisan polling (which flooded the airwaves the last 2 weeks) was hopelessly wrong. And fundamentals expectations was biggest loser?
Given that every candidate whom I expected to win did win, and every candidate I thought would lose did lose ... yeah. With the sole exception being sorta Herschel Walker, whom I thought was going to win by a few votes. But he might yet do so in the run-off.
I have a story here. The boy and his three freshmen roommates had a dorm room equipped with a landline. (This was a few years back.) They never learned the number nor touched the phone. One day late in the year it rang. They looked at each other in fear. It was like a horror movie to them, a phone that just rings, you don't know who is on the other end. Nobody mustered the courage to pick it up.
https://www.npr.org/2022/11/04/1134434712/planet-money-tries-election-polling They touch on the problem with cell phones and polls.
Not really. Those are probability distributions based upon both poll aggregation and a regression to some perceived mean informed by the economy and historical partisan lean (Cook). That forecast is basically polls + hedging a bit for historical observations of elections under these types of conditions. For the “polls only” forecast, 538 adjusts polls for historical partisan lean and quality of those pollsters. The higher the quality, the greater the weighting and more narrow the confidence interval on the poll estimate. Their median polls only expectation: 50 GOP senate seats 229 GOP house seats That is very, very close to where we’ll end up, so overall, the polls performed well. Some polls performed terribly. Namely, the GOP backed polls we’ve been seeing over the past month. But these polls weren’t given as much weight in the 538 model. A couple reasons the polls + historical hedge underestimated Dems: 1) we’re in a hyper partisan environment, so the number of swing voters has declined substantially. 2) the current GOP is really awful by any sort of historical standards. It’s not only miles away from what the opposition wants on specific issues. It’s miles away from what many of its own voters want. If a model weighs conditions like GDP growth, inflation, state/district partisan lean 1/3 or whatever and you end up with a 1.5pt adjustment on the polling estimate, that may not be the case in this environment. That appears to be true of this election.
Your last paragraph is spot-on. If all the forecasting was just aggregating the polls, then you'd by WAY off, which is what happened to Real Clear Politics. So then, you look at accuracy of polls and weight them. A good deal of analysis goes into this, but it isn't like interpreting quantum mechanics. Not everyone will agree exactly on the weight that should be given to each poll, but they'll be pretty close. This is all quite rote, and anybody with an excel sheet should be able to be much more accurate than Real Clear Politics just by abiding by these principles. The "magic sauce" is taking into account these external conditions that are not consistent from year-to-year, and weighing that correctly. The weights can even be different from poll-to-poll as conditions and populations in each State differ. But I think the point remains, the elections are weird when Trump is on the ballot. And I don't think the data from 2016 & 2020 is going to be very predictive of 2024, because I could see him depressing the conservative vote turnout instead of boosting it.
Can this also be the What Media Got Wrong thread? I post this because I'm convinced that a good chunk of the media is still in 2005. Also generational: things learned from prior cycles are over-applied to the current situation. And maybe things that worked 20 or 30 years ago don't work now.— Elizabeth Spiers (@espiers) November 9, 2022
If a moderator wanted to change the title to "What Polling (AND PUNDITS!!!) Got Wrong," I'd be okay with that. Caps and exclamation points optional.
The aggregation “sauce” really comes down to understanding each poll’s weighting method. A poll is only as good as who it thinks will actually show up to vote and in what proportion by subdemo. That’s already baked into the 538 “poll only” forecast. This is our 4th national election in the age of Trump, so those estimates are actually getting easier/better. We’r so polarized that a greater percentage of seats are easy calls weeks out than even 10 years ago. What has become more difficult is to translate seat estimate ranges into chamber control because the polarization is so close to a 50-50 split. You can put together a dummy model. It would only consider three things: 1) partisan lean of each state/district from Cook 2) a small incumbent advantage 3) an adjustment for the national generic ballot poll applied universally to all races. That dummy model would have given the GOP maybe 52 Senate seats and 232 House seats. That isn’t rally that far off from what will end up being the final result. It would have missed on PA, GA and NV in the Senate. If you made a slightly more sophisticated dummy model that gave weighted outcomes to tight races, it probably would have given 1 of these three to Dems anyway. The electoral map is just very close to the control tipping point.
I went with the caps and exclamation points. It seems right. I think some in the media got caught up with the history of the midterms, expecting the worst for the president's party. Also, they underestimated the reverse Midas touch that Donald Trump seems to have. Everything he touches turns to, errr... to number 2.
I think that was a factor. I mean, it's part of what Kazuma means by the "like it's 2005" point. But that's a smaller part of the "manufacturing consent" thesis which is still in play. Pundits seem to be herd creatures. Once they get a narrative going, they're going to run with it regardless of how little connection it has to evolving events.
ETTD is the better way. Everything Trump Touches Dies. And yes on the midterm narrative. Yes, I'm aware the party loses seats, but a significant number of people were acting as if Dems were going to be obliterated.