.
How the Barbecue Sauce is Made
Last week I saw a poll from CIVIQS, with low ratings for Biden. Even-lower than
fivethirtyeight's numbers.
I was not familiar with CIVIQS, so I searched fivethirtyeight, to see if fivethirtyeight considers CIVIQS reputable.
In 2020, fivethirtyeight
did report results from CIVIQS, giving CIVIQS a "B-minus" letter grade.
So then, I checked fivethirtyeight's Joe Biden Popularity page--
to see whether fivethirtyeight incorporates-in-their-model... "B-minus" polls.
Indeed they do:
Ipsos B-,
AP-NORC [no grade],
McLaughlin & Assoc. C/D,
NewsNation [no grade],
American Research Group C+,
Echelon Insights B/C,
Redfield & Wilton Strategies B/C,
Navigator Research B/C,
Léger B/C,
SurveyMonkey C.
(those were all used by fivethirtyeight... since June 1, 2022).
And so-- I waited & watched-- to see if fivethirtyeight would incorporate CIVIQS' very-low Joe Biden numbers... into fivethirtyeight's model.
Either the low CIVIQS number would bring-down Biden's overall 'average' in five38's model...
or maybe five38's other sources would report equally-low numbers.
or maybe five38 would omit CIVIQS' result... which would call into question fivethirtyeight's methodology.
The CIVIQS poll never appeared in five38's model.
~and so...~
Today I saw another CIVIQs poll. Again, CIVIQS' numbers for Joe Biden's popularity... are even-lower than fivethirtyeight's.
CIVIQS:
fivethirtyeight:
8% of People who told CIVIQs they neither Approve nor Disapprove of Biden...
are counted as "Approve" by fivethirtyeight. fishy?
And so I looked-into fivethirtyeight's methodology. fivethirtyeight sez:
However, there are some types of polls we don’t include, such as:
...
Polls like Civiqs ... produced using MRP (short for “multilevel regression with poststratification”), a modeling technique that produces estimates based on survey interviews that are then used to calculate probabilities; these probabilities are then projected onto the composition of the population in question. While this is a valid technique for understanding public opinion data, we exclude them because we consider them more like models than individual polls. (As an analogy, we think of this as using someone else’s barbecue sauce as an ingredient in your own barbecue sauce.)
I understand why Averaging a list of Averages can produce unreliable single-result. A model based on other models could suffer the same flaw. But that's not-quite what fivethirtyeight is saying, here. The 'modeling' element in CIVIQS's poll isn't 'a model' (a poll of polls) in the same sense as fivethirtyeight. CIVIQs is merely using a statistical model to adjust their OWN raw survey-group replies-- to better-match the actual composition of the population as-a-whole. To use fivethirtyeight's own metaphor... CIVIQs already put "barbecue sauce" on the numbers. And so fivethirtyeight refuses to serve those numbers to the public-- because fivethirtyeight would insist-on adding their own "barbecue sauce" too.
The irony of this-- and the flaw in fivethirtyeight's analogy-- is that fivethirtyeight's signature claim--
is that fivethirtyeight's Mix Of All The Other Barbecue Sauces is
Better Than any individual sauce.
I'm not saying fivethirtyeight shouldn't exclude CIVIQS. They should. They're doing the right thing by considering CIVIQS a rival-equal in the marketplace. But... by the SAME exclusionary logic... fivethirtyeight would have-to exclude-from-their-model any pollster which puts any statistical "sauce" on their poll. And... I'm pretty sure... that's ALL OF THEM. They all finesse their surveys... at some level... such as deciding who-to-poll... in an attempt to make their sample-groups 'reflect' the larger population. fivethirtyeight doesn't deny this. They write:
I don't think this shatters fivethirtyeight's credibility. I do think it shows cracks. And they're trying to 'patch' the problem, rather than admit their model runs on bad data.
~and then...~
Lastly my perusal of fivethirtyeight's fine-print... uncovered this:
What's that about?
Who? Whut? How must a pollster transgress fivethirtyeight's "pillars of philosophy", to be excluded??
fivethirtyeight doesn't BAN blatantly partisan polls. They merely put a "flag" symbol next to them:
So... who on earth... was soooo bad that they fell-outside these lenient rules?
Fuck you fivethirtyeight. You're so cunty & cutesy... that you Don't Even
Name the ONE organization you banned. You "Voldemort"...
They Who Shall Not Be Named... behind a
link... to the fucking biased FAKE NEWS Washington Post.
But-- here's the thing-- back when Fox News was banned for being 'partisan'-- Fox said Trump was neck-in-neck with Biden.
While fivethirtyeight and ABC News and all the other polls said...
Put aside your whether you agree with the ideology of Fox News. Fox's polls were more ACCURATE.
fivethirtyeight has abandoned their mission & become the thing they once despised.