The Anatomy of a Biased Poll

YouGov, CBS News's pollster of choice, released a poll showing that former Vice President Joe Biden leads President Donald Trump by 10 points: 52% to 42%. Their sample consisted of 957 registered voters with a margin of error of plus/minus 3.6 points.

According to YouGov, Trump leads Biden by 10 points (47% to 37%) among independents and by 88 points (93% to 5%) among Republicans. So, why is he behind by 10 points overall? The sample—i.e., the group of people that YouGov polled—was roughly 41% Democratic, 30% Republican, and 27% independent.

Compare YouGov's sample to the actual breakdown of the electorate: 41% independent, 31% Democratic, and 26% Republican. YouGov's numbers, when adjusted to fit reality, indicate a much tighter race: 45.9% Biden to 44.4% Trump (my math is below).

President Donald Trump

The massive over-representation of Democrats in YouGov's sample is a textbook example of why conservatives and impartial observers reflexively discount public-opinion surveys. As I noted last week, polls are useful only when considered alongside other factors.

In other news, YouGov found that 57% of respondents watched none of the Democratic convention or "not much" of it. It remains to be seen whether this week's Republican National Convention will enjoy better viewership.

***

My math:

Biden: (.41 x .37) + (.31 x .95) + (.26 x .05) = .4592 = 45.92%

Trump: (.41 x .47) + (.31 x .03) + (.26 x .93) = .4438 = 44.38%

Comments

  1. Elijah AsdourianAugust 25, 2020 at 8:45 AM

    This gallup data doesn't measure "the electorate," it measures American adults. A full quarter of Americans aren't registered to vote at all.

    Because this is a poll of registered voters, and not all adults, every single data category, not just political party, is skewed.

    61.5% of the people polled were above 45, when they're only 51.8% of adults.
    70.7% of the poll was white, when they're only 63% of adults.
    37.9% of whites held bachelor's degree, when the real figure is about 32%.
    53.4% of the poll was women.
    Even using the gallup data, scroll down to the "republican leaning" and "democratic leaning" disaggregated independents, redo the math, and you get Biden 47.7% to Trump 40.5%.
    If you redo the math for all of these "biases", you would get 1) a supposedly unbiased poll, and 2) a totally useless one, because not all groups vote at the same proportions.

    Redoing the math accounting for all the "biases", you get
    Adjusted for Age
    Biden: (.482 x .58) + (.518 x .48) = 52.8%
    Trump: (.482 x .32) + (.518 x .48) = 40.2%
    Adjusted for Race
    Biden: (.37 x .76) + (.63 x .41) = 54%
    Trump: (.37 x .18) + (.63 x .52) = 39.4%
    Adjusted for Whites' Education
    Biden: (.37 x .76) + (.63 x .68 x .37) + (.63 x .32 x .48) = 53.6%
    Trump: (.37 x .18) + (.63 x .68 x .57) + (.63 x .32 x .45) = 40.2%
    Adjusted for Sex
    Biden: (.492 x .48) + (.508 x .54) = 51%
    Trump: (.492 x .47) + (.508 x .38) = 42.4%

    By the theory of the article, these calculations are all correcting for bias. But unless YouGov is a pro-Biden on party registration, education, and sex pollster, while simultaneously being a pro-Trump pollster on race and age, I don't think bias correctly describes what's going on here.

    ReplyDelete
    Replies
    1. The EditorAugust 26, 2020 at 9:16 AM

      What is the likelihood that someone who is not a registered voter would take ~15 minutes to respond to a Gallup poll regarding partisan affiliation? We can assume that that the answer is "Not very high."

      Thus, the Gallup data is a reasonable substitute for the electorate more broadly.

      Delete
    2. Elijah AsdourianAugust 26, 2020 at 10:19 AM

      By the same logic, what is the likelihood that someone who is not a registered voter would take ~15 minutes to respond to a YouGov poll regarding partisan affiliation? We can also assume that the answer is "Not very high." Thus, the YouGov data is a reasonable substitute for the electorate more broadly.

      Now, you might say that the Gallup poll didn't ask about presidential choice, while the YouGov poll did, but of course if the Gallup poll is taking ~15 minutes (where did we find this number?) it's not just asking about party affiliation either.

      More broadly, data from Ballot Access News, which records data from the 31 states which require registered voters to register with parties, has the country at 40% democratic, 29% independent, and 29% republican (this is from official voter registrations).

      The Gallup data, when compared to the data we actually have here, is not a good sample of registered voters, while the YouGov poll aligns almost perfectly with the data. Here's a link from conservative The Hill reporting this in February: https://thehill.com/homenews/campaign/485255-there-are-fewer-registered-republicans-than-independents-for-the-first-time

      Again I submit that we can't call this bias

      Delete

Post a Comment

Popular posts from this blog

Trump Favored in Pennsylvania

Image
The RealClearPolitics polling average in Pennsylvania is Biden +3.7, with polls ranging from Trump +2 (InsiderAdvantage) to Biden +7 (Quinnipiac). I predict that President Donald Trump will narrowly win the state. Consider Muhlenberg College’s most recent poll for the Morning Call , which pegs Biden’s support level at 49% and Trump’s at 44%. 2016 election in PA Consider also that Muhlenberg’s final 2016 poll had Clinton up 48% to 42%, a six-point lead. Trump ended up winning the state by a little less than a point. The cross-tabs tell us why Muhlenberg was wrong: they found Trump only marginally leading males (45% to 43%), whites (49% to 42%), and independents (41% to 38%). The final results for these categories, per CNN exit polls , were 57% to 40%, 56% to 40%, and 48% to 41%, respectively. Thus, I am skeptical of some of the findings from Muhlenberg’s most recent poll, which shows that Trump’s lead is 49% to 43% among men and 52% to 44% among whites. Further, Muhlenberg has Trump ...
Read more

Ron Johnson Saves Taxpayers at Least $89 Billion

Image
By opposing a $1,200 check to each American and $500 for each dependent, Senator Ron Johnson (R., Wisconsin) pushed Congress toward a $600 check for each American making less than $75,000 and $600 for each dependent. In so doing, Johnson saved the taxpayer at least $89 billion. Ron Johnson ( Milwaukee Journal Sentinel ) Based on my calculations, there are at least 191,400,280 Americans making less than $75,000 a year, as there are 255,200,373 adults and 75% of income earners make less than $75,000 a year. Carrying over that 75% statistic, there are also at least 49,835,000 dependents (as there are 66,447,000 people under 18). These numbers are a bit low, as they yield a sum of 241,235,280—i.e., a cost of $144.74 billion after the Johnson intervention. The actual cost  of the handouts will be $166 billion. Prior to Johnson's intervention, the cost of the $600 checks to people making less than $75,000 a year would have been at least $229.68 billion (191.4 million times $1,200) an...
Read more