I used to run political campaigns; as part of that, I commissioned polls. I have a good sense of how polling works, what it’s good for, what makes a poll valid. Martel’s implication that 612 people is an absurdly low sample size is not only erroneous, it’s a staple of those who want to cast doubt on research for political purposes.

Skepticism is always warranted. It is not, however, valid to take issue with basic math.

Let me lift a good analogy from the Washington Post. To test the temperature of soup, you don’t have to eat the whole bowl. You stir it up and take a taste. Similarly, to get an accurate poll, you don’t need to ask everyone the question – you just need to gather a random sampling of people.

Polling takes a random selection of people and then weights the results, as is well-explained here. Since a random sample will not reflect the demographics of the full population, results are recalculated to place more emphasis on bigger population groups, a process called weighting. Common factors considered are economic status, race, gender, location and educational history – though some polls may consider additional things to weight.

The Fairleigh Dickinson poll released yesterday had a sample size of 612 people, all within the state of New Jersey. 612 people is a very standard size for a poll, particularly one taken within one state. Polls bear diminishing returns as you add people; that is, adding an additional 100 people to a 600-person poll has much less of a difference than adding 100 to a 100-person poll. Not that you’d want to do a 100-person poll. The margin of error for a poll of that size is 10%, as opposed to 4% for a 600-person poll.

Let’s look at recent polling for the 2012 race (polling, I might add, that Martel has referred to at least six times in the last month without comment). In Iowa, twenty-seven polls have been commissioned since May of 2011. The average sample size for all of those polls? 528 people.

One savvy commenter on Martel’s post raises a good point: isn’t the fact that this is restricted to one state a bigger problem for applying it nationally? In general, yes – limiting an opinion poll to one state reduces its randomness. If we were polling for President, sampling only in Texas or Illinois would certainly impact the validity of the results. This poll, of course, is testing knowledge as much as opinion, for which I’ll give it some leeway.

But the sample size is beyond question. There’s one final piece of supporting evidence that’s worth mentioning: the long, solid history of correlative results from polls using the same methodology. Nate Silver, the Times‘ polling guru and author of the always great FiveThirtyEight blog, regularly does a post-mortem after big elections, rating how various pollsters did. After last year’s midterms, he ranked eight; in June of 2010, he ranked dozens more. His post-midterm findings reflect the accuracy of poll results. Quinnipiac, the firm that did the best, conducted 21 polls and ended up with an average error of 3.3 percent. And bear in mind, these polls could have been up to three weeks before Election Day. Results like that are hard to argue with.

The polling firm that did worst in Silver’s survey was Rasmussen Reports. Silver delineates Rasmussen’s “cavalier attitude toward polling convention,” mostly done in the interest of cost savings, and closes with “the methodological shortcuts that the firm takes may now be causing it to pay a price in terms of the reliability of its polling.” Methodology is everything, and firms that follow that methodology – like, say, a 600-person sample size – see predictive results.

Oh, and any guess which media company regularly relies on Rasmussen polls? Fox News.

One final resource: the National Council on Public Polls has twenty questions journalists should ask about polling. It’s no longer online, but the preceding link goes to a cached version. Worth the read.

RasmussenReports.com explains:

Republican primary voters at this early stage of the game now give billionaire developer Donald Trump the edge over presumptive favorites Mitt Romney and Mike Huckabee in the race to be the GOP’s presidential nominee in 2012.

A new Rasmussen Reports national telephone survey of Likely Republican Primary Voters finds Trump with 19% support, just ahead of the former governors, Massachusetts’ Romney at 17% and Arkansas’ Huckabee with 15%. (To see survey question wording, click here.)

Former Alaska Governor Sarah Palin and ex-House Speaker Newt Gingrich are tied for fourth place with nine percent (9%) each, closely followed by Texas Congressman Ron Paul at eight percent (8%). Rounding out the list are former Minnesota Governor Tim Pawlenty (5%) and Mitch Daniels, the current governor of Indiana, at three percent (3%). Five percent (5%) like some other candidate, and 11% are undecided when presented with this list of candidates.

In January, before Trump began making noises about seeking the GOP nomination, Romney led the pack among likely primary voters with 24% support, followed by Palin (19%), Huckabee (17%) and Gingrich (11%).

One wonders if these poll results would be much different if it were conducted after yesterday’s media circus. Either way, these results are sure to be seen by many as a rather strong indictment of the GOP presidential field. Perhaps recent news will benefit the presidential aspirations by more “conservative” candidates like Tim Pawlenty? Only time will tell.

Read the entire write-up at Rasmussen Reports (H/T DailyCaller.)

OK Cupid’s statistics crew (which, given that the site was founded by mathematicians, is fairly formidable) concludes that a bigger variety of opinions within a party renders it increasingly powerless as they try to cater to their internal political spectrum. This is bad news for the Democrats, as their research shows that there is a large chunk of the party who self-identify as economically and socially conservative. Compare this to the Republicans, who almost completely into the socially conservative, economically liberal category. The study concludes:

Unlike in many things, size here is a liability. Yes, a political party that’s this wide-open is probably a more intellectually stimulating organization to be a part of, and it has a lot more potential power. But bigger base is also just that many more competing viewpoints Democratic politicians must cater to and that many more different viewpoints in play among the actual elected officials themselves.

The introduction of a dating website to the statistics community sounds like another nail in the coffin of objective research. OK Cupid has no responsibility to be objective. The New York Times reports that OK Cupid has about 1.1 million active users, but for the purposes of this survey, the website’s sample size is 172,853 people. Because it has a potential data set of one million people, it could pick and choose subjects to generate the results their agenda require in a much simpler fashion than, say, Daily Kos pollsters could. Not to mention that the group is self-selecting. The type of person that would have a profile on a dating site is probably a bit more socially liberal than the average.

On the other hand, their sample size is two orders of magnitude more than the 4-digit numbers common at organizations like Gallup and Rasmussen. That’s about an extra 99,000 people.

Not that OK Cupid is the new Gallup, but its foray into political research presents an exciting possibility for the future of public polling and statistics. Social networking websites have a built-in survey audience answering potential questions voluntarily– imagine the power of harnessing that through a website that can escape the potential of being self-selecting more than OK Cupid, like Facebook. Facebook already lets people identify politically and list interests– information that could very well give an accurate pulse of the nation– and boasts 400 million users around the world. Of course, the issue with politics on Facebook is that it’s possible to put almost anything in the “political views” box with impunity (for example, my current political views on Facebook are “Rod Blagojevich”), but with the audience there, the details of how to collect accurate information– via radio buttons and not open-ended categories, for example– can be worked out later.

According to a recent Rasmussen poll only 43% of people polled agreed with the assessment that global warming was linked to extreme weather, down from 55% in 2008. Meaning, among other things, the country is evenly split between those folks who think extreme weather is part of a natural cycle (or maybe God really is “just hugging us closer!”) and the people who believe global warming is the reason much of the East Coast is in a deep freeze. Perhaps for the sake of a peaceful holiday dinner table, 14% of the people surveyed did not have an opinion on the matter. Rasmussen has put together a nifty little video on the poll results, below. Should you prefer the Hollywood version, please scroll down.

According to a Politico feature, Scott Rasmussen‘s polling firm Rasmussen Reports has come under increasing fire from liberal bloggers and pundits, many of whom see it as a conservative front and talking point factory. Surprisingly, liberal polling and statistics maven Nate Silver came to Rasmussen’s (qualified) defense:

According to Silver, Rasmussen’s “lean” towards Republicans is a defensible sign of polling methodology, and their election polling “has tended to be quite accurate in the past.” Some of the concerns raised by Rasmussen’s liberal critics in the Politico article — for instance, that Rasmussen polls tend to assume smaller numbers of young and minority voters than other polls based on their “likely voter” model — could ring hollow if Rasmussen’s older, less diverse snapshot of the electorate turns out to be accurate in the 2010 midterm elections.

Of greater concern, says Silver, is Rasmussen’s subject matter – ”they have a knack for issuing polls at times which tend to dovetail with conservative media narratives” — and wording:

I’m not saying that Rasmussen’s question wording is always biased. It isn’t. And I’m sure you could find a couple of cases where the wording tend to portray the liberal argument more favorably. But cases like these happen consistently enough with Rasmussen that I’d say it’s a concern. And when they do use unorthodox question wording, nine times out of ten it favors the conservative argument.

In a blogosphere obsessed with black-and-white dichotomies and competitions towards hyperbole, it’s refreshing to read nuanced, decidedly non-shouty takes like Silver’s.

Also: here is a video, circa July, of Jimmy Fallon “slow jamming” Rasmussen Reports and calling them an “outlier;” this apparently amused Scott Rasmussen.

(h/t FiveThirtyEight. image via Washington Independent)

Last night on The Daily Show, Jon Stewart highlighted a graphic on Fox & Friends relating to global warming and, loosely, Climategate. But the numbers didn’t add up – or did they?

This was first brought to light by Media Matters yesterday. The image in question is a full-screen graphic of Rasmussen poll results on Fox & Friends December 4. It seems to show that 59% of respondents said it is “somewhat likely” that “scientists falsify research to support their own theories on Global Warming,” 35% say “very likely” and 26% say “not very likely.” Here’s exactly what co-host Steve Doocy said while reading the results: “About 60 percent of you say somewhat likely, 35 percent say very likely, so you got 90…you got a lot of people right there thinking…it is likely, although 26 percent say not very likely.”

As Stewart says: “So attacking scientists for falsifying data to support their theories on global warming, you’ve cited a poll that adds up to 120%.”

So it’s a mistake – the numbers add up to 120, let’s move on. Except Fox News is defending it. Lauren Petterson, executive producer of Fox & Friends, tells Politico “she sees no error in the graphic.” Petterson says, “We were just talking about three interesting pieces of information from Rasmussen. We didn’t put on the screen that it added up to 100 percent.”

The problem with this explanation is it ignores that the poll actually gave those answers as possibly responses, and, obviously, the numbers were different than F&F showed. As TVNewser points out, here are the actual results:

35% Very likely
24% Somewhat likely
21% Not very likely
5% Not at all likely
15% Not sure

Those numbers are completely different. The “somewhat likely” figure is actually just 24% while the “not very likely” number is 21%. Instead, someone decided to combine the somewhat and very likely percentage and conclude that equals “somewhat likely” and add up “not very” and “not at all” to get the “not very likely” figure. Why? Cynics would say it makes it look like more people believe “scientists falsify research to support their own theories.” Or you could chalk it up to a simple mistake. But it’s strange to see an outright defense of such a misleading graphic. (Petterson told Politico: “The mistake I do see is we could have been a little clearer here.”)

The rest of the Daily Show segment was spent with Stewart “exposing” that Fox & Friends co-host Gretchen Carlson is actually really smart – which is not always on full display on the show. “From now on I want to see you give it 120%,” he joked.

Here’s The Daily Show last night:

—–
» Follow Steve Krakauer on Twitter

Luckily for hypothetical Barack Obama, if the election were held today, somebody would have to get 45% of the vote in order to beat him.

While a poll like this unquestionably bespeaks a deep dissatisfaction among Americans, it doesn’t tell the whole story. According to a recent NBC/Wall Street Journal poll, not only is there a lot more of this sandwich to go around, President Obama gets a relatively small bite of it.

That poll pegs Republican Party favorability at 25% versus 46% unfavorable, while the Democrats are at 42% favorable, 36% unfavorable. On health care reform, 64% disapprove of the job Republicans in Congress are doing. By a 46%-38% margin, voters hope the Democrats retain control of congress in 2010.

The Republican Party’s most visible star, Sarah Palin, has a 27% approval rating in this poll, enough to eke past Nancy Pelosi. Even in a CNN poll that has her at 44% approval, 71% say she’s not qualified to be president.

What this poll really means is that the country is messed up, and people want it to get fixed more quickly than it is, as quickly as they hoped it would. President Obama is only losing to candidate Obama. The thing is, problems like the economy and the wars in Iraq and Afghanistan were never supposed to be solved in nine months.

The Republican strategy of trying to win every news cycle may be taking a toll on Obama now, but they need to offer Americans something they like better. They’re certainly not doing that now.

The Rasmussen poll finds that “45% of adults say they would be at least somewhat likely to vote for Obama if he was up for reelection right now. Forty-nine percent (49%) say they would be unlikely to vote for the president’s reelection.” Ouch. Side note: more women like him than men.

Of course, it should go without saying that a poll like this is held in a vacuum, which does not include real world contributing factors like opposing candidates or running mates. Not to mention the next chance voters will get to check the President’s name off on a ballot is still three years off — needless to say, a lot can change in three years! — but still, probably not exactly the way Obama wanted to arrive at election day 2009!

For her part, Arianna Huffington, whose coverage of Obama during the campaign was generally, shall we say, positive, also appears to have decamped into the less-than-thrilled party. Huffington has penned a new post titled ‘The Audacity of Winning vs. The Timidity of Governing.’ From the post:

How did the candidate who got into the race because he’d decided that “the core leadership had turned rotten” and that “the people were getting hosed” become the president who has decided that the American people can only have as much change as Olympia Snowe will allow?

How did the candidate who told a stadium of supporters in Denver that “the greatest risk we can take is to try the same old politics with the same old players and expect a different result” become the president who has surrounded himself with the same old players trying the same old politics, expecting a different result?

The audacity of reality? No doubt the difference the next 36 months can make will help people make up their minds on that one.

Further proof: 68% say they paid close attention the Balloon Boy story.

Additionally, the majority of people thought too much attention has been paid to both Michael Jackson’s death, and the Obama’s personal life. So apparently most of the country would actually prefer to be watching PBS Newshour but has yet to fully confront the addiction and can’t quite get off Fox, et. al.

According to a recent Rasmussen poll, there’s a strong connection between individuals’ political leanings and their attitudes towards the media overall. “While 58% of Democrats view journalists favorably, 70% of Republicans and 55% of unaffiliateds do not.”

Overall, 43% of Americans have a favorable view of journalists, versus 54% who don’t — although twice as many have “very favorable” views of them as “very unfavorable.”

One curious aspect of these numbers is that by some metrics, conservative anchors and talkers are the media right now: recently, Fox News topped CNN, HLN, and MSNBC combined in primetime viewership, and the likes of Air America can’t compete with the number of listeners drummed up by conservative talk radio.

Rather than pontificate about where exactly the poll results are coming from, we’re opening it up to you. What do you think these numbers mean? What do Democrats value about journalists that Republicans don’t? Does the media have a liberal or conservative bias, or neither? And how do you explain the Republican media disapproval/Fox News paradox?