Bizzare MSNBC Poll

NBC-WSJ poll: New Clinton lows - First Read - msnbc.com

This poll was on Drudge with the heading: NBC-WSJ POLL: NEW CLINTON LOWS. The leader was that this new poll showed Hillary dropping in the polls, but the second para was a real winner: The poll was conducted Monday and Tuesday this week by Hart-McInturff and surveyed 700 registered voters, which gives the poll a margin of error of +/- 3.7 percent. In addition, we oversampled African-Americans in order to get a more reliable cross-tab on many of the questions we asked in this poll regarding Sen. Barack Obama's speech on race and overall response to last week's Rev. Jeremiah Wright dustup.

How can you advertise a poll as showing Hillary is slipping, but then say that the poll oversamples blacks? It goes without saying that a poll that oversmapled blacks would show Hillary with lower numbers. Also, this is the first time I have ever seen a poll where oversampling is clearly defined. You usually have to dig through the fineprint at the end of the poll to find out that Democrats were oversampled two to one to Republicans in one of MSNBC's polls.

Also, looking at the WSJ's analysis of the poll at Democrats Are Tied in New Poll - WSJ.com, there is no mention of oversampling blacks. Interesting.

[RadioPatrol]

seems a bit shady .....

[SamSpade]

No, it makes sense if you want a good cross-tab with a sample that small with regard to a smaller population with a reliably known smaller size.

For example, if you're going to split your questions down among three choices - say, which of the three candidates had the best chance to unite the country along racial lines - and THEN split again among black and white voters - in a sample as small as 700 - the number of regsitered black voters will trail off significantly.

This is because

a) Blacks comprise 12.5% of the population and
b) Are less likely to be registered voters than their white counterparts.

This means in the above question, it's possible that each slot might only carry one or two dozen black respondents.

It would be better to oversample, and then reweight it accordingly.

[ImnoMensa]

The way polls work.

First decide what answer you want
Then go where you will get that answer.

[SamSpade]

Originally Posted by ImnoMensa

The way polls work.

First decide what answer you want
Then go where you will get that answer.

I can undertstand why you might think that - but if you want a sample to fairly represent smaller sub-samples, you MUST over-sample first, and then correct it by reweighting your data afterwards. I do this all the time. In fact, I'm reweighting some data as we speak.

[PsyOps]

Originally Posted by SamSpade

I can undertstand why you might think that - but if you want a sample to fairly represent smaller sub-samples, you MUST over-sample first, and then correct it by reweighting your data afterwards. I do this all the time. In fact, I'm reweighting some data as we speak.

It would have made more sense to just take (as an example) two different polls; one of all black people and one of all white people, then reweight the numbers.

[SamSpade]

Originally Posted by PsyOps

It would have made more sense to just take (as an example) two different polls; one of all black people and one of all white people, then reweight the numbers.

We do stuff like that too. I don't see either way as being wrong.

[PsyOps]

Originally Posted by SamSpade

We do stuff like that too. I don't see either way as being wrong.

Well, I'm learning a little about polls that's for sure. It seems to me the numbers would be more accurate to poll in terms of percentages of populations and let the numbers speak for themselves. Or maybe I just don't understand reweighting.

[SamSpade]

Originally Posted by PsyOps

Well, I'm learning a little about polls that's for sure. It seems to me the numbers would be more accurate to poll in terms of percentages of populations and let the numbers speak for themselves. Or maybe I just don't understand reweighting.

You just scale your results accordingly.

Let's take a simple example. Let's say we had a sample of 300 voters and we wanted to know which candidate - of the three remaining - they favored. And then let's say we wanted to know how *Asian* Americans felt.

Well, they comprise about 4.4% of the population. Any RANDOM sample of 300 Americans is going to yield somewhere in the vicinity of 15 Asians, tops. Split that three ways in a cross tab, and you get a handful of Asians proportionally representing the nation. And that's WAY too small a sample. Suppose just 4 or 5 answer "I don't know". Isn't THAT going to blow your findings all to hell, especially if most Asians DO KNOW who they support.

So you oversample, and scale your results back in proportion to the known population. That way, instead of 5 "I don't knows" which comprise a third of your universe - maybe you'd get a smaller proportion, and that would be more accurate.

Most polling outlets do try to be fair, because they compete. But I always like to see methodology. Opinion polling is always the hardest, because you have to assume that people aren't lying - that the sample of people WHO COOPERATE still is representative of the whole - and that you haven't overlooked something really vital in your methodology. For example, overnight telephone polls tend to be VERY inaccurate, but ones conducted over several days starting with Thursday tend to be good - because lots of people are home on Thursday nights. Exit polling at polls can be poor because in many precincts, the bulk of the voting is done in the evening, and a sample of people who don't work during the day isn't representative of the whole population.

I've got no beef with oversampling, if they scale it accordingly.

Originally Posted by PsyOps

It would have made more sense to just take (as an example) two different polls; one of all black people and one of all white people, then reweight the numbers.

I agree. If what they were looking for is how blacks specifically feel about Wright, then poll just Blacks. It's not fair t oversample blacks and then use the poll to indicate Clinton is slipping.