Statistics Essentials For Dummies (50 page)

Although the pairs of probabilities being compared are different for the two methods of checking for independence, the overall conclusions should always agree. In this example, your suspicions were right — no matter how you slice it, your player is more likely to make the second shot when he misses the first than when he makes the first.

Chapter 12
:
A Checklist for Samples and Surveys

In This Chapter

Defining and getting good samples from the target population

Crafting and administering good surveys

Making appropriate conclusions

Surveys are all around you — I guarantee that at some point in your life, you'll be asked to complete a survey. You're also likely to be inundated with the results of surveys, and before you consume their information, you need to evaluate whether they were properly designed. In this chapter, I present a checklist you can use to evaluate or plan a survey.

The survey process can be broken down into a series of ten elements that should be checked:

1. Target population is well defined.

2. Sample matches the target population.

3. Sample is randomly selected.

4. Sample size is large enough.

5. Nonresponse is minimized.

6. Type of survey is appropriate.

7. Questions are well worded.

8. Survey is properly timed.

9. Personnel are well trained.

10. Proper conclusions are made.

This list helps you carry out your own survey or critique someone else's survey. In the following sections I address each item and discuss its role in getting a good survey done.

The Target Population Is Well Defined

The
target population
is the entire group of individuals that you're interested in studying. For example, suppose you want to know what the people in Great Britain think of reality TV. The target population is all the residents of Great Britain.

Many researchers don't do a good job of defining their target populations clearly. For example, if the American Egg Board wants to say "Eggs are good for you!" it needs to specify who the "you" is. For example, is the Egg Board prepared to say that eggs are good for people who have high cholesterol? What if one of the studies the group cites is based only on young people who are healthy and eating low-fat diets — is that who they mean by "you"?

If the target population isn't well defined, the survey results are likely to be biased. The sample that's actually studied may contain people outside the intended population, or the survey may exclude people who should have been included.

The Sample Matches the Target Population

When you're conducting a survey, you typically can't ask every single member of the target population to provide the information you're looking for. The best you can do is select a good
sample
(a subset of individuals from the population) and get the information from them. A good sample
represents
the target population. The sample doesn't systematically favor certain groups within the target population, and it doesn't systematically exclude certain people, either.

The best scenario for selecting a representative sample is to obtain a
sampling frame
— a list of all the members of the target population — and draw randomly from that. If such a list isn't possible, you need some mechanism that gives everyone in the population an equal opportunity to be chosen to participate in the survey. For example, if a house-to-house survey of a city is needed, an updated map including all houses in that city should be used as the sampling frame.

The Sample Is Randomly Selected

An important feature of a good study is that the sample is randomly selected from the target population. Randomly means that every member of the target population has an equal chance of being included in the sample. In other words, the process you use for selecting your sample can't be biased.

The biggest problem to watch for is convenience samples. A
convenience sample
is a sample selected in a way that's easiest on the researcher — for example call-in polls, man-on-the-street surveys, or Internet surveys. Convenience samples are totally nonrandom, and their results are not credible.

For surveys involving people, reputable polling organizations such as the Gallup Organization use a random digit dialing procedure to telephone the members of their sample. This excludes people without phones, of course, so this kind of survey does have a bit of bias. In this case, though, most people do have phones (over 95%, according to the Gallup Organization), so the bias against people who don't have phones is not a big problem.

The Sample Size Is Large Enough

You've heard the saying, "Less is more"? With surveys, the saying is, "Less good information is better than more bad information, but more good information is better."

If you have a large sample size, and the sample is representative of the target population (meaning randomly selected), you can count on that information to be pretty accurate. Exactly how accurate depends on the sample size, but in general a bigger sample leads to more accurate information (assuming the data is well collected).

A quick and dirty formula to calculate the accuracy of a survey is to divide by the square root of the sample size. For example, a survey of 1,000 (randomly selected) people is

accurate to within
, which is 0.032 or 3.2%. This per-

centage is called the
margin of error
. (Note that this formula is just a
rough
estimate. A better estimate can be found using the formulas from Chapter 7.)