Determining the minimum number of respondents needed for a survey is an important part of the survey design process. The number of respondents directly impacts the reliability and validity of the survey results. With too few respondents, the results may not be an accurate representation of the target population. With too many respondents, time and resources may be wasted collecting more data than is needed.
Quick Answers
Here are some quick answers to common questions about minimum survey sample size:
- There is no universal minimum – it depends on factors like population size, margin of error, confidence level, and response rate.
- As a general rule of thumb, most experts recommend a minimum of 100 respondents for surveys of the general population.
- For smaller or more specialized populations, as few as 50 respondents may be sufficient if there is little diversity in the population.
- For populations over 1 million, 384 respondents is considered the minimum to achieve a 95% confidence level with a 5% margin of error.
- Online surveys typically need more respondents than other methods because of lower response rates.
What Factors Determine Minimum Sample Size?
There are four key factors that determine the minimum number of survey respondents needed:
- Population Size – The total number of people in the group you want to survey. The larger the population, the more respondents you need.
- Margin of Error – The amount of potential error you are willing to accept. Typically this is 5% or less.
- Confidence Level – The level of confidence you want in the survey results, typically 90% or 95%.
- Response Rate – The estimated percentage of people who will complete the survey. Anticipated response rate affects sample size needs.
Statisticians have developed formulas that take these factors into account when calculating the ideal survey sample size. We’ll look at some of these next.
Sample Size Formulas and Calculators
While general rules of thumb can provide a starting point, statistical formulas give a more precise way to determine ideal survey sample size. They incorporate the key factors listed above.
Here are some commonly used sample size formulas:
- General Formula: n = z2 * (p) * (1-p) / c2
- For Populations Over 10,000: n = z2 * p(1-p) / e2
- For Populations Under 10,000: n = z2 * p(1-p) * N / e2 * (N-1) + z2 * p(1-p)
In these formulas, z represents the z-score based on the confidence level (1.645 for 90% confidence, 1.96 for 95% confidence), p is the estimated proportion of the population with the attribute in question, e is the desired margin of error, and N is the total population size.
There are also a number of easy-to-use sample size calculators available online that do the math for you once you input the necessary factors. Some popular calculators include:
Rules of Thumb for Minimum Sample Size
While formulas and calculators provide the most accurate way to determine sample size needs, general guidelines can also be useful. Here are some common rules of thumb:
- For small populations under 100, survey the entire population.
- For populations under 1000, a minimum sample of 50 may suffice.
- For populations under 5000, a minimum sample of 100 is recommended.
- For populations over 5000, minimum samples of 200-500 are recommended.
- For general populations over 1 million, 384 is considered the minimum sample.
- For online surveys, calculate for double the sample size needed for other methods to account for lower response rates.
It’s important to note that these general rules should be used as starting points. The ideal sample size still depends on your specific population, desired margin of error, and other factors.
How Large of a Sample Size is Needed for Different Margins of Error?
The margin of error is one of the key factors determining sample size needs. Here’s an overview of approximate sample sizes needed for different margins of error:
| Margin of Error | Sample Size Needed |
|---|---|
| ±1% | Over 10,000 |
| ±2% | Around 4,000 |
| ±3% | About 1,100 |
| ±4% | Around 600 |
| ±5% | Roughly 400 |
| ±6% | Approximately 267 |
| ±7% | Around 204 |
| ±8% | About 174 |
| ±9% | Around 158 |
| ±10% | Roughly 138 |
These sample sizes are based on a 95% confidence level for general populations over 10,000. As you can see, wider margins of error require smaller sample sizes, while tighter margins of error require very large samples.
How Does Population Diversity Impact Minimum Sample Size?
The more diverse or varied the target population is, the larger the sample size needs to be to achieve representative results. Key factors that increase diversity include:
- Larger population size
- Wider geographic distribution
- More demographic segments (age, gender, income, etc)
- More diverse opinions, behaviors, preferences, etc.
When there is a high degree of diversity, a larger sample is needed to capture all the relevant subgroups and variability. For a homogeneous population, a smaller sample may be sufficient.
For example, surveying high school students in a single district may require fewer respondents than a survey of all US teens across geographic regions, ethnicities, public/private schools, etc.
How Does the Method of Data Collection Impact Minimum Sample Size?
Survey delivery method can significantly impact the number of completed responses. Therefore, it should be considered when determining minimum sample size needs.
Some key differences in response rates by survey method:
- In-person surveys: 50-70% response rate
- Paper or mail surveys: 10-30% response rate
- Phone surveys: 10-20% response rate
- Online surveys: 10-15% response rate
- Email surveys: 1-5% response rate
Since online, email, and mail surveys have much lower response rates, the initial sample size needs to be substantially larger to account for non-responses. Phone and in-person methods allow for smaller sample sizes.
Does Sample Size Vary by Question Type?
In most cases, all the questions in a survey can use the same sample size determination. However, if certain questions relate to a subset of the population with unique characteristics, they may require a specific minimum sample of that subgroup.
For example, a survey going to all adults could ask “Have you ever served in the military?” That subset question would need a separate minimum sample of military veterans specifically to draw conclusions about that group.
Multi-part surveys with screening questions can also require adjustment to minimums for different sections. Online survey tools make it easy to adjust sample sizes as needed for different segments.
How Can I Ensure Reaching Minimum Sample Size?
Once you have determined the ideal minimum sample size, there are several techniques you can use to ensure reaching it:
- Over-sample – Invite more participants than the minimum needed to account for non-responses. The over-sample rate should be based on typical response rates for your survey method.
- Follow up – Send email/phone reminders to increase completed responses from initial non-responders.
- Extend field period – Keep the survey open longer to allow more time to reach the minimum sample.
- Offer incentives – Providing rewards for participating can boost response rates.
- Keep questions short – Fewer and simpler questions reduce survey abandonment.
Careful tracking of completed responses vs. minimums needed allows you to gauge whether these steps are required to hit your target sample size.
What are the Consequences of a Sample Size That is Too Small?
Surveys with inadequate sample sizes are prone to significant errors and limitations:
- Results may not be representative of the population as a whole.
- There is a wider margin of error around estimates.
- It is easier for bias to skew results.
- There is reduced ability to analyze subgroups or segments.
- Results lack statistical power and significance.
- Findings may be dismissed as unreliable or invalid.
- Groups with small representation may be ignored.
- Patterns and correlations are harder to identify.
The negative consequences underscore the importance of determining an appropriate minimum sample size for reliable, accurate survey results.
When Might a Small Sample Size be Sufficient?
In certain cases, a survey with a relatively small number of respondents may be acceptable:
- Pilot studies or preliminary research where high precision is not needed.
- When the full target population is constrained in size.
- For internal use and informal feedback collection.
- If the population is very homogeneous with little variability.
- To study issues where incidence rates are expected to be high.
- When additional qualitative data will supplement the survey results.
The key is being aware of the limitations and not over-generalizing or exaggerating findings based on small samples. Disclose the sample size and margin of error when reporting results.
Conclusion
Determining survey sample size requirements takes careful consideration of the target population characteristics, research objectives, desired precision, available resources, and survey method limitations. While gaining more responses is often better, response fatigue and other factors can diminish data quality if the sample size is too large.
Expert guidance recommends focusing first on obtaining an optimal minimum sample size. Statistical formulas, online calculators, and rules of thumb can help identify reasonable minimums. But researchers should also rely on their judgement of the specific situation and objectives when deciding on appropriate survey sample sizes.