[ 4 minute read ]

While data collection is increasingly being done automatically for website traffic (e.g. Google Analytics), social media engagement, and click-throughs via link trackers, there is often a need to collect data manually. One way to bridge the divide between "big data" and more personalized information gathering such as interviews, is to conduct online surveys. In this first part on best practices for survey design, we focus on defining the goal of the survey as well as how to develop questions. These considerations are important for maximizing survey responses, reducing bias in answers, and maximizing actionable insights.

1. Defining the Goal of the Survey

The results of a survey should be used to develop insights that will help your non-profit organization improve in a specific way. If you have more than one issue to improve upon, choose the one that’s most urgent or top of mind. For example, “how do we raise more funds from annual donors?”or “how do we recruit more volunteers from high schools?” Surveys can also be used to determine how well your organization is doing in a specific area. For example, “how well is our new outreach program working?” or “how satisfied are our employees in the workplace?”

Once you’ve developed the goal of the survey, this will also indicate the target audience. The target audience is often donors (potential and/or existing), but can also include volunteers, employees, clients, partners, or the general community.

2. Considerations for Developing Questions

With the goal of the survey and its target audience defined, we can begin designing the survey. For illustration purposes throughout this article, let’s use “improving donor satisfaction” as our goal with “our existing donor base” as the target audience.

a. Types of Questions

There are four main types of questions in a survey:

• perceptions, e.g. How friendly are our staff members?
• behaviours, e.g. How many organizations do you donate to annually?
• future behaviours, e.g. How likely are you to join our monthly giving program?
• demographics, e.g. What age group do you belong to?

Not every survey has to have all four types of questions, however, demographics should likely appear every time. This way, you can determine if there are differences between groups (e.g. people in 20s vs. people in 30s) or if there are general trends within groups. In turn, this information will help you serve a particular stakeholder group more effectively.

• Write questions that are clear, precise, and relatively short

This will ensure that the respondent is able to answer your questions confidently and without misinterpretation

• Do not use “loaded” or “leading” questions

For example, “don’t you agree that we provide great member services?”, is a leading question. This question is worded as to bias the respondent into answering in a way that favors perception of the member services. It would be better worded as “how satisfied are you with our member services?”

• Do not use double-barreled questions

An example of a double-barreled question is, “do you want us to phone and e-mail you more often?” Although the answers can be structured to reflect all the possible combinations, this type of wording can still be confusing both to the respondent and the one analyzing the answers. Perhaps the respondent wants to be phoned more often but not e-mailed, for example. Therefore, to avoid confusion, separate the above example question into two questions: “do you want us to phone you more often?” and “do you want us to e-mail you more often?”

• Avoid double negatives

For example, “do you agree or disagree that we shouldn’t offer more perks for donors?” The “shouldn’t” is a negative that makes it unnecessarily more difficult for the respondent to process the question. Therefore, replace the above with “do you agree or disagree that we offer more perks for donors?”

In our next blog article, we explore how to write effective answers, which is just as important at effective question design to reduce bias and maximize actionable insights. We also explore general survey design considerations, deploying the survey, and how to maximize return rates.

In the meanwhile, do you have any questions on the above tips? Comment below or e-mail us!