Learning the Craft of Surveys
- Learn the craft. Survey-building is a craft, so study up on survey design. Luckily for you, there’s a free Coursera course on Questionnaire Design that started on June 1, 2015. I can attest that the lectures are useful.
- Don’t be afraid to start small and develop more nuanced surveys over time. You’ll learn what sorts of questions and approaches actually work for you.
- Consider representative, quota, or cluster sampling rather than trying to get responses from everyone. Don’t know what that is? Take Solid Science: Research Methods for free on Coursera, starting this August 31, 2015. It’s well worth it for library research.
- Why do this? Nobody wants to take surveys, unless they’re underemployed; is there a compelling reason they should take yours? Is the benefit really worth the collective time?
- Keep it short.
- Seriously, the best way to get responses is to keep it to 4-5 focused questions.
- A 20-30% response rate is good, especially if you don’t offer prizes. The more focused you can make the invitation look, the better your results may be.
Keep it Useful
- Mix it Up. Don’t just ask the same questions over and over in a yearly survey. Unless your survey is well-designed by a social scientist, in line with the library’s strategic plan, and you have the tools to analyze longitudinal data, you’re not going to make good use of the data.
- Don’t duplicate. Don’t collect data you could collect elsewhere (usage stats, gate counts). If you see that some questions don’t change much year to year, consider rotating questions in and out so that the survey stays both short and informative.
- Always run a pilot: check your question wording with a few trial respondents before sending the whole thing out. Feel free to change or eliminate questions that aren’t returning useful answers.
- Surveys get stronger if multiple institutions or social scientists do them together.
Good Survey Design
- Important stuff first. Put demographic questions on the last page; getting topical the topic is usually more important.
- Get partial data. Even when you put the important things first, unfinished surveys are normal. Choose a survey program that captures partially-completed pages. SurveyMonkey doesn’t return page results until respondents click ‘next’, while SurveyGizmo seems to capture even partial pages.
- Consider your pages. The fewer pages you have, the more likely people will complete the survey. But if one page has too many questions, they may also stop. It’s a balancing act!
- Stay phone-savvy. Check how easy the survey is for smartphone users. I learned the hard way that a long survey may scare mobile users away.
- Get IRB Review. If you plan to publish or present results as ‘scientific research,’ submit the survey to your campus IRB board. An anonymous survey may be judged as ‘exempt’ from further review, but at least you’ve had the IRB take a look.
- And/or, respect ethical principles. Often customer surveys, usability studies, educational surveys, or personal surveys of friends online don’t require an ethics review. But it’s good to live by theBelmont Principles anyway: design surveys that respect individuals, are just, do no harm, and benefit others.
- Even if there is no ethics review required, maximize the benefit and minimize the harm.
- Don’t collect identifying data. Google Apps and Qualtrics let you extract usernames or demographic data from campus accounts—but that’s likely a violation of privacy. Don’t collect data that could be leaked, and safeguard the data you do collect.
Survey Analysis and Results
- Have a goal. As my colleague Amanda Rinehart has recommended: a library survey is strongest if you can map each question to a specific hypothesis. Don’t just throw questions into the dark; instead, make sure you can act on the answers to each question you ask.
- Think before questioning. If analyzing by gender, race, or age isn’t useful, don’t ask those questions. Keep questions closely tied to your hypothesis or survey goal, as you can always survey a different subset of users later.
- Show the value. Value the time others put into your surveys; make sure you do something for users with the results, and make the link clear!
Any other suggestions? Add them to the comments below, or share your thoughts on these tips over at LITABlog!