User surveys are a quick and inexpensive way to test the user experience of a product, as long as the questions are carefully crafted. Keep in mind that surveys measure what a user thinks, not what they do. Be careful to avoid bias as much as you can. It is impossible to avoid bias entirely, but carefully designed questions help. A large sample size can help to overcome the effect of bias.
What Are User Surveys?
User surveys can give you a lot of useful information about users such as who they are, what they want, what they buy, where they buy it, what they think of the product, and problems they may have with it. Respondents can take surveys on paper, but most are done online.
The information from surveys can help you understand what the user wants or needs, but it can also help your team avoid designing the wrong product. Survey data can also give stakeholders confidence that the design is moving in the right direction.
Types of Data
There are two types of data available from user research, quantitative and qualitative. Quantitative data gives you numbers and is more objective, but less detailed. Qualitative data is more subjective and gets to the beliefs, experiences, and feelings of the user, but is more complicated and more expensive to analyze.
Designing the Survey
Before you begin to write questions, be very clear about what you are testing. Develop research questions to use as a framework to design the survey. The type of questions you use in your survey depends upon the data you are looking for. A mix of quantitative and qualitative data will give you a good picture of user opinions.
The following are some general guidelines for writing survey questions:
- Questions should follow a logical flow with questions on a similar topic grouped together.
- Stick with neutral questions to avoid leading the user in one direction or the other.
- Use easy and simple questions with clear language, no jargon, and no abbreviations.
- Use a funnel structure for your survey. Start off with general questions, put more complex questions in the middle of the survey, and then more general questions at the end.
- Respect the anonymity of users. Find out how much personal information you may share.
- Screen the audience with a few introductory questions to make sure you get the respondents you want. For example, if your product is for parents, you want to screen out people who don’t have children.
- Let users skip questions that don’t apply to them and include a don’t know option. Keep it short to avoid overwhelming users.
- Show the user how far they’ve gone and how much more they have left to do.
Writing the Questions
The questions you write depend upon the kind of data you want. If you are looking for
Quantitative data, use closed-ended questions like checkboxes, yes/no answers, multiple-choice, and Likert scale. Closed-ended responses have higher response rates, but there is a danger of leaving out useful, detailed responses. Open-ended questions yield qualitative data, with more context and description, like short essay questions. Open-ended questions take longer to analyze making them more expensive.
Some other things to keep in mind include:
- Use only one concept per question. It is impossible to separate the two parts of the question if you do.
- Avoid leading questions or priming respondents in preceding questions.
- Be careful not to introduce bias into the survey.
The Problem of Bias
Beyond the constraints of time and money, bias is the biggest problem with research. Bias means favoring or being prejudiced against something based on personal attitudes. Bias is usually based on limited information and is often unconscious. One important thing to keep in mind about bias is that it is almost universal and UX Researchers need to be constantly checking for it in their research designs.
Kinds of Bias
There are many different kinds of bias that can affect research, but one way to separate them is into researcher bias and participant bias. Researcher bias refers to attitudes and assumptions of the researcher that can influence the behaviors of test subjects and affect results. Participant bias is the attitudes and beliefs of the test subjects that can give false results.
Researcher Bias
When researchers fail to examine their assumptions and beliefs before designing studies, it can lead to researcher or investigator bias. Here are some common forms that can take.
- Confirmation bias—Confirmation bias happens when you start out looking for evidence to prove an assumption instead of gathering evidence and forming a theory that reflects the data.
- Culture Bias—The researcher interprets results according to their own cultural attitudes.
- False consensus bias—False consensus bias is the assumption that others think the same way you do or the assumption that anyone who doesn’t agree with you is abnormal. It is easy to overestimate the number of people who agree with an idea or design feature, so it is important to identify and articulate assumptions to check for false consensus bias.
- Primacy bias—Primacy bias is the tendency people have to remember the first participant better than the others.
- Recency bias—Recency bias is the tendency to remember the last thing you heard.
- Unconscious bias—Unconscious bias occurs when a UX Research’s personal prejudices and stereotypes impact the study. It becomes a problem when subjects within a limited set of identity profiles are chosen based on researcher assumptions. This can lead to a lack of representation in the subject and skew the results of the research.
- Availability bias—trying to fill the study quickly and not vetting the participants carefully enough. When this happens, you don’t get objective results.
- Wording bias or framing effect—researcher asks the question in a way that suggests an answer.
- Sunk cost fallacy—The sunk cost fallacy comes into play when time and money have been invested into a project and the team is resistant to changing course. It can be useful to break the project down into smaller phases and designate testing points along the way. This will help the team decide whether to continue or go back and change things based on new insights.
Participant Bias
Researchers need to be careful to construct their tests to produce good results. Here are some types of participant bias that can skew test results.
- Social desirability bias—participants answer what they think researchers want to hear rather than how they really feel. This happens when people want to be liked. They over-report socially desirable behaviors and characteristics and under-report undesirable behaviors and characteristics.
- The Hawthorne Effect—Participants are aware they are being observed, so they are extra careful and don’t necessarily demonstrate how they would actually use the product.
- Response bias -The tendency of participants to want to make themselves look good.
- Acquiescence bias—The tendency of participants to say yes.
Finding Participants
Once you have designed your survey, next you will need to find people to fill it out. Customer email lists and product registrations can be useful resources. Make it worth your respondents’ while to fill out your survey by giving them incentives like gift cards, discounts, early access to new features, or free trials.
Design your incentive program carefully because it can get expensive. It can also introduce bias because you might get more positive responses because of the incentives. Also, the incentive might attract respondents who are not representative of the users you are looking for.
Run Pilot Tests
Before you take your survey live, run a pilot test. You can change questions if you see problems. Ask respondents if any of the questions were unclear or there were any options missing.
Putting the Survey Together
It is very easy to create surveys with the many digital tools available. Some of the most popular are the following:
- Survey Monkey
- Google Forms
- Survey Gizmo
- Wufoo
You have to pay for most survey compiling tools. If you work in a large company, see if other teams already have a survey tool you could use that the company pays for.
Where to Learn UX Design
If you would like to learn more about UX design and research to switch to a new career, one of the best ways to do that is to sign up for classes. You can choose classes that meet in-person or online to learn design software and other applications. Some people prefer to attend brick-and-mortar sessions when learning new information, but that isn’t always available. Live online classes have a similar set-up with a real-time, remote instructor who can answer questions and take control of your monitor—with permission—to show you how to do things. Training is part or full-time and available weekdays, weeknights, or weekends.
The best way to prepare for a career shift to a field like UX design is to enroll in a bootcamp or certificate program. These are intensive training courses that run from a few weeks to a few months and another plus of training is that you will leave class with a professional-quality portfolio that you can show to prospective employers.