Question marksIn both usability testing and user testing, you want to learn more about what a person is thinking. While watching them perform the tasks that you’ve specified can offer unique insights, so does the ability to find out the ‘why’ behind those actions. By using the right usability and user testing questions, you can get more insights for every test that you run.

Best practices

Before we focus on the actual questions you can use in your testing sessions, I want to outline some general best practices for user testing questions. They can help you to get more details answers from your user testers.

Ask open-ended questions

Can this page be improved?
How would you improve this page?

Closed-ending questions tend to get very short (often yes/no) responses. These answers aren’t particularly useful in a usability testing context because we want to find out the ‘why’ behind the actions of a tester. Open-ended questions, on the other hand, do often offer such insights, as the invite the tester to elaborate on his or her answer.

Ask neutral questions

How frustrating did you find that widget?
What did you think about that widget?

When you’re asking a tester a leading question, it will subtly push him/her towards a particular answer. Thoughts like “Am I supposed to find this widget frustrating?” will start popping up and those can skew your results. By asking your questions in a neutral fashion, you’re leaving it up to them whether or not they find a particular element frustrating, enjoyable or just dull.

Don’t ask for their intent

Would you buy this product?
Observe their behavior

Usability tests often contain questions that ask for a user’s intent. These questions often start with “Would you …”. Given that ‘intending to do something’ and ‘actually doing something’ is not quite the same thing, it makes little sense to ask your users such questions. Instead, you should try to focus on observing their actual behavior, and be asking them follow-up questions about this afterward.

Run a pilot test

Determining which questions will work best for a usability test can be a complicated task. Particularly when you dealing with a target audience that you’re not intimately familiar with.

Given that a set of 10 remote unmoderated user testing videos will often cost your upwards of $500, you might want to consider running a so-called pilot test. In such a test you can find out whether users understand the tasks and questions that you’ve set out for them. Try putting the user test out there for only a handful of users first (and check the videos to see if they got it all) before launching your full-blown campaign only to find out that nobody understood a particular question.

Amy SchadeAmy Schade of NN Group:
Pilot testing (a session or two before the real test) helps fine-tune usability studies, leading to more reliable results. It provides an opportunity to validate the wording of the tasks, understand the time necessary for the session, and, if all goes well, may even supply an additional data point for your study.

Example questions

At the core, UX is about ensuring that users find the maximum value in whatever it is that you’re providing to them. Peter Moville visualized this succinctly in his User Experience Honeycomb.


Useful

Why do you think someone would use this product or service?

Responses to this question could allow you to find new ways that customers are using your products. It could help you find out new keywords to target or dedicated landing pages to set up.

What are the three things you like best about the page?

Collecting user feedback can be an efficient way to do competitive research or find out what works well in other parts of your website. By asking for three things instead of one, you increase the chance of getting valuable new input from this question.

What are the three things you like least about the page?

Elements of the page that are mentioned here could be excellent input for A/B tests. What happens with your conversion rates if you remove these elements or change them significantly?


Usable

How would you improve the page?

This is perhaps one of the most valuable questions in the set. It often results in a broad set of responses, ranging from feedback on the value proposition to specific tips on how to improve the call-to-action buttons.

How could we make this page easier to use?

Other blogs about usability testing questions often phrase this question like “Overall, how easy to use do you find this page?”. Unfortunately, that question often results in short replies like ‘Easy’. By phrasing the question differently, you increase the chances of receiving more detailed responses.

What should be added or removed to make this page more clear?

By specifically asking for elements that should be added or removed you are guiding your testers towards certain answers. Often, this question is used when looking for new ideas on what to A/B test on a website.


Desirable

What could help persuade you to buy this product or service?

By posing the question like this, you get people to actually considering buying the product or service. Answers often include specific ways to help persuade your visitors, which might help to improve, for instance, your landing pages.

How could we improve the overall appearance of this page?

Had the question been phrased as “What do you think of the visual design of this page?” that would likely have resulted in short and subjective responses. By phrasing it like this, you still allow your testers to tell you what they think about the design but guide them to phrase the feedback in a more constructive way.

What do you think of the name, appearance, and branding of this page?

This question might not be particularly useful to established business. However, it can be a great way to help startups find out how their target audience perceives these aspects of their organization.


Findable

What confused you most about this page? Why?

By asking specifically for confusion elements, you’re increasing your chances to find elements that impact the bounce rate (for instance, of a landing page). If you learn what confuses visitors, and later remove that, you are likely to decrease the bounce rate of that page.

Is there anything missing on this page?

While it seems that most pages have too much content, they sometimes tend to be missing important information. This question can help you uncover these missing elements.

How intuitive and helpful is the navigation system?

On most websites, the navigation system is the primary way of getting from one page to another. If for whatever reason, the navigation system isn’t intuitive or helpful, that would greatly harm the chances of that page to convert a visitor to a customer.

How well organized did you find the page?

A page that has content all over the place will be hard to scan. Such disorganization can lead to high bounce rates and low conversion rates.


Accessible

What might keep people from using this product or service?

Lightbulb new ideasFinding out what could prevent people from using your product or service can be a great inspiration for content marketing or FAQ items. Assuming your product or service is, in fact, able to help these people too, you could talk about that on the page.

How easy to read did you find the page?

Text that is hard to read is unlikely to convert any visitors. Common issues here included small font sizes and low contrast between the background and the text.

What did you think about the color scheme that was used?

Using an unattractive color scheme can really harm the first impression that visitors have of a page. Furthermore, it might hinder people suffering from color blindness in their ability to understand or use the page properly.


Credible

Does the page look trustworthy or not? Why?

When a page doesn’t look trustworthy, that will certainly hinder its ability to convince a visitor to become a customer. Common feedback includes having a bad design or lacking an SSL certificate to secure the page.

What’s your first impression of this page?

The first impression of a page is critical, as the next page is only one click away. By improving your first impression, you’re positively impacting your SEO rankings, Quality Score, and bounce rate.

What would make you trust this site more?

By specifically asking for improvements, you might stumble upon aspects of the page that you hadn’t uncovered via other questions.

Peter Coville of Semantic Studios:
The honeycomb hits the sweet spot by serving several purposes at once. First, it’s a great tool for advancing the conversation beyond usability and for helping people understand the need to define priorities. Is it more important for your web site to be desirable or accessible? How about usable or credible?


Valuable

What do you think this site offers?

This is another question that can really help you to reduce your bounce rate. If your user testing shows that visitors are having trouble determining what it is that you are offering, that is a major red flag.

How do you think this product or service is going to help you?

With this question, you are having the user testers focus on the benefits of the products, instead of the features. The feedback you get from this user testing question might come in handy when crafting new USPs or improving your value proposition.

Would you use this product or service today?

This one gets your testers in the here and now. Instead of asking them whether or not they might use this at some point in time, they now have to think about ways to actually use this product right now.

Who do you think the intended audience is?

Sometimes, a website or landing page just seems targeted to the wrong audience. This issue can sometimes be difficult to spot without external user feedback because the people around (and your customers) are often well within the intended target audience.

What would entice you to return to this page?

Often, returning to a page can be an important sign that it hit the right spot. By asking your respondents what would get them to return to this page, they might point out important flaws that are preventing them from doing so at this moment.


Conclusion

We’ve listed several best practices on usability testing questions. If, in addition to using these best practices, you’ll also select some of the questions that fit your goals best, odds that this will improve the responses that you’ll receive during your sessions. The responses will likely be more detailed and contain more valuable UX insights for your clients, which is, of course, the end goal.