Usability Testing: How to Quickly Find Issues & Fixes in Your Product
A Beginner's Guide
Thanks for reading. If you enjoy this piece, consider subscribing and sharing with your friends.
Have you ever looked at your metrics dashboard and seen something concerning? Yeah, me too. Let’s imagine you’ve narrowed down the possible culprits and one hypothesis you have is that your product or feature is confusing or designed in a way that causes friction for users to accomplish a task.
For example, if you observe people entering an onboarding flow, but not completing it and want to know why. Or, if you own an e-commerce site and see that your conversion rates are low during the checkout process despite having free shipping.
An easy and quick way to identify such frictions is to run a simple usability test. You don’t need to be a UX Researcher to do it. This quick guide should give anyone the tools to start practicing and perfecting usability testing to quickly get your product back on track.
What is Usability Testing?
Usability testing is a way to quickly test a product to understand if the product functions as expected in the minds of its users. Have you ever seen the comical video of the girl watching as all the different shaped blocks are put into the square hole? Yeah, usability testing is essentially that.
It is a way to evaluate the usability or ease of use of a product. The researcher will usually give tasks to the user and then observe them performing the task, often asking the user to express their thoughts throughout. After a few usability tests, the researcher will be able to find patterns in the data and provide suggestions to the team to improve the usability of a product, thus improving metrics such as conversion.
When Should I Use a Usability Test?
When performing user research, there are many different methodologies to choose from. Knowing when to run a usability test is as important as knowing how to run one.
In most cases, it should not be used to test if you understand the users’ problem correctly. I don’t believe it even should be used to test if the solution to the users’ problem is correct. Instead, usability testing should be employed to understand if the solution is functional and easily usable.
If you are about to run a usability test, but don’t truly understand the user problems you are trying to solve, stop! A different research method should be used, but that’s a post for another time.
Preparing for a Usability Test
Finding People to Test
It is best practice to find people closely aligned with your users or customers. For example, if you are building an app for car insurance brokers, you don’t want to ask an accountant to participate. In this case, only car insurance brokers can give you the most relevant feedback.
To find the right people to test your product, you can create a quick 4–5 question screener or survey asking people about their behaviors, attitudes, or demographics. If you are testing something that will benefit your current customers/users, you can recruit from your customers/users list and email them or ask them if they want to participate directly on your app or website.
However, if you need to test something like a new onboarding flow, you may want to recruit people who don’t currently use your product. You can post the screener to Craigslist, Facebook groups, Subreddits, or other platforms you think your users will hang out on.
In writing a screener, you want to write the questions in a way that targets the segment of users you need to better understand. For example, if you need to talk to power users, you should recruit for the behaviors they exhibit instead of a user who only uses your platform once in a blue moon. Once you collect responses, you can sort through the submissions and pick the people that best align with your users.
Providing Compensation
You will have the most success finding people to test if you provide compensation. I generally give a $50-$100 gift card for consumers and a bit more for B2B customers. If you don’t have a budget for that, you can get creative. I have used t-shirts in the past and have even gotten some people to do it for free. Just remember that the way you incentivize will attract different types of people.
A Few Tools Needed
You will need a device for the participant to test your prototype. You may want to also screen record, record the face of the participant, and record audio. Having these recordings not only helps when you are analyzing the data but also to sell members of your team on your recommendations when presenting the data. You can pretty easily hack some of these things together by using free tools. You don’t have to have a professional usability lab to do this.
Conducting a Usability Test
Discussion Guide
You will want a discussion guide before you start testing. This helps set the stage for a successful usability test. You can then have the guide as a cheat sheet as you run the research. Be consistent in asking the questions from one participant to the next.
An Overall Research Question
This will guide you on what you should be trying to answer. Don't talk about this with the participant, but keep it at the top of your guide to remind you throughout the usability test. An example of an overall research question could be, “How can we improve our onboarding process to ensure users complete the signup flow?”
Introductions to Put the Participant at Ease
Once the participant is in the (virtual) room with you, put them at ease. Engage in small talk for the first couple of minutes. Ask them how their day is going, what they do for work, etc. Even small talk will give you insights into how they think and behave.
Next, explain that you are testing the product, not the participant. You don’t want them to feel embarrassed if they “don’t get it.” Tell them you are looking for honest feedback and that they can’t hurt your feelings. Lastly, explain to them that you will be having them navigate through a product and that you will be giving them tasks to complete. As they do so, tell them to think out loud so you know what is going through their mind. You will probably have to remind people to do this throughout.
Consent Form
This is the boring stuff, but it will protect you from the minuscule chance someone wants to sue you. A consent form will usually state that you have permission to use the data they provide, will keep their feedback anonymous, and that any ideas generated during the study will be the company’s to keep. You can Google templates for consent forms, but if you are concerned, you can have your company lawyer write one up.
Tasks
You will need tasks for the participants to complete so you can test the usability of each action they take. If you are testing the user flow of an e-commerce solution, give them a task by saying, “Ok, now I want you to purchase a pair of shoes and go all the way through checkout.”
Then shut up and watch them do it! Watch for where they get hung up, make mistakes, click in the wrong place, get confused, etc. Create tasks for each part of the design you want to test. Just be careful not to be too specific in your task. You don’t want to give away the answer on how to complete it. As the user makes mistakes or ponders their next click, ask them what they are thinking or why they did a certain thing.
Summarize
After the participant finishes all the tasks, talk through their experience with them. You will usually uncover more insights here. Make sure to keep recording until they leave the room. Sometimes people are more honest when they feel the “test” is over. As they leave, make sure to give them their promised incentive.
Analyze the data
You will need to do usability tests with multiple participants. Once you start to hear the same thing consistently, you can probably stop. This is called “saturation.” Most people see patterns after 5–10 participants. Once all the data is collected, you can pick out the patterns to present to the team.
If you want to add a bit of quantitative findings into the mix, you can report back the number of mistakes people make or how long it takes. It’s not uncommon for companies to have a running usability program so you can track your progress over time.
One thing to note is the 5-10 participant rule is per user segment. If you are testing multiple segments (e.g. SMBs vs Enterprises), you may need to double your participants. Similarly, after each significant design iteration, you may want to run additional usability tests to ensure the product is still intuitive.
Get Started
There you have it. You are now all set to run your first usability test. Pretty soon you will be uncovering user insights to solve some of those pesky issues that have been annoying your users for months. Before you execute your study, below are a few research do's and don'ts to ensure you get the most reliable data possible.
Do:
Listen and allow for silence. If the participant isn’t talking 80% of the time, you are talking too much. You are learning from them, not the other way around.
Observe. You can’t always trust what people tell you. Their actions can give you the answers that their words can’t or won’t.
Ask Good Questions. Questions such as, “Why did you do that?” or “What did you expect to happen when you clicked there?” or “Tell me more about what you are thinking right now.” are questions that will uncover good insights.
Simplify. Ask your questions in a way that a child can understand. It is human nature to act smarter than you are and your participants will do the same, often resulting in bad data. Avoid jargon at all costs!
Clarify. Clarify what the participant is saying or doing to ensure you understand correctly. They will often expand on their original comments or actions.
Don’t:
Ask leading questions. “Do you think this is a good design?” “Why do you like this feature?” I can tell you the response to those questions before the participant answers. You are the researcher and should be able to infer the answer to these questions based on how the usability test goes.
Ask them to predict future behavior. This is impossible to answer with 100% accuracy. Instead, ask about their past behavior, which helps you to predict future behavior.
Sell or persuade. You want pure, unbiased data. As soon as they feel that you have a preference or agenda, they will tell you what they think you want to hear.
Ask multiple questions at once. “What are your first thoughts when seeing this screen and have you seen a screen like this before?” These types of questions are confusing.
Don’t draw conclusions immediately. You don’t want to draw conclusions after one usability test. Try your best to not let one participant influence another test. View the data holistically to not bias your findings.