User Testing: A Step-by-Step Guide, Templates and Examples

In this handbook, we cover everything related to conducting usability research and testing the usability of a website, app, or prototype. We take a closer look at what user research entails, why it is important, what methods there are, and how you can get started with it.

What is usability?

Usability or user-friendliness is the extent to which a user can use a product in his own way without obstacles, doubts, or questions. With a user-friendly product, the use of the product is smooth and the user does not experience frustration.

Definition of usability

The ISO 9241-11:2018 standard provides the following usability definition:

“Usability is the degree of effectiveness, efficiency, and satisfaction of a defined group of users to achieve a specific goal in a specific context.”

Usability consists of three parts:

• Effectiveness. The extent to which users can use the product to achieve their goal(s).
• Efficiency. The effort that users have to make to achieve a certain goal.
• Satisfaction. This refers to the thoughts of the user while using the product. This includes the user's opinion about the ease of use, the added value of the website or app, and its applicability.

Example user test script 

Curious what kind of questions you can ask during a usability test? Then download the sample usability test questionnaire.

Real user test example

Conducting usability tests 

For every phase in the development process of a website or app, testing with real users is of great value. The five biggest benefits are:

• Discover what users encounter. Convenience serves people, now more than ever. When users want to order a product, but the ordering process does not work, they order it from the competitor with three clicks. That leaves the owner of the webshop with one question: why did the user abandon its shopping cart? Through user testing, you discover the frustrations of your user audience that would normally go unnoticed, providing you with the advantage of knowing who your user is, how they want to receive your product, and how you need to provide it.
• A better end product with prototype testing. Save time and money by conducting usability testing during the development process. Is it clear what they can do with the app or website? What do they encounter? What would they use it for? By collecting and processing this feedback during the development process, you build a product that better suits the end user.      
• Create empathy for the end user. It often occurs that a team that has been working on a project for a long time, find it increasingly difficult to empathize with the end user, because they know the product inside out as an expert and no longer a first time user. User testing forces you to see the product through the eyes of the end user, which gives you a lot of valuable information for the rest of the development process.      
• Valuable addition to quantitative data. Tools such as Google Analytics and Hotjar show what users do, but not why they do it.. A heatmap can show you that users are dropping out during the ordering process, but not why users are still considering. Usability testing removes that uncertainty and provides insight into how the ordering process can be improved.      
• A better product for everyone. Whether you have just launched your product, or have been in the market for some time, through testing you can find out to what extent the product meets users' expectations. Is something missing in the interface? Do we lose money by having inefficiently arranged the ordering process?    

Benefits of a better UX

The ultimate goal of usability testing is to make the experience for the user as good as possible. The usefulness of a better user experience or user experience (UX) has been demonstrated by countless studies. Improving the user experience generally brings the following five benefits:

• Reduced workload for customer service employees. With a good user experience, the website or app answers questions that users normally have. 
• Loyal customers generate more sales. A good user experience ensures more satisfied customers who are less likely to switch to the competitor. Depending on the changes made, a better UX can increase sales by more than 100%! Would you like to know more about the relationship between user testing and conversion ? Download our whitepaper. 
• Competitive advantage. By really putting yourself in the shoes of the user, you can further strengthen your position compared to the competition. 
• Risk reducing. When you put a lot of money and time into the development of a website or app, you want to make sure that users don't drop out because they can't figure it out.
• Progress can be tracked. Characteristic of a good user experience is that usability tests are carried out structurally. By keeping track of usability metrics via, for example, the System Usability Scale, you can measure the progress of a website or app. 

Four methods of user testing

Usability research can be performed in different ways. When someone talks about usability research, they almost always talk about performing usability or user tests. Did you know that usability research has been conducted since the 1980s? 

The research methods for usability research can be distinguished on two dimensions. The first dimension is the location of the usability study: online or in a usability lab. The second dimension concerns the way in which the research is conducted: with or without a moderator.

In a usability lab

By taking the tests in a usability lab, you have maximum control over the equipment that is used and you can quickly intervene when there are technical problems. The main drawback is that the user is not in their natural environment, which affects the answers and makes users less honest.

It also costs more time and money to recruit users and have them travel to the usability lab. User testing in a usability lab is almost always accompanied by a moderator who interacts with the tester / respondent during the test. 

Remote / Online

A remote user test means that the tester can participate in the usability research online. In practice, this often means that the user test takes place from home or at the office. The main advantage of this is that the user takes the test in their natural environment, which leads to fairer feedback and makes it easier to reach difficult respondent profiles.

Another added benefit of remote user testing? It is often many times cheaper than in a usability lab. Where a test in a lab costs on average between €5,000 and €10,000, this can be done from €650 via an online platform such as User Sense. A world of difference!
 

Moderated

This usability research method is also known as moderated user testing. The moderator guides the test session from start to finish. The test can take place both face-to-face and online. When the test takes place online, the moderator uses an online tool to communicate with the users.

The biggest advantage is that a moderator can ask in-depth questions and thus gather more insights. On the other hand, an (inexperienced) moderator can influence the results of the research with leading questions.

If this happens, it will have a negative effect on the reliability of the results - something you logically want to avoid.

Moderated usability research is most suitable if:

• You want to test and validate a concept or idea
• You  want to test a wireframe, mock-up or non-interactive prototype
• You want to make a day out of testing, with the entire (customer) team watching the execution of the user tests

Unmoderated

Of course, it doesn't make sense to have testers travel to a usability lab and then not talk to them. This also means that this variant of usability research almost always takes place online. This variant is therefore called remote unmoderated usability testing (URUT) .

The biggest advantage of this variant of usability research is that the tester cannot be influenced by a usability researcher during the execution, which means that the tester exhibits its most natural behavior during the user test. In our experience, this often leads to fairer feedback and greater willingness to test, because the tester can perform the usability test at a time of his/her choosing.

Unmoderated remote usability research is most appropriate if:

• You want to improve an existing app or website based on qualitative research (= user testing)
• You want to test a clickable prototype so changes can be made before the app is built
• You want to conduct international usability research

This variant of user testing is almost always carried out via a platform such as that of User Sense. Are you looking for a tool to perform user testing? Then we advise you to take a good look at how the quality of the testers is guaranteed and the size of the pool of the country in which you want to test. 

How much does usability research cost?

The costs for conducting usability research start at € 650 and can sometimes rise to € 10,000. The price of the research depends on the chosen research variant, the location of the research and whether respondent recruitment is necessary. In general,  UX research via a platform such as User Sense is almost always cheaper than on-site studies. You can check our pricing here.