February 22th, 2018
Usability testing data is a sure-fire way to eliminate guess-work and focus on making the UX design process more effective.
These types of user tests can help you figure out if your audience can adequately use your website or mobile app.
Additionally, they help identify specific issues, such as desired actions which are hard to complete.
Usability testing is a valuable research technique which provides data development teams can leverage to find practical solutions to the problems the testers uncovered.
Despite all of these benefits, UX test results can feel overwhelming because:
- There’s a huge volume of data that needs to be processed
- Over-thinking the results often leads to paralysis by analysis, essentially taking no action
- Misinterpreting the data can occur quite often.
Due to these three major obstacles, turning usability testing data into solutions is not an easy task, but it’s definitely achievable. We want to help you become more proficient by using the four-step process we’ll present down below.
Let’s get into it.
1. Structuring and Organizing the Data
This may seem like a no-brainer, but it’s essential for you to structure and organize the data so you can avoid clutter later on during the process.
We recommend keeping track of key user testing metrics, such as:
- Time on task
- Success/failure ratios
- System usability scale (SUS), which is a reliable tool for measuring usability that consists of ten item statements with five rating response options each, alternating positively framed and negatively framed
- Net promoter score (NPS), which is a loyalty metric that can be correlated with the usability of your site or app
- Single ease question (SEQ), which is a scale that measures how difficult users find a specific task.
Additionally, you should also organize the data based on where the issue occurred (homepage, login page, main menu, search bar, etc.), the task the user was performing when he/she encountered the issue, and lastly, have a detailed description of the issue.
This can help you figure out if certain areas are more problematic than others.
Note that you don’t need to include the positive observations when structuring and organizing the data. Those should be logged separately.
When our users use the Userlytics platform for user testing, we provide the functionality to write annotations on each user session, hyperlinked to the result and timeline moment, and aggregate the annotations automatically so that you can easily review the UX issue areas from a high level, as well as download the clip annotations and/or create usability test highlight reels. You can also share the annotations, the user session videos and the quantitative user experience metrics with colleagues or clients through a configurable link you can even private label with your own logo and color scheme. Organizing the data in this way allows for efficient analysis for you and your team.
2. Prioritizing the Issues
Let’s say the usability tests revealed more than 200 issues that need to be addressed. The smart approach in a situation like this is prioritizing the issues so that the development team focuses on the critical ones, rather than on those which can be postponed.
The severity of each issue is usually defined by three factors:
- Frequency – how many times it occurred during the user testing
- Impact – how it affected the user who was trying to accomplish it
- Task importance – rated based on how important it was for the user so he/she can continue the usability testing
Here’s the method you can use to prioritize the issues:
Define the impact score for each issue. For example, 1 can define a suggestion from the user, 2 means it’s a minor issue, 3 is a significant issue and 5 is usually when the user cannot accomplish the task.
Find out the frequency of the issues. This is easy to do. You need to divide the total number of occurrences by the total number of tests.
Determine the severity of each issue. One way of calculating this is calculated by multiplying the variables mentioned above.
You can create a spreadsheet to keep track of all the usability issues and then add 3 columns for each one to start prioritizing them for this next step by sorting according to the severity column.
After you do all these steps, you will have a clear idea of what are the main issues your team needs to start working on.
3. Generating the Solutions
To maximize the chances of finding the best solutions, you first need to generate several key ideas.
You need to keep in mind that a solution can address multiple issues, which is great because you could kill many birds with one stone.
Here are the steps you need to follow at this stage:
- Generate multiple solutions for each issue. This applies only to the cases where the solution is not obvious. For this step, you can bring in your entire team to collaborate and brainstorm solutions together.
- Rewrite them by making the solutions specific. The keyword we want to stress here is specific. If the idea you came up with is not clearly defined, then it’s not an actionable solution. Anything which is too vague and abstract needs to be rewritten or deleted from your list.
- Lastly, don’t forget to mark in the spreadsheet if a solution resolves multiple issues.You need to create a different sheet where you’ll write down the solution you came up with and the number associated with the issue.Now that you have the solutions, it’s time to go over the final step.4. Prioritizing the SolutionsIn an ideal world, each issue would have one perfect solution.But what do you do when an issue can be resolved in many different ways? How do you choose the best solution?Here’s a reliable way to prioritize the solutions:
- Calculate how effective each solution is. You can do this by adding up all the issue severities which can be resolved by a solution.
- Calculate the complexity of each solution. To figure this out, you need to ask yourself: What resources do I need to allocate for implementing this solution? Do I have enough manpower? The more effort you need to put into the solution, the more it’s going to take for the issue to be resolved.
