Conducting a usability test is very simple and, in its most classic format, requires little logistics and a very low cost. The simplest way would be to sit next to a user and ask them to do something – a task – with a product or interface (a sketch, wireframe, finished design, or website, mobile application, totem, etc.).
If it's so simple, does that mean anyone can do it? In theory, yes.
In practice, a user test can present some complexity, and the results and interpretation can be affected by a series of common errors. Below are some errors we've seen over time and tips to avoid falling into the trap.
The basis for starting to test is to define what will be tested, how, and with whom. Whether the usability test is remote or in-person, the testing protocol should include details such as:
- the welcome message / introduction,
- specific tasks, if any,
- instructions for the user,
- instructions for the moderator or facilitator, and
- important aspects to observe.
Being systematic does not mean spending too much time on this planning stage; it means being rigorous to get the most out of the testing process.
Due to the nature of the test, this situation occurs more often in in-person mode than remotely: the moderator follows the protocol to the letter and misses opportunities.
To systematize and obtain comparable results, it is important to follow the guideline and protocol. However, the moderator of an in-person usability test should have the autonomy and agility to identify opportunities beyond the initial instructions, and take advantage of these opportunities while the user is available to uncover new findings.
Let's say we are testing an e-commerce site where we need to validate some hypotheses, among other test objectives. If we follow the instructions, we can probably validate the assumptions, but if we let the user perform other tasks freely, other findings will surely emerge.
Whatever type of usability test you are conducting, in-person or remote, it is important to test the protocol to validate the instructions, ensure that the tasks to be completed are understood, and that the language used is simple and clear.
In other words, you have to test the test. Otherwise, if we make a mistake in choosing a word and only realize it after the moderator has conducted several sessions, we are wasting time and money, as the number of participating users will need to be increased.
Once we were told, "every time we make drastic changes and think we're improving, we're crashing the call center." This means that, although the design and development team in question was making improvements and simplifying the interface, the frequent user already knew the software's "quirks" and was used to pressing Enter 3 times in a row to reach the main screen. Changing this flow meant disrupting their routine. They would get lost and call the call center.
Sometimes, it's a necessary short-term step to generate medium-term experience improvements. Other times, it's a risk that must be taken if the user experience is to be improved in the long run. In this case, an important segment to consider is new users who, if they have problems, it's because they don't know the interface and are not used to it. Therefore, the learning period is slow.
The focus is on testing, gaining insights, and concrete results to ultimately improve an interface and user experience, not on filtering databases.
Spending too much time and resources filtering customer databases to recruit THE perfect user and choosing them with tweezers is not always a profitable process. Perhaps it's better to invest these resources in building more testing scenarios and testing with a wider variety of users to get more findings.
Here we could delve deeper, and we will in the future, but a detail to plan carefully is the choice of words in communication with the user during testing, the formulation of instructions, and avoiding providing "help" or "tips" during task execution.
During usability testing, we can identify a series of problems. In some cases, the problems are obvious: the user cannot find information, does not see where the shopping cart is located, does not understand a label, etc.
In other situations, the problem is more complex, and identifying that there is a problem and what its causes are is the task of the moderator and the analyst or consultant (it can be the same person).
Proposing solutions to a usability problem can be a challenging task, but one thing is certain: it is the job of the team behind the interface, not the user.
Once again, user testing is not a focus group (where participants might be asked to think of possible alternatives).
If your testing protocol includes a question like "How would you make this step simpler?", you should consider removing it and instead focus on better observation, analyzing the results, or improving the design and retesting.
Thinking that usability testing is THE source of information that will tell you what to do to optimize results and relying solely on this methodology is illusory.
Some other research methodologies and relevant data sources you should analyze are:
- traffic metrics (from Google Analytics, for example),
- heatmaps from applications like Clicktale or Crazyegg,
- navigation sequence recordings,
- case analysis based on requests received via email or social media,
- content performance analysis,
- qualitative research through interviews,
- satisfaction surveys,
- benchmarking,
- research with internal users such as customer service executives or salespeople, etc.
While a user test can be exploratory in an initial stage, it is important to define indicators when systematizing the process and results.
What will be evaluated? Some result indicators for a usability test:
- Success rate in task completion.
- Time needed to complete a transaction.
- Time needed to find information.
- Task abandonment rate.
When conducting a usability test for a new interface, there is a challenge regarding the proposed interaction flows to be tested. Having only one alternative can severely limit the test results.
Should the transactional flow have 3 or 4 steps? Is the call to action clear and concise? It is preferable to make these types of design decisions by comparing the effectiveness of several alternatives.
Congratulations, we've started testing, we have a list of pending tasks and work priorities for the coming weeks. Now what?
If we are improving the user experience on an existing digital interface, there is surely still much to do. If, on the contrary, we are designing a website or application, it is important to integrate testing as part of the design process and test in the main pre- and post-launch stages.
In both cases, let's remember that the test itself is not an end but a means to design remarkable experiences.
Usability testing with users is not the same as user research. It's not the same as "going out into the street," in startup parlance, to test a minimum viable product (MVP). It's not a browser compatibility test, an A/B test, a satisfaction survey, a focus group, a taste or preference test, etc.
Understanding the differences between each method and knowing how to explain internally or evangelize ensures that all involved stakeholders speak a common language.
Be careful about wanting to test alternatives like a yellow button versus a red button. To test small differences and obtain significant data from a larger sample, it is preferable to consider A/B testing. Usability testing is not the best methodology to measure the impact of this type of variable.
Thinking that a successful usability test means a successful transaction, amazing conversions, or sales once the interface is in production, is to assume a causal relationship while ignoring the impact of other variables.
We have encountered situations where, as a result of a user test, a transactional flow is improved (and validated as improved), but once "live," the results are disappointing. Few users actually use the respective flow, which generates few transactions, and ultimately, the internal team declares that the objectives were not met; it's almost a failure.
Does this mean the test result was wrong? Or that the transaction is poorly designed? Not necessarily. Low result indicators (or KPIs, key performance indicators) can be attributed to other causes, such as poor information architecture: within the sea of information on the site, the transaction gets lost.
Unlike the testing instance, where the user must complete the task of carrying out a transaction, an online payment for example, starting from step 1 and ending at step x, when the payment is in production and the user must go through the homepage, access to the transaction "competes" with other content elements. Another example: when the new functionality is live, the launch coincides with hosting problems, so the page loading time increases and users abandon the transaction.
To avoid this type of false cause-and-effect problem, it is important to analyze in depth what is happening and to have complementary sources of information, both qualitative and quantitative.
If you're thinking that a usability test will eliminate all risks associated with a major design change, ensure the success of a production launch, or be synonymous with an increase in sales, you should reconsider the process and make sure you're choosing the right method for the right objective.
If, on the contrary, it's clear to you, then you are in a position to communicate to your client and team why you are conducting a user test and what they should expect from the results.
What other common errors have you observed in user usability tests?
By 
![Digital Marketing Strategy: Mistakes to Avoid [Part 1]](/content-images/f8abfb6b-portada-marketing-digital.jpg)