Usability testing can be used to establish problems that need solving, validate a hypothesis or to test a solution. Depending on what's required, there are different types of testing availability.
However, not every change needs to be tested. This page is deisgned to highlight when should and shouldn't use testing, and how we should do it when we do.
Plenty of design changes don’t require upfront hypothesis validation. If the design change is due to any of the following it may not need validation from users:
Colour, font, shading, button types, and anything in uStyle is a given. We want to achieve site consistency for brand reasons.
Functionality, UI, hierarchy are not 'look and feel' consistency.
Moving the first two Broadband designs ('Oldest stlye' and 'Old style') to be more ‘in line’ with the third design ('New style') is an on-going project of consistency.
Changing the colours of the first design or moving to a consistent layout would not require testing.
Similarly, changing the row-data presentation in the second example to be more consistent with the third example would not require testing.
There are certain web-design standards, and in our aim for a consistent user experience, maintaining web consistency is an important part of that.
The following is a non-exhaustive list of design conventions:
A basic UI convention is the distinction between active elements and inactive elements, with inactive elements being 'greyed out'.
The below car insurance has an overlay that must be interacted with to continue with the page below, but only part of the page is greyed out.
Five clickable elements on this page aren’t greyed out, despite not being clickable:
The ten basic usability heurisitcs, like basic design rules, give us a number of usability principles that must be sense-checked against. Designs that do not meet these requirements give us a clear problem to be solved without the requirement of validation.
Keep users informed about what is going on through appropriate feedback within reasonable time. In the example below the user cannot see what impact selecting a dropdown option has because the overlay covers the results table:
"Did updating those filters do anything?"
Speak the users' language with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
"Large as in size of people or house? Doesn’t it depend what they’re doing? What’s a Plusnet safeguard?"
If a user makes a mistake, let them undo, redo, go back, etc...
"I wanted more details...and now i've left the site....how do I get back?”
Users should not have to wonder whether different words, situations, or actions mean the same thing.
"Is a plan and a tariff the same thing?”
Even better than good error messages is a careful design which prevents a problem from occurring in the first place.
"No deals, why did you present it as an option?"
Minimise the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part to another.
"I’ve filtered these results, but by what?"
llow users to tailor frequent actions. 1-button checkout via Amazon, or Add to my wishlist via Airbnb.
"I personalised all these details yesterday, why has it just reverted?"
Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
"What am I supposed to do on this page?"
Error messages should be expressed in plain language, precisely indicate the problem, and constructively suggest a solution.
"Ahhhh....the ol' requested format..."
Any such help information should be easy to find, focused on the user's task, list concrete steps to be carried out, and not be too wordy.
Sometimes design is dictated - commercial or regulatory or business.
If there is flexibility around how these are implemented we can enter the design process. Sometimes there isn’t though, and they have to be implemented.
"The brand red clashes with this provider's red."
If our change isn’t down to one of the above then it’s problem solving, so we need to structure our approach to design differently - we need to understand the problem first.
A problem statement is the design brief, but first we need a good problem.
Data tells us about results, not causes. It tells us that it looks like there is a problem, but now what the problem is. Data comes from:
Similarly business objectives are things we desire to happen. They tell us what we want to the outcome to be. Objectives sound like:
These tell us there is a problem, but we don’t know what it is.
A real problems is something we’ve observed a significant number of occasions.
Testing designed to steer design, or validate assumptions about what problems users are encountering, is known as ‘formative’ testing. It’s open-ended, and the tasks are simply to run a comparison, or to go through a journey start to finish.
Any type of testing, whether it be face-to-face, guerilla, remote, or interviews, can help define the problem. This allows us to frame the problem with a problem statement. A problem statement is:
A concise description of the issues that need to be addressed - it is specific, measurable and explains what it impacts. There are no assumptions or solutions.
Once the problem is clearly defined, start trying to solve it (one problem at a time), but remember to validate the solutions as soon as possible.
'Summative' usability testing can help validate. Summative testing gives the user a particular task to complete that tests the solution against the problem. Again, any type of testing can perform this function.
User testing is great to validate flows, designs and interactions, but it's not exhaustive. There might be bugs, or solutions that don't work in certain scenarios. This is where in-house testing and the device lab comes in.
So what type of testing should we use, and when?This article by Nielsen Norman Group is a great starting point to understand testing methods and when to apply each.
There are over 20 types of validation, including quant methods, but here at uSwitch we predominantly use the following:
Interviews are designed to understand behaviours, needs and pain-points around a particular task.
They are open-ended and can help define design personas - archetypes that can be used as a reference point for any design changes for that product.
See the interview checklist
Interviews are most commonly used at the earliest stage of product development to determine the needs/problems that the product will address.
Face-to-face testing can be used in a formative or summative sense, and can be combined with an interview and simple exercises, like card sorting.
This is the highest-fidelity form of validation. The feedback you receive will be of the highest quality, but it will also take you the most time.
Face-to-face testing consists of preparing a particular task, putting it in front of real users, and seeing how they get on.
As it’s face-to-face, you can also interview users in this format, or ask them to complete exercises such as:
See the testing checklist for a step-by-step guide to setting up and running a test. The most crucial components are:
Anytime. For an existing product where the objective is to discover problems, an open-ended test asking users to complete the most common journey. This should be done at least once a year.
For a new product or feature face-to-face testing is always a good idea. The objective here is to test the solution against the problem using particular tasks.
A lower-fidelity version of face-to-face testing, guerilla testing sacrifices quality for speed. The feedback you receive will be of lower quality as the participants may not fit user types, and the testing environment is more ‘ad-hoc’ (think coffee shops, sandwich places).
Just like face-to-face testing, guerilla testing involves preparing a particular task, putting it in front of real users, and seeing how they get on. You can also do interviews and get quant feedback but again, the feedback will be lower quality.
See the guerilla testing checklist for a step-by-step guide to setting up and running a test.
The number of participants should be at least 5 for guerilla testing, but you can use your discretion.
Guerilla testing is perfect at the early stage of a design, when you want to sense check a particular flow, component, or wording.
Any fidelity of design can be used. Guerilla testing works best when kept relatively informal, and the participant feels at ease.
Asking your colleagues in the office for feedback. This is the quickest but lowest quality form of validation.
Like guerilla testing this is an informal type of testing, but the quality is considerably lower owing to familiarity with the product, bias based on personal relationships and a mismatch between persona types and testers.
In-house testing is perfect for bug testing, seeing if anyone can ‘break’ your design, and last-minute tweaks. It is not for designed to surface flow or interaction changes.
See the bug-testing checklist
A few days before release
Copyright uSwitch Limited 2016 - 2017. Github