Finally - a practical definition of a Usability Bug
Updated: Dec 3, 2018
Smart testers know how to test to confirm the spec, and how to test for functionality, reliability, performance, compatibility, and scalability. Perhaps your organisation even has different categories in your bug tracking system for these different kinds of defects. But how on earth does a tester test for usability?
In today's world, usability is a major competitive differentiator.
Usability is a notoriously tricky to test for. It took me many years and many discussions to finally figure out a way to define a test for usability. This post describes the solution I found.
It seems everyone has their own opinion on usability. Whose opinion is the most important? How long do you want to take to resolve differences of opinion?
It's great if your company has UI experts, Usability Labs, User experience experts, 1-way mirrors, focus groups, surveys and personas validated by reliable customer profile data.
Is a usability problem a bug that should even be reported by a tester?
Smart testers have a vital role to play. Testers are the folks who get the final say before a product goes out to customers. Testers have the final look at what the customer's first impression will be.
A simple way to test usability is to notice when you need to "pause" when using the product.
If you pause when using a product, even for a second, this means that you are not sure what to do next. Excellent software must support flow. So if the tester can find and get eliminated every place where a user "pauses", you can end up with a beautifully operating product.
Pausing is a symptom of a usability problem.
Here is the advice I give to testers:
Report as a "bug", everything that gives you as tester "pause". Highly usable software should draw the user along. "Pause" bugs may appear too minor to fix from a dev point of view, but keep at it and progressively eliminate them. The result is smooth flow for your users.
How do you test for Pause?
Try ( this is hard) to use the product to do a task, as if you were not familiar with it.
Be aware of yourself as you test the product. Notice when you need to stop to figure out what to do -even for a second.
Watch when you see others use the product. Note when they pause ( non-testers may assume its their fault when they don't know what to do next - No! its a usability bug!)
Use the product naively, don't be afraid to make mistakes ( a real user will make mistakes, especially to start with), and when you find a place where you hesitate and try to figure out what to do next - this can be reported as a usability bug.
Examples of what gives users' pause:
ambiguous or hard to follow error messages
inconsistent use of words or icons
broken links - or links that lead to unexpected places
back button fails
But of course, reporting usability is all for nought, if devs deprioritize and never fix "pause" bugs!
Be sure you have both your team and management support if you are going to be recording usability bugs.
Be sure you are reporting usability bugs at the right time in the release cycle - when software is solid enough but while there is still time to fix them.
Your credibility rides on ensuring the pauses you find, are on reasonable user work flows. You as a tester are prioritising pause bugs by deciding which ones to open and which ones to let go.
Usability is just as important as functionality.
Its too important not to be tested for.
Pause and report.