- Observers have a very specific role in usability test sessions. In fact, you might want to show them this video, so that they understand what's required of them. There's a big difference between passive and active observation. Passive observation is what we do when we're watching TV. We sit back and let the images on the screen entertain us. Active observation is different. When you're actively observing, it's more like being in a classroom, where you're learning things and taking notes.
Usability sessions are all about active observation. You're watching for certain behaviors, and taking notes on the things that you see the participant doing. At the end of a usability session, your writing hand should hurt, and you should have several pages of notes. Your head should be full of all the interesting things you've seen. The best way of doing active observation is just to write down what you see without processing it too much. Focus on the participant's actions and quotes, what they do and what they say.
Stay away from trying to write down your interpretation of the reasons they did things or any potential fixes. If you're thinking about fixes, your mind isn't on the session anymore, and you're no longer actively observing. They'll be plenty of time to talk about solutions as a team after the sessions. Everyone on the team can, and should, observe usability sessions. Developers, designers, project managers, marketing people, and especially managers should all be present.
The information you get from watching real users is priceless. It lets you know where the pain points are in your product, and it also suggests ways of resolving them. Usability sessions give you real data, rather than just relying on team members' opinions. If there are only a couple of people observing, it might be okay for you to be in the same room as the moderator and the participant. However, you have to take a vow of silence. The moderator is the only person who should be interacting with the participant.
This is because the moderator knows the overall test plan and knows what types of interaction might cause problems or introduce bias into the study. If observers have questions, they should write them down, and then at the end of the session, they should hand them to the moderator to ask the participant. The reason for this is that even if observers think they're asking good questions, in reality, their questions are typically driven by a product idea or philosophy they have, rather than by the behavior they've observed.
So they tend to ask leading questions. Leading questions are ones like, "Don't you think that such and such?" Polite participants might find it hard to say, "No, I don't." Another common one is, "Tell me what you liked about such and such a thing." Well, what if they didn't like anything about it? If there are more than two extra observers, it's best for the observers to sit in a different room with a video and audio feed.
That allows the observers to come and go without interrupting the participant. This also takes the stress off the participant because there are less pairs of eyes watching their every move.
- What is usability testing?
- Finding the right participants
- Making a screener
- Asking the right questions
- Avoiding bias
- Making a task list
- Creating the test environment
- Running a pilot study
- Moderating sessions
- Capturing real-time observations
- Analyzing and reporting your results