- Once you're in the room with the participant, almost every interaction you have with him should follow a script. That may sound strange, considering you hope to see interesting new behaviors from your participants, but there's a difference between seeing those behaviors, and reacting to them. I've already mentioned that you should use a script for the introductory part of the study, to make sure you cover all the important points. But, what does a moderator do after these opening formalities? My advice to you is don't do anything.
In most usability studies, I tend to use the Think-Out-Loud protocol. That is, I ask participants to think out loud, as they go through the tasks they're performing for me. This removes the need for most questions you might feel like asking. Any question you ask, interrupts the participant's concentration. It might even make them change their behavior. For instance, if you were to ask: "Which option are you thinking of choosing?", you draw the participant's attention to the options.
They may have been looking at and thinking about a completely different part of the screen. Even when participants are following the Think-Out-Loud protocol, there will be quiet times. That's okay. Your idea of an uncomfortable silence is based on regular conversations. When a participant is thinking, you should wait quite a bit longer than you normally would, before interrupting them. The participant may even have forgotten that you're in the room. I tend to sit to the side of the participant, and slightly behind them, so they don't feel inclined to try an engage me in conversation.
You introduce the Think-Out-Loud protocol as part of the initial script. Then, when you hand the participant their first task, say: "Please read this out loud, and then go ahead and do it, and remember to think out loud as you go through the session." If the participant is quiet and has obviously forgotten to think out loud, remind them, by saying: "Please remember to think out loud." Don't ask, "What are you thinking?", or other variations. They may not be thinking anything at all at that point, but before you say anything, be sure they really have forgotten to think out loud.
You don't want to disturb them while they're paying attention to something on the screen, or focusing on a problem. If the participant asks you a question during the study, your first reaction may be to answer it, in order to help them out. However, before you do that, try redirecting them with your own question, like: "What do you think you should do?", or "What would you normally do here?" Once you have their answer, you may choose to provide some information. For instance, what the frequently asked questions page might say, or what help desks might tell them.
But doing so changes things, because now you've given the participant help they normally wouldn't have had. Obviously, it's completely fine to help the participant out if they're asking you a procedural question, or whether they can take a break. You'll undoubtedly have questions as a result of what you see. However, by just writing down the question and waiting, you'll get answers to many of those questions, based on what the participant does next. It's not really best to save the rest of your questions until the end of the task, or better still, until the end of the session.
- What is usability testing?
- Finding the right participants
- Making a screener
- Asking the right questions
- Avoiding bias
- Making a task list
- Creating the test environment
- Running a pilot study
- Moderating sessions
- Capturing real-time observations
- Analyzing and reporting your results