How (much) to Intervene in a Usability Testing Session
All things considered, I suggest that a useful way to look at this issue is in terms of the purpose of a usability test. If the purpose is scientific hypothesis testing, or collecting quantitative data for comparison against other products or usability goals, then experimental control, including non-intervention, is of paramount importance. For example, the process of responding to a tester’s questions would clearly invalidate various quantitative measurements, such as task completion time.
However, if the purpose is to gather qualitative data so as to identify significant user interface problems and recommend design solutions, then openly interacting with users in various ways may not only be useful but also sometimes necessary for collecting and understanding the required diagnostic information. In short, because scientific research and formative evaluation may have significantly different goals, quite different standards can apply.
By Howard Tamler. Provides a good overview of the reasoning for/against an increased level of involvement with testers.