Technical Translation: Usability Strategies for Translating Technical Documentation

Interactive Observation 
Interactive observation is a type of indirect observation where the part of 
the system or computer is played by a member of the evaluation team. 
Commonly known as the “Wizard of Oz” (Faulkner 1998:122), this ap-
proach makes users think that they are using a real system, but in reality all 
of the system’s responses and actions are performed by a human. This 
method is effective in that it does not require a fully functional version of 
the system and it can be implemented reasonably cheaply in comparison to 
the expense of producing a fully-functioning system. However, this ap-
proach is formative and is more suited to situations where an actual soft-
ware system is being produced. The effort required to create an interface, in 
terms of design and labour coupled with the additional staff requirements to 
conduct the experiment make this approach difficult to implement. In any 
case, this approach is of limited applicability to documentation as it is aimed 
at investigating the way the software interface works. 
184
Interaction logging 
tools operate in a similar manner except that they 
record the entire interaction in real-time. What makes interaction logging 

---

Approaches to Empirical Evaluation
Verbal Protocols
clamations and other information which may arise during the course of an 
experiment. One particular variety of verbal protocol is the think-aloud 
protocol which involves users saying what they are thinking, feeling, plan-
ning etc. as they perform tasks and use the interface. This can provide a 
valuable insight into what users want to do, what they think they are doing 
and what their responses are when something unexpected occurs. It is also 
possible to gain an insight into how users remember commands, plan and 
execute tasks and how they recover from errors (Preece 1993:113). Verbal 
protocols of this type are generally used in conjunction with audio or video 
recording (Preece 1994:621). 
Although think-aloud protocols (TAP) are ideal for formative usability 
testing where the wealth of qualitative data they can provide is extremely 
useful in understanding the nature of the interaction, in the case of purely 
summative
evaluations where, for example, the speed at which users work 
is being measured, TAP is less applicable, chiefly because summative 
evaluations require quantitative data. It can also be argued that TAP may 
hinder evaluation rather than aid it. This can be attributed to a number of 
factors. Firstly, as we discussed in Chapter 3 the human cognitive system 
can realistically deal only with one response to a stimulus at a time even 
though it can process several inputs or stimuli. Indeed, the process of divid-
solving and verbalising thoughts where high levels of accuracy are required 
for both. Similarly, it is also held that the very act of putting into words 
what it is a user is doing will affect the way the user performs the task (Dix 
1998:427; Downton 1991:334). While many agree that this double-tasking 
will degrade performance on both tasks, there is some conflicting evidence 
that the think-aloud protocol may actually improve performance of the 
task. If this were proven to be true, it could be because the verbalisation 
process focuses a user’s mind on the task and helps users rationalise the task 
better. Nevertheless, the additional strain of performing two complex and 
demanding tasks such as putting thoughts into words can result in lower 
performance and some users will simply be unable to verbalise their 
thoughts (Preece 1994:622). 
There is also the problem of silence caused by the fact that users are ei-
ther unaccustomed to thinking out loud or because all of their concentra-
tion is being devoted to performing the task; some users may simply forget 
to speak. This problem is discussed by Dumas & Redish (1993:278-281) 
185
Verbal protocols are spoken records of users’ comments, observations, ex-
ing one’s attention equally between two tasks is unreliable at best, but 
extremely difficult when performing two complex tasks such as problem-

---

Assessing Usability 
who point out that while some people have no problems whatsoever in 
producing an “unedited stream of consciousness”, others either mumble or 
do not speak. The authors make the interesting point that users need to be 
taught how to think out loud and that they may need to be reminded to do 
so. This in itself can become a source of stress for users who may already 
feel pressurised as a result of the tasks (Preece 1994:622).
In view of these problems, 
retrospective
or 
post-event protocols
are
sometimes used to elicit verbal data from users. Instead of commenting 
on
their actions while they perform them, users are shown a video of the 
experiment and are asked to comment on their activities. This approach pro-
duces different results in terms of the type of information users provide. Ac-
cording to Preece, users tend to rationalise or interpret their actions or even 
justify them (Preece 1994:623). Rather than simply stating what they were 
doing or thinking, users tend to explain what they are doing and why. 
Dumas & Redish, however, do make the point that retrospective proto-
cols frequently yield more suggestions as to how to improve the interface as 
compared to think-aloud protocols (Dumas & Redish 1993:279). However, 
we are not interested in using the evaluation to improve quality, merely to 
assess it. 
It is clear from the preceding paragraphs that observational methods are 
extremely useful in gathering comprehensive information on the way users 
work with an interface. While these methods produce large volumes of data 
which can be more difficult to analyse, the sheer detail and insight they 
provide more than compensates for this (Preece 1993:119). By using indi-
rect observation we avoid such negative effects as the Hawthorne effect and 
we are provided with a permanent record of the experiment. In order to 
ensure that participants' task performance during the experiments is as rep-
resentative of real-life as possible, think-aloud protocols are to be avoided as 
they can affect the way tasks are performed. Retrospective protocols are of 
limited use and are really only of benefit when the purpose of the evalua-
tion is improvement rather than quantification; in the case of this study, we 
are concerned with the latter.