Technical Translation: Usability Strategies for Translating Technical Documentation

Usability Evaluation Procedures 
In the previous sections we discussed the various approaches, methods and 
techniques used in usability evaluation. This section examines how these 
various factors are implemented as part of a usability evaluation. The pur-
pose of this section is to discuss how and why these tools are used in an 
evaluation. The procedures for conducting evaluations will also be discussed 
nation of user guides. 
196
with reference to a number of case studies which relate directly to our exami-

---

Usability Evaluation Procedures
Practical Usability Evaluation 
While having a firm understanding of the methods and tools described in 
previous sections is, without doubt, essential in conducting usability evalua-
tions, the success or failure of an evaluation depends on a variety of prepa-
rations being carried out. Without these preparations, the test will stumble 
awkwardly on, with no clear goal, purpose, transparency or logic. In the 
following sections, we discuss the various practical tasks that form a central 
part of usability evaluation. 
Measuring Usability 
As we discussed in Chapter 4, usability involves not only how well users 
can use an interface, but also how users perceive the interface and what 
their subjective opinions of the interface are (Dumas & Redish 1993:184). 
We also know that the usability of an interface can be measured in absolute 
terms on the basis of predefined performance criteria or it can be compared 
against that of another interface. However, the term performance is vast 
and can refer to a multitude of elements and factors of interface usage and 
the execution of tasks. Unless we define which aspects of performance we 
want to measure, it will be difficult, if not impossible, to determine the us-
ability of an interface. 
Wixon & Wilson (1997:664) present a list of what they term “usability 
attributes” which are characteristics of an interface. These attributes can be 
used to categorise and quantify the various facets of an interface’s perform-
ance. The attributes proposed by Wixon & Wilson include: 
usefulness 
learnability 
efficiency 
error rates 
memorability 
first impressions 
advanced feature usage 
satisfaction or likeability 
flexibility 
evolvability 
It is apparent that there are some questionable inclusions in this list such as 
“usefulness” which, as we have already discovered in Chapter 4, is quite 
separate from usability in that it is a social or commercial factor which does 
not effect how well users can use something. 
197
•
•
•
•
•
•
•
•
•
•

---

Assessing Usability 
On the basis of these categories it is possible to compile a list of meas-
urement criteria for use in usability evaluations. These criteria make it pos-
sible to count or measure individual behaviours on the part of the user. 
Dumas & Redish (1993:184) point out that counting instances of user be-
haviour requires careful attention but, importantly, that it does not require 
judgmental decisions; either the event occurred or it did not provided, of 
course, such events are observable and discrete. Wixon & Wilson 
(1997:666) provide the following list of measurement criteria: 
time to complete a task 
number of tasks completed 
number of subtasks completed 
number of errors per unit of time 
number of steps required to 
number of negative reactions to
Dumas & Redish (1993:185) provide their own list of criteria which, al-
though similar in certain respects to the criteria of Wixon & Wilson, are 
more detailed in their formulation. These criteria include: 
time to complete a task 
time spent navigating menus 
time spent in online help 
manual 
time spent reading manual 
time spent recovering from errors 
choices 
choices key 
choices 
the number of repeated errors 
198
the number of help screens
the number of other errors 
the number of incorrect icon
time needed to complete a task 
after a specified period of time
away from the system 
complete a task 
number of times users access
number of commands or icons 
documentation or technical 
support
the number of incorrect menu
the number of calls to technical
productively
versus time spent working 
time spent recovering from errors 
interface
remembered after task 
completion
time spent finding information in 
the number of incorrect choices
in dialog boxes 
the number of incorrect function 
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•

---

Usability Evaluation Procedures
support or for assistance 
viewed
the same help screens 
consulted 
the number of times a quick
each visit to the manual 
observations of frustration 
observations of confusion 
observations of satisfaction 
It is clear from these lists, not to mention other similar lists such as the one 
compiled by Preece (1994:405), that there is a wide range of criteria which 
can be measured. Obviously, recording data for all of these criteria would 
be extremely time-consuming, if not overwhelming. Indeed, Wixon & 
Wilson (1997:667) maintain that the number of criteria should not over-
whelm the test team. They go on to say that 2-3 criteria are sufficient to 
measure usability. In contrast, Dumas & Redish (1993:185), while ac-
knowledging the impracticality of using all of the criteria, do not restrict 
the number like Wixon & Wilson. Instead, they say that not all of the crite-
ria are applicable to each test and that only those that relate directly to the 
product should be used. 
If we refer back to Chapter 4 where we detailed the ways in which 
Iconic Linkage can improve usability, we can see that the other main at-
tributes of interest are: learnability, retention of information over time, 
comprehensibility, accessibility of information and speed of processing. In 
the following section we will look at previous studies to see how they used 
the procedures, methods and criteria we have just discussed.