Technical Translation: Usability Strategies for Translating Technical Documentation

Results of Usability Evaluation 
For clarity of presentation, the data collected during the empirical study is 
presented in tabular form in Appendixes 6-8. The following sections con-
tain an analysis and discussion of the results and their implications. 
Analysis of Results 
Having collated and recorded the data gathered during the main study and 
observed the differences between the Control and Experimental groups, the 
next step is to ascertain whether such differences are statistically significant 
or simply due to chance. 
The nature of this study and the fact that only 10 subjects were used, 
poses a number of problems for any attempts at statistical analysis. Gener-
t-Test
. However, such tests require at least 20-30 observations for the results 
to be statistically reliable. This means that the data from the main study 
cannot be analysed using the more common statistical tests. 
This problem can be overcome, however, by using a Rank-Sum test, 
also known as the Wilcoxon-Mann-Whitney exact test. It is used for com-
paring two populations to detect “shift alternatives”. That is, the two distri-
butions have the same general shape, but one of them is shifted relative to 
the other by a constant amount under the alternative hypothesis (see be-
low). This test can be used for either continuous or ordinal categorical data. 
comparing groups of sample size 10 or smaller per group” (Helsel & Hirsch 
1993:119).
At the simplest level, the Wilcoxon-Mann-Whitney exact test is used to 
determine whether one group tends to produce larger observations than a 
second group. The null hypothesis of this test states that the probability of 
one group (x) producing a higher value than another group (y) is 50%.
239
later in order to complete the post-test survey. The post-test survey was 
ally, the reliability of data is calculated using methods such as a 
Student’s
It is one of the most popular nonparametric tests for detecting a shift in 
location between two populations and is “the only form appropriate for 

---

Assessing Usability 
This can be written as: 
H
0:
Prob[x>y] = 0.5 
Helsel & Hirsch (
ibid
.) state that the alternative hypothesis is one of the fol-
lowing: 
H
1:
Prob[x>y] 
 0.5
(two-sided test where x might be 
higher or lower than y) 
H
2:
Prob[x>y] > 0.5 
(x is expected to be larger than y) 
H
3:
Prob[x>y] < 0.5 
(x is expected to be smaller than y) 
totic p-values would not reliable, so exact p-values were calculated 
throughout using 
StatXact 4.01
from Cytel Software Corporation, Cam-
bridge MA. 
In performing statistical analyses on the three sets of data, i.e. task times, 
error rates and user satisfaction levels for each group, it was necessary to de-
cide upon the level of detail to be used. While analysing, for example, data 
for individual error criteria or times for individual tasks would be interest-
This problem is exacerbated by the small sample size used in this study. 
Consequently, it was decided to analyse the overall data for each subject, 
i.e. total task times, total error rates and overall satisfaction ratings.