An Introduction to Applied Linguistics

Resource Book. London and New York: Routledge. This volume provides a 
comprehensive and interestingly-presented treatment of the issues involved in designing 
and validating language tests. It includes a thorough examination of different models of 
language ability, along with detailed guidance on test construction and delivery. Practical 
issues of test administration and staff training, often glossed over or ignored in testing 
texts, are discussed in some detail, and issues of fairness, ethics and standards are 
also addressed. The volume includes a range of tasks aimed at encouraging readers to 
engage with the material as well as a selection of key readings that illustrate the concepts 
discussed.
Hughes, A. (2003) Testing for Language Teachers (second edition). Cambridge: 
Cambridge University Press. This widely-used text is a clearly-written practical guide 
to the principles and practice of language testing. It covers a range of essential topics, 
including test purpose, relationships between teaching and testing, stages of test 
development, and techniques for testing different language skills.
McNamara, T. (1996) Measuring Second Language Performance. London: Longman. 
This book is an in-depth examination of the issues and problems involved in assessing 
language performance. The first half of the book looks at approaches and theories in 

---

266 An Introduction to Applied Linguistics
performance assessment and identifies the dimensions of an adequate model of language 
performance. The second half of the book is concerned with ways in which Rasch 
measurement technology can be used to model variability in language performance and 
rater behaviour.
McNamara, T. and Roever, C. (2006) Language Testing: The Social Dimension. Malden, 
MA and Oxford: Blackwell. In this book, the authors explore the social dimensions of 
language testing from a validity theory perspective. They critique the lack of a social 
perspective in psychometric approaches to testing and outline various ways in which 
social dimensions of language use can be incorporated into test construction. The uses 
and abuses of language tests as policy instruments are also critically examined. The book 
provides a thorough and theoretically-grounded overview of the issues and problems 
associated with a socially situated view of language assessment.
Hands-on Activity
Question 1
In a short ESL vocabulary test, the test developers wanted to be sure to include 
vocabulary items at a wide variety of difficulty levels, and so they included items 
such as the following: ‘The main focus of the course’, ‘People’s stereotypes of other 
social groups’ and ‘My name is Sandy’. The examinee had to select the correct 
word (the underlined one) from four choices. Since the items were intended to 
reveal different ability levels, the test developers looked at the item difficulty of 
each of the items. They found a range of difficulties for the three items (100, 74 
and 42). Complete Table 15.4 with your analysis of where each item fits, the level 
of predicted item difficulties (low, medium, high) and actual item difficulties.
Item, predicted and 
actual results
Beginning
Intermediate
Advanced
Item
Predicted
Actual item difficulty
Table 15.4 Summary of predictions and results for ESL vocabulary items
Question 2
Test developers were attempting to develop two forms of a language test that they 
could claim measured the same thing. They started by developing three forms 
of the test, each of which had the same types of items. They gave the three tests 
to a group of students and they found that the correlation between Form 1 and 
Form 2 was 0.74. The correlation between Form 1 and Form 3 was 0.75, and the 
correlation between Form 2 and Form 3 was 0.85. Which two forms should the 
test developers choose?
Question 3
Test developers were attempting to write a test of ESL Business English, and 
therefore as one part of the validity study they needed to find evidence that the 

---

267
Assessment
test was measuring this construct. They did this, in part, by examining the scores 
obtained on the tests by four different groups of examinees. The four groups were 
chosen because of their predicted differences in ability in Business English. Can 
you predict which group the researchers hoped would perform the best on the 
test, the second best, etc? Place each of the four groups in the appropriate place in 
Table 15.5:
• Native speakers of English, business majors (NSBM).
• Non-native speakers of English, business majors (NNSBM).
• Native speakers, non-business majors (NSNBM).
• Non-native speakers, non-business majors (NNSNBM).
Predicted test 
performance
Lowest scores
Third-highest 
scores
Second-highest 
scores
Highest scores
Group
Table 15.5 Developing a test of ESL Business English
Question 4
In order to improve their testing of language ability, many testing programmes 
are attempting to develop tests that are delivered and scored by computer, and 
as a part of the development process, validation studies are conducted. One type 
of validation study looks at the correlation of performance on the computer-
delivered test with performance on a similar paper and pencil test. If the purpose 
of the computer-delivered test is to assess language ability better than has been 
possible with paper and pencil tests, what kind of relationship should be expected 
as the ideal outcome from such a study and why? (A strong correlation would 
be approaching 1 (for example, Pearson’s r = 0.92), and a low correlation would 
approach 0 (for example, Pearson’s r = 0.23).)
Question 5
In a large test revision project, testing researchers were hoping to develop a better 
test of academic language ability. To do so they tested several different tests, 
including tests of listening, vocabulary, grammar and reading. They found that 
they only had enough time to include either the reading or the grammar test in 
the final version, and they found that the correlations of the grammar test with 
the other language tests were somewhat stronger than those between the reading 
test and other tests, but in the end they chose to include the reading test rather 
than the grammar test. Why do you think they would do this?
Question 6
As a teacher in a language programme, you have developed an achievement test for 
your course that helps you to focus your teaching on particular reading strategies 
throughout the semester, and that seems to assess your students in a manner 
that is fair and useful as input to the grading process. In looking for a means of 
reporting outcomes to an external agency, your programme administrator takes a 
look at your test and then starts questioning its validity for the purpose that she 
is concerned about. She wants you to do a correlational study of your final exam 
with another reading test to see if it is valid. What should you tell her?

---

Suggested Solutions
Chapter 2, 
Grammar
1 Form. In English, a specific number or number-like phrase in English used as an 
adjective is always singular before the noun it modifies.
2 Meaning. This sentence is accurate and meaningful, but it is not likely the 
meaning that the student intends. The -ing participle refers to the cause, not 
the experiencer of the emotion.
3 Use. It is not as acceptable to use a stative verb (for example, want) with the 
passive voice as it is to use a verb denoting physical action (for example, score).
4 Meaning. The logical connector ‘on the contrary’ usually denies a proposition. 
A connector like ‘in contrast’, to compare two things, would be better.
5 Use. The historical present modal form, that is, will, is likely to be used in a 
response to a request, for example, ‘Of course, I will’.
6 Meaning. Few has a negative connotation. Although signals a contrast. One 
would therefore expect either a quantifier with a positive connotation to be 
used, such as a few or a causal connector, such as because. Thus we would expect 
either: ‘Although he had a few close friends, he was very lonely’ or ‘Because he had