The Common European Framework in its political and educational context What is the Common European Framework?

No 12.
(a)
Data from the qualitative techniques Nos 6, 7 or 8 can be put onto an
arithmetic scale with Rasch.
No 12.
(b)
Tests can be carefully developed to operationalise proficiency descriptors
in particular test items. Those test items can then be scaled with Rasch
and their scale values taken to indicate the relative difficulty of the
descriptors (Brown et al. 1992; Carroll 1993; Masters 1994; Kirsch 1995;
Kirsch and Mosenthal 1995). 
No 12.
(c)
Descriptors can be used as questionnaire items for teacher assessment of
their learners (Can he/she do X?). In this way the descriptors can be
calibrated directly onto an arithmetic scale in the same way that test
items are scaled in item banks.
No 12.
(d)
The scales of descriptors included in Chapters 3, 4 and 5 were developed
in this way. All three projects described in Appendices B, C and D have
used Rasch methodology to scale descriptors, and to equate the resulting
scales of descriptors to each other.
No 12.
In addition to its usefulness in the development of a scale, Rasch can also be
used to analyse the way in which the bands on an assessment scale are
actually used. This may help to highlight loose wording, underuse of a band,
or overuse of a band, and inform revision (Davidson 1992; Milanovic et al.
1996; Stansfield and Kenyon 1996; Tyndall and Kenyon 1996).
Appendix A: developing proficiency descriptors
211
Test A
Test B
Test C

---

Select annotated bibliography: language proficiency scaling
Alderson, J.C. 1991: Bands and scores. In: 
Discusses problems caused by confusion of purpose 
Alderson, J.C. and North, B. (eds.): Language 
and orientation, and development of IELTS speaking 
testing in the 1990s, London: British Council/
scales.
Macmillan, Developments in ELT, 71–86.
Brindley, G. 1991: Defining language ability: 
Principled critique of the claim of proficiency scales 
the criteria for criteria. In Anivan, S. (ed.) 
to represent criterion-referenced assessment.
Current developments in language testing, 
Singapore, Regional Language Centre.
Brindley, G. 1998: Outcomes-based assessment 
Criticises the focus on outcomes in terms of what
and reporting in language learning 
learners can do, rather than focusing on aspects of
programmes, a review of the issues. Language 
emerging competence.
Testing 15 (1), 45–85.
Brown, Annie, Elder, Cathie, Lumley, Tom, 
Classic use of Rasch scaling of test items to produce 
McNamara, Tim and McQueen, J. 1992: Mapping a proficiency scale from the reading tasks tested in 
abilities and skill levels using Rasch techniques. 
the different items.
Paper presented at the 14th Language Testing 
Research Colloquium, Vancouver. Reprinted in 
Melbourne Papers in Applied Linguistics 1/1, 37–69.
Carroll, J.B. 1993: Test theory and behavioural 
Seminal article recommending the use of Rasch to 
scaling of test performance. In Frederiksen, N., 
scale test items and so produce a proficiency scale.
Mislevy, R.J. and Bejar, I.I. (eds.) Test theory for a 
new generation of tests. Hillsdale N.J. Lawrence 
Erlbaum Associates: 297–323.
Chaloub-Deville M. 1995: Deriving oral 
Study revealing what criteria native speakers of 
assessment scales across different tests and 
Arabic relate to when judging learners. Virtually the 
rater groups. Language Testing 12 (1), 16–33.
only application of multi-dimensional scaling to
language testing.
Davidson, F. 1992: Statistical support for 
Very clear account of how to validate a rating scale 
training in ESL composition rating. In Hamp-
in a cyclical process with Rasch analysis. Argues for 
Lyons (ed.): Assessing second language writing in 
a ‘semantic’ approach to scaling rather than the 
academic contexts. Norwood N.J. Ablex: 155–166.
‘concrete’ approach taken in, e.g., the illustrative
descriptors.
Fulcher 1996: Does thick description lead to 
Systematic approach to descriptor and scale 
smart tests? A data-based approach to rating 
development starting by proper analysis of what is 
scale construction. Language Testing 13 (2), 
actually happening in the performance. Very time-
208–38.
consuming method.
Users of the Framework may wish to consider and where appropriate state:
•
the extent to which grades awarded in their system are given shared meaning through
common definitions;
•
which of the methods outlined above, or which other methods, are used to develop such
definitions.
Appendix A: developing proficiency descriptors
212

---

Gipps, C. 1994: Beyond testing. London, Falmer 
Promotion of teacher ‘standards-oriented 
Press.
assessment’ in relation to common reference points
built up by networking. Discussion of problems
caused by vague descriptors in the English National
Curriculum. Cross-curricula.
Kirsch, I.S. 1995: Literacy performance on three Simple non-technical report on a sophisticated use of 
scales: definitions and results. In Literacy, 
Rasch to produce a scale of levels from test data. 
economy and society: Results of the first 
Method developed to predict and explain the 
international literacy survey. Paris, Organisation 
difficulty of new test items from the tasks and 
for Economic Cooperation and development 
competences involved – i.e. in relation to a 
(OECD): 27–53.
framework.
Kirsch, I.S. and Mosenthal, P.B. 1995: 
Interpreting the IEA reading literacy scales. In 
More detailed and technical version of the above 
Binkley, M., Rust, K. and Wingleee, M. (eds.) 
tracing the development of the method through three 
Methodological issues in comparative 
related projects.
educational studies: The case of the IEA 
reading literacy study. Washington D.C.: US 
Department of Education, National Center for 
Education Statistics: 135–192.
Linacre, J. M. 1989: Multi-faceted Measurement. 
Seminal breakthrough in statistics allowing the 
Chicago: MESA Press.
severity of examiners to be taken into account in
reporting a result from an assessment. Applied in
the project to develop the illustrative descriptors to
check the relationship of levels to school years. 
Liskin-Gasparro, J. E. 1984: The ACTFL 
Outline of the purposes and development of the 
proficiency guidelines: Gateway to testing and 
American ACTFL scale from its parent Foreign 
curriculum. In: Foreign Language Annals 17/5, 
Service Institute (FSI) scale.
475–489.
Lowe, P. 1985: The ILR proficiency scale as a 
Detailed description of the development of the US 
synthesising research principle: the view from 
Interagency Language Roundtable (ILR) scale from 
the mountain. In: James, C.J. (ed.): Foreign 
the FSI parent. Functions of the scale.
Language Proficiency in the Classroom and Beyond.
Lincolnwood (Ill.): National Textbook 
Company.
Lowe, P. 1986: Proficiency: panacea, framework, Defence of a system that worked well – in a specific 
process? A Reply to Kramsch, Schulz, and 
context – against academic criticism prompted by the