The Common European Framework in its political and educational context What is the Common European Framework?

Appendix B: The illustrative scales of descriptors
218

---

descriptors in relation to different educational sectors, language regions and
target languages in order to identify descriptors with a very high stability of
values across different contexts to use in constructing holistic scales
summarising the Common Reference Levels.
7.
Performance assessment by all participating teachers of videos of some of the
learners in the survey. The aim of this assessment was to quantify differences in
severity of participating teachers in order to take such variation in severity into
account in identifying the range of achievement in educational sectors in
Switzerland.
Interpretation phase:
8.
Identification of ‘cut-points’ on the scale of descriptors to produce the set of
Common Reference Levels introduced in Chapter 3. Summary of those levels in a
holistic scale (Table 1), a self-assessment grid describing language activities (Table
2) and a performance assessment grid describing different aspects of
communicative language competence (Table 3). 
9.
Presentation of illustrative scales in Chapters 4 and 5 for those categories that
proved scaleable.
10. Adaptation of the descriptors to self-assessment format in order to produce a Swiss
trial version of the European Language Portfolio. This includes: (a) a self-
assessment grid for Listening, Speaking, Spoken Interaction, Spoken Production,
Writing (Table 2); (b) a self-assessment checklist for each of the Common Reference
Levels.
11. A final conference in which research results were presented, experience with the
Portfolio was discussed and teachers were introduced to the Common Reference
Levels. 
Results
Scaling descriptors for different skills and for different kinds of competences
(linguistic, pragmatic, sociocultural) is complicated by the question of whether or not
assessments of these different features will combine in a single measurement
dimension. This is not a problem caused by or exclusively associated with Rasch
modelling, it applies to all statistical analysis. Rasch, however, is less forgiving if a
problem emerges. Test data, teacher assessment data and self-assessment data may
behave differently in this regard. With assessment by teachers in this project, certain
categories were less successful and had to be removed from the analysis in order to
safeguard the accuracy of the results. Categories lost from the original descriptor pool
included the following:
Appendix B: The illustrative scales of descriptors
219

---

a)
Sociocultural competence
Those descriptors explicitly describing sociocultural and sociolinguistic competence. It
is not clear how much this problem was caused (a) by this being a separate construct
from language proficiency; (b) by rather vague descriptors identified as problematic in
the workshops, or (c) by inconsistent responses by teachers lacking the necessary
knowledge of their students. This problem extended to descriptors of ability to read
and appreciate fiction and literature.
b)
Work-related
Those descriptors asking teachers to guess about activities (generally work-related)
beyond those they could observe directly in class, for example telephoning; attending
formal meetings; giving formal presentations; writing reports & essays; formal
correspondence. This was despite the fact that the adult and vocational sectors were
well represented.
c)
Negative concept
Those descriptors relating to need for simplification; need to get repetition or
clarification, which are implicitly negative concepts. Such aspects worked better as
provisos in positively worded statements, for example:
Can generally understand clear, standard speech on familiar matters directed at him/her,
provided he/she can ask for repetition or reformulation from time to time.
Reading proved to be on a separate measurement dimension to spoken interaction and
production for these teachers. However, the data collection design made it possible to
scale reading separately and then to equate the reading scale to the main scale after
the event. Writing was not a major focus of the study, and the descriptors for written
production included in Chapter 4 were mainly developed from those for spoken
production. The relatively high stability of the scale values for descriptors for reading
and writing taken from the CEF being reported by both DIALANG and ALTE (see
Appendices C and D respectively), however, suggests that the approaches taken to
reading and to writing were reasonably effective.
The complications with the categories discussed above are all related to the scaling
issue of uni- as opposed to multi-dimensionality. Multi-dimensionality shows itself in a
second way in relation to the population of learners whose proficiency is being
described. There were a number of cases in which the difficulty of a descriptor was
dependent on the educational sector concerned. For example, adult beginners are
considered by their teachers to find ‘real life’ tasks significantly easier than 14 year
olds. This seems intuitively sensible. Such variation is known as ‘Differential Item
Function (DIF)’. In as far as this was feasible, descriptors showing DIF were avoided
when constructing the summaries of the Common Reference Levels introduced in
Tables 1 and 2 in Chapter 3. There were very few significant effects by target language,
and none by mother tongue, other than a suggestion that native speaker teachers may
Appendix B: The illustrative scales of descriptors
220

---

have a stricter interpretation of the word ‘understand’ at advanced levels, particularly
with regard to literature.
Exploitation
The illustrative descriptors in Chapters 4 and 5 have been either (a) situated at the level
at which that actual descriptor was empirically calibrated in the study; (b) written by
recombining elements of descriptors so calibrated to that level (for a few categories like
Public Announcements which were not included in the original survey), or (c) selected on
the basis of the results of the qualitative phase (workshops), or (d) written during the
interpretative phase to plug a gap on the empirically calibrated sub-scale. This last
point applies almost entirely to Mastery, for which very few descriptors had been
included in the study.
Follow up
A project for the university of Basle in 1999–2000 adapted CEF descriptors for a self-
assessment instrument designed for university entrance. Descriptors were also added
for sociolinguistic competence and for note taking in a university context. The new
descriptors were scaled to the CEF levels with the same methodology used in the
original project, and are included in this edition of the CEF. The correlation of the
scale values of the CEF descriptors between their original scale values and their values
in this study was 0.899.
References
North, B. 1996/2000: The development of a common framework scale of language proficiency. PhD thesis,
Thames Valley University. Reprinted 2000, New York, Peter Lang. 
forthcoming: Developing descriptor scales of language proficiency for the CEF Common
Reference Levels. In J.C. Alderson (ed.) Case studies of the use of the Common European Framework.
Council of Europe.
forthcoming: A CEF-based self-assessment tool for university entrance. In J.C. Alderson (ed.)
Case studies of the use of the Common European Framework. Council of Europe.
North, B. and Schneider, G. 1998: Scaling descriptors for language proficiency scales. Language
Testing 15/2: 217–262.
Schneider and North 1999: ‘In anderen Sprachen kann ich’ . . . Skalen zur Beschreibung, Beurteilung und
Selbsteinschätzung der fremdsprachlichen Kommunikationmsfähigkeit. Berne, Project Report,
National Research Programme 33, Swiss National Science Research Council.