The Common European Framework in its political and educational context What is the Common European Framework?

Assessment
189

---

the person concerned over a period of time. Very many school systems operate on this
basis. 
The term ‘guided judgement’ is here used to describe the situation in which that
impression is guided into a considered judgement through an assessment approach.
Such an approach implies (a) an assessment activity with some form of procedure, and/or
(b) a set of defined criteria which distinguish between the different scores or grades, and
(c) some form of standardisation training. The advantage of the guided approach to
judging is that if a common framework of reference for the group of assessors concerned
is established in this way, the consistency of judgements can be radically increased. This
is especially the case if ‘benchmarks’ are provided in the form of samples of performance
and fixed links to other systems. The importance of such guidance is underlined by the
fact that research in a number of disciplines has shown repeatedly that with untrained
judgements the differences in the severity of the assessors can account for nearly as
much of the differences in the assessment of learners as does their actual ability, leaving
results almost purely to chance.
The scales of descriptors for the common reference levels can be exploited to provide
a set of defined criteria as described in (b) above, or to map the standards represented by
existing criteria in terms of the common levels. In the future, benchmark samples of per-
formance at different common reference levels may be provided to assist in standardisa-
tion training.
9.3.11
Holistic/analytic
Holistic assessment is making a global synthetic judgement. Different aspects are weighted
intuitively by the assessor.
Analytic assessment is looking at different aspects separately.
There are two ways in which this distinction can be made: (a) in terms of what is looked
for; (b) in terms of how a band, grade or score is arrived at. Systems sometimes combine
an analytic approach at one level with a holistic approach at another.
a)
What to assess: some approaches assess a global category like ‘speaking’ or ‘inter-
action’, assigning one score or grade. Others, more analytic, require the assessor to
assign separate results to a number of independent aspects of performance. Yet
other approaches require the assessor to note a global impression, analyse by dif-
ferent categories and then come to a considered holistic judgement. The advantage
of the separate categories of an analytic approach is that they encourage the asses-
sor to observe closely. They provide a metalanguage for negotiation between asses-
sors, and for feedback to learners. The disadvantage is that a wealth of evidence
suggests that assessors cannot easily keep the categories separate from a holistic
judgement. They also get cognitive overload when presented with more than four
or five categories.
b)
Calculating the result: some approaches holistically match observed performance to
descriptors on a rating scale, whether the scale is holistic (one global scale) or ana-
lytic (3–6 categories in a grid). Such approaches involve no arithmetic. Results are
reported either as a single number or as a ‘telephone number’ across categories.
Other more analytical approaches require giving a certain mark for a number of dif-
Common European Framework of Reference for Languages: learning, teaching, assessment
190

---

ferent points and then adding them up to give a score, which may then convert into
a grade. It is characteristic of this approach that the categories are weighted, i.e. the
categories do not each account for an equal number of points.
Tables 2 and 3 in Chapter 3 provide self-assessment and examiner assessment examples
respectively of analytic scales of criteria (i.e. grids) used with a holistic rating strategy (i.e.
match what you can deduce from the performance to the definitions, and make a judge-
ment).
9.3.12
Series assessment/category assessment
Category assessment involves a single assessment task (which may well have different
phases to generate different discourse as discussed in section 9.2.1.) in which perfor-
mance is judged in relation to the categories in an assessment grid: the analytic
approach outlined in 9.3.11. 
Series assessment involves a series of isolated assessment tasks (often roleplays with
other learners or the teacher), which are rated with a simple holistic grade on a labelled
scale of e.g. 0–3 or 1–4.
A series assessment is one way of coping with the tendency in category assessments
for results on one category to affect those on another. At lower levels the emphasis tends
to be on task achievement, the aim is to fill out a checklist of what the learner can do on
the basis of teacher/learner assessment of actual performances rather than simple
impression. At higher levels, tasks may be designed to show particular aspects of profi-
ciency in the performance. Results are reported as a profile.
The scales for different categories of language competence juxtaposed with the text in
Chapter 5 offer a source for the development of the criteria for a category assessment.
Since assessors can only cope with a small number of categories, compromises have to
made in the process. The elaboration of relevant types of communicative activities in
section 4.4. and the list of different types of functional competence outlined in section
5.2.3.2 may inform the identification of suitable tasks for a series assessment.
9.3.13
Assessment by others/self-assessment
Assessment by others: judgements by the teacher or examiner.
Self-assessment: judgements about your own proficiency.
Learners can be involved in many of the assessment techniques outlined above.
Research suggests that provided ‘high stakes’ (e.g. whether or not you will be accepted
for a course) are not involved, self-assessment can be an effective complement to tests and
teacher assessment. Accuracy in self-assessment is increased (a) when assessment is in
relation to clear descriptors defining standards of proficiency and/or (b) when assessment
is related to a specific experience. This experience may itself even be a test activity. It is
also probably made more accurate when learners receive some training. Such structured
self-assessment can achieve correlations to teachers’ assessments and tests equal to the
correlation (level of concurrent validation) commonly reported between teachers them-
selves, between tests and between teacher assessment and tests.
Assessment
191

---

The main potential for self-assessment, however, is in its use as a tool for motivation
and awareness raising: helping learners to appreciate their strengths, recognise their
weaknesses and orient their learning more effectively.
Self-assessment and examiner versions of rating grids are presented in Table 2 and in
Table 3 in Chapter 3. The most striking distinction between the two – apart from the
purely surface formulation as I can do . . . or Can do . . . is that whereas Table 2 focuses on
communicative activities, Table 3 focuses on generic aspects of competence apparent in
any spoken performance. However, a slightly simplified self-assessment version of Table
3 can easily be imagined. Experience suggests that at least adult learners are capable of
making such qualitative judgements about their competence.