Measuring student knowledge and skills

Measuring Student Knowledge and Skills
22
OECD 1999
Almond and Mislevy (1998) note that such variables can take on one of five roles in an assessment.
They can be used to:
– limit the scope of the assessment;
– characterise features that should be used for constructing tasks;
– control the assembly of tasks into booklets or test forms;
– characterise students’ performance in or responses to tasks; or to
– help to characterise aspects of competencies and proficiencies.
Some of these variables can be used to help both in the construction of tasks and in the understand-
ing of competencies, as well as in the characterisation of performance.
A finite number of the task characteristics most relevant for measuring student performance have
been selected for constructing and marking tasks. These characteristics are components of the reading
process that will be manipulated in the OECD/PISA reading literacy survey to simulate and evaluate the
interactive nature of the reading process. They include:
–
Situation: Since adults do not read written texts in a vacuum but within a particular situation, it is
important to identify a range of situations from which materials for this reading literacy assessment
can be sampled. It should be remembered that one of the goals of OECD/PISA is to move beyond
classroom-based texts to include a range of materials that students will also encounter outside
their classrooms.
–
Texts: While no one would doubt that a reading literacy assessment should include a variety of
material, what is critical to the design and interpretation of the scores that are produced is the
range and specific features of the text material which is included in constructing the tasks. Thus, a
broad range of both continuous and non-continuous text types are included in OECD/PISA, and
consideration will be given to having students read these materials both singly and in combina-
tion. That is, they might be asked to read two continuous texts covering a similar topic, or a contin-
uous text and a non-continuous text such as a graph or table.
–