Measuring student knowledge and skills

Recommended distribution of score points across science processes
Scientific processes
% scored points
Recognising scientifically investigable questions
10-15
Identifying evidence needed
15-20
Drawing or evaluating conclusions
15-20
Communicating valid conclusions
10-15
Demonstrating understanding of science concepts
40-50

---

Scientific Literacy
69
OECD 1999
It may well be that the topics of some tasks mean that the balance is tipped more towards assess-
ment of understanding (Process 5), with the opposite occurring within other tasks. Where possible, items
assessing Processes 1-4 and items assessing Process 5 will occur within each task, both to achieve the aim
of covering important scientific concepts that students are likely to have developed from their school
science curricula or outside school, and because the ability to use processes is very highly dependent
upon the situation in which they are used (hence the processes need to be assessed in relation to a range
of concepts). The aims of OECD/PISA suggest that both conceptual understanding and the combination
of scientific knowledge with the ability to draw evidence-based conclusions are valued learning out-
comes. The recommended target of roughly equal numbers of score points assigned to these two main
kinds of learning outcomes should serve these aims.
As already noted, all types of items will be concerned with the application of scientific concepts that
are likely to be developed in students through their school science curricula. Where the OECD/PISA
science items differ from some – but by no means all – school science assessment is in their requirement
that the concepts be applied in real-life situations. Similarly, the ability to draw evidence-based conclu-
sions appears among the aims of many school science curricula. The OECD/PISA assessment will require
the application of the processes in situations which go beyond the school laboratory or classroom. The
extent to which this is novel to students will depend on how far applications in the real world are part of
the curriculum they have experienced.
In relation to the areas of application, Table 11 shows that there will be as even a spread as possible
across the three main groups.
In relation to the selection of situations, OECD/PISA will aim to spread the items evenly across the
four identified situations: the personal, the community, the global and the historical.
The situations represented in the tasks will be defined by means of stimulus material, which may be
a short written passage, or writing accompanying a table, chart, graph or diagram. The items will be a set
of independently scored questions requiring the types of response indicated in Figure 19. Note that
these include the ability to show understanding of scientific concepts by applying them in the areas of
application. The responses required will relate to the situations and areas of application presented in
the stimulus.
Until trials have taken place and the results have been analysed, it is not possible to be certain
about, for example, the extent of uniformity across tasks in terms of the number of items within them,
whether they relate to more than one area of application, or how items of different formats will be
arranged. However we can summarise what is envisaged at this stage:
– All tasks will be extended, not single item tasks; they will include items assessing one or more of
the scientific processes (Figure 14), knowledge of scientific concepts (Figure 15) and knowledge
relating to one or more of the areas of application of science (Figure 16), and will require answers
on paper (writing or drawing).
– Most tasks will be presented in written form, certainly for the survey in 2000, although the use of
stimuli in other forms will be investigated for the year 2006, when science is the major element.
Table 11.