Measuring student knowledge and skills

Example item with a constructed context
Which of the following numerical representations could be used to solve the problem? Bill
weighed 107 pounds last summer. He lost 4 pounds, and then he gained 11 pounds. How much
does he weigh now?
a)
107 - (4 + 11) = A
b)
(107 - 4) + 11 = A
c)
(107 + 11) + 4 = A
d)
-4 + 11 = 107 + A
e)
(107 - 11) + 4 = A
Figure 8.
Example item with a constructed context
Which of the following numerical representations could be used to solve the problem? Bill
weighed 107 pounds last summer. He lost 4 pounds, and then he gained 11 pounds. How much
does he weigh now?
a)
107 - (4 + 11) = A
b)
(107 - 4) + 11 = A
c)
(107 + 11) + 4 = A
d)
-4 + 11 = 107 + A
e)
(107 - 11) + 4 = A
Figure 9.
Example item with a “virtual” context
Would it be possible to establish a coinage system (or a stamp system) based on only the
denominations 3 and 5? More specifically, what amounts could be reached on that basis? If
possible, would such a system be desirable?
Figure 9.
Example item with a “virtual” context
Would it be possible to establish a coinage system (or a stamp system) based on only the
denominations 3 and 5? More specifically, what amounts could be reached on that basis? If
possible, would such a system be desirable?

---

Mathematical Literacy
53
OECD 1999
how the answer was reached. The key feature of open constructed-response items is that they allow stu-
dents to demonstrate their abilities by providing solutions at a range of levels of mathematical complex-
ity. The item in Figure 12 is an example.
For OECD/PISA, 25-35 per cent of the testing time devoted to mathematics will be allocated to open
constructed-response items. These items require marking by a trained person who implements a mark-
ing rubric that may require an element of professional judgement. Because of the potential for disagree-
ment between markers of these items, OECD/PISA will implement marker reliability studies to monitor
the extent of disagreement. Experience in these types of studies shows that clear marking rubrics can be
developed and reliable scores can be obtained.
The first survey cycle of OECD/PISA will make use of a task format in which several items are linked
to common stimulus material. Tasks of this format give students the opportunity to become involved with
a context or problem by asking a series of questions of increasing complexity. The first few questions
are typically multiple-choice or closed-constructed items while subsequent items are typically
open-constructed items. This format is appropriate for all competency classes.
A seal has to breathe even if it is asleep. Martin observed a seal for one hour. At the start of
his observation the seal dove to the bottom of the sea and started to sleep. In 8 minutes it
slowly floated to the surface and took a breath.
In 3 minutes it was back at the bottom of the sea again and the whole process started over
in a very regular way.
After one hour the seal was:
a)
at the bottom
b)
on its way up
c)
breathing
d)
on its way down
Figure 10.