Educational Leadership - September 2015

the right thinking and those with the wrong thinking can answer a question in the same way, it’s not very valuable as a diagnostic tool. The aim is always to have students with the right thinking and students with the wrong thinking give different answers.

The following question about mea-
surement in science, from Stanford
University science education professor
Jonathan Osborne (2011), illustrates
how we can do this:
Janet was asked to do an experiment to
find how long it takes for some sugar to
dissolve in water. What advice would
you give Janet to tell her how many
repeated measurements to take?

A. Two or three measurements are always enough.

B. She should take five measurements. C. If she is accurate, she only needs to measure once.

D. She should go on taking measurements until she knows how much they vary.

E. She should go on taking measurements until she gets two or more the same.

This question has a single correct response (it’s D), and so students have a 20 percent chance of answering the question correctly. Obviously, in a class of 30, six students would likely guess the correct response.

But this question is so well designed that students who don’t understand the main point are highly unlikely to get the correct answer. The incorrect responses—what test developers call distractors—are so plausible that they’re attractive to students with incomplete understandings.

One of the best ways to develop
such questions is to start from the
partial or incomplete understandings
that students have and generate ques-
tions that help identify which students
have which misconceptions. The aim
is to create what Philip Sadler (1998)
calls distractor-driven assessments.
People with deep knowledge of
a subject, but little knowledge of
teaching it to school-age students,
often find it difficult to generate good
distractors. They know the subject, but
they don’t know the difficulties that
students have with it. This latter kind
of knowledge—what Lee Shulman
(1986) called pedagogical content
knowledge and Deborah Ball and col-
leagues call knowledge for teaching
(Hill, Rowan, & Ball, 2005)—is
developed only through sustained
experience working with students.

I suggested earlier that we could check whether students know what an adverb is by asking them to identify the adverb in the following sentence:

The boy ran quickly across the street.

However, students with an incorrect
understanding can still answer the
question correctly. Many students
believe that an adverb usually follows
the verb, so they may identify quickly
as an adverb simply because it follows
the verb ran. Offering an alternative
version of the sentence, with the
adverb moved to the end, will likely be
more effective:
The boy ran across the street quickly.
(A) (B) (C) (D) (E)

Students who believe that an adverb typically comes immediately after the verb will now give a different response than those with a correct understanding.

Hinge Questions with
Multiple Correct Responses

Another way of making it less likely that students get the right response to a question for the wrong reason is by offering multiple correct options. When a question has multiple correct answers, the chances of a student getting the correct response by guessing drop markedly.

For example, the teacher might ask
a class to identify all adverbs in the fol-
lowing sentence:
Fred ran the race well, but unsuccessfully.
(A) (B) (C) (D) (E)

If students aren’t told how many of the options are correct, then there are 32 possible responses. (Students have to make a separate choice about the correctness of each of the five options, so there are 2 x 2 x 2 x 2 x 2 combinations.) Students are far less likely to guess the correct answer when faced with 32 possible choices as opposed to the one-in-five chance that the single correct response offered.