a multiple-choice section that
didn’t require writing but
couldn’t measure extended
thinking about the novel.
Another measure was an
assignment in which students
wrote open-ended questions
at the end of each chapter;
this task revealed students’
thinking about the plot
without requiring much
formal writing. The teacher
combined grades from all
three measures to give a richer
picture of plot understanding
for all students in the class.
School-Level and Policy Decisions
If our main concern is to know for
certain whether a school has reached a
goal on a particular achievement
construct (for example, a certain level of
reading or mathematics performance),
then we might want to use a compensatory approach combining multiple
measures of that construct. If false negatives are a major concern—for example,
if severe consequences are in place for
failing to meet a standard—then we
might want to use complementary
multiple measures so that a school can
pass by meeting the standard on any
one measure. But if we are convinced
that each of several measures is vital to
quality, we’ll probably want to use a
conjunctive approach in which a
school must pass all measures.
When states design high school
graduation policies, a multiple-measures policy can help them avoid
defining achievement narrowly as
performance on one test. Darling-Hammond, Rustique-Forrester, and
Pecheone (2005) reported that graduation rates stayed the same or declined
slightly from 1998 to 2001 in five states
that required students to pass an exit
exam (Indiana, North Carolina, New
York, Florida, and South Carolina).
Four states that used a multiple-measures approach to graduation
during that time (New Jersey,
Wisconsin, Pennsylvania, and
Connecticut) fared better: Rates stayed
the same in three states and rose in one.
In addition, the graduation rates for
these four states were higher overall in
2001 ( 73–86 percent) than those for
the five exam-only states ( 51–67
percent).
Evaluations of school programs are
also best accomplished by using
multiple measures of different
constructs. For example, suppose a
district wanted to evaluate its K–12
science curriculum. One obvious
measure would be performance on
state exams mapped to district science
standards—both absolute levels of
achievement (status) and the amount
of change in achievement (growth).
To make wise decisions about the
science curriculum, however, the
district would probably want to
include other measures—for example,
the number of graduates who go on to
major in science or work in a scientific
field, how students perceive the importance of science or how confident they
feel as science learners, student participation in science-related clubs and
activities, and so on. These are all
different constructs, but they all have a
bearing on making decisions about the
science program.
Multiple Measures for
Meaningful Decisions
EL
© STEFANIE FELIX
The term multiple measures can
mean many things. What’s
important is that multiple
measures result in meaningful,
useful decisions. A clear understanding of the many faces of
multiple measures helps us
think about the logic used in
each case. Wise actions can
only result if the measures and
logic are right for their
intended purposes.
References
American Educational Research
Association, American Psychological
Association, & National Council on
Measurement in Education. (1999).
Standards for educational and psychological
testing. Washington, DC: Authors.
Chester, M. D. (2005). Making valid and
consistent inferences about school effectiveness from multiple measures.
Educational Measurement: Issues and Practice,
24( 4), 40–52.
Darling-Hammond, L., Rustique-Forrester,
E., & Pecheone, R. L. (2005). Multiple
measures approaches to high school graduation. Stanford, CA: Stanford University
School Redesign Network. Available:
www.srnleads.org/data/pdfs/multiple
_measures.pdf
National Council on Measurement in Education. (1995). Code of professional responsibilities in educational measurement.
[Online]. Available: www.natd.org/Code
_of_Professional_Responsibilities.html
No Child Left Behind Act of 2001, Pub. L.
No. 107-110, Sec. 1111(b)( 3)(C)(vi).
Available: www.ed.gov/policy/elsec
/leg/esea02/ pg2.html
Schafer, W. D. (2003). A state perspective on
multiple measures and school accountability. Educational Measurement: Issues and
Practice, 22( 3), 27–31.
Susan M. Brookhart is Senior Research
Associate at the Center for Advancing
the Study of Teaching and Learning
(CASTL), Duquesne University, Pittsburgh, Pennsylvania. She is the author of
Exploring Formative Assessment (ASCD,
2009) and How to Give Effective Feed-back to Your Students (ASCD, 2008);
susanbrookhart@bresnan.net.
