Educational Leadership - November 2012

that states and districts can increase
responding to the new systems.

If We Do This, Let’s Do It Right

accuracy. One basic step would be to
Another important detail is the
Test-based teacher evaluations are
require that at least two or three years
accuracy of the large administrative
probably the most controversial issue
of data be accumulated for teachers
data sets used to calculate value-added
in U.S. education policy today. In the
before counting their value-added
scores. These data sets must be con-
public debate, both sides have focused
scores toward their evaluation (or,
tinually checked for errors (for example, almost exclusively on whether to include
alternatively, varying the weight of
in the correct linking of students with
value-added measures in new evaluation
value-added measures by sample size).

teachers), and teachers must have an
systems. Supporters of value-added
Larger samples make for more precise
opportunity to review their class rosters scoring say it should dominate evalua-
estimates and have also been shown
every year to ensure they are being eval- tions, whereas opponents say it has no
to mitigate some forms of systematic
error (Koedel & Betts, 2011). Value-
added estimates can also be adjusted
uated for the progress of students they
actually teach.

legitimate role at all. It is as much of a
mistake to use value-added estimates
carelessly as it is to refuse to consider
(“shrunken”) according to sample
them at all.

size, which can reduce the noise from random error (Ballou, Sanders, &

By themselves,

Error is inevitable, no matter which
measures you use and how you use
Wright, 2004).

Second, even when sample sizes are larger, states and districts should directly account for the aforementioned confidence intervals. One of the advantages of value-added models is that, unlike with observations, you can actually measure

value-added data
are neither good nor
bad. It is how we use
them that matters.

them. But responsible policymakers will do what they can to mitigate imprecision while preserving the information the measures transmit. It is not surprising that many states and districts have neglected some of these steps.

They were already facing budget cuts
some of the error in practice. Accounting
and strained capacity before having
for it does not, of course, ensure that the
to design and implement new teacher
estimates are valid—that the models are
Finally, each state should arrange for evaluations in a short time frame. This
measuring unbiased causal effects—but
a thorough, long-term, independent
was an extremely difficult task.

it at least means you will be interpreting research evaluation of new systems,
Luckily, in many places, there is still
the information you have in the best
starting right at the outset. There are
time. Let’s use that time wisely. EL

possible manner. The majority of states few prospects more disturbing than
and districts are ignoring this basic
the idea of making drastic, sweeping
requirement.

changes in how teachers are evaluated but never knowing how these changes

Continually monitor results

have worked out.

and evaluate the evaluations.

All these exercises should be accom-
This final recommendation may sound
panied by a clear path to making
like a platitude in the era of test-based
changes based on the results. It is dif-
accountability, but it is too important to ficult to assess the degree to which
omit. States and districts that implement states and districts are fulfilling this rec-
new systems must thoroughly analyze
ommendation. No doubt all of them are
the results every single year. They need performing some of these analyses and
to check whether value-added estimates would do more if they had the capacity.

(or evaluation scores in general) vary systematically by student, school, or teacher characteristics; how value-added

EL Online

scores match up with the other components (see Jacob & Lefgren, 2008);

and how sensitive final ratings are to changes in the weighting and scoring of the components. States also need to monitor how stakeholders, most

For another perspective on
the use of value-added data,
see the online-only article
“Value-Added: The Emperor with
No Clothes” by Stephen J. Caldas at
www.ascd.org/el1112caldas.

notably teachers and administrators, are

References

Baker, E., Barton, P., Darling-Hammond, L., Haertel, E., Ladd, H., Linn, R., et al. (2010). Problems with the use of student test scores to evaluate teachers (Briefing paper 278). Washington, DC: Economic Policy Institute.

Ballou, D., Sanders, W., & Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29( 1), 37–65.

Chetty, R., Friedman, J., & Rockoff, J. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood (NBER Working Paper 17699). Washington, DC: National Bureau of Economic Research.

Corcoran, S. (2010). Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. New York: Annenberg Institute.