whether teachers determined as highly
effective are also ( 1) teachers with more
years of experience, ( 2) teachers whose
supervisors or peers would also be classified as highly effective, ( 3) teachers
who received high scores on their
teacher licensure tests, ( 4) teachers who
have higher levels of education, or
( 5) teachers who have received teaching
awards and honors, are National Board
certified, and the like.
Moreover, personnel in the districts
and schools that have implemented the
model do not seem to be using the data
in the expected and promoted ways.
This is largely because of the confusing
data reports and a lack of professional
development opportunities to help
teachers and administrators understand
the model’s output.
Limitation 3:
Lack of Transparency
There has been insufficient external
examination of the EVAAS model to
inform recommendations or regulatory
decisions about its use, benefits, and
risks. The question here is whether
there have been enough empirical
studies conducted to warrant the federal
and state education policies mandating
the use of this system.
The model’s developers have not
completely opened up their system—in
particular, the computational algorithms
used to analyze test data—to external or
peer review. Nor have they released any
value-added data they have collected to
enable other researchers to verify the
claims they make. This makes scientific
research by external statisticians nearly
impossible, limiting researchers’
capacity to make sound recommendations about the model to inform education policies and provide consumers
with the facts they need to make their
own “regulatory” decisions.
In 1997, developers asserted that they
had undertaken “extensive efforts” to
This model
discriminates
against teachers
who have
larger classes.
increase understanding of the system,
formerly known as the Tennessee Value-Added Assessment System (TVAAS),
and they explained the system in great
detail. They also stated that “detailed
external reviews from both the statistical
and educational evaluation communities
have confirmed that the properties of
the TVAAS results are as claimed”
(Sanders, 1998, p. 26)—but they didn’t
provide citations or references to these
external reviews. Four sets of external
reviewers examined the assessment
system in depth: Two reviewers praised
the system, one reviewer raised significant points of contention, and the last
reviewer was one of the model’s developers (Sanders & Wright, 2008).
Educating the
Education Consumer
In all fairness, all value-added models
are flawed, especially when it comes to
their reliance on standardized tests and
the assumptions about what these tests
can reveal. The EVAAS model is the
most sophisticated, or the least inferior,
of these models.
Nevertheless, should the issues that
contaminate the practicality of the
EVAAS model warrant its removal from
the market? Yes, at least until external
reviewers can verify the model’s
assumptions about what standardized
tests can reveal, validate the inferences
drawn about students and teachers,
begin necessary internal and external
research studies, answer commonsense
questions, and inform consumers about
the system’s benefits and risks.
We need to take our education health
as seriously as we take our physical
health. Education consumers should get
to know the model before education
policymakers force them to blindly
accept it, simply because the theory
behind it makes sense. And they should
have the opportunity to learn about the
benefits and risks of the EVAAS
approach because, in the end, they—
and not the software developers or the
system builders—will experience the
side effects. EL
References
American Educational Research Association
(AERA). (2000). AERA position statement
on high-stakes testing in PreK- 12 education.
Available: www.aera.net/?id=378
Battelle for Kids. (n.d.). High-school value-added project. Retrieved February 1, 2007,
from www.battelleforkids.org.
Bracey, G. W. (2007, May 1). Value
subtracted: A “debate” with William
Sanders. The Huffington Post. Available:
www.huffingtonpost.com/gerald-bracey/
value-subtracted-a-debate_b_47404.html
SAS. (2007). Dr. William L. Sanders. Available: www.sas.com/govedu/edu/bio
_sanders.html
Sanders, W. L. (1998). Value-added assessment. The School Administrator, 55( 11),
24–27.
Sanders, W. L., & Horn, S. P. (1994). The
Tennessee Value-Added Assessment
System (TVAAS): Mixed-model methodology in educational assessment. Journal
of Personnel Evaluation in Education, 8( 3),
299–311.
Sanders, W. L., & Wright, S. P. (2008, April
14). A response to Amrein-Beardsley (2008):
“Methodological concerns about the Education Value-Added Assessment System.”
Available: www.sas.com/govedu/edu/services
/Sanders_Wright_response_to
_Amrein-Beardsley_ 4_ 14_2008.pdf
Audrey Amrein-Beardsley is Assistant
Professor in the College of Teacher
Education and Leadership at Arizona
State University, Phoenix; audrey
. beardsley@asu.edu.
