it’s possible to attribute gains or losses in
student achievement solely to the
quality of instruction, independent of
other school and life factors. This is the
fundamental assumption on which the
system relies, which, if we were to seriously consider it, might cause a model
recall.
Test Data Irregularities
The EVAAS model requires complete
and high-quality longitudinal test data
that most states currently do not have.
Students sometimes miss
tests. Student test score data
are often not linked to
teacher names. Students are
often misreported by class
and grade level. And sometimes data show that students
jump from the top to the
bottom of the class, or vice
versa, from one year to the
next, which is nearly impossible in actuality.
Data errors like these,
which are often caused by
student mobility, missing test
scores, data-processing
errors, or students incorrectly
bubbling in their score
sheets, affect thousands of
student records. Model developers claim these things don’t
matter, that the system can operate
regardless.
Student Risk Factors
The EVAAS model does not control for
student risk factors, making it the only
sophisticated assessment model that
does not account for such things as
family income, ethnicity, and other
student background variables.
Developers of the model state that the
effects of these factors on student
growth are negligible. Yet educators
know too well that student background
variables unquestionably affect student
achievement and the progress students
make from year to year. How could the
achievement gap continue to widen if
these factors play no role?
Class Size
Statistical errors in test results
frequently occur when fewer students
are in a given class, a problem that
prevents truthful claims about the
quality of teachers with class sizes below
a certain number. In the EVAAS model,
general and special education teachers
who teach smaller classes are more
likely assumed to be average. An ineffective teacher who teaches a large
class might be penalized for being below
average, whereas an equally ineffective
teacher who teaches a smaller class may
go undetected. The larger the class, the
more “accurate” the estimate. This model
makes the process of evaluating teacher
quality unfair, discriminating against
teachers who have larger classes.
Grades and Subjects Tested
Only students in certain grade levels
must take the standardized accounta-
bility tests. This situation subjects
teachers to accountability measures in
some grades, but not in others. In addition, many of these tests only assess
students’ reading and mathematics
skills, exempting teachers who teach
other subjects from being held accountable in similar ways.
Teacher Effect
The EVAAS model is also incapable of
controlling for out-of-school learning
and the effects one teacher might have
on another. Let’s say students
complete a standardized test
in the spring in one teacher’s
classroom. They complete the
school year still learning from
that teacher, spend three
months in the summer losing
or gaining variable amounts
of knowledge, enter the class-
room of a new teacher in the
fall, and then take the “post-
test” the following spring
under the tutelage of the new
teacher. It is impossible to
prove that the losses or gains
posted from the previous year
to the next are solely a result
of the current teacher’s efforts.
Although system developers
argue that their system can
factor these effects out, this
remains unclear—and unlikely.
© JAMES YANG
The issue becomes more convoluted
when students enter middle and high
school and switch teachers and classrooms daily, sometimes taking classes in
the same subject areas during the same
semester. For example, if a student is
taking geometry and algebra the same
semester, who is to say that the geometry teacher was more or less effective
than the algebra teacher or that the
value the geometry teacher added to the
student’s learning about math had
nothing to do with what the student
learned in algebra? Who is to say that a
