Educational Leadership - November 2009

Some countries place great emphasis on motivating students to do well on PISA assessments, as demonstrated by these test-prep books in a German airport book stand.

pedagogical philosophies. As former Commissioner of Education Statistics Mark Schneider said, the tests are “blunt instruments. . . . A dozen factors could be behind a nation’s test score” (Cavanagh & Manzo, 2009, p. 16).

Nations vary greatly in the extent of their efforts to motivate students to do well on the assessments. In Germany, where PISA has likely received more attention than in any other country, PISA-prep books can be found in airports. Observers at a school in Taiwan reported that on PISA testing day, parents gathered with their children on the school grounds urging them to do well. The students then marched into the school to the national anthem and heard a motivational speech from the principal (Sjøberg, 2007).

Can low-scoring and middle-scoring nations learn anything from the high scorers? Mostly, no. After A Nation at Risk appeared in 1983, Secretary of Education Terrel Bell dispatched a team to Japan. The effort came to naught, no doubt in part because Japanese schools

NAEP has nothing
to say about
education quality
at the district
or school level.

and U.S. schools are embedded in vastly different cultures.

W. Norton Grubb and an OECD team observing schools in Finland, which ranks at the top on PISA, found some things the United States could likely adopt—for example, the interlocking system in which teachers and specialists work to head off learning problems early on. But they also noted some things we could not adopt without also adopting other large segments of the Finnish social system, such as comprehensive health care and public housing. Grubb (2007) pointed out, “The Finns take it as

axiomatic that both high-quality schooling and nonschool programs are necessary for equity” (p. 109).

A Better Way

PHOTO BY SVEIN SJØBERG

To be related to school quality, tests must be sensitive to instruction. Most of the tests used for accountability today aren’t—in fact, the manner in which they are constructed prevents them from being sensitive to instruction. That means that schools under the gun to raise test scores increasingly rely on strategies that get immediate, but short-lived results. Evaluation based on instruction-insensitive tests cannot help but reduce the quality of teaching (and teacher morale).

The best assessment system, but a difficult one to bring off, begins with teachers rather than with external measures that are imposed on them. The state of Nebraska developed such a system—the School-based Teacher-led Assessment and Reporting System (STARS)—based on instruction-driven measurement as opposed to the dysfunctional, measurement-driven instruction that predominates elsewhere. (Alas, it appears to have been almost eclipsed by the statewide program installed to meet NCLB requirements.) It is that kind of system—not NAEP, TIMSS, PISA, or similar tests—that will tell us what we need to know about our schools. EL

References

Bracey. G. W. (2009). Education hell: Rhetoric vs. reality. Alexandria, VA: Educational Research Service.

Cavanagh, S., & Manzo, K. K. (2009, April 22). International exams yield less-than-clear lessons. Education Week, 28( 29), 1, 16–17.

Chaddock, G. R. (2004, December 7). Math + test = trouble for U.S. economy. Christian Science Monitor. Available: www . csmonitor.com/2004/1207/p01s04-ussc .html