The Problems of Accountability, Teacher Evaluation, and Merit Pay
From Chapter V of Knowing The Learner
In a collective social enterprise such as publicly supported education, all participants — teachers, students, parents, supervisors and administrators, government officials, and providers of educational resources — should be accountable for their actions. But what forms precisely should that accountability take for each of these stakeholders? The public mind today is focused primarily on teacher accountability. At the time of the writing of this book, teachers are being held accountable for the performance of their students on high-stakes, norm-referenced tests. Hiding in the shadows behind this practice is the assumption that we can know the effects of teaching on students’ test performance. Hiding more deeply is an assumption of causality — the assumption that teachers’ actions or lack thereof are a cause of the level of student performance on these tests. It seems straightforward that there should be some relationship between teachers’ actions and students’ attainment. The whole educational enterprise is founded on this assumption. But what can we soundly infer about the actual relationship?
A large number of factors would have to be considered to build any such causal equation. To begin with, there are the numerous human attributes that need to be taken into account (including student intention, motivation, interest, previous learning, learning habits, and cultural factors such as level of competency in language in which test is administered, and socioeconomic well-being). These constitute competing explanations for level of success on assessments and tests. Some of these (e.g., interest) might themselves be considered worthy goals for learning, often more worthy than stated course content itself. Many teachers take on the challenge of developing these as capabilities, but, of course, HSNR tests neither measure nor take these capabilities into account. Consequently, the exclusive use of such instruments reduces the evaluation of teaching and learning to improperly defined content and format.
Then there are questions of the effects of poverty/wealth, cultural enrichment/ deprivation, and sense of security/insecurity on student performance. Might such effects have as great or greater an influence on student performance than the experience of spending a period of time with a particular teacher? What proportion of these factors can a teacher reasonably be held accountable for? How can we tease out the effects of such factors?
We must consider the attributes of teachers as well. How long have they been teaching? What preparation do they have? What resources do they have available? What support are they obtaining from colleagues and from the school community as a whole? These would all have to be entered into the equation. Valid and reliable assessment and research instruments would need to be developed for each of these attributes and then applied and integrated into interpretations of students’ performance. The Institute of Educational Sciences sets a premium on funding research that employs randomized control trials (RCT) as a basis for inferences of causality in educational settings. But RCT studies struggle persistently, and unsuccessfully, with the problem of over-determination — that is, situations “…where a number of factors are present, any one of which could have produced the same result that is being attributed to the intervention” (Cook, Scriven, Coryn, & Evergreen, 2010).
In actuality, no school has the resources to conduct studies to permit the inference of causality. It turns out that the attempt to tease out these attributes in order to establish causal effects of a teacher’s action is not a feasible enterprise. In fact, it constitutes a monstrous pseudoscience. Therefore, on what basis do states and schools make the inference that the teacher or the teacher’s method is responsible for students’ performance on a test? The answer is on no sound rational or empirical basis at all! There is no way that research to establish cause (as a feature of instruction) and effect (as the performance on a HSNR test) could be carried out other than in highly restricted artificial educational settings. The cost would be too great, the conditions could not be met, and the inferences could be easily challenged on the basis of the argument that the results could have been attained without the method of instruction used (Klees, 2016).
But all such attempts at relating student performance to teacher performance neglect the most fundamental questions of all. Do the assessment instruments provide information on the attainment of targeted learning outcomes, those to which teachers’ efforts should be directed and for which teachers might truly feel accountable? Do any of the assessments used in educational programs measure what is most important (e.g., core capabilities)? Are they accurate and dependable measures of the learning goals that constitute the program’s aim? The reader should not be surprised at this point to realize that, for the most part, none of these conditions are met. Again, we are led to the sad realization that those who engage in these virtually omnipresent processes of establishing teacher accountability are resting on the cultural belief that HSNR tests are the best available way to measure the value of instruction. Once the pseudo-scientific attribution of causality is discarded, the entire justification for attempts by school districts, states, and federal agencies to manipulate educational programs based on HSNR tests collapses, its lack of foundation evident and its potential for harm exposed. Any thought of basing teacher evaluation or teacher accountability on such instruments should be immediately and roundly dismissed. Naturally, any proposals for merit pay based on HSNR test results must quickly be allowed to fall by the wayside as well.
The strongest basis for sound educational decision-making that we have at present is the considered judgments of seasoned, dedicated teachers based on their knowledge of their students and these teachers’ facility in working with targeted learning goals. Is it wise to lose these treasures in deference to policies and authorities operating in ignorance at a hyper distance from the teaching learning process?
Systematic educational assessment and community-based educational evaluation such as we have proposed in the preceding chapters give a sound basis for accountability in the context of professional development and program improvement. The most productive next step that we can take at present towards teacher accountability is a purely educational one. It is part of the realization of teacher as learner. It can be accomplished in the context of helping teachers learn how to use assessment information to help students attain the practical learning outcomes for their curriculum. As education is a social enterprise, teachers who cannot learn to help students attain the targeted learning goals that make up the curriculum must inevitably chose to leave of their own volition in the face of public reports of assessment results. Accountability of teachers and students (indeed, all facets of educational programs from the student to the state) are critical. But, first, the instruments and procedures of accountability must themselves be made accountable. They must show their efficacy in supporting educational activities and events.