Accountability in Education (Excerpt From Knowing The Learner)

The Problems of Accountability, Teacher Evaluation, and Merit Pay

From Chapter V of Knowing The Learner

In  a  collective   social   enterprise  such   as  publicly   supported  education,  all participants — teachers, students, parents, supervisors and administrators, government officials,  and  providers of educational resources — should be  accountable for their actions. But  what  forms  precisely  should that  accountability take  for  each  of these stakeholders? The  public  mind  today  is focused primarily on  teacher accountability. At the  time  of the  writing  of this  book, teachers are  being  held  accountable for the performance of their  students on  high-stakes, norm-referenced tests.  Hiding  in the shadows behind this  practice is  the  assumption that  we  can  know   the  effects  of teaching on  students’ test  performance. Hiding  more  deeply is  an  assumption  of causality  — the  assumption that  teachers’  actions  or lack  thereof are  a cause  of the level of student performance on these  tests. It seems  straightforward that there  should be some  relationship between teachers’  actions  and  students’ attainment. The whole educational enterprise is founded on this assumption. But what  can we soundly infer about the  actual  relationship?

A large  number of factors  would have  to be considered to build  any such  causal equation. To  begin   with,  there   are  the  numerous  human attributes that  need   to be  taken   into  account (including student  intention, motivation, interest, previous learning, learning habits,  and cultural factors such as level of competency in language in  which   test   is  administered,  and   socioeconomic  well-being). These   constitute competing explanations for level of success  on assessments and  tests. Some of these (e.g., interest) might  themselves be considered worthy goals for learning, often  more worthy than  stated   course content itself.  Many  teachers take  on  the  challenge  of developing these  as capabilities, but, of course, HSNR tests neither measure nor take these  capabilities into  account. Consequently, the  exclusive use  of such  instruments reduces the  evaluation of teaching and  learning to improperly defined content and format.

Then  there  are  questions of the  effects  of poverty/wealth, cultural enrichment/ deprivation, and  sense   of  security/insecurity on  student performance. Might  such effects   have   as  great   or  greater  an  influence  on  student  performance than   the experience of spending a period of time  with  a particular teacher?  What proportion of these  factors  can a teacher reasonably be held  accountable for? How can we tease out the  effects  of such  factors?

We must  consider the  attributes of teachers as well.  How  long  have  they  been teaching? What preparation do they have? What resources do they have available? What support are they obtaining from colleagues and from the school community as a whole? These  would all have  to be entered into  the  equation. Valid and  reliable  assessment and  research instruments would need  to  be  developed for each  of these  attributes and  then  applied and  integrated into  interpretations of students’ performance. The Institute of Educational Sciences  sets  a premium on  funding research that  employs randomized control trials  (RCT) as a basis  for inferences of causality  in educational settings. But RCT studies struggle persistently, and  unsuccessfully, with  the  problem of over-determination — that is, situations “…where  a number of factors  are present, any one  of which  could  have  produced the same  result  that is being  attributed to the intervention” (Cook,  Scriven, Coryn, & Evergreen, 2010).

In  actuality,   no  school   has  the   resources  to  conduct  studies  to  permit the inference of causality. It turns  out  that  the  attempt to  tease  out  these  attributes in order  to establish causal  effects  of a teacher’s  action  is not  a feasible  enterprise. In fact, it constitutes a monstrous pseudoscience. Therefore, on what  basis do states  and schools make  the inference that the teacher or the teacher’s  method is responsible for students’ performance on a test? The answer is on no sound rational or empirical basis at all! There  is no  way  that  research to establish cause  (as a feature of instruction) and  effect  (as  the  performance on  a  HSNR test)  could  be  carried   out  other   than in highly  restricted artificial  educational settings. The  cost  would be  too  great,  the conditions could  not  be  met,  and  the  inferences could  be  easily  challenged on  the basis  of the  argument that  the  results  could  have  been  attained without the  method of instruction used  (Klees, 2016).

But  all such  attempts at  relating student performance to  teacher performance neglect the most fundamental questions of all. Do the assessment instruments provide information on the attainment of targeted learning outcomes, those  to which  teachers’ efforts  should be  directed and  for which  teachers might  truly  feel  accountable? Do any of the assessments used  in educational programs measure what is most important (e.g.,  core  capabilities)? Are they  accurate and  dependable measures of the  learning goals  that  constitute the  program’s aim? The  reader should not  be  surprised at this point  to realize  that,  for the  most  part,  none  of these  conditions are  met. Again, we are  led  to the  sad  realization that  those  who  engage in these  virtually  omnipresent processes of  establishing teacher  accountability are  resting   on  the  cultural belief that  HSNR tests  are the  best  available way to measure the  value  of instruction. Once the  pseudo-scientific attribution of causality  is discarded, the  entire  justification for attempts by school  districts, states,  and  federal  agencies to manipulate educational programs based   on  HSNR tests  collapses, its  lack  of  foundation  evident and  its potential for  harm  exposed. Any  thought of  basing   teacher evaluation or  teacher accountability on  such  instruments should be  immediately and  roundly dismissed. Naturally,  any  proposals for merit  pay  based  on  HSNR test  results  must  quickly  be allowed to fall by the  wayside as well.

The  strongest basis  for  sound  educational decision-making that  we  have   at present is the  considered judgments of seasoned, dedicated teachers based  on their knowledge of  their  students and  these   teachers’  facility  in  working with  targeted learning goals. Is it wise to lose these  treasures in deference to policies and authorities operating in ignorance at a hyper  distance from the  teaching learning process?

Systematic   educational  assessment  and   community-based  educational evaluation  such   as  we  have   proposed  in  the   preceding  chapters  give  a  sound basis for accountability in the context of professional development and program improvement. The  most  productive next  step  that  we  can  take  at  present towards teacher accountability is  a  purely   educational one.  It  is  part  of  the  realization  of teacher as learner. It can  be  accomplished in the  context of helping teachers learn how  to  use  assessment information to  help  students attain  the  practical learning outcomes for  their  curriculum. As education is  a  social  enterprise, teachers who cannot learn  to  help  students attain  the  targeted learning goals  that  make  up  the curriculum must  inevitably chose  to leave  of their  own  volition  in the  face of public reports of  assessment results. Accountability of  teachers and  students  (indeed, all facets of educational programs from the student to the state)  are critical. But, first, the instruments and procedures of accountability must  themselves be made  accountable. They must  show  their  efficacy in supporting educational activities  and  events.

Page with Comments

  1. Thank you Hunter for this illuminating perspective on the issues we tackled in our discussion of accountability in education. Establishing direct cause is something that even ‘hard’ science finds problematical. What we can do, as you have pointed out, is to obtain information on how well discrete learning goals have been attained. If we do this before and after an instructional sequence or program is offered we will have evidence relevant to the question of whether the instruction has been successful. If so, well and good, we need not look further into the nature of the instruction ( Although we should remember that success in attaining the learning goals is only supporting evidence and not proof that the instruction was the cause of the attainment). If the learning goals have not been attained, this also does not pinpoint the cause of the failure. The instruction may have been carried out poorly, or it may be that the instruction was not appropriate for the targeted learning goals. It may also be that the assessment was not appropriate to the learning goals or not properly carried out. Unless all of these potential causes for assessment results are ‘evaluated’ we can not properly assign responsibility to the teacher for the lack of success. In any case assessment results associated with the aggregated test scores produced by high-stakes, norm-referenced tests, contain no information on the attainment of discrete learning goals, and so do not provide the information needed to evaluate or improve educational activities. This is why we say in the book that assessment and evaluation must first be made accountable before we try to make teachers accountable for assessment results.

  2. The above two quotes illustrate the concept that most struck me the first time I read through KTL, relating to accountability. At the beginning of the section entitled “The Problems of Accountability, Teacher Evaluation, and Merit Pay,” Paul and Will refute the claim that there is causality in education, saying there are a number of factors situated between the teacher’s communication and and the student’s reception of the communication that may be as important or more important than the communication itself, giving examples of student intention, motivation, interest, etc. This is extremely radical! We live in a society that thrives on causality and accountability – how can we justify paying taxes to support a school system which cannot show us that is doing its function, educating its students? The problem is not that there is no way to hold educational institutions accountable, however, only that we are going about it in the wrong way. This is where discrete learning goals come in. Discrete learning goals are the kernel of truth from which we can base a claim of causality. If developed with an eye towards simplicity and assess-ability, some of the noise is removed from the data, some of the chaos of schooling. Once reoriented towards discrete learning goals, we can again hold schools and teachers accountable in good faith.

Leave a Reply

Your email address will not be published. Required fields are marked *