Reply To: Private: Educational Science in Action: Evaluating an Assessment Instrument

A Renewal of Education Forums Active Conversations Private: Educational Science in Action: Evaluating an Assessment Instrument Reply To: Private: Educational Science in Action: Evaluating an Assessment Instrument

#1181
Paul Zachos
Moderator

Cubes & Liquids (C&L) may be used by teachers to support their daily practice. It is also sufficiently refined that it may be used as a research instrument. For an example of the latter, we are now using C&L to develop concepts and practices relevant to validity and reliability of educational measures working at the level of practical learning goals and outcomes. Teachers may also be seen as engaged in research when they are systematically building or refining assessment instruments.

The questions associated with daily practice vs. research are distinct.

For daily practice the primary question is, “How well are my students performing with regard to some learning goal?”

For research the questions are more of the nature of, “How confident am I in the results of this assessment? How valid are they, how dependable? What is the likelihood that the judgment made (i.e. of level of attainment of the learning goal) is accurate?”

In our sister conversation [Interpreting Educational Data] a request was made to see the comments made by raters regarding their judgments of level of attainment of a learning goal. This is part of evaluating the accuracy of judgments. For research purposes then it makes sense for raters to be generous in their comments (i.e. in giving reasons for their judgments) and for all stakeholders in the process to be attentive to those reasons.