The Measures of Effective Teaching Project (MET) is a research collaborative of academicians, teachers, and education organizations committed to investigating
better ways to identify and produce effective teaching. The complete funding for this endeavor is provided by the Bill & Melinda Gates Foundation. Nearly 3,000 teachers from across various states of America had volunteered to have their classrooms be used in the project.
Generally speaking, teachers work by themselves. They develop their own teaching methods to implement the curriculum as best as they can for their students. Anytime outside guidance is conferred to them, it is often saturated by vague teaching standards, overly numerous and often trivial learning objectives, and testing systems that measure only some of the outcomes that educators value for students. Success is ambiguously defined. Researchers involved in the MET project believed that current evaluation systems were not being used to improve teacher support. There was little faith that traditional evaluation measures and practices could provide usable information to guide more effective teaching.
It is very hard to support effective teaching without sufficient information regarding the actual teaching practice. The MET project has sought to build and test measures of effective teaching so that school systems can clearly understand and then close the gap between their expectations for effective teaching and the actual teaching occurring in classrooms. There are nine principals that MET project used in this manner. The first set of three principles included, as they are defined: setting expectations, using multiple measures & balancing weights. The expectations must be rooted in an agreed upon system regarding a teacher’s knowledge, skills, and behaviors that enable better student learning. This helps to benefit the entire system by providing a shared language to talk about teaching, set priorities, and target support. Having different measures should reflect the multifaceted nature of effective teaching. The MET project developed measures to reflect all key aspects of its definition of effective teaching: student surveys to assess the supportiveness of the instructional environment; content tests to assess teachers’ knowledge of their subject and how to teach it; observation instruments to assess teachers’ classroom practice; and student assessments to measure the learning gains of a teacher’s students. What counts most gets the most attention. When combining measures into a single index, we have found that approaches that allocate between 33%-50% of the weight to student achievement measures are sufficient to indicate meaningful differences among teachers. Moreover, balanced weights avoid the risks posed by too narrow a focus on one measure.
The next set of principles dealt with ensuring high quality data by monitoring validity, ensuring reliability and assuring accuracy. Teachers who demonstrate skills and score high on a measure should experience more success in helping students learn than teachers with lower scores. The MET project found that students learn better in the classrooms of teachers with better observation scores, better student survey results, and prior success raising student test scores. Validity is an ongoing process. Having low reliability means lack of trust in the system. The MET project has learned much about how to reliably measure teacher practice. Paramount among these is the need to observe more than one lesson and include more than one observer for each teacher. In addition, it was learned that short observations to supplement full-lesson observations can increase the reliability of observation ratings. Without the combination of reliability and accuracy, the results are essentially wrong. It is important to be able to accredit the correct student test scores & survey responses to the corresponding teacher. Whenever the MET project collected student data from a classroom it verified with the teacher the names of the students in the class.
Ultimately, if a new educative initiative yields positive data, then there is significant possibility for investing in it going forward. The first involves making important distinctions from the results. MET project data suggest that teachers’ effectiveness is unlikely to be distributed equally among several performance categories. Additionally, it was determined that many of the teachers had similar performances in the center of the distribution data, thereby, suggesting a large middle category of effectiveness. Rather than trying to make fine distinctions among teachers in this vast middle, efforts would be better spent working to improve their practice, therefrom. Next, obtaining support & feedback can be a bonus in addition to what information is gleaned from multiple measures because it gives the teachers opportunities to enhance their professional development by reviewing how & what they did during the MET project study. Lastly, having access to pure, rich data can allow decisions at all levels to be made by a school system. The responsibility for improving teaching should not rest with teachers alone. Measures of effective teaching enable school systems to better support teachers’ improvement needs. Sound measures help school systems know where to target professional development and whether the supports work. As an example, a school district could focus its professional development support on rigorous instructional techniques for which teachers showed the most room for improvement rather than classroom management skills that most teachers had clearly mastered. States and districts have learned a great deal in the last few years about how to create better teacher development and evaluation systems. But there’s still much to learn as these systems are implemented and improved over time and aligned to new expectations for students.
Prepared by Krishan Jeyarajasingham, MD
Gates-Continued Progress Nov 2015