Why Consistency is Not Possible in Experienced Teacher Evaluations

Abstract

One of the three functions of the Ontario performance appraisal is “to provide for fair, effective, and consistent teacher evaluation” (Ministry of Education, 2002, p. 3). Without consistency, the appraisal is no longer fair or effective. Through the application of Grounded Theory, it was discovered that consistency in implementation was not occurring. This paper discusses how the central phenomenon - principal consideration of task – causes inconsistency in implementation. As long as the process is not followed, and school principals apply alternate expectations and procedures, the performance appraisal cannot be used to determine the competency of teachers.

Keywords: Teacher evaluation, principals as assessors, grounded theory

Introduction

The Ontario Ministry of Education’s performance appraisal and its process was developed with thought given to the needs of teachers, schools, school boards, and pupils. This is reflected in the stated purposes:

• To ensure that students receive the benefit of an education system staffed by teachers who are performing their duties satisfactorily
• To provide for fair, effective, and consistent teacher evaluation in every school
• To promote professional growth (Ministry of Education, 2002, p. 3)

The task of administering the Ontario Performance Appraisal was assigned to the school principal. As noted by Hayman and Sussman (1986), “[a]t the bottom line, supervisors are responsible for ensuring that the work done within their units is done effectively. For this reason, it is appropriate to hold principals accountable for managing the remediation or removal of incompetent teachers within their schools” (p. 111). It was hoped that through consistent implementation, principals could help teachers become aware of areas that require attention improve classroom practice. During the performance appraisal process, the principal should encourage the teacher to examine the evidence by citing specific examples noted in practice (McLaughoin & Pfeifer, 1988). The Ontario Ministry of Education embedded four opportunities for sharing and discussion within the performance appraisal process. If the process had been implemented as designed, it would have been an invaluable tool to benefit practice, student learning, and developing schools into true learning communities.

Problem Statement

In the article Experienced Teacher Evaluation through Performance Appraisals: Is Consistency Possible? (Barnett, 2012), the author reviewed the finding that the Ontario performance appraisal was not being implemented in a consistent, nor fair, fashion. While this initial paper reviewed what occurred for the different teachers and compared their experiences, it also argued that consistency is impossible if the focus of the appraisal is on the needs of the teacher. As noted, “it is entirely understandable for the process to be altered so that a true picture of a teacher’s competency can be generated”. The article still stressed the importance of consistency in following the basic tenants of the appraisal process (Barnett, 2012). “Expecting the performance appraisal to run the exact same in all situations is unrealistic” (Barnett 2012).

While the above is a justifiable reason for altering an implementation, the inconsistency discovered during the research was not due to any attempt at adapting the performance appraisal to best meet the developmental needs of teachers.

Research Questions

If the performance appraisal was not being altered to benefit teacher growth or promote a clearer understanding of teacher competency, why was it altered? Thus, this research sought to answer the question: What is the cause of the inconsistency in implementation of the Ontario teacher performance appraisal process?

Purpose of the Study

Evaluation systems which focus on accountability and professional growth are necessary in order to meet the needs of the teachers, school, and society at large (Stronge, 1997). The Ontario Ministry of Education created a teacher performance appraisal that incorporated formative assessment and summative evaluation. If implemented as described it would promote collaborative sharing and learning partnerships between administration and teaching staff. It is important to understand why it was not implemented as designed. Without this understanding, steps cannot be taken to remedy the situation.

Research Methods

As “qualitative researchers are concerned with process as well as product” (Fraenkel & Wallen, 1993, p. 380), a qualitative design was an ideal methodology for discovering why consistency was not occurring during the appraisal implementation process. Grounded theory was chosen as it not only identified a central phenomenon, but also allowed causal factors and barriers to success to be noted. Having established that consistency was not occurring (Barnett, 2012), the substantive level theory developed would also explain why consistency is not possible during experienced teacher performance evaluations.

Language-based data was generated via interviewing teachers who had undergone the performance appraisal. The population (school board) was chosen because the appraisal process was fully entrenched and most often enacted by experienced principals, thus minimizing the possibility of inconsistency being caused by newness or unfamiliarity. To secure participants, non-probability sampling was employed. Using convenience sampling, a request for volunteers was distributed to acquire initial participants. Snowball sampling then occurred providing the researcher with voluntary participants who represented a wide range of experience, age, sex, and/or teaching position. As this study was concerned with the experiences of teachers, the teacher participant was considered the expert.

During the interviews, open coding identified the categories and elements of each category. Following the guidelines for grounded theory research, the data collection occurred in a “zigzag process – out to the field to gather information form interviews, analyze the data, back to the field to gather more information, analyze the data, and so forth” (Creswell, 1998, p. 57). Nine teachers participated before saturation of the data occurred.

Though all teacher participants were employed by the same board of education, their schools were scattered over an area of 4864km2. Teaching experience ranged from 31 years to one year. Three participants taught in the Secondary panel (Grades 9-12). Of the remaining teachers, four taught primarily in the Intermediate division (Grade 7 & 8), one taught primarily in the Junior division (Grades 4-6) and the final participant taught in the primary division (Grades 1-3). Two taught in special education classrooms, one taught drama, one taught English, and one primarily taught French as a Second Language. The rest were classroom teachers responsible for a wide breadth of subjects. To protect confidentiality, all names and identifying information were changed.

After saturation occurred, the research engaged in the identification of common themes and subthemes through comparative analysis. This caused the central phenomenon, barriers and causal factors to become known. In an attempt to better understand the identified causal factors and barriers, seven principals participated in additional interviews. These final interviews produced data that triangulated the findings of the teacher participants.

Findings

The performance appraisal was not being consistently implemented following the ministry’s guidelines nor was it implemented in a consistently altered manner. What was to occur was clearly outlined in the ministry’s appraisal manuals. The procedure included notice of evaluation in September, a pre-observation meeting with the teacher to discuss the pending observation and any areas of need as determined by the teacher, an observation of the teacher’s performance in the classroom, a post-observation meeting to suggest areas for growth or areas of concern, and a standardized form to be completed by the principal and filed at the board office. A successful teacher was mandated to go through the process twice during their evaluation year. Support, mentoring, and additional evaluations were added to aid those who were deemed unsuccessful. These teachers were required to go through the process at least three times and had the timeframe extended to allow for professional development to also occur.

This study found that pre-observation meetings were 5 minutes in length to an hour in length, occurred in a designated locale or in the school hallway, were collaborative or were initiated in a top-down manner. In some pre-observation meetings, the ministry documents were shared with the teacher, in other meetings only a few of the forms were shared, while in other meetings no documents or forms were shared with the teacher about to undergo the evaluation process. In one situation, the teacher was asked to write a draft of her summative observation report and bring it with her to the pre-observation meeting. The first observation occurred on time, or was delayed, or rescheduled. In one instance the teacher was never observed and in another situation the observation was continually rescheduled until it finally occurred a year and a half after the process was initiated. Principals arrived on time for the observations, were late and left early. The post-observation meeting did not always occur. On numerous occasions, the second round of the process was omitted entirely. Of the nine participants, only two experienced two observations and had the required appraisal meetings, though variation in these also occurred. The process was not consistently implemented.

Rubrics to aid principal assessors were created and included in the manual, 2002. These ministry standards were also not consistently applied. Though the ministry provided rubrics to define the evaluative ratings, the principals relied on their own subjective interpretations to apply the rating results. The grade 1 teacher received a G (Good) rating because her principal told her he did not believe in giving an E (Exceptional). A mid career teacher was told she was exceptional but was given a G because according to her principal, Es were meant for those at the end of their careers. In this instance the rating was not of the teacher’s teaching but of her age and her years of service. Another mid-career teacher in the study did receive an E rating. The rationales for the ratings varied based on the principal’s personal belief. Issues of favoritism also arose – a favorite teacher received a high rating she felt was unjustified while a person who routinely challenged the principal received a low rating.

The ministry attempted to create a process that would ensure consistent evaluation.(2001) created the standards for evaluation and Regulation 98/02 and 99/02 (Ministry of Education 2002) outlined the mandatory competencies based on the standards and provided a framework for evaluation. Steps were clearly outlined. Given this attention to description of process, implementation and purpose, why was consistency impossible to employ? This research has discovered each principal-assessor viewing the task differently caused inconsistency.

How the principal perceives the evaluation task affects the manner in which it was implemented. If it was viewed as “one-more-duty-to-do” implementation lost its focus on collaboration with the teacher and as a vehicle for the promotion of professional growth. It was viewed as a competency-check that had to be submitted to the board office. Simply, it became less about the teacher’s competency and growth, and more about the principal appearing to those at the board office to be doing his/her job. In the principal’s mind, it became a competency-check on their ability to do their job as assigned.

The strength of this central phenomenon depended on the causal factors of a) principal- leadership style, b) previous experience with the purpose of appraisals and principal subjectivity in terms of the purpose of the appraisal and c) their personal relationship with the teacher being evaluated. Principals who were more comfortable working in a top-down, principal-controlled setting tended to view the appraisal as a task to be completed instead of a collaborative tool to help teachers. If principals had completed other appraisals that were based on assessing competency, principals assigned the same purpose to the new appraisal system. The principal’s relationship with the teacher also influenced the central phenomenon. If the principal was unable to separate personal emotions from the evaluative process, teachers who were liked usually received a good evaluation while those who were not liked received a lower evaluation. The evaluation not only became seen as a duty to do, but also as a duty that would provide a punitive report, thus elevating the principal by diminishing the teacher. These three casual factors – principal leadership style, previous experience / subjectivity concerning purpose, personal relationship with the teacher – influenced the principal’s perception of the evaluation. The more present each causal factor was, the more apt the principal was to view the appraisal as simply something he/she had to complete and thus responded by conducting the appraisal in a top-down, authoritative manner.

Barriers. Three barriers were found to prevent the principal from applying the process as described by the ministry: time, support and principal subjectivity. As the process was added to the principal’s duties without removing a different duty to compensate for time, the evaluation process became viewed as one more thing to do during a busy day at work. Further, even if the principal managed to reorganise the day(s) to be able to conduct the appraisal, consistency could not be guaranteed. It is the nature of the principal’s position to be called away from the office to deal with student or school emergencies, impromptu visits by parents, or to deal with staff concerns. It should not be a surprise that every principal interviewed provided clear support for Time being a significant barrier to consistent implementation. What deserves attention, however, is that this barrier was derived from the teacher interviews. The lack of Time was so obvious an issue that every teacher noted the impossibility of the task assigned to the principal. They did not express anger at the principal, but commiserated. As a special education teacher with 15 years of experience said, “if principals are indeed supposed to do this then somehow they have to be given the time to do it, to do a proper job”. This concern over the lack of time to implement the appraisal appropriately is a theme in the Canadian literature (Black, 2003; Bolger & Vail, 2003). As Cowans (2004) noted, it “is proving to be just as onerous for its administrators as it is for its victims…it can only be implemented in the most superficial manner” (p. 5).

The second barrier was support. No funding was provided to allow the principal to hire a substitute in order to free his / her time. No training took place to stress the importance of consistency and no professional development on how to implement the process or apply the evaluative scales occurred. Lastly, no attempt was made to alleviate some of the other daily pressures of the principal’s job so they could adequately engage in this task.

The final barrier was principal subjectivity. Without training, support or time, principals freely applied their personal beliefs. The fact that most teachers did not experience the second round of the appraisal process was most often due to the fact that the principal considered it unnecessary. Some teachers were even informed by the principal that the second round was a “waste of time” as the principal had enough data to submit a report to the board. In doing so, lost was the ministry’s belief that the teacher requires two appraisals in order to demonstrate their range of abilities. Anyone can have a bad day. Inconsistent application of the evaluative ratings due to principal subjectivity is another serious issue. A high evaluative ranking being considered a reward given to experienced teachers for years of service instead being based on actual ability, diminishes every evaluative ranking assigned and angers the younger and mid-career teachers who are also exceptional. As a result, the teachers saw the process as a “hoop to jump through” instead of a valuable tool to aid them in their own professional development. Upon receiving the final report, instead of reflecting many filed it away, not to be read or looked at again. Schmoker (1999) clearly states: “Data helps us to monitor and assess performance. Just as goals are an essential element of success, so data are an essential piece of working towards goals” (p. 35).

Appraisal data need to be received, heard, and acted upon (McLaughlin & Pfeifer, 1988). When teachers do not attempt to improve practice, stagnation occurs (Beerens, 2000).

Conclusions

Given the potential serious consequences of a poor performance appraisal, consistency in implementation is essential. By providing a standard format and mandating is application, the Ontario Ministry of Education attempted to ensure consistent evaluation of Ontario teachers. The task of administering the Ontario Performance Appraisal was assigned to the school principal. As noted by Hayman and Sussman (1986), “[a]t the bottom line, supervisors are responsible for ensuring that the work done within their units is done effectively. For this reason, it is appropriate to hold principals accountable for managing the remediation or removal of incompetent teachers within their schools” (p. 111). Having competent teachers who are engaged in professional growth strengthens the entire system; it has beneficial implications for the school, the students, and the school boards. Unfortunately, the strong appraisal process the ministry developed was not supported through training, funding, relief time, or any form of compensation. It became a stressful task they simply had to do. As a result, the principals interviewed were quick to explain why teacher competency was important due to the accountability placed on their shoulders. When queried on how the appraisal was used to engage teachers in their own professional development, initially they were silent.

Increased time, funding, and training would improve consistency in implementation. Principal subjectivity, however, can only be challenged through a monitoring system. While adding a top- down monitoring system to ensure that appraisal’s steps are followed and that it is not implemented in a top-down manner may make sense to many, the irony of adding top-down monitoring to prevent top-down implementation should be evident. A better suggestion would be to put in place steps to alter the principals’ perception. If the principals could be provided the time to network and share their thoughts with other principals and senior administration. If the central phenomenon causing the inconsistency is how the principal perceives the evaluation, what truly is required are opportunities to change their perception. If instead of seeing it as one more duty to insert into an already busy day, the principal sees it as an opportunity to share with his/her staff, talk about teaching with teachers, and be involved in professional growth, consistent implementation may follow. While it is true the causal factors may still be in place, alleviating some of the barriers to consistent implementation would also be beneficial and offer implicit support to the new perception of the appraisal as a valuable tool when implemented as designed.

Acknowledgements

The authors declare that there is no conflict of interest.

References

• Barnett, J. A. (2012). Experience Teacher Evaluation through Performance Appraisals: Is consistency possible? Procedia Social and Behavioral Sciences, 69, 1853-1860.

  Google Scholar

  Crossref

• Beerens, D. R. (2000). Evaluating teachers for professional growth: Creating a culture of motivation and learning. Thousand Oaks, CA: Sage Publications.

• Black, C. (2003, April). Action research and teacher performance appraisal. Paper presented at the proceedings of the annual meeting of the American Educational Research Association, Chicago, IL.

• Bolger, D., & Vail, H. (2003, May). Perspectives on “Supporting Teacher Excellence”: the Ontario teacher performance appraisal system. Paper presented at the annual meeting of Canadian Society for the Study of Education, Halifax, NS.

• Cowans, J. (2004). The end of PLP: now what about TPA? Update – Ontario Secondary School Teachers’ Federation, 31(8).

• Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among the five traditions. Thousand Oaks, CA: Sage Publications.

• Fraenkel, J. R., & Wallen, N. E. (1993). How to design and evaluate research in education (2nd ed.). New York, NY: McGraw-Hill, Inc.

• Hayman, B., & Sussman, S. (1986). The future of performance appraisal for certified education staff in the school boards of Ontario. Toronto, ON: Queen’s Printer for Ontario.

• McLaughlin, M. W., & Pfeifer, R. S. (1988). Teacher evaluation: Organizational change, accountability, and improvement. In E. S. Hickwood, S. B. Lawton, K. A. Leithwood., & D. F. Musella (Eds.), Making a difference through performance appraisal (pp. 113-140). Toronto, ON: The Ontario Institute for Studies in Education.

• Ministry of Education. (2002). Supporting teaching excellence: Teacher performance appraisal manual and approved forms and guideline. Toronto, ON: Queen’s Printer for Ontario

• Smocker, M. (1999). Results: The key to continuous school improvement (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

• Stronge, J. H. (1997). Evaluating teaching: A guide to current thinking and best practice. Thousand Oaks, CA: Corwin Publications.