At a recent RSE TASC meeting in Albany, we were presented with the following selection of evaluation “shorts” as part of an engagement activity leading to the topic of evaluating professional development. The facilitator led us through a process by which we took “I” time to read each short and then turned to a partner to process and share learning. By activating and engaging the group members in this manner, all of the participants were able to learn content, focus their thinking on the topic at hand and engage in dialogue which allowed for processing of information and making connections to current work. I began to think about the power of using “shorts” such as these at staff meetings to engage participants in thinking more deeply about targeted topics and applying new learning to their work without necessarily having to read pages of articles or journals. Each short in itself could be a catalyst for a change in thought when paired with an effective processing activity such as those found on the National Faculty for School Reform website, www.nsrfharmony.org/protocol/a_z.html. A few of my favorites are Save the Last Word for Me, 4A’s Text Protocol, and Final Word. Give it a try at your next meeting! Here are the shorts to get you started….
1. History of Professional Development Evaluation as told by Guskey
Unlike many fields that have a history of steady improvement built on a continually expanding knowledge base, professional learning of educators has a mixed history at best. Sure, we have occasional success stories based on anecdotal evidence. Case studies here and there depict experience that participants considered “effective” because there experience offered useful ideas or were relevant their on-the-job responsibilities. What we do not have however, is strong and convincing evidence from activities and program improvements in diverse context that resulted in better practice and improved student learning.
2. Assessment vs. Evaluation
Although the terms are often used interchangeably, there is an important difference between assessment and evaluation. Chittenden (1991) defines assessment as the process of collecting and organizing information or data so that the learning may be evaluated or judged. Evaluating uses the results of assessment to make judgments about growth (how much and individual has changed) and progress (Owen, 1994). Yet assessment and evaluation are closely related. Assessment is possible without evaluation; however evaluation is not possible without assessment.
3. Are Professional Development Providers too far removed from Student Outcomes to Assess Impact?
Some educators contend that improving student learning outcomes represents too lofty a goal for many professional learning activities. Those in education service agencies and state or district office, for example often indicate they are too far removed from classroom interaction to expect their efforts to consistently reach that level. Because of this distance, they should only be accountable for providing evidence that the professional learning activities they plan and coordinate improve educator’ knowledge and skills. But if professional learning activities increase educators’ knowledge and skills but result in no change I school or classroom practice and no improvements in student learning, would we consider these activities successful?
4. Collection of Useful Data that fits the Realities of our Work
Decision makers need useful data to make informed judgments about the results of professional learning. In most cases, sophisticated research methodologies are impractical and not the answer. Instead, school system and school leaders need resources and tools that fit with the realities of their work environments and enable them to gain greater understanding about whether professional learning is solving their problems, improving their school, team, and individual performance, and helping more students to achieve standards.
5. Proof vs. Evidence
Keep in mind, too, that good evidence is not that hard to come by if you know what you’re looking for before you begin. If you do a good job of clarifying your goals up front, most evaluation issues pretty much fall into line. The reason many educators think evaluation at Levels 4 and 5 is so difficult, expensive, and time-consuming, is because they are coming in after the fact to search for results. It is as if they are saying, “We don’t know what we are doing or why we are doing it, but let’s find out if anything happened” (Gordon, 1991). If you don’t know where you are going, it’s very difficult to tell if you’ve arrived.