By Jim Hollis
“If you learn only methods, you’ll be tied to your methods, but if you learn principles you can devise your own methods.”
– Ralph Waldo Emerson
Technology allows us to look beyond methods. As Emerson presaged nearly two centuries ago, methods can be either a barrier or a catalyst for change, according to the context. When used to handle basic tasks for instructional leaders, technology not only increases productivity and efficiency, but it also illuminates principles that drive instructional practice.
Schools are intrinsically risk-averse. This poses a natural challenge to innovation because innovation involves risks. The general consensus among educators is a fear that kids will suffer when new program ideas are unsuccessful. Yet the effectiveness of programs is often analyzed by a “gut reaction,” or even worse, by a review of state test results – when adjustment of practice is too late. District leaders want to know when school or district goals are being implemented, so that further interventions or changes in practice can be made without relying on exit tests.
In the hands of a trained observer, the handheld computer (including mobile devices like the Blackberry or iPhone) can be one of the most powerful tools for identifying, improving and sustaining high levels of instruction. By collecting observational data and transmitting it to a database via the Internet, the handheld device can help to build a comprehensive picture of a school’s instructional activity.
Research conducted by Pivot Learning Partners, a nonprofit school reform organization that works to improve student achievement and narrow the achievement gap, shows technology-based classroom observation tools help teachers, principals and district officials both introduce and utilize a common language for observation protocols and a shared method for examining results.
We conducted a study of observational practices in schools spanning 20 different districts, analyzing more than 1,000 classroom observations, in the 2008-2009 school year. The classroom observation tool we used, Walk’bout 3.0, is a Web-based data collection tool that can be added to most wireless, handheld and smartphone devices.
Built in conjunction with the Association of California School Administrators, the Walk’bout tool (available at www.acsa.org/WalkboutTraining or www.pivotlearningpartners.org) gathers data on classroom observations, shapes the data into reports and graphs, and transforms data to insight that can be used to improve instructional practice. By transforming mobile data tools into data collection terminals that aggregate new knowledge about classroom instruction, this device can truly liberate principals and teachers from their desks and overcome the ambiguity of what is happening in the classroom.
The Walk’bout 3.0 is based on research gathered over the last five years about what works in the classroom. The tool enables observations to be downloaded from the device to an offsite online database, turning data into information that is placed back into the hands of the observer in seconds.
The more quickly data is available, the more immediate the impact. Our research found that when schools and districts combined best practice strategies (such as Marzano, Bloom’s taxonomy and differentiated instruction) with technology to help aggregate results, they increased their adherence to goals and the number of observations. The Walk’bout tool helped observers know what to look for and how to identify it.
Monitoring Brawley Elementary
We monitored performance closely over one year in Brawley Elementary School District. A small district in the southeast corner of California, Brawley comprises four elementary schools (two K-3 schools and two grade 4-6 schools) and one middle school (grades 7-8). It serves approximately 3,680 students, 81 percent of whom are Hispanic, 15 percent white, and 3 percent African-American. There are approximately 400 certificated and classified district employees.
Brawley’s main focus was improving proficiency in math. After using the Walk’bout 3.0 for one year, Brawley was able to move more of its students into higher levels of proficiency (very proficient and above proficient) than in the previous two years combined.
An important part of this improvement resulted from an increased number of classroom observations and discussions about data. Principals reported that the “ease of use” and “convenience” of using the tool, stocked with protocols and best practices, streamlined classroom observations and discussions with teachers. This is consistent with the research literature, which finds that one of the main characteristics of good observation protocols is increasing the frequency with which they are carried out.
Collecting and reporting these results frequently to teachers will maintain the perception of walkthroughs as a fair process. Frequent walkthroughs translate into more opportunities to see different teaching and learning strategies, often resulting in higher teacher flexibility that, in turn, translates into more openness to coaching and to learning new techniques.
Before Brawley undertook the work, Pivot Learning informed teachers about how the data would be used and what was being observed. We maintained strict confidentiality of individual teacher data to uphold the critical issue of fairness by sharing individual observations only with the individual teacher. Prior to viewing their own data, we encouraged teachers to look at a summary report of their entire grade level or subject.
After a few weeks of conducting walkthroughs, schools in Brawley began to aggregate observations. They presented reports at school- or grade-level meetings and in faculty committees, where they examined the data together as a team. Although individual teacher-level reporting was available during the initial period, we discouraged its use in the belief that teachers need to first establish feelings of comfort with the process before analyzing their own performance.
We quickly identified areas of strength as well as areas in need of additional growth in the aggregated reporting for further discussion or professional development. With very carefully crafted focus questions, the Walk’bout helped to identify what was taking place in the classroom. It also assisted teachers to help students achieve at their highest levels.
Our study also found that classroom observations were a common practice in all of the focal districts. However, administrators conducted observations by hand and recorded them on paper. These pencil-and-paper observations reported on classroom activity, but they did not necessarily correlate with best practices, lacked a longitudinal view of classroom instruction, and were largely anecdotal.
By contrast, when we developed an aggregated view of multiple visits by a single observer and matched it against AYP gains over the same period, similar characteristics and principles became clear. Although specific components of these walkthroughs varied in purpose from district to district, the most effective walkthrough strategies shared certain common elements. They include the following:
• Brevity: A walkthrough strategy should be designed to increase the number of classrooms that principals visit, and visits must be brief. While the typical walkthrough lasts about 10 minutes, we found that observers who averaged three to five minutes were more likely to conduct more observations over the course of the year than those observers conducting seven- to 10-minute observations. Walkthroughs do not replace longer observations. Rather, they supplement these observations by providing a high number of classroom practice “snapshots” that can reveal patterns and instructional issues throughout the school over time.
• Focus: Because the walkthrough is an adaptable strategy that can be used for a variety of purposes, it must be clear for all participants what is being observed and why. A clear focus ensures that there is a common understanding. As one school principal described it, a clear walkthrough focus “puts teachers and principals on the same page in terms of expectations.”
A clear focus helps teachers identify the key factors and techniques that principals use during the observation, and develop reflective questions that structure the feedback session. This participation goes a long way toward reassuring teachers that the walkthrough is a strategy for support, not for evaluation.
• Dialogue: A third common feature of the well designed walkthrough is that it results in a dialogue between the principal and the teacher. This dialogue begins with the principal giving feedback about what was seen and heard. Feedback often takes the form of reflective questions, such as: “Why did you group your students for that activity?” and “How did you develop the criteria for posting student work?” The goal of the dialogue is twofold: to encourage teachers to reflect on their classroom practice, and to inform the principal how to best support it.
Principals may also engage in discussions about schoolwide trends that they have identified, and make suggestions for classroom visits and for specific professional development activities that can improve instructional practice (Downey et al., 2004). Principals and teachers who talk openly about what matters in the classroom significantly increase the possibilities and strategies for continuous improvement.
The longitudinal view
There is still some debate in the literature on whether an observation tool can have much predictive validity for measuring changes in student learning. However, Walk’bout demonstrates that a tool that combines research-based best practices with a longitudinal view, and fosters increased dialogue, produces great results.
If we believe that practices and principles can be reliably measured and serve as predictors of instructional growth, it is essential to have an instrument that can gather and collect this growth for principals over a period of time. Longitudinal analysis is simply not possible when using traditional paper classroom walkthroughs.
The importance of purpose
In our research, we attached all of the academic standards to the protocols, allowing principals to tie each classroom observation to teacher practice for specific subject matter (at, above or below grade level). Some of the districts under study took a different approach, choosing to focus on programs and customizing observation protocols, helping to:
• Evaluate the effectiveness of a school program;
• Assess the performance of a teacher or a school;
• Provide feedback to teachers for professional development; and
• Conduct research on classroom practices for professional learning communities.
Although not a formal part of individual teacher evaluation, walkthroughs place high importance on what takes place in classrooms as measured through short, snapshot observation (KDE/Gates Phase II). From these “drop-in” observations, data can be generated as charts and graphs and analyzed quickly using handheld technology.
The learning and teaching indicators are easily observed and consist of items pertinent to teacher and student engagement, types of classroom activities, lesson planning and instructional differentiation, and grouping arrangements. Kelley and Finnegan (2003) suggest that prompt, meaningful feedback and principal support for accountability goals should be a primary objective of any online training or curriculum.
Furthermore, there are ethical considerations that arise under the limited viewpoint provided by single observations. Recording snapshot data for meaningful and fair results requires multiple observations in each classroom of the school. A strategy that may not have been in practice on the observation day may very well be observed on a subsequent visit. An important component of effective educational leadership and accountability design is pertinent and relevant information with which to drive data decisions.
In Brawley, as in many other districts, principals took a more horizontal approach to dialogue with the teachers in order to accommodate these new practices learned in the reporting process. They were also more willing and able to discuss which practices worked and which practices and approaches were shared by their colleagues. More frequent face time in the classroom by the principal, armed with best practices and a longitudinal view into classroom activities, proved to be a powerful strategy for improving instruction.
References
Downey, C.; Steffy, B.E.; English, F.W.; Frase, L.E. & Poston Jr., W.K. (2004). The three-minute classroom walk-through: Changing school supervisory practice one teacher at a time. Thousand Oaks, CA: Corwin Press.
Fuller, Loeb, Arshan, Chen and Yi. (2007). California Principals’ Resources: Acquisition, Deployment and Barriers. University of California, Berkeley and Stanford University Policy Analysis for California Education.
KDE/Gates Phase II: Using handheld technology to improve student learning. Frankfort, KY: Office of Leadership and School Improvement, Kentucky DOE.
Kelley, C. & Finnegan, K. (December 2003). “The effects of organizational context on teacher expectancy.” Educational Administration Quarterly, 23(5), 603-634.
For more information about the Walk’bout tool and training, visit www.acsa.org/WalkboutTraining or www.pivotlearningpartners.org.
Jim Hollis is director of Technology and the Innovation Incubator for Pivot Learning Partners (formerly Springboard Schools).