Upcoming workshop on evaluating professional learning

Evaluation is the key to determining the impact of teacher professional development (Guskey, 2000). Shaffer Evaluation Group will deliver an interactive concurrent session on evaluating professional learning at the 22nd Annual SURN Leadership Conference, June 18, 2018, in Williamsburg, VA. The SURN Leadership Conference offers a forum to learn and exchange ideas on the campus of the William and Mary School of Education. Leading education writers, researchers, and consultants provide international and national perspectives on their work. The opportunity to share with other educators, discuss topics with leading researchers, reflect on the prior school year, be inspired, and plan for the upcoming year are all reasons individuals and school district teams attend.

During this 90-minute session, school and district leaders can improve their ability to measure and strengthen the impact of professional learning initiatives on teacher beliefs, dispositions, and practices. Participants will learn strategies to develop a plan and discover low- or no-cost assessment instruments for documenting and monitoring the effectiveness of professional learning efforts on student performance outcomes. Importantly, the session will help school and district leaders meet ESSA requirements, which call for professional development activities to be “regularly evaluated for their impact on increased teacher effectiveness and improved student academic achievements, with findings of the evaluations used to improve the quality of professional development” (ESSA SEC 8002).

Registration for the SURN Leadership Conference is now open -- come join us!

Evaluating Literacy Interventions

We are evaluating several educational initiatives led by school districts that seek to improve reading

achievement by refining their approach to literacy intervention, specifically targeting students with disabilities. One approach used by two of our district partners is the Leveled Literacy Intervention (LLI) deployed specifically by special education teachers. Districts invest in ongoing teacher professional learning and resources to bolster special education teacher’s access to high quality reading resources and confidence to deliver targeted and intensive reading interventions.

During initial phases of implementation, ensuring the intervention is delivered with fidelity is a priority for project teams. In our role as evaluators, we collaborated closely with school district staff to develop tools, such as classroom observation protocols, and often train district staff to utilize those tools. Training a team of observers builds inter-rater reliability for consistent identification of strategies observed across classrooms. This type of data is useful for identifying program elements that are being implemented successfully, and those that are not. Analysis of fidelity data allows project teams to target continued support and professional learning to close the gaps. Formative data collection, such as professional development feedback, annual staff surveys, and focus groups, provides additional insight on classroom implementation successes and challenges and overall efficacy with the intervention. Triangulation of the data sources enhances data validity and reliability, ensuring our district partners are able to make informed decisions.

Improving mathematics instruction through evidence

We are evaluating several educational initiatives led by school districts that seek to improve student achievement in mathematics by refining their approach to mathematics instruction.

One approach commonly used by our district partners is the Concrete-Representational-Abstract (CRA) model for teaching mathematics, a research-based, three-part, systematic approach designed to build student understanding. At the concrete step, the concept is introduced using manipulatives (base ten blocks, fraction bars, counters) to solve a problem. During the representational stage, students use drawings (pictures, tallies, dots, graph paper) to represent the concrete objects to solve the problem. The abstract step problem solving moves to using symbols, operations, and formulas to solve a problem. Implementing this systematic approach to teaching mathematics has been shown to build student understanding of mathematical concepts.

We frequently support implementation by gathering data on fidelity of implementation through the analysis of teacher lesson plans using a rubric and/or classroom walk-through data with an observation protocol. In both cases, the data collection instruments are tightly aligned with the instructional model. Monitoring the observed frequency of specific instructional indicators provides formative feedback to the project team, which is used to plan for continued professional learning and/or support specific areas of need and monitor implementation progress over time.