Improving mathematics instruction through evidence

We are evaluating several educational initiatives led by school districts that seek to improve student achievement in mathematics by refining their approach to mathematics instruction.

One approach commonly used by our district partners is the Concrete-Representational-Abstract (CRA) model for teaching mathematics, a research-based, three-part, systematic approach designed to build student understanding. At the concrete step, the concept is introduced using manipulatives (base ten blocks, fraction bars, counters) to solve a problem. During the representational stage, students use drawings (pictures, tallies, dots, graph paper) to represent the concrete objects to solve the problem. The abstract step problem solving moves to using symbols, operations, and formulas to solve a problem. Implementing this systematic approach to teaching mathematics has been shown to build student understanding of mathematical concepts.

We frequently support implementation by gathering data on fidelity of implementation through the analysis of teacher lesson plans using a rubric and/or classroom walk-through data with an observation protocol. In both cases, the data collection instruments are tightly aligned with the instructional model. Monitoring the observed frequency of specific instructional indicators provides formative feedback to the project team, which is used to plan for continued professional learning and/or support specific areas of need and monitor implementation progress over time.

Strategic Planning in Charles City, Virginia

When Charles City Public Schools in Virginia was looking for a consultant to assist with strategic planning, they contacted the Shaffer Evaluation Group. We're working closely with school district officials in the development of outcome-focused performance measures that will enable them to assess progress toward accomplishing their strategic goals and objectives. We look forward to continuing to work with them on collecting and/or analyzing performance data across the strategic plan period, and reporting results in a manner that is highly visible and easily accessible to school district staff, students, parents, and the Charles City community.

Strategic planning is more important than ever for school districts who are working to succeed in the face of multiple internal and external challenges. A well-crafted strategic plan clearly defines what an organization intends to achieve. Bryson (2011) defines strategic planning as a “deliberate, disciplined approach to producing fundamental decisions and actions that shape and guide what an organization is, what it goes, and why.” Strategic planning is a process that requires stakeholders to engage with one another, consider where they are, where they want to be, and how to get there (Bryson & Alston, 2004). Evaluators are often asked to support school districts and organizations with their strategic planning process due to their expertise in data collection and performance measurement. Evaluator support may include: data collection from stakeholders and associated analyses during the planning stage; development of SMART performance measures used monitor progress of selected strategies or objectives; and creating data collection guides used by school district staff during implementation. A well-crafted strategic plan should effectively link the organization’s stakeholders to the overarching mission and vision of the school district with clear and measurable performance outcomes of the strategic plan.

Evaluation Capacity Building with AmeriCorps

As evaluators, we often fulfill a variety of roles within partner organizations. In recent years, accountability expectations continue to increase, while budgets have often declined. As a result, our work as facilitators for building internal capacity within our partner organizations has become a more frequent request. Such work is defined within the literature as Evaluation Capacity Building (ECB). Stockdill, Balzeman, and Compton (2002) define ECB as “the intentional work to continuously create and sustain organizational processes that make quality evaluation and its uses routine.” This year, the SEG team had the opportunity to support Virginia AmeriCorps programs through small group virtual training events and 1:1 technical assistance to enhance understanding of theory of change, evidence-based practices, performance measurement, and data collection in order to meet the Corporation for National and Community Service (CNCS) evaluation expectations. By increasing internal evaluation capacity, programs can ensure they are collecting high quality evidence to refine their interventions, demonstrate impact, and “tell the story” of their work.