Shaffer Evaluation Group: Pioneering Educational Innovation and Research through Rigorous Evaluation

By: Patricia Moore Shaffer, PhD

Principal & CEO

At Shaffer Evaluation Group, we pride ourselves on driving educational innovation through meticulous, insightful, and impactful evaluation. Our track record speaks volumes about our capability to conduct rigorous educational evaluation, and we'd like to share a few standout projects that illustrate our expertise and dedication. These projects not only showcase our ability to handle complex evaluations but also highlight our commitment to fostering educational equity and excellence.

1. Boston College/DevTech Research Group: Coding as Another Language

Overview: The Coding as Another Language curriculum, which utilizes coding app ScratchJr, integrates computational thinking and computer science with learning about math and literacy through a collaborative, project-based process. 

Our Role: To assess the effectiveness of the Coding as Another Language curriculum, we designed and implemented a randomized control trial that met What Works Clearinghouse standards for study rigor. This study design allowed us to measure the effectiveness of the curriculum and its impact on student performance in literacy, mathematics, and coding skills. This evaluation was supported through an Educational Innovation and Research (EIR) grant from the U.S. Department of Education.

Key Findings:

• Second-grade student participants demonstrated significantly higher coding skills compared to the business-as-usual control group.

• The Hedge’s g effect size of 0.39 indicated a medium effect size of the impact of the curriculum on second grade students’ coding skills.

2. GrapeSEED: Oral Language Program for Pre-K

Overview: Preschoolers must be able to understand language at an oral level to be expected to understand it at the text level. GrapeSEED is a research-aligned, teacher-led program that helps preschool students build oral language and critical listening skills.

Our Role: We conducted several studies in the U.S. and abroad to assess the program's impact on student oral language proficiency. Our mixed-methods approach included surveys, interviews, classroom observations, and test score analysis.

Key Findings:

• GrapeSEED enabled at-risk preschool students to grow faster than their national non-risk peers in every category of language fundamentals.

3. Teacher National Board Certification    

Overview: National Board Certification is the most respected professional certification available in education and provides numerous benefits to teachers, students and schools. It was designed to develop, retain and recognize accomplished teachers and to generate ongoing improvement in schools nationwide.

Our Role: We conducted a quasi-experimental study that evaluated student outcomes in Mathematics and Reading in classrooms taught by Board-certified and non-Board-certified teachers in a southeastern school district.

Key Findings:

• Students taught by Board-certified teachers had significantly higher academic achievement in the subject areas of Mathematics and Reading. The gain in test scores was particularly large for students in Mathematics classes taught by Board-certified teachers.

• Students of color benefited more than their white peers from instruction by Board-certified teachers.

Conclusion

Shaffer Evaluation Group's extensive experience in evaluating innovative educational programs makes us uniquely qualified to conduct an Educational Innovation and Research (EIR) grant evaluation. Our proven methodologies, coupled with our commitment to educational equity and excellence, ensure that we can provide valuable insights and actionable recommendations to drive innovation and improve student outcomes.

Whether it's boosting student performance in STEM or improving early literacy skills, Shaffer Evaluation Group is dedicated to making a positive impact in education. Let's work together to create a brighter future for all students through innovative and effective evaluation!

---

Interested in applying for an Education Innovation and Research (EIR) grant?

If you are interested in applying for the EIR Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). Most recently, SEG collaborated with Boston College on their EIR Early-phase grant, which tested the efficacy of their Coding as Another Language curriculum. SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/

Please note the deadline for submitting the intent to apply is June 6, 2024, and the deadline for submitting the application is July 22, 2024. The mid-phase and expansion tiers have different deadlines. 

Education Innovation and Research (EIR) Project: Coding as Another Language (2019-2023)

By: Courtney Hagan, PhD

Evaluation Project Manager, Higher Ed

The Project

In partnership with the DevTech Research Group at Boston College, SEG evaluated their Education Innovation and Research (EIR) project ‘Coding as Another Language’ (CAL). The overall aim of this project was to improve computational thinking, coding skills, and language arts comprehension of early elementary school students. To achieve this, a delayed treatment design cluster RCT was implemented through randomization of 20 schools to either a control or treatment group. In the treatment group, schools implemented the Coding as Another Language curriculum using the ScratchJr platform. Teachers were provided with professional development to learn the curriculum and, throughout the school year, delivered up to 24 lessons to their students.

SEG's Role

SEG worked with project staff to design the evaluation plan according to What Works Clearinghouse standards. The evaluation plan was designed to meet standards without reservations through addressing randomization, attrition, and baseline equivalence. SEG worked with DevTech to facilitate the process of randomization and measure attrition and baseline equivalence. 

The Evaluation Plan

The evaluation plan also included an outcome evaluation and implementation evaluation. Throughout the project, data were collected from both students and teachers. Student data supported the outcome evaluation, and student assessments and state test scores were used in a hierarchical linear model to determine whether participating in the CAL curriculum had an impact on computational thinking, coding skills, and language arts comprehension. Teacher data were used to support the implementation evaluation. Teachers were asked to complete surveys and participate in focus groups to determine the threshold for fidelity of implementation.

---

Interested in applying for an Education Innovation and Research (EIR) grant?

If you are interested in applying for the Education Innovation and Research (EIR)

Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/

Please note the deadline for submitting the intent to apply is June 6, 2024, and the deadline for submitting the application is July 22, 2024. The mid-phase and expansion tiers grant competitions have different deadlines. 

Solicitation Calls for the Education Innovation and Research (EIR) Program Early Phase Grant

By: Courtney Hagan, PhD

Evaluation Project Manager, Higher Ed

About the EIR Program Grant

The US Department of Education has issued an invitation for applications to the Education Innovation and Research (EIR) program. The aim of this grant program is to support the “creation, development, implementation, replication, or scalability of evidence-based innovations to improve student achievement and attainment for high-needs students.” For this particular call, the EIR program is interested in supporting projects that “propose services and activities that help students recover from the COVID-19 pandemic, accelerate learning and academic achievement, reimagine schools, and transform our education system.”

Grant Requirements

The EIR grant is broken down into three different phases: early-phase, mid-phase, and expansion. Each phase carries different expectations and requires different levels of evidence to support the application. Early-phase grant submissions need to introduce a new practice based on prior research that suggests the new practice has “promise.” Potential grantees must also submit a logic model that outlines goals, objectives, outcomes, and project components to meet the required evidence standard.

In addition to supporting evidence, early-phase grant applications require the evaluation to be either an experimental or quasi-experimental design. Either design requires meeting What Works Clearinghouse standards, ensuring that a strong research design was implemented, and therefore, allowing conclusions being drawn from how the implementation impacts outcomes. The designs offered in the grant application should provide a statistically significant effect of the intervention on student outcomes.

---

Interested in applying?

If you are interested in applying for the EIR Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). Most recently, SEG collaborated with Boston College on their EIR Early-phase grant, which tested the efficacy of their Coding as Another Language curriculum. SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/

Please note the deadline for submitting the intent to apply is June 6, 2024, and the deadline for submitting the application is July 22, 2024. The mid-phase and expansion tiers have different deadlines.