Evaluation of NSF LSAMP Awards

The Louis Stokes Alliances for Minority Participation (LSAMP) program is a National Science Foundation (NSF) program that provides funding to alliances of colleges and universities to increase the number of STEM baccalaureate and graduate degrees awarded to populations historically underrepresented in these disciplines.

Evaluation is a key component of the NSF LSAMP program. Evaluation helps to ensure that a LSAMP project is meeting its goals and objectives and achieving outcomes for participating students. Evaluation also helps project teams identify areas where strategies and activities can be improved.

There are a number of different ways to evaluate LSAMP projects. One common approach is to use a mixed-methods approach, which combines quantitative and qualitative data. This approach can provide a more comprehensive understanding of the impact of the project while providing rich process findings to inform project implementation. Quantitative data can be used to measure the project's impact on student outcomes, such as academic performance, retention, and graduation rates. Qualitative data can be used to understand the students' experiences in the program, and the factors that influence their success.

Some common outcome measures used to evaluate LSAMP projects include:

• Sense of belonging: This outcome can be measured using surveys or interviews that ask students about their feelings of connection to the STEM community and their sense of belonging at their institution.

• STEM identity: This can be measured using surveys or interviews that ask students about their identification with STEM fields and their commitment to pursuing a STEM career.

• STEM self-efficacy: This can be measured using surveys or interviews that ask students about their confidence in their ability to succeed in STEM courses and careers.

In addition to these student measures, the impact of LSAMP on faculty and the institution can be measured. It is important to note that the best way to evaluate an LSAMP project will vary depending on the specific goals and objectives of the project. However, the measures described above can provide a good starting point for developing an evaluation plan.

In the Shaffer Evaluation Group's evaluation of the Central Florida STEM Alliance LSAMP project, a mixed methods evaluation design collects and analyzes data on project implementation and outcomes. The outcome evaluation utilizes both quantitative and qualitative data to identify student, faculty, and institutional impacts from this LSAMP project. Student outcome indicators include graduation rate, transfer rate, STEM self-efficacy and identity, and sense of belonging. The implementation evaluation monitors activity levels of students, faculty, and the participating institutions, using these to determine correlations to short-term student outcomes during the program, such as persistence and retention. The implementation evaluation also collects information to identify barriers or challenges that have impacted implementation, track improvement in service delivery, and assess the overall reach of the services provided. It also identifies actions taken by project staff to ensure the sustainability of strategies/activities beyond the grant funding period.

Evaluation of a LSAMP project can help ensure that it is meeting its goals and objectives and that it is making a positive impact on the lives of underrepresented students in STEM. In your LSAMP grant application, including a robust evaluation plan will improve the competitiveness of your application.

Are you applying for a LSAMP grant? Shaffer Evaluation Group welcomes the opportunity to partner with your institution on this grant opportunity. Once you have a draft grant application in hand, reach out to us for assistance with the evaluation plan.

References

• Tinto, V. (1987). Leaving college: Rethinking the causes and cures of student attrition. Chicago, IL: University of Chicago Press.

• Byars-Winston, A. M., Estrada, M., & Bell, C. (2016). Racial and ethnic identity, STEM self-efficacy, and STEM persistence: A meta-analysis. Educational Psychologist, 51(2), 109-129.

Postsecondary Student Success Program Grantwriting: A Step-by-Step Guide

By Patricia Moore Shaffer

Principal & CEO

The Postsecondary Student Success Program (PSSP) grant, administered by the U.S. Department of Education, aims to support initiatives that improve student outcomes and success in institutions of higher education (IHE). The purpose of this program is to equitably improve postsecondary student outcomes, including retention, transfer (including successful transfer of completed credits), credit accumulation, and completion, by leveraging data and implementing, scaling, and rigorously evaluating evidence-based activities to support data-driven decisions and actions by institutional leaders committed to inclusive student success. To successfully apply for this grant, institutions should follow a step-by-step process that includes careful planning, thorough research, effective proposal writing, and meeting the evaluation requirements.

Step 1: Understand the Grant Program

Before starting the application process, it is crucial to thoroughly understand the Postsecondary Student Success Program grant program. Review the program's guidelines, objectives, and eligibility criteria. Familiarize yourself with the specific requirements and priorities outlined by the U.S. Department of Education, including the evaluation requirements. Visit the Department of Education program page for easy access to these materials.

Step 2: Identify Institutional Needs

Identify the areas within your institution that require improvement and align with the grant program's objectives. Conduct a comprehensive needs assessment to determine the specific challenges faced by your institution and collect data and evidence to substantiate these needs. This may include student enrollment and retention rates, graduation rates, student satisfaction surveys, and any other information that demonstrates the need for the proposed activities.

Step 3: Develop a Project Plan

Based on the identified needs, develop a plan that outlines the project's goals and objectives, and identifies the strategies and activities you propose to implement. Ensure that your plan aligns with the grant program's objectives and priorities. For instance, if your institution aims to enhance student advising and support services, your project plan may include the implementation of a comprehensive advising program, peer mentoring initiatives, and academic support resources.

At this step, your institution should also decide whether it is best positioned to seek an early-phase or mid-phase/expansion grant. Early-phase grants provide funding to IHEs to develop, implement, and test the feasibility of a program that prior research suggests is likely to improve relevant outcomes, for the purpose of determining whether an initiative improves student retention and completion of postsecondary students. Mid-phase or expansion grants are supported by moderate or strong evidence and provide funding to IHEs to improve and/or expand initiatives and practices that have been proven to be effective in increasing postsecondary student retention and completion.

Step 4: Evaluation Plan

Develop a comprehensive evaluation plan that outlines how you will measure the success and impact of your project. For this grant application, there are expectations of rigor for the evaluation design. The evaluation plan will be judged on its ability to produce evidence about the project’s effectiveness that would meet the What Works Clearinghouse standards with or without reservations while also providing performance feedback and permitting periodic assessment of progress toward achieving intended outcomes. Evaluation plans for mid-phase grant applications also must demonstrate the extent to which the evaluation will provide guidance about effective strategies suitable for replication or testing in other settings.

Step 5: Budget Preparation

Prepare a detailed budget that clearly outlines the costs associated with your proposed activities. Ensure that your budget aligns with the grant program's guidelines and restrictions. Provide a breakdown of expenses, including personnel, equipment, supplies, and any other necessary resources. Justify each expense and demonstrate how it directly contributes to achieving the project's objectives.

Step 6: Proposal Writing

Write a compelling and well-structured proposal that clearly articulates your institution's needs, project plan, and expected outcomes. Address each component of the grant application, including the abstract, absolute priority, significance, project design, project personnel, management plan/budget, and evaluation plan (see Step 4). The mid-phase application also requires a description of the program scaling strategy. Use clear and concise language, provide evidence to support your claims, and adhere to the specified word limits.

Step 7: Review and Revise:

Before submitting your application, carefully review and revise your proposal. Ensure that it is free from grammatical errors, typos, and inconsistencies. Seek feedback from colleagues or experts in grant writing to improve the clarity and coherence of your proposal. Make any necessary revisions to strengthen your application.

Step 8: Submit the Application:

Once you have completed all the necessary steps and reviewed your application, it is time to submit it. Follow the instructions provided by the grant program for submitting your application. Ensure that you meet all the deadlines and include all the required documents and attachments.

Writing a successful Postsecondary Student Success Program grant application requires careful planning, thorough research, effective proposal writing, and meeting the rigorous evaluation requirements. By following this step-by-step process, institutions can increase their chances of securing funding to implement innovative strategies that enhance student success in postsecondary education. Remember to align your proposal with the grant program's objectives, provide compelling evidence, and develop a comprehensive evaluation plan that measures the impact of your project. With dedication and attention to detail, institutions can create a strong application that demonstrates their commitment to improving student outcomes and success.

Shaffer Evaluation Group welcomes the opportunity to partner with your institution on this grant opportunity. Once you have a draft project plan in hand, reach out to us for assistance with the evaluation plan.

The Benefits of Holistic Advising in Community Colleges

By Patricia Moore Shaffer

Principal & CEO

In today's rapidly changing educational landscape, the importance of effective advising programs cannot be overstated. Particularly in community colleges, where students often have multiple responsibilities, including working full-time, parenting, or returning to school after a lengthy break, a holistic advising program can play a crucial role in supporting their academic and personal growth. This blog post explores this approaching to advising, highlighting its benefits and providing examples of its positive impact on students' success.

A holistic advising program at a community college encompasses a comprehensive approach that addresses students' academic, career, and personal needs. It goes beyond traditional academic advising by considering students' individual circumstances, aspirations, and challenges. By providing tailored guidance and support, this program aims to empower students to make informed decisions, set achievable goals, and navigate the complexities of college life successfully.

A holistic advising program recognizes that academic success is a cornerstone of students' college experience. Advisors work closely with students to develop personalized academic plans, ensuring they are aware of degree requirements, course sequencing, and available resources. Advisors may conduct regular check-ins with students to monitor their progress, identify potential obstacles, and provide timely interventions. As a result, students feel supported and are more likely to stay on track towards graduation.

In rural areas, students often face limited access to career opportunities and resources. A holistic advising program addresses this challenge by offering comprehensive career guidance. Advisors assist students in exploring potential career paths, identifying internships or job-shadowing opportunities, and developing essential skills. Advisors may organize career fairs, invite guest speakers, and facilitate networking events to connect students with local employers. This proactive approach enhances students' employability and equips them with the necessary tools to succeed in their chosen fields.

Recognizing that students' personal well-being significantly impacts their academic performance, a holistic advising program prioritizes their overall development. Advisors provide a safe and supportive environment for students to discuss personal challenges, mental health concerns, or any other issues affecting their college experience. Advisors may collaborate with campus counseling services to ensure students have access to the necessary support systems. By addressing students' emotional well-being, the program fosters a positive learning environment and enhances their overall college experience.

A holistic advising program at a community college is a vital component of student success. By considering students' academic, career, and personal needs, this program provides comprehensive support that empowers students to thrive academically and personally. Through personalized academic planning, career guidance, and attention to personal development, students are better equipped to overcome challenges and achieve their goals. As community colleges continue to evolve, investing in holistic advising programs will undoubtedly contribute to the overall success and well-being of their students.