How to Measure Effectiveness in FSCS Grants

Historically, Full-Service Community School (FSCS) grants required grantees to report on the program’s progress twice a year through a series of performance measures identified in your grant proposal and finalized during your grant’s first year. In 2024, the United States Department of Education is redesigning the FSCS grant’s reporting expectations—moving towards a results-based accountability (RBA) framework that is shifting from a focus on compliance to reflection and growth. But the challenge for grantees remains the same: How do you measure grant performance in a way that simultaneously meets federal requirements and promotes continuous improvement for your program?

Metis Associates has been partnering with FSCS grantees since the first year of the grant program in 2018. In 2024 we are partnering on FSCS grants with both school districts and non-profit partners across the United States. As program evaluation experts, we leverage several strategies for measuring the success of our FSCS clients that we hope you’ll make a part of your performance measurement process no matter what type of grant award you’re going after.

Create resonant performance measures rooted in your theory of change

While it’s common for performance measures to be comprised of escalating targets tied to a baseline (e.g., a 5% over baseline in Year 1), your performance measures should align with your theory of change and capture the outputs and activities that drive your program’s intended outcomes. Outcomes could include family members served and student attendance rates or the number of professional development opportunities offered.

To this end, the targets within a given measure should resonate with those carrying out the work being measured and the thresholds should feel meaningful. For example, do a certain number of teachers need to be trained in a key pedagogy for a school-wide instructional shift to occur? Do perceptions of school safety need to reach a certain level before school climate no longer hampers student and teacher attendance?

Questions like these will be easier to answer with a strong logic model, for several reasons. First, the linkages between your activities and outcomes will already be clarified. Second, you’ll have already identified the most important activities to include in your performance measures. Third, you’ll be able to communicate the relevance of the performance measures to a broad range of stakeholders. 

Performance measures should also represent the value that the community school model brings to a traditional school setting. For example, your performance measures might address underlying questions like, “What can an FSCS-funded school coordinator offer a school that a traditional leadership team (i.e., one without a school coordinator) cannot accomplish?” and  “How does a partnership within a community school framework look different from a traditional partnership that a school principal might build with their organization?” by seeking to measure the activities of a coordinator or the strength of a partnership. 

Finally, the measures should be written to support the long-term sustainability of the grant-funded activities. This means developing measures that are written flexibly enough to accommodate shifts in who may carry out key activities as a grant matures and school staff and administrators take on greater responsibilities. 

Make your data collection participatory and supportive of continuous improvement

To ensure that the performance measurement process not only satisfies grant requirements but is also locally meaningful, it should be both participatory in nature and supportive of continuous improvement. 

Participatory process

First, school-based staff should be invited into the performance measurement process and given the necessary training and support to become active collectors and users of evaluation data.

When we partner with our FSCS and other clients on evaluation, we support this level of involvement in multiple ways, from frequent meetings with clients, to the cultivation of advisory committees, to co-creating dedicated data collection instruments, which we iteratively improve as stakeholders’ access to data changes. For one longstanding client, for example, Metis created a series of attendance trackers to measure an unduplicated count of students, family members, and community recipients of community school services, thereby addressing the grant’s requirement to report the total (i.e., unduplicated) individuals served on an annual basis. 

Second, data collection should be designed to elevate the voices of those served by the community school model, such as teachers, parents, community members, and students. This can include the use of existing data sources, such as school climate and social-emotional health surveys administered by school districts, or feedback forms provided by external partners as part of their own self-evaluation efforts (e.g., “exit ticket” or satisfaction surveys). 

When extant data are not available, evaluation partners should work with school stakeholders to deploy supplemental surveys. For example, we’ve developed FSCS-specific surveys for families, advisory boards, and school staff, as well as co-created with our clients activity-specific feedback forms for both student, family, and community enrichment activities and professional development activities for teachers and staff.  

Supportive of continuous improvement

To support continuous improvement throughout a grant’s lifecycle, transparency around the evaluation process should be maintained and findings should be shared back with on-the-ground stakeholders in a timely manner. Together, along with the establishment of a trusting client relationship and clear lines of communication, performance measurement can be transformed into a process that is locally meaningful. We recommend:  

1. Creating clear analysis plans outlining how collected data are processed. 
2. Delivering survey, interview, and other data collection findings outside of the bi-annual reporting requirements required by the grant as quickly as possible. 
3. Including staff in the identification of successes and challenges—those carrying out the work are often best-positioned to reflect on the grant’s progress. 
4. Including FSCS-funded school coordinators and other key staff in the development of Annual Performance Reports and other federally-required deliverables (such as mid-year and local evaluation reports).

Best practices beyond performance measurement

At Metis, we believe strongly that working with an evaluation partner can add considerable value to an FSCS grant beyond contributions to performance measurement. For our community school clients, we’ve identified two essential techniques to strengthen implementation, further sustainability, and facilitate the objective documentation of program successes: fidelity inventories and rigorous impact studies. 

Fidelity inventories

A fidelity inventory is a quantifiable assessment of the extent to which a given community school is embodying a specific community school strategy. While we anchor fidelity inventories in the specifics of our client’s model, we also draw on established frameworks such as the Coalition for Community Schools’ Community School Standards. 

Our fidelity inventories are comprised of a series of statements that describe concrete school-based activities, such as “partners meet regularly with school staff” or “advisory boards are convened on a monthly basis,” developed through an intensive collaborative process with our clients and based on the logic model, document reviews, interviews, and facilitated workshops. Once deployed, school stakeholders are asked to assess how their school is enacting these strategies, through a joint process of communal reflection which results in a series of quantifiable metrics. Through this exercise, we are able to help you better understand how your program design works in practice and the ways in which it is adapted by local staff to each schools’ own context. 

Additional uses of the fidelity inventory include identifying school-specific and cross-school capacity building needs, and identifying high-implementing schools that can offer leadership, intervisition, and peer learning to others.

School stakeholders have remarked that the process of reflection has allowed them to better understand the holistic nature of their community school strategies as well as where communication and transparency may be lacking. The results of a fidelity inventory can also help clients understand the extent to which their models change amid local contexts. This is an especially important lens for FSCS grantees that have received Multi-Local Education Agency FSCS grants and are faced with the prospect of bridging multiple districts and communities with different needs, school cultures, and processes.

Rigorous impact studies

Rigorous impact studies are only applicable for certain clients whose programs have sufficiently matured, but when incorporated in a community school network’s evaluation design, it can objectively ascertain whether the outcomes of students attending a given community school (or set of schools) are better than the outcomes of students attending a comparison school (or set of schools). 

While a study of this type requires considerable financial investment, it also requires lesser involvement of school staff (given that school and district-level data become the focus), and can be a good choice if your grant requires external validation. One of our clients found that attendance, credit accumulation, and gains in English as a New Language students’ English-language proficiency were positively associated with the introduction of their community school model, as compared to other schools within their local district.

New opportunities in performance measurement for FSCS grants

We believe that performance measurement for Full-Service Community Schools is moving in a transformative and exciting direction, shifting from a focus on compliance and accountability to that of reflection and growth as grantees begin using Results Based Accountability to document their grant’s metrics. 

For recent grant cycles, these expectations have already shifted, with an additional—and important—focus on equity through the encouraged disaggregation of measures by student and teacher characteristics. We anticipate that the same motivations that drive effective performance measurement now will become even more relevant: specifically, a commitment to continuous growth, involving stakeholders in the performance measurement process, and remaining theory driven. 

If you’re looking for a partner to help you get the most out of these new approaches to performance measurement, we’d love to join you in your continuous improvement journey. Please reach out to start a conversation.

Get In Touch