When the COVID-19 pandemic disrupted learning for millions of students nationwide, Georgia moved swiftly to address learning loss and support student well-being. Through the American Rescue Plan Act, the state allocated approximately $85 million to expand access to high-quality afterschool and summer enrichment programs for K-12 youth. The Georgia Statewide Afterschool Network (GSAN), in partnership with the Georgia Department of Education (GaDOE), launched the Building Opportunities in Out-of-School Time (BOOST) Grants Program in 2021 to distribute these funds through a competitive grant process designed to reach youth most impacted by the pandemic.
Designing a Comprehensive Three-Year Evaluation
To independently assess the effectiveness of BOOST and inform continuous improvement, GSAN issued a competitive RFP for evaluation services in February 2022. Metis Associates was selected from ten applicants to serve as the evaluation partner, bringing expertise in youth development programming, participatory evaluation methods, and statewide data collection.
Metis developed a comprehensive cross-site evaluation with three interconnected components. The Implementation Study documented service delivery, youth satisfaction, partnerships, and lessons learned across all 96 grantees. The Outcome Study assessed learning acceleration, connectedness, and well-being outcomes for participating youth. The Systems Study examined the quality and effectiveness of program oversight, administrative support, and long-term sustainability planning.
Building Evaluation Capacity Through Participatory Approaches
To ensure the evaluation reflected the perspectives of grantees and youth, Metis facilitated two advisory groups. The BOOST Evaluation Advisory Group brought together GSAN staff and ten grantee representatives who met quarterly throughout the three-year evaluation to provide feedback on data collection tools, reporting systems, and evaluation findings. The Youth Evaluation Advisory Group engaged middle and high school students in learning evaluation methods and contributed directly to the research by helping to develop focus group protocols and moderate family focus groups during virtual case study visits, thereby bringing youth perspectives to the data collection process.
Metis also delivered extensive, customized technical assistance to strengthen grantee evaluation capacity. This included working closely with individual grantees to develop SMART outcomes, identify appropriate assessment tools, and establish data-collection systems. The evaluation team paid particular attention to supporting statewide grantees in collecting and reporting data across their many subgrantee sites.
Documenting Significant Reach and Impact
Over three years, BOOST demonstrated substantial reach across Georgia. In Year 3 alone, 96 grantees operated 1,429 academic year sites and 669 summer sites, serving over86,000 youth during the academic year and nearly 83,000 during summer months. The program extended into 115 of Georgia’s 159 counties, with youth participants residing in 141 counties representing 89% of the state
The evaluation revealed strong program effectiveness across all service areas. The majority of grantees met or exceeded at least one target outcome in both the academic year and summer programming. Learning acceleration outcomes were achieved for approximately 70,000 youth during the academic year and 54,000 during the summer. Well-being and connectedness outcomes benefited more than 75,000 youth, while enrichment outcomes were achieved for nearly 35,000 participants.
Youth satisfaction data showed high levels of engagement, with 89% of academic-year participants and 93% of summer participants reporting satisfaction with their overall program experience. Grantees showed statistically significant gains in the implementation of best practices across all six quality improvement areas, including service delivery, program sustainability, and evaluation and data use.
Supporting the Transition to Sustained State Funding
The evaluation findings proved instrumental when federal ESSER funding concluded in 2024. Armed with comprehensive data on BOOST’s impact, GSAN successfully advocated for continued state investment. The Georgia legislature appropriated $11.5 million annually for BOOST 2.0, ensuring the program’s sustainability beyond federal relief funding.
Metis continues to serve as the evaluation partner for BOOST 2.0, adapting the evaluation design to meet evolving program needs. Current work includes providing evaluation technical assistance to help grantees strengthen their local outcomes measurement, administering an annual BOOST 2.0 Grantee Survey, and analyzing grantee outcomes and implementation progress for annual reporting.
Beginning in the BOOST 2.0 initiative’s second year, the evaluation will expand to examine gains in student achievement on state reading and math assessments through a data-sharing partnership with the Georgia Department of Education. This enhanced design will enable Metis to connect program participation with academic performance outcomes, providing critical evidence of BOOST’s impact on learning acceleration—a core program objective.
This ongoing partnership enables GSAN to demonstrate accountability to state funders and to support continuous quality improvement across Georgia’s out-of-school time system. The BOOST evaluation exemplifies Metis’s commitment to participatory evaluation methods, the development of local evaluation capacity, and the production of actionable findings that inform both program improvement and policy decisions. Through this multi-year partnership, Metis has helped establish a model for statewide OST program evaluation that other states are now looking to replicate.
Georgia’s statewide afterschool evaluation shows how the right evaluation design can surface what’s working across dozens of sites at once. See how our program evaluation services can bring that same clarity to your initiative.