What Is a Formative Evaluation?

When many people hear the word “evaluation,” the first thing that comes to mind is reviewing final outcomes to see if a project or program has been successful in meeting its goals and objectives. While looking at what happened at the end is important, there is also much value in better understanding how you got there—especially when seeking to improve your offerings or services while they’re still happening. 

What is a formative evaluation?

Formative evaluation—a term sometimes used interchangeably with “process evaluation” or “implementation evaluation”—is an evaluation conducted during the planning or implementation phase of a program in order to provide feedback that shapes program improvement. We regard formative evaluation as a unique opportunity to dig more deeply into the inner workings of a program and identify its successes, challenges, and opportunities.

Whereas outcome evaluations or impact evaluations are primarily done after a program is completed in order to assess its effects, formative evaluations provide data that can be reviewed while the program is still in progress—and while there is still time to make changes. This involves data collection and analysis at incremental steps throughout the life of a program, often followed by corresponding adjustments to planning or implementation activities. Formative evaluations can begin during a program’s planning stages, or they might be launched at the start of a certain program activity. Essentially, they can take place at any time while the program is operating.   

What does a formative evaluation look like?

While there is no single formula for conducting a formative evaluation, all begin with an essential first step: establishing what you hope to learn and why. After all, formative evaluation is not about final outcomes or accountability to a funder; rather, it’s rooted in the goal of program improvement. 

A formative evaluation might answer questions like:

• Do program delivery staff and leadership share the same desired program goals and objectives?
• Is the program reaching the intended audience?
• Are program resources being used effectively?
• What challenges might constrain consistent and accurate data collection, and how can they be addressed?

Determining your list of questions should be a collaborative process involving key stakeholders such as organizational leaders, board members, program staff, funders, partners, educators, youth, parents, and community members. When all stakeholders collectively articulate organizational and programmatic goals, formative evaluations produce meaningful data that can improve programs and organizations to better serve intended audiences. 

3 common methodologies of formative evaluations

There are many methodological approaches to formative evaluation, but these are three best practices in the field.

1. Qualitative data collection

While tracking quantitative data such as clients served, resources expended, or services offered has an important role to play in program evaluation, it is often the qualitative data that provides the most actionable insights into the “why” behind the quantitative data. 

You can collect qualitative data through methods such as interviews, focus groups, surveys, and observations—each of which provides valuable insights into the inner workings of a new or ongoing program. Program stakeholders, including staff, leadership, or community members, can provide nuanced explanations of what is happening on the ground to shape processes and outcomes. For example, these kinds of questions are perfect openings for formative qualitative inquiries: 

• Do program delivery staff have the resources they need to implement programs as designed? 
• Do community members feel that an intervention is adequately addressing their needs?
• What challenges make it harder for staff to implement the professional development lessons they have learned?

2. Participatory approaches

We believe that participatory approaches to formative evaluations are not only more equitable, but that they make for more valid, reliable, and actionable research. Those closest to the inner workings of a program have unique insights that can explain progress—or lack thereof. 

Metis often works with clients to identify representative stakeholders—parents, teachers, community-based organization staff, or any other relevant community members—who can serve on an advisory board. An advisory board meets regularly to lend a grass-roots voice to the evaluation process, helping the evaluators identify community needs and review research instruments and data. 

For example, advisory board input might tell us that a previously identified community need has been replaced by a higher priority. Members can also tell us what languages we need for survey translations, or even what words to use in the same language: Do youth refer to the program by the same name as the funder? Do community members call their meeting location a bodega or a corner store? A participatory process ensures that the questions we ask are both valid and culturally relevant to the communities being served—contributing to stronger response rates and more representative data.

3. Logic models

Metis also uses tools such as logic models to facilitate formative evaluations. As the W.K. Kellogg Foundation states, a logic model is “a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve.” If you haven’t articulated a logic model yet, we can help you create one that accurately reflects your program’s available resources and desired results. 

Once developed, a logic model can become a critical resource in the formative evaluation process, acting as a point of reference for understanding where and why intended outcomes may fall short—and how they can be addressed. For example, a logic model can help you answer the following questions:

• Are any identified resources unavailable? 
• Are any of our activities missing outcomes?
• Is there a desired outcome that lacks a corresponding activity? 

Once you identify potential shortfalls, they can be mapped onto the logic model to identify particular areas that need further attention. Specifically, having this knowledge is an invaluable opportunity to make programmatic or administrative changes—retrain staff, change partners, seek additional resources, provide new offerings—that can get programming back on track.   

Should you hire internal or external evaluators?

Before undertaking a formative evaluation, organizations must decide whether an external or internal evaluator best suits their needs. An internal evaluator is someone who already works for your organization, typically in some kind of evaluation, data management, or reporting role. An external evaluator like Metis is an outside consultant or organization that specializes in program evaluation.

An internal evaluator already knows your organization well, including your mission, organizational structure, history, and target population. They know what data are available and where to get them, and can often turn around information effectively and efficiently to support program improvement. Further, many nonprofits and educational organizations lack the staffing or resources to hire an external evaluator, making an internal evaluator a natural option.

On the other hand, external evaluators contribute an outsider’s perspective to your programming and evaluation efforts. They can bring a sense of objectivity, as they are less entrenched in organizational politics and can be franker about their observations. This can be beneficial when seeking external validation for evaluation findings (such as from a potential funder) or when trying to use a more neutral convener to engage stakeholders or community members. In addition, external evaluators often bring strong training and experience in evaluation methods and have a wider array of instruments and methodological approaches in their toolboxes.

Whichever approach is right for your organization or program, formative evaluation is invaluable when trying to improve your offerings and advance your mission. Producing quality programming is an iterative process, and formative evaluation can help your program get the data it needs—while you can still use it for improvement. Learn more about how Metis conducts evaluations, or reach out to start a conversation.

Get In Touch