Developing a Plan for Outcome Measurement

 

Overview

Welcome to the e-learning lesson on developing a plan for outcome measurement. The world of evaluation uses countless words and varying terminology to describe the process of evaluating outcomes. For the purpose of this e-learning, we will be discussing outcome measurement as a means of exploring the impacts or results of a program or initiative. An outcome is a measurable and observable change in individuals, groups, organizations, systems, or communities. Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended results. The main questions addressed in outcome measurement are: What has changed in the lives of individuals, families, organizations, or the community as a result of this program? Has this program made a difference? How are the lives of program participants better as a result of the program? By the end of this lesson, you will be able to: understand how to identify measurable outcomes and outcome chains, develop a logic model, and identify performance indicators and performance targets to support the outcome measurement process.

Outcomes communicate value.

Outcomes are the reason nonprofit organizations strive to deliver programs and build their capacity. Most importantly, however, proper evaluation techniques provide your organization with proof of their value to existing funders, potential funders, and the larger community. Whether this value is communicated in dollars or the number of individuals served, quantifiable performance measures are becoming important in the increasingly competitive social service industry.

Successful outcome measurement hinges on a cohesive plan.

Preparation and planning is a key part of the outcome measurement process. The better the planning, the more impact the outcome measurement will have on your organization and your organization's bottom line. By the conclusion of this lesson you will be able to:

CHAPTER 1: Getting Started

Your outcome measurement efforts will benefit greatly from some planning at the front end. Before you jump head first into the outcome measurement process, convene stakeholders and organizational leadership to discuss the important questions. Where will you focus your efforts? What are you hoping to accomplish through the process? Who will be involved in the process? How will you engage members of your organization in the process? What is your timeline? What resources are at your disposal? Once you have answers to these questions, you can begin to look more closely at the specific outcomes and performance indicators that will drive the outcome measurement process.

Narrow your scope and clarify your goals.

In starting out, it's important not to try to measure too much. Start slowly and learn from your experience. Don't try to perform outcome measurement at the same time for everything you do. Pick one program or service as a starting point. Questions that will help you brainstorm where to begin include:

Once you have decided what program to evaluate, explore the purpose for the outcome measurement processes. There are a variety of reasons that could motivate outcome measurement, including:

Assemble a well-rounded team.

The outcome measurement team will require a well-rounded group with varying skill sets and a devoted leader. As you begin to assemble your outcome measurement team, consider who on your staff possesses the following skills:

The outcome measurement process can often feel intimidating or threatening to program staff, as outcome measurement is sometimes a response to poor performance or part of a corrective action plan. To allay your staff's concerns about outcome measurement, involve them in the process whenever possible.

Outcome measurement is an investment.

Developing an outcome measurement plan will require an investment of time and resources. For organizations first learning outcome measurement, it may take as long as three years to develop a comprehensive outcome measurement system. Timing is also important to funders. Make sure to consider when your funders' reporting cycles are so that you are producing outcome measurement results at a time that aligns with their requests for information about your programs' impacts. Keep in mind that there are times when conducting outcome measurement may not be a good idea. If your organization is in crisis or severe financial trouble, outcome measurement cannot be a priority.

In addition to investing time, outcome measurement may require an investment of other resources, including:

CHAPTER 2: Understanding Outcomes

Prior to beginning the process of data collection, you'll need to define the outcomes you want to accomplish. As a nonprofit organization, your intended outcomes focus on the impacts or changes that your clients experience as a result of your programs and services. When crafting outcomes, consider the who, what, and how of the initiative. Who will be impacted by the initiative? What will change as a result of the initiative, and how will it change? When identifying outcomes, it can be helpful to organize your thoughts in the form of an outcome chain that logically links your services to client learning, behavior, and, eventually, results.

Outcome statements capture the "who," "what," and "how."

Formulaic, straightforward outcome statements can help you develop your outcomes. When crafting an outcome statement, consider the details of the initiative you are providing, the recipient of that service or program, and the intended impact of that service on your target recipients. For instance, a youth-serving nonprofit might identify the following outcome statements in relation to their afterschool programming:

Outcome chains document correlative relationships.

Outcome chains can help organize your thinking about what you hope to achieve. They require you to put your program theory to work and articulate how your activities will bring about the impacts in the organizations with whom you are working. Outcome chains create a logical progression of the short-term, intermediate, and long-term outcomes that lead to your goals. Consider the example of a large nonprofit that provides training to smaller nonprofit organizations. The provision of training can create the following chain, linking reactions, learning, behavior, and results:

Not all organizations can and will measure all the different outcomes noted in an outcome chain. Consider the example of the outcome chain above. Your organization may not have the tools and resources to evaluate outcomes as they relate to reaction, learning, behavior, and results. If your circumstances do not allow you to evaluate all areas, focus on the earlier outcomes noted in the outcome chain. There is no point in measuring for results if you cannot point to the series of outcomes that impacted those results.

Develop realistic, informed outcomes.

When crafting outcomes, consider the following:

CHAPTER 3: Logic Models and Outcome Measurement

An organization should have a well-developed logic model in place before they finalize a comprehensive outcome measurement plan. A logic model maps out an overview of an organization's tools and resources, the services they provide, and the intended impacts of these services. A basic logic model documents inputs or resources, activities, outputs, and short-term, intermediate, and long-term outcomes. Inputs or resources are the assets that an organization is prepared to invest to support or implement a program, including things like money, staff, and equipment. Activities capture the methodologies an organization plans to use in order to implement a project, while outputs describe activities in more finite, numerical terms such as the units of service provided. Lastly, outcomes capture the changes, benefits, and overall impact that the program or initiative has had on an organization's client population. Once a well-developed logic model is in place, an organization can begin to identify performance indicators that will help to measure the organization's progress towards its intended outcomes.

Ask the right questions.

When looking to identify the elements of your organization's logic model, consider the following:

Logic models capture underlying assumptions and program theory.

This connection between the activities you provide and the outcomes you hope to accomplish is known as the program theory; it articulates the assumptions about the ability of certain activities to drive particular changes. Many nonprofit providers use logic models as a tool to reflect the program theory underlying their programs. One contribution that outcome measurement can make in your own organization is to demonstrate whether or not the program theory underlying your organization's activities and initiatives is valid.

Logic models document relationships.

While not all logic models look the same, they all serve the same purpose: to graphically capture the assumptions and cause and effect relationships that drive your organization's work on a project.

Download a sample logic model template and test your understanding of the different elements of a logic model using the activity on the right.

CHAPTER 4: Developing Performance Indicators

The next step in putting together an effective outcome measurement plan focuses on deciding how you are going to make your intended outcomes measurable by defining a set of performance measures or indicators. Because outcomes can be broad in nature and somewhat vague, performance indicators serve as a bridge, connecting intended outcomes and the actual data collection process. Identifying performance indicators and creating targets for performance help your organization to determine whether you have, in fact, had a measurable impact on your clients and reached your programmatic goals.

Indicators are specific and observable.

In order to serve effectively as a bridge to data collection, indicators must be specific items of information that describe observable and measurable characteristics or changes in corresponding outcomes. Indicators must be measures that can be seen, heard, counted, reported, or enumerated using some type of data collection method.

Performance indicators answer questions like, "How you will know when changes have occurred?" and "How you will know when you have achieved the outcomes?" Thinking ahead to possible data collection methods will tell you if your indicators are specific enough. Ask questions like these to determine whether your indicators will work:

Be thorough, yet realistic when identifying indicators.

Each outcome you identify will generally have one to three supporting indicators depending on the complexity of the outcome. When selecting indicators, keep things simple and be sure to collect indicators that are useful, meaningful, and informative. Keep in mind that you will be responsible for measuring these indicators through your data collection process. Consider the following checklist:

Targets, comparative standards, and baselines give meaning to indicators.

When developing indicators, it is often helpful to identify targets, baselines, and comparative standards.

Summary

Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended results. Preparation and planning is a key part of the outcome measurement process. The better the planning, the more impact the outcome measurement will have on your organization and your organization's bottom line. You should now understand how identifying outcomes and outcome chains, developing a logic model, and identifying performance indicators and performance targets are integral parts of planning for the process of outcome measurement. Thank you for taking the time to learn about developing a plan for outcome measurement.

Outcome measurement proves correlation, not causation.

Keep in mind that outcome measurement explores what your program provides, what its intended impacts are, and whether or not it achieves them. It will not prove that the changes that take place are a result of your program. In order to prove direct causation, an organization will need to take part in experimental research and a controlled study to link your specific programs and activities to results. Naturally, this requires considerable time, effort, and resources.

Learn more about outcome measurement at the links below.