1 - Logic Models and Outcome Measurement - Evaluating Training and Technical Assistance

Logic Models and Outcome Measurement

You need Adobe Flash Player to view some content on this site.

An organization should have a well-developed logic model in place before they begin to develop a comprehensive evaluation plan. A logic model maps out an overview of an organization’s tools and resources, the services they provide, and the intended impacts of these services. A basic logic model documents inputs or resources, activities, outputs, and short-term, intermediate, and long-term outcomes. Inputs or resources are the assets that an organization is prepared to invest to support or implement a program, including things like money, staff, and equipment. Activities capture the methodologies an organization plans to use in order to implement a project, while outputs describe activities in more finite, numerical terms, such as the number of training hours provided. Last, outcomes capture the changes, benefits, and overall impact that the program or initiative has had on an organization’s client population. Once a well-developed logic model is in place, an organization can begin to analyze its stated outcomes and develop performance measures and a detailed evaluation plan.

Clearly defined outcomes become organizational goals and hypotheses.

Organizations may find it helpful to analyze their activities and outputs through the “if/then” lens. When developing outcomes, an organization should ask itself, “If we provide these activities and outputs, what do we hope will then happen?”

The answer to this question should provide an organization with short-term, intermediate, and long-term outcomes.

Short-term outcomes are those outcomes that will occur while clients are receiving your services, including things like knowledge gain or changes in attitude in the organizations that you work with. Achievement of short-term outcomes can generally be measured using Kirkpatrick’s second level of evaluation.

Intermediate outcomes are those that occur within the client organization itself, including changes in behavior or skill-gain that you expect to result from the training and technical assistance you provided. Achievement of intermediate outcomes is usually measured through tests for learning and observations of changes in behavior, Kirkpatrick’s second and third levels of evaluation.

Long-term or end outcomes refer to the resulting ability of a client organization to operate more efficiently and effectively by serving more people, or becoming more sustainable in accomplishing its larger purpose. Achievement of long-term outcomes can be measured through Kirkpatrick’s fourth level of evaluation.

Use the if/then exercise to identify outcomes.

You need Adobe Flash Player to view some content on this site.

Logic models document relationships.

While not all logic models look the same, they all serve the same purpose: to graphically capture the assumption and cause and effect relationships that drive your organization’s work on a project.

Download a sample logic model template and test your understanding of the different elements of a logic model using the activity on the right.

Elements of a Logic Model

You need Adobe Flash Player to view some content on this site.

Build from a foundation of data.

Experienced training and technical assistance providers know that in order to prove the effectiveness of their services, they must incorporate evaluation into all that they do and build off a foundation of data collection. Organizations may decide to collect this information through in-person or online surveys, or through site visits to client organizations.

Conducting regular surveys and needs assessments with your client population can help you to determine client demographics, experience, training and technical assistance needs, motivations, job satisfaction levels, and baseline performance.

While these surveys are incredibly helpful in providing insight into what sort of training and technical assistance opportunities would most benefit the client, these surveys also offer long-term value, providing points of comparison that your organization can reference throughout the evaluation process.

Site visits can also present training and technical assistance providers with important insight into how client organizations are performing and operating. Site visits can be an excellent source of qualitative information, most of which is not easily conveyed through surveys.