Evaluation

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player

You have fielded a request, analyzed an organization’s capacity, and improved the performance of a beneficiary organization by delivering high quality technical assistance. The final phase of technical assistance is evaluating the services you provided. Evaluating TA services is important for two reasons. It proves your effectiveness and impact as a TA provider, and it helps you improve your skills and services. Evaluations are typically conducted following a TA engagement, but it is not uncommon to conduct evaluations periodically throughout a long-term engagement. This chapter will cover the purpose of evaluations, as well as ways to conduct a thorough evaluation of your TA services.

Evaluating TA services proves your effectiveness and impact, and improves your skills and services.

Evaluation can be done in many ways. You can do a verbal “check-in” onsite immediately following the TA session, it can be done through a web-based survey a few days later, or you can request that the beneficiary organization complete an evaluation form or interview periodically throughout a long-term engagement. See the interactivity for information about low-cost web-based survey platforms, if you choose to use that method. Sample data to collect includes reaction to the TA provider, reaction to the delivery method, knowledge gained, behaviors or practices changed, next steps, areas for improvement, and follow-up TA needed.

Click to open interactivity A number of low cost online tools provide easy interfaces for building surveys and viewing reports online.

A number of low cost online tools provide easy interfaces for building surveys and viewing reports online.

Check out this article for a more detailed comparison of these online survey tools.

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player

Data should be documented and analyzed, and used to make adjustments to the overall technical assistance plan if necessary.

There are unique challenges to measuring a long-term TA engagement. The goal of TA is to improve efficiency or management practices which in turn allows for expanded or enhanced direct services. But there is not always a clear relationship between TA provided and, for example, additional children served in an after-school program. Below is list of sample indicators of TA success and when you might be able to see or document those indicators.
Short Term Results (1 year or less) - Short term results can be realized at or near the point of execution of technical assistance. Short term results are often simply the outputs of a planned TA activity, such as a strategic plan or an installed financial system. Take care to consider only the indicators that describe a result of the TA activity, rather than the completion of the capacity building activity.
Examples:

  • Beneficiary organization has begun keeping minutes and attendance of board meetings (as a result of board development).
  • Beneficiary organization has developed systems to help manage the organization's finances more effectively (as result of technical assistance in accounting).
  • Beneficiary organization implements a new outreach strategy (as a result of technical assistance in outreach).

Intermediate Term Results (1 to 2 years) - Intermediate term results indicate success in the goal of improving sustainability of the beneficiary organization. The beneficiary organization could begin to show long term results such as expanded social services, but those results will be established with additional time.
Examples:

  • Beneficiary organization increases the number of volunteer hours contributed by all unpaid staff/volunteers.
  • Beneficiary organization produces information on program outcomes such as number of clients served.
  • Beneficiary organization diversifies their revenue sources.
  • Beneficiary organization increases the amount of funding attracted from grants or contracts from Federal, state, or local sources (amount of funds from grants or contracts).
  • Beneficiary organization increases the number of clients served.
  • Beneficiary organization expands services to include a new group of service recipients or geographic area.
  • Beneficiary organization increases the number of community stakeholders (citizens, organization and government representatives) participating in meetings to coordinate youth activities.

Long Term Results (Over 2 years) - Long term results indicate success in the achievement of the ultimate goal of improving and/or expanding services to clients. Measurement of long term results is helpful as it allows the TA provider and beneficiary organization to “keep their eyes on the prize.” The indicators of these results need to be measured throughout the engagement, so you want to a) select measures for which you can find data and b) get a "baseline" measure or benchmark that will be used to measure future changes.
Long term results will be specific to the program services offered by the beneficiary organization. The examples below are for a youth-serving organization.
Examples:

  • Increased youth participation in programs that provide for positive youth development. Indicators: # of target youth engaged in programs for            months out of the year, average number of contact hours.
  • Reduced youth violence. Indicator: # and % of target youth involved in violent crime.
  • Reduced gang involvement. Indicator: # and % of target youth involved with gangs.
  • Reduced gun violation arrests and confiscations.
  • Improved emotional well-being. Indicator: # and % of target youth who showed improved self esteem, personal power, and/or decreased alienation.
  • Increase social competencies and problem solving skills. Indicator: # and % of target youth with conflict resolution skills.