Strategies for Quality Assurance and Boosting Response Rates

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player

As data is collected and entered into a storage mechanism, checking for errors and data quality is an important step that is easily overlooked. Build in time to review data and follow up about discrepancies in your overall timeline; the more data you collect, the more time you will need to assure its quality. Additionally, low response rates can threaten an outcome measurement effort. Follow-up surveys with organizations that no longer actively participate in a capacity building program are especially vulnerable to this problem.

Use these strategies

  1. Double entry.  This entails setting up a system to collect data twice and then compare for discrepancies.  This can be costly and time-consuming, but is the most thorough method for quality control.
  2. Spot checking.  This entails reviewing a random sample of data and comparing it to the source document for discrepancies or other anomalies.  If discrepancies are found, the first step is to identify any patterns (data entered in a particular time period or by a specific staff person; data associated with a particular beneficiary organization; a specific type of data that is incorrect across many records—for example, if all data for additional persons served at an organization was formatted as a percentage instead of as a whole number).   The capacity builder may need to review all the data entered, especially if there is no discernible pattern to the errors.
  3. Sort data to find missing, high, or low values.  If you are using a database or spreadsheet function, identifying outliers (those pieces of data at either extreme) is very easy, whether through the use of formulas or sorting functions.
  4. Use automation, such as drop-down menus.  Automating data collection provides a uniform way to report information and makes sorting and analyzing data much easier.  For example, organizations reporting the number of additional persons served will all use the same language to report the outcome, whereas without such automation the language could vary significantly from report to report.  Additionally, more sophisticated forms can pre-populate performance goals from an existing database, which reduces data entry errors made by those filling out the forms.
  5. Format a database to accept only numbers.  Whether organizations are filling out forms directly or your staff is entering data from a handwritten form, formatting your data fields to accept only numbers reduces errors related to typos.
  6. Review data for anomalies.  This strategy requires that a staff person who is familiar with the organization’s capacity building interventions and who has a good eye for detail reviews the data collected and identifies anomalies.  Some of these anomalies may not appear with general sorting.
  7. Discuss data discrepancies with the organization.  If, after implementing any of these quality assurance mechanisms, discrepancies remain unexplained, take the data back to the organization for discussion and clarification.
Click to open interactivity Organize data using Microsoft Excel.

Organize data using Microsoft Excel.

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player

Boosting your response rates can help ensure sufficient and timely data.

It’s important to have response rates that are as high as possible considering your circumstances.  If response rates are low, you may be excluding valuable opinions, feedback, and responses that can help you shape future training and technical assistance programs.  You may also get an inaccurate picture of how your current program is proceeding.  The following strategies can help you increase your response rates.

  1. Tie data collection to project milestones.  Throughout the course of the capacity building relationship, it is relatively simple to require organizations to report desired data.  For example, an evaluation could be due as a requirement for moving on to the next phase of the project, such as releasing funds for a capacity building project or approving a consultant to begin work.   However, once the organization exits the capacity building program, the capacity builder loses this leverage.
  2. Conduct an exit interview once the engagement is complete.  Participation in this interview can be mandated in a memorandum of understanding.  An exit interview is close enough to the intervention that the organization may still be invested in maintaining its relationship with the capacity builder and follow through on the commitment.  However, the organization may not have realized all its possible outcomes and therefore, the data may not capture some of the ripple effects where outcomes are realized after the data has been collected.
  3. Stay in touch.  By holding monthly meetings or conference calls with organizations after they exit the program, the capacity builder can maintain more informal connections and provide reminders.  The organizations have access to advice and support and may be more likely to participate in a follow-up data collection effort.  Establishing a community of practice among organizations so they have even more reason to be in touch with each other (and you) is one way to implement this strategy.
  4. Provide the outcome data to the organization.  Offer organizations a short, summary report card of the data you collect from them and demonstrate how it can be used as a marketing tool.  This summary can be invaluable to a program and may increase the number of responses you get to your data surveys.  If you can use the merging functions available in software like Microsoft Word and Outlook, generating report cards for tens or even hundreds of organizations may take just a few hours.
  5. Offer multiple collection methods.  Be available to complete the survey on the phone with the organization.  Be available to go to the organization’s headquarters and do it in person.  Be prepared to offer language translation if necessary, offer the survey electronically, or mail the survey with a stamped envelope.  The easier it is for an individual to complete the survey, the more response rates will increase. 
  6. Be culturally competent.  Capacity builders may take great steps to ensure that training and technical assistance is culturally appropriate and should extend to data collection efforts.  Moreover, if you are engaging a third party to collect data—a consultant or a team of interns, for example—remember that being a third party means they have not had the benefit of getting to know an organization and its staff through the course of the capacity building engagement.  Language barriers, cultural differences, and individual preferences can influence whether you are likely to get a response.
  7. Introduce your external data collectors.  If you are working with those third parties, introduce them to the organizations you are working with early on.  Maintaining a relationship helps improve response rates, but the lack of a relationship will hurt response rates.  As a caveat, be sure to maintain confidentiality about the results, especially if a third party is collecting direct feedback about your services.