Validity and Reliability

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player

Validity and reliability are two critical concepts in implementing effective outcome measurement systems. These concepts can be illustrated through the example of a bathroom scale. Let’s say Jennifer weighs herself every day and finds that the scale always reads 150 pounds. Assuming she is not losing or gaining weight, and that she actually does weigh 150 pounds, then the scale is valid and reliable. It’s valid because it displays an accurate reading, and it’s reliable because it displays this reading each time it’s used. The scale could be reliable without being valid. For example, if Jennifer’s weight is displayed each day as 135 pounds, then the scale is not providing a valid reading because the number is not accurate. It doesn’t measure Jennifer’s true weight. If, however, the scale always reads 135 pounds, and she is not gaining or losing weight, then it is still reliable because it reflects a consistent reading.

Validity is the accuracy of the information generated.

The primary advantage of surveys is their cost in relation to the amount of data you can collect. Surveying generally is considered efficient because you can include large numbers of people at a relatively low cost. There are two key disadvantages: First, if the survey is conducted by mail, response rates can be very low, jeopardizing the validity of the data collected. There are mechanisms to increase response rates, but they will add to the cost of the survey. We will discuss tips for boosting response rates later in this lesson. Written surveys also don’t allow respondents to clarify a confusing question. Thorough survey pre-testing can reduce the likelihood that problems will arise.

Reliability refers to consistency.

Reliability can also be thought of as the extent to which data are reproducible. Do items or questions on a survey, for example, repeatedly produce the same response regardless of when the survey is administered or whether the respondents are men or women? Bias in the data collection instrument is a primary threat to reliability and can be reduced by repeated testing and revision of the instrument.

You cannot have a valid instrument if it is not reliable. However, you can have a reliable instrument that is not valid. Think of shooting arrows at a target. Reliability is getting the arrows to land in about the same place each time you shoot. You can do this without hitting the bull’s-eye. Validity is getting the arrow to land on the bull’s-eye. Lots of arrows landing in the bull’s-eye means you have both reliability and validity.

Click to open interactivity Test your knowledge of validity and reliability.

Test your knowledge of validity and reliability.

You need Adobe Flash Player to view some content on this site.

Click to install Adobe Flash Player