you may as well just throw darts at a dart board and hope you get lucky.

Posted by  Steven Jarrett, Ph.D.

If you’re just getting started with employee assessments, you might have heard the term validation being used when discussing selection systems.  In the context of organizations and their use of assessments in the hiring process we are often referring to criterion-related validation. Criterion-related validation is the empirical relationship between a predictor and a criterion. Now, let’s repeat that in English. Validation will show an organization how well their selection system is able to predict future job performance. In essence, the stronger the relationship between the assessment and performance the more likely your selection system will predict successful employees. So now that we know what validation evidence is, let’s discuss some common questions around validation studies:

1) As an organization, how can we perform a validation study?

There is a long answer and a short answer, the short answer is you collect scores on the predictor (e.g., how people scored on the selection test or interview) and how they are performing on the job (e.g., how many widgets made in a day, how the supervisors rates their performance) and you determine if the two scores are correlated with one another.

2) All of my current employees took the selection test and I just finished performance evaluations, so I have all the data I need, right?

Well yes and no, it is important that when you do a validation study that the performance ratings are separate from any administrative or developmental agendas within the organization. The best practice would be to use the data from the selection test, but actually have supervisors rate each individual’s performance for the purpose of the validation study alone. That way, the ratings will be less biased. For example, an employee may be showing average performance, but the supervisor really feels he/she deserves a raise because it has been 3 years since his/her last one. When completing performance evaluations they may rate the individual higher to ensure that he/she gets a raise, but if the ratings are for research purposes only the supervisor will be less likely to skew the ratings.

3) How about I give my top 3 performers the assessment and if they score high I know it will work?

Will you? Will you know if the assessment works? Let’s say for a minute that we took this strategy and the top 3 performers all received a perfect score. This would clearly mean that the assessment was doing its job, right?  Wrong. Because if I then gave that same assessment to the bottom 3 performers and they also got a perfect score, it would mean the assessment was doing nothing. The assessment would have no ability to differentiate between bottom and top performers.

When doing a validation it is important to get a WIDE variety of performance levels because as important as it is for high performers to score well, it is equally important that low performers score poorly. If this is not the case then the assessment does not allow you to discriminate between high and low performers, which is the whole point of using the assessment.

4) Makes sense, so I will give it 3 top performers AND 3 bottom performers?

Well using my previous logic I just told you, this plan sounds perfect. However, the other thing to consider is to ensure that the assessment puts people in the right category (high or low) over the course of hundreds and even thousands of candidates.  For example, if you ask me to guess heads or tails 3 times and I guess it right each time, would you believe that I can guess it 100% of the time? No, you would just call me lucky. The real test would be to make me do it 100, 500, or 1,000 times and the odds are that I would be right closer to 50% of the time. This is what statisticians call having the appropriate power to detect differences. Thus, we recommend that validations studies are conducted with a minimum of 100 people (of different performance levels) and it should be noted that the more people you have the more confident you can be in the results.

To recap: So what is validation? It is the best way to determine if the way you are hiring individuals into your organization works or not. Without it, you may as well just throw darts at a dart board and hope you get lucky.