Welcome to part 5 of our guide to recruitment, the last instalment looking a recruitment from an academic perspective by Dexter Davies Smith. If you are just joining us, we have so far covered:
Part 1 – The True Cost of Recruitment
Part 2 – What to consider when evaluating a selection method (I would recommend reading this to fully understand this instalment)
Part 3 – Traditional Recruitment Methods
Part 4 – Psychometric Approaches to Recruitment
In this instalment we look at two interactive approaches to recruitment. First we have Situational Judgment Tests, which are growing in popularity, and secondly Assessment Centres which, depending on the specific provider and course, will use a combination of the selection methods we have discussed and more.
Situational Judgment Tests
Situational Judgment Tests (SJT) are a relatively new selection method. They can be custom designed and allows companies to put prospective candidates into a hypothetical situation, giving them several options to choose from. They can also ask candidates to predict what they think will happen as a result of their choices. This takes into account cognitive ability, personality & any competencies that the company design it to.
Face Validity: Due to the obvious link between the assessment and the role candidates are being hired for, SJT have naturally high face validity.
Predictive Validity: To go along with the high face validity, SJT also have very good predictive validity. Early research indicates that SJT can sometimes reach similar predictive power as cognitive ability tests when well designed.
Adverse Impact: There is little research into the adverse impact of SJT tests and there is little reason to expect adverse impact from SJT. The mechanisms underlying SJT are likely to be general mental ability, personality & work knowledge. This might suggest a bias towards those with more previous experience which isn’t always a good indicator of who would be the best employee long term.
Practical Review: SJT again score well for practical concerns, they can be administered online which automatically scores and ranks the candidates or this may be an exercise you want the candidates to complete verbally with you (although beware of human judgment coming in, the scoring should be based purely on their answers to the SJT). The interactive nature also means it is a more enjoyable candidate experience which is a growing priority for major corporations whose candidates may also be customers.
Overall there is clear reason why SJT are becoming so popular and will no doubt continue to do so.
Assessment Centres are a big business these days, with companies willing to invest the big bucks to ensure they get the best candidates. The definition of what qualifies as an “assessment centre” is surprisingly loose so the label assessment centre will actually cover a wide range of approaches. In general, candidates are sent on a 1 – 2 day course where they complete a series of assessments and these assessments are used to identify which candidate should be hired.
Face Validity: 2 days-worth of assessment with the professionals to identify top talent, how much higher can face validity get?
Predictive Validity: Assessment centres use a varying range of selection methods, several of which we have discussed and seen that they provide predictive validities of above 30%. Surely using a matrix of these tools will provide a higher overall predictive validity?
Not according to the literature. Whilst some researchers claim to get predictive validity of above 50%, that would appear to be a gold-standard approach that few are achieving. 3 Meta-analysis of Assessment Centre’s predictive validity found they predict between 8 to 14% of job performance. That is in the same range as a single un-structured interview.
Adverse Impact: There is little evidence that I am aware of on the impact of assessment centres but as they vary so much this may be hard to say. It would also depend on the methods used for that centre and the amount that human judgment, with all its biases, plays in the final decision.
Practical Review: This result would surprise most and disappoint many. There is research going on as to why assessment centres appear to not deliver the high standards we would intuitively expect. A key concept that has emerged is that whilst assessment centres are complex psychometric processes, using multiple methods of selection, they are also a political, social and administrative event with lots of room for human influence to detract from the selection methods involved.
The clearest evidence of this can be seen in the different methods of combining the multi assessment scores for each candidate. If a purely mechanical approach is used adding the scores together in a statistical manner, the predictive validity of assessment centres sharply rises, however less that 5% of centres use this. The vast majority use a “wash up”, this involves all of those involved in the selection process discussing the candidates and their performance to decide which candidate should be hired. This allows all the biases we have covered to creep back in.
We now draw to a close to this guide to recruitment. It is by no means comprehensive, but hope it has fulfilled a few main aims, which were:
- To show that recruitment is critical for the survival and prosperity of an organisation because, at the end of the day, if you take away the people, there is no company.
- To generate thought around why we use the recruitment methods we do; the traditional approach is so well ingrained in many western organisations that most appear to use them because that is simply all they know.
- To provide a glimpse into what academic research has to say about different methods of recruitment and how these results can fuel more accurate recruitment going forward.
There really is no one-size-fits-all approach to recruitment and anyone who claims to sell a magic formula that will make it flawless may be a stronger marketer than recruiter. However there are tools that move recruitment from the just better than pot-luck lottery of human judgment into the reliable realms of scientific identification.
As you depart from this blog series, if I could leave you one piece of advice it would be do two things; firstly, stop and reflect on the recruitment methods you use and question why you are using them; secondly, identify the top performers in your organisation and maybe put them through these recruitment methods, see if there is a commonality in their results – this could prove a great starting point for what you want to identify in future recruitment processes.