Assessment Centre Design:
Easier Said Than Done
Assessment centre designers used to assume that the different exercises were separate opportunities for candidates to exhibit relatively stable competencies. The expectation was that each competency would be marked relatively similarly across exercises. There was also the expectation that, within each exercises, people would do quite well on some competencies and poorly on others. Repeated studies have shown the opposite to be the case. Within each exercise each person has fairly similar scores on the competencies being measured. But, across exercises the same competency gets quite different scores in different exercises. This phenomenon known as the exercise effect is beginning to be better understood and its practical implications are being identified.
An article in the latest issue of the Journal of Applied Psychology summarises clearly the latest thinking on assessment centres. The exercise effect is best seen as a genuine reflection of both candidates' variation in performance across exercises and the relative sameness in their performance across dimensions in any particular exercise. For example, a person might do well at a presentation, appearing generally strong on all the competencies measured. On the other hand, they might do less well on a piece of analytical work, appearing generally weak on all the competencies.
One explanation for this variation in general performance between exercises might be that, in each exercise, a particular attribute like cognitive ability, self-confidence or experience is dominant. In the presentation, the self-confident person also appears quite clever and, indeed, more clever than the genius who becomes tongue-tied by embarrassment. Performance in an assessment centre can be seen as made up of exercise-specific performance and some performance that is stable across exercises and driven by broad general traits. The practical implication of this is that, if they are to work properly, assessment centres must be comprehensive samples of the role that the candidate will be doing. With so much variation across exercises, it is absolutely vital that the exercises reflect fully the role.
We also know that quite subtle variations in a situation or exercise can provoke quite profound differences in response. For example, a lawyer who appears to be reasonably assertive in dealing with people of the same profession might appear overawed when presenting ideas to a client.
The practical implication of this is that, if they are to work properly, assessment centres must be accurate samples of the role that the candidate will be doing. If you measure performance in off-the-shelf exercises there is no reason to suppose that it will bear any close relationship to performance in the actual role the person will be doing. Nor does it provide a realistic preview to candidates. Without such a preview, successful candidates are quite likely to find the role fails to match their expectations. They can become de-motivated and leave, resulting in all the costs of having to go through the recruitment process again.
How can Human Assets help?
Assessment centre design is a skilled task that should be based on a thorough understanding of the background research and literature and a track-record of specialist experience. We regularly see assessment centres that might have some superficial credibility but that in reality are probably next to useless. At worst, the exercises do not try to simulate the role but consist of generic management development tasks like deciding who will be thrown out of a lifeboat! Others use off-the-shelf exercises, performance in which is likely to be very different to performance in the role. Still others use exercises that simulate the role but fail to cover its full complexity. Sometimes the restricted coverage is because a full range of exercises is thought to be too daunting.
Dr Charles Woodruffe is author of the CIPD's book on assessment centres (for details, click here), and we are acknowledged experts in the design of assessment centres. We bring clients our experience to ensure that you have assessment centres that incorporate the latest thinking and research, incorporating diversity considerations throughout. A Human Assets- designed assessment centre will offer the best possible prediction of performance and ensure that talent management starts with the accurate identification of talented people.
If you have any doubts about whether your assessment centre matches up to today's highest standards, please contact our consultants for a free exploratory conversation on +44(0)20 7434 2122 or by email enquiries@humanassets.co.uk
Reference
Charles E Lance et al. Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology, April 2004, Vol 89 No 2, pps 377-385.