Choosing the right Situational Judgement Test: Video-based or Written?

Choosing the right Situational Judgement Test: Video-based or Written?

The Situational Judgment Test (SJT) has been in use since the 1920s but it is only in recent years that its popularity has significantly increased as a predictor of job performance. Advances in technology have meant SJTs are often presented in video/DVD format in contrast to the more traditional written form but what do we know about the advantages and disadvantages of these mediums? Does it matter which one we choose?

Filip Lievens and Paul Sackett from the US, help throw some light on the subject in the study they carried out comparing the two mediums in a selection setting. Lievens and Sackett developed a video-based SJT as part of a medical admission exam. The SJT was aimed to measure interpersonal and communication skills e.g. handling complaints of a patient or handling bad news. The scene froze at a certain point and candidate medical students had to answer a question related to the scene in multiple choice format. The results of the video-based SJT were compared with a written version of this test which contained only the verbal content of the video.

The results showed that:

The video-based SJT was better able to predict success in an interpersonal situation than the written SJT. This is likely to be due to videos/DVDs conveying important cues such as body language and voice tone. A written format is unable to provide such a rich portrayal of the situation.

The correlation between the SJT and cognitive ability was lower for the video-based format than the written format. This finding suggests that performance in written SJTs is more reliant on cognitive ability (as opposed to other job relevant abilities) than it is for video-based SJTs.

Applicants liked the different formats equally - face validity was high for both the video-based and written format.

These results suggest that, despite the additional cost, it makes sense to invest in video-based SJTs for measuring interpersonal skills. The video-based assessment provides a richer, more detailed portrayal of the realities of the interpersonal situations faced on the job. As a result the video-based SJT is a better predictor of success in interpersonal situations than the written SJT and provides significant value over and above cognitive ability tests.

How can Human Assets help?

At Human Assets, we are experts in the field of assessment. We design bespoke SJTs, as well as advising organisations on the best way of presenting these to ensure maximum validity. We also evaluate the validity of SJTs through trialling and by statistically comparing successful candidates' SJT score with their future job performance.
If you would like to find out more please contact our consultants on +44 (0)20 7434 2122 or at enquiries@humanassets.co.uk

Lievens, Filip & Sackett, P (2006). Video-based versus Written Situational Judgement Tests: A Comparison in Terms of Predictive Validity. Journal of Applied Psychology, 91(5), 1181 1188

Goldstein, H. W, Yukso, KP, Braverman, EP, Smith, DB, & Chung, B (1998). The role of cognitive ability in the subgroup differences and incremental validity of assessment centre exercises. Personnel Psychology 51, 357-374

Newsletter: October 2007