Information processing accounts for situational judgment test responding

Summary

From political leaders to CEOs to front-line employees, there is widespread recognition that the capacity to make effective decisions and accurate judgments is a sign of competence and expertise. For example, consider the following scenario and potential responses one might take:

You and someone from another department are jointly responsible for coordinating a project involving both departments. The other person is not carrying out his share of the responsibilities. How would you deal with this situation?

1. Discuss the situation with your manager and ask him to take it up with the other person’s manager.
2. Remind him that you need his help and that the project won’t be completed effectively without a full team effort from both of you.
3. Try to find out why he is not doing his share and explain to him that this creates more work for you and makes it harder to finish the project.
4. Get someone else from his department to help with the project.

Which of these actions do you believe would be most effective? Least effective? Why? How did your reach that conclusion? How would you interpret a person’s responses to this question?

The scenario and response options shown above is an example of an item that might be found on a situational judgement test (SJT). SJTs most commonly present respondents with a series of scenarios that reflect possible circumstances, tasks, or decisions that might be experienced in a particular context. Each scenario is accompanied by a set of possible actions that the respondent is asked to evaluate and choose among. Because of their practical and predictive value, SJTs are commonly used in many personnel assessment contexts including selection and training evaluation. Nevertheless, there are many fundamental questions related to the use and design of SJTs. Perhaps most significantly, there is still uncertainty regarding how to interpret the meaning of “performance” on an SJT. Similar uncertainties exist regarding how the construction or presentation of an SJT item might influence respondents’ choices and subsequent interpretation of scores.

The main purpose and motivation of this research is to better understand the judgment and decision-making processes underlying SJT responding. Through studying the psychological mechanisms of SJT responding, I hope to provide insight and potential recommendations that can be used to inform the scoring, interpretation, and design of SJTs. To this end, I have proposed a formal theory and computational model of what I believe represents the cognitive processes used by respondents to answer SJT items known as SiRJ (Situated Reasoning and Judgment). Simulation studies conducted with the model reveal considerable correspondence with existing empirical evidence. I have also gathered additional data using eye tracking that provides further support for several of the model’s predictions.