Everyone points this tool at the wrong problem
Search for situational judgement tests and you will find two kinds of page: vendors selling candidate screening, and practice sites selling candidates a way to beat the screening. The entire visible market treats the SJT as a gate.
Which is odd, because the properties that make an SJT useful at the gate are not actually its most distinctive ones. Plenty of instruments can sort people once. Very few can measure the same person twice, years apart, against an identical standard — and that is the thing an SJT can do that almost nothing else in the toolkit can.
What changes when the job changes
Selection and development ask different things of an instrument. It is worth being precise about which properties suddenly start mattering.
| Property | For selection | For development |
|---|---|---|
| Comparing whom? | Different people, one moment | The same person, two moments |
| Scoring consistency | Nice to have | Load-bearing — a drifting standard destroys the delta |
| Repeatability | Irrelevant — you assess once | Essential — the whole design depends on a second sitting |
| Cost per sitting | Matters at volume | Matters twice over |
| Practice effects | A cheating risk | A measurement-validity risk — different problem, different fix |
| Faking | The central threat | Weaker incentive — nobody is being hired or fired on the result |
Note the last row, because it is the one people get backwards. In hiring, faking is the central threat — candidates have every incentive to present the best version of themselves. In development, that incentive largely evaporates: nobody is being hired or fired on the strength of a baseline, and the participant usually wants an accurate read as much as the organisation does. The development context is, in this narrow sense, a cleaner measurement environment than the hiring context.
The practice-effect problem, stated properly
Here is the objection any competent psychometrician will raise, and it deserves a straight answer rather than a reassuring one: if you test someone twice, they may score higher the second time simply because they have been tested before. If that is what is happening, your beautiful pre/post delta is an artefact and your programme may have done nothing at all.
Three things reduce the effect for an SJT specifically.
- There is no answer key to memorise. The items are judgement dilemmas, not knowledge questions, and the scoring weights are never disclosed to the participant. There is very little to rehearse even if someone wanted to.
- The re-measure uses a parallel form. Same competency structure, same difficulty calibration, different scenarios. The participant is not seeing the test again; they are seeing its twin.
- The interval is long. Six to twelve months sits well beyond the window in which recall meaningfully inflates a score.
And here is the part most vendor pages leave out: reduced is not eliminated. A residual practice effect almost certainly remains, and if your evaluation has to survive genuine scrutiny — a board, a regulator, a grant condition — the only way to quantify it is a comparison cohort that takes both sittings and receives no development in between. Their delta is your noise floor. We will help you design that, and we would rather you knew about it from us than from the person in the meeting looking for a hole.
What an SJT does not do
It measures decisions in realistic dilemmas. It does not measure on-the-job outcomes, and the two are not the same thing — a leader can choose well in a scenario and still fail to act under real pressure, with real relationships and real fatigue in the way.
It also does not tell you how someone is experienced by their team. That is what 360-degree feedback is for, and an SJT is not a substitute for it. The honest position is that these instruments are complements: the 360 tells you how a leader lands, the SJT tells you what they do. Run both where budget allows, and use the SJT as the instrument of record for the specific claim “this capability moved by this much”, because it is the only one of the two that can make that claim cleanly.
For the underlying science — construct validity, scoring, and how our scenarios were built — see our SJT methodology page. For how this fits into a development programme, see the capability baseline.
Frequently asked questions
Can a situational judgement test be used for development rather than hiring?
Yes, and structurally it is better suited to development than most people realise. A selection SJT has to sort people; a development SJT has to track the same person over time. The second job requires two properties that SJTs happen to have and most alternatives do not: algorithmic scoring, so the standard does not drift between sittings, and low enough delivery cost to be run twice on a whole cohort. An assessment centre has the first property but not the second. A 360 has neither, because its rater panel changes between cycles.
What is the difference between an SJT and a personality test?
A personality test asks you to describe yourself; an SJT puts you in a situation and observes what you choose. That difference matters most in exactly the settings where the stakes are high: a motivated respondent can shift a self-report personality profile substantially, because they know which answer looks good and nothing stops them giving it. An SJT presents dilemmas where the "good" answer is not obvious, and the scoring weights are not disclosed. Personality instruments describe dispositions, which are relatively stable; SJTs capture judgement, which is exactly the thing development is supposed to change.
Do people just score higher the second time because they remember the test?
Practice effects are real in any repeated assessment and must be handled, not waved away. Three things reduce them for SJTs specifically. There is no answer key to memorise — scenarios are judgement dilemmas and the scoring weights are never revealed, so there is very little to rehearse. The re-measure uses a parallel form: same competency structure and difficulty calibration, different scenarios. And the interval between sittings is typically 6–12 months, well outside the window in which recall meaningfully inflates scores. Reduced is not eliminated. An evaluation that has to withstand real scrutiny should include a comparison cohort that receives no development, which is the only way to quantify the residual effect.
Are SJTs actually valid?
SJTs have a substantial validity literature behind them and are endorsed as an assessment method by bodies including the US Office of Personnel Management. They predict job performance better than unstructured interviews and better than self-report personality questionnaires, and they show smaller subgroup differences than cognitive ability tests. That said, validity is a property of a specific instrument for a specific purpose, not of a method in the abstract — a badly built SJT is a bad test. Ask any vendor, including us, what their instrument was validated against and on whom.
Should we use an SJT instead of 360-degree feedback?
Use both if you can, because they answer different questions. A 360 tells you how a leader is experienced by the people around them — which is genuinely important information and often the thing that motivates a leader to change at all. An SJT tells you what decisions that leader makes when facing a realistic dilemma. For the specific claim "this capability moved by this much over twelve months", the SJT is the stronger instrument, because the 360 delta is contaminated by rater turnover and rating drift. For the question "how is this person landing with their team", the 360 is irreplaceable.