The report that gets filed away
Every L&D function has produced this document. It is well designed. It says 95% completion, 4.2 out of 5 satisfaction, 1,200 learning hours delivered. It is sent to leadership, acknowledged politely, and never referred to again.
The instinctive diagnosis is that the report needs better numbers, so the next version has more charts. But the problem was never presentation. Those metrics describe what happened to the programme — it ran, people showed up, they did not hate it. The reader wants to know what happened to the people. No amount of polish converts the first into the second.
Keep the vanity metrics. Just stop misusing them.
The usual advice is to abandon completion rates. That is wrong, and following it costs you a genuinely useful signal. A completion rate that drops to 40% is telling you something real and urgent about delivery. A satisfaction score that collapses is telling you the facilitator or the content has a problem. These are good operational metrics — early-warning systems for a programme that is breaking.
The error is one of job description. Operational metrics answer “should we fix the programme?” Evidence metrics answer “should we keep funding it?” Most L&D reporting fails because it brings the first kind of metric to the second kind of meeting.
It is also worth being clear about satisfaction specifically: high scores are close to meaningless as evidence. Participants reliably rate enjoyable programmes well and useful-but-uncomfortable ones poorly. Low scores are informative; high ones are not.
Why better learning analytics will not save you
This is the expensive detour, and a lot of budget disappears into it. The reasoning goes: our data is too coarse, so let us get finer data. More xAPI events. Engagement heat maps. Time-on-module. A learning-analytics platform that tracks every interaction.
At the end of that project you will know exactly which video someone watched, for how long, and at what time of day. You will still have no idea whether they now delegate better. Activity data does not become capability data at any resolution — the two are different categories of thing, and the gap between them is not measured in pixels.
The missing layer is not a higher-fidelity version of what you already collect. It is a measurement you are not currently taking at all.
What to measure instead
Capability, before and after
An objective reading of what people can actually do, taken before the programme and again 6–12 months later with the same instrument. This is the evidence layer. Everything else on this list supports it.
Dosage, honestly recorded
Who actually received what, and how much. Half of every cohort attends half of everything. A capability delta read against assumed exposure rather than real exposure will mislead you about what worked.
Behaviour, observed by others
Manager and peer input on whether anything changed on the job. Softer than capability data and vulnerable to rater drift — but it is the layer closest to the work, and it corroborates.
Business metrics, reported with a caveat
Retention, engagement, productivity. Include them, and state out loud that attribution is uncertain because a dozen other things moved in the same window. Owning that caveat is what makes the rest of the deck credible.
Where to start if the budget is already tight
Do not try to instrument the whole function. Pick the next cohort that has a real budget attached and a sceptical audience waiting, and measure that one properly: baseline before the first session, re-measure with the same instrument 6–12 months later, report the deltas including the ones that are flat.
One cohort with a clean pre/post comparison changes a budget conversation more than a year of dashboards covering everyone — because it is the first piece of L&D reporting in the room that answers the question that was actually asked.
The method in full, including which instruments hold up and which quietly do not: how to measure leadership development impact.
Frequently asked questions
What is wrong with completion rates as a training metric?
Nothing, as long as you use them for what they are: an operational health check on whether the programme actually ran. A 40% completion rate tells you something is broken in delivery, and that is worth knowing. The failure begins when completion is presented as evidence of impact. Counting participants is attendance, not measurement. It describes what happened to the programme, not what happened to the people in it — and the person asking about impact wants the second thing.
What should we measure instead of completion rates?
Capability, measured before and after the programme with the same instrument. That is the layer that sits between learning (did the content land?) and business results (did revenue move?), and it is the layer almost every L&D function is missing. Keep completion and satisfaction as operational metrics — they tell you whether delivery is healthy. Add a capability baseline as the evidence metric. The distinction matters: operational metrics tell you whether to fix the programme, evidence metrics tell you whether to keep funding it.
Won't better learning analytics solve this?
No, and this is the most expensive mistake in the category. More xAPI events, finer-grained engagement tracking and better LMS dashboards produce more detailed data about learning activity. But the gap is not a resolution problem, it is a category problem: no amount of activity data becomes capability data. You can know exactly which video someone watched, for how long, at what time — and still know nothing about whether they now delegate better. The missing layer is a different measurement, not a higher-resolution version of the one you have.
Are satisfaction scores worth collecting at all?
Yes — as an early-warning signal about delivery, not as evidence of effect. A satisfaction score that collapses tells you something is genuinely wrong with the facilitator, the content, or the room. A high satisfaction score tells you almost nothing, because participants routinely rate enjoyable programmes highly and unenjoyable useful ones poorly. Research on training reactions has long shown that satisfaction correlates weakly with actual learning transfer. Collect it, act on the lows, and never put it on the slide that answers "what changed".
How do we start measuring capability without a big budget?
Start with one cohort, not the whole function. Take an objective capability baseline before the next programme begins, run the programme as planned, and re-measure the same people 6–12 months later with the same instrument. One cohort with a clean pre/post comparison is worth more in a budget conversation than a year of dashboards covering everyone. It also costs a fraction of an assessment centre, which is the traditional way to get objective behavioural data and the reason most organisations never get it.