The vanity metrics problem
Learning management systems are excellent at generating numbers: enrolment counts, completion rates, average scores, login frequency. The problem is that many of these numbers measure activity rather than learning, and optimising for the wrong metric produces the wrong behaviour.
High completion rates achieved by making courses trivially easy to complete tell you nothing about whether learners retained anything useful. High enrolment numbers with low engagement suggest a compliance checkbox culture, not genuine learning.
Here are the metrics that actually tell you whether your learning programmes are working.
Leading indicators of learning quality
Knowledge retention over time
A learner who scores 85% immediately after completing a module and 40% on the same assessment 30 days later has not learned effectively. Spaced repetition assessments — short tests at 7, 30, and 90 days post-completion — give you a far more accurate picture of actual knowledge acquisition than completion-time scores alone.
Learner-generated questions
When learners ask questions — in discussion forums, through support channels, in live sessions — the topics and frequency of those questions reveal where the content is failing to achieve understanding. Clustering learner questions and mapping them to specific content modules is one of the most useful feedback loops for content improvement.
Assessment attempt distribution
How many attempts do learners typically need to pass an assessment? A module where 70% of learners pass on the first attempt and 30% on the second or third is functioning well. A module where 40% of learners require 4+ attempts, or give up before passing, signals either poorly designed content or an unrealistic pass threshold.
Business-level impact metrics
Performance correlation
For skills-based training, can you correlate completion of specific learning programmes with on-the-job performance outcomes? Compliance training completion with incident rates? Sales training completion with conversion rates? This correlation analysis is the strongest evidence base for learning ROI.
Time-to-competency
For onboarding programmes, how long does it take a new hire to reach defined competency benchmarks? Tracking this over cohorts and programme iterations tells you whether your onboarding learning journey is improving.
Knowledge transfer to practice
Manager observation, performance reviews, and skill assessments in real work contexts validate whether learning completed in the LMS translates to changed behaviour on the job. This data is typically outside the LMS, but it is the only data that truly measures learning effectiveness.
Building a data culture around learning
The shift from activity metrics to learning quality metrics requires investment in both data infrastructure and analytical capability. Your LMS reporting may not surface these metrics natively — you may need to build a data pipeline that combines LMS data with HRIS data and business performance data.
The teams that do this well treat the learning function as a product that is continuously measured and improved, not a compliance obligation to be ticked off. The difference in learning outcomes is significant.