Knowledge Tracing: How AI Models What a Learner Knows

Every teacher knows the gap between a mark and a mind. A learner can score 7 out of 10 on Monday and fail the same idea on Friday — or get a question right by guessing and wrong by a careless slip. The number on the page is a snapshot of performance. What we really want is an estimate of knowledge: a quiet, continuously updated model of what each learner has actually mastered. That estimate is what knowledge tracing provides, and it is one of the oldest and most validated ideas in educational data science.

What is knowledge tracing?

Knowledge tracing is the practice of estimating a learner's hidden mastery of a skill from the answers they give over time. The idea was formalised by Albert Corbett and John Anderson in 1995, working on intelligent tutoring systems at Carnegie Mellon.¹ Their insight was simple but powerful: mastery is a latent state we cannot observe directly, but every answer is a noisy clue about it. By treating each response as evidence, a model can maintain a running probability that a learner has learned a given skill — and update that belief with every new question. Instead of a single score, the learner gets a living estimate that rises with genuine understanding and resists being fooled by a lucky guess. Where a gradebook records what already happened, knowledge tracing maintains a current best guess about what the learner can do next — the difference between a photograph and a live feed, and the foundation every adaptive learning system is built on.

Why isn't a grade enough?

A grade compresses a rich learning history into one number, and in doing so it throws away exactly the information a teacher needs. It cannot tell a slip from a misconception, or a confident answer from a guessed one. It says nothing about which foundation is missing beneath a wrong answer, so two learners with the same mark may need completely different help. Knowledge tracing keeps that structure. By modelling skills separately and updating them response by response, it distinguishes "hasn't learned this yet" from "knew it, had a bad day" — and points to the specific prerequisite to revisit rather than the whole topic. Two learners can both score 6 out of 10 and still be in entirely different places: one has a shaky grasp of every step, the other has mastered most of the topic but stumbles on a single missing idea. A single mark hides that difference; a per-skill estimate makes it the starting point for teaching.

How does Bayesian Knowledge Tracing work?

The classic model — Bayesian Knowledge Tracing, or BKT — treats each skill as a hidden on/off state and tracks the probability it is "on" using four interpretable parameters: the chance the learner already knew it, the chance they learn it at each step, the chance they slip (know it but answer wrong), and the chance they guess (don't know it but answer right).¹ After every response, the model applies Bayes' rule: a correct answer raises the mastery estimate, a wrong one lowers it, and the slip and guess terms stop the model from over-reacting to a single data point. The learner is considered to have mastered the skill once the probability crosses a high threshold, often around 0.95. The beauty of BKT is that every number means something a teacher can read.

What did deep knowledge tracing change?

In 2015, a Stanford team led by Chris Piech showed that recurrent neural networks could trace knowledge across many skills at once, capturing how practising one idea shifts mastery of related ones — something classic BKT, with its one-skill-at-a-time view, could not.² Deep Knowledge Tracing was more accurate on large datasets, and it kicked off a decade of neural variants. But it came with a cost that matters enormously in a classroom: a neural network can output a confident probability without being able to say why. A model might predict that a learner will miss the next question without pointing to the gap responsible, which leaves a teacher with a number but no next step. The field has been negotiating that accuracy-versus-interpretability trade-off ever since, and it is the central design tension behind every adaptive system that aims to do more than rank learners — it has to explain them too.

Where does the data come from?

Modern knowledge tracing is trained on diagnostic questions — multiple-choice items whose wrong options are designed so that each one reveals a specific misconception. The largest public benchmark is the NeurIPS 2020 Education Challenge, built on more than 20 million real answers from the Eedi platform, which drew nearly 400 teams competing to predict learner responses.³ That dataset matters because it links two ideas: it is not enough to know that a learner is struggling — the pattern of which wrong answers they choose tells you what they misunderstand. A well-designed distractor turns a wrong answer into a signal: the option a learner picks names the misconception they hold. Good knowledge tracing and good question design are therefore two halves of the same diagnostic loop — the model is only as informative as the questions feeding it, which is why the quality of the item bank matters as much as the sophistication of the algorithm.

A probability a teacher can't explain isn't a diagnosis — it's just a better-dressed grade.

Why does interpretability matter for teachers?

This is where the research meets the staffroom. An adaptive system can be highly accurate and still be useless to a teacher if its reasoning is locked in a black box. A teacher cannot act on "the model is 0.31 confident"; they can act on "this learner is missing equivalent fractions, which is why the ratio questions are collapsing." At CogniTrace we treat interpretability as a design requirement, not an afterthought: the system is built so that every estimate comes with the missing foundation behind it and the next step to rebuild it. The goal isn't to predict failure — it's to make the reason for a struggle visible early enough to do something about it. An interpretable estimate also earns trust: a teacher who can see why the model flagged a learner can overrule it when it is wrong, which is exactly what keeps a human in charge of the classroom.