Traditional AI usually means systems whose behavior comes from rules and structures that people write down, while modern AI usually means machine learning, where a model derives its behavior from examples. The difference is easiest to see in a task from education research called knowledge tracing, where a rule-structured model and a neural network were tested on the same student data, and the neural network scored higher on predicting answers.
Key takeaways
- Traditional AI encodes expert knowledge by hand; machine learning extracts patterns from data, and the two are often combined.
- Bayesian knowledge tracing tracks each skill as learned or not learned, and building its skill maps takes expert effort.
- In a 2015 study, a recurrent neural network reached an AUC of 0.86 on one benchmark, against 0.69 for the best earlier result.
- A 2016 follow-up found that an improved Bayesian model performed about as well as the neural network.
- Neural models need large datasets, which small classrooms rarely produce.
What separates the two approaches
A proposal dated August 31, 1955, by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon for a Dartmouth summer research project on artificial intelligence is a common starting point for the field. Much of the work that followed was symbolic: researchers wrote facts and rules, and programs reasoned over them. Every behavior could be traced to a line someone had authored, which made such systems easy to inspect and hard to extend, because each new situation needed a new rule.
Machine learning reverses the workflow. A developer supplies examples and a model structure, and training adjusts internal numbers until predictions fit the examples. The result can handle cases nobody listed in advance, but the reason for a given output is spread across thousands or millions of parameters. These are tendencies, not clean categories. Many deployed products mix a learned component with hand-written rules, for example a model that proposes an answer and a rule that blocks it when a policy applies.
Knowledge tracing: one problem, two eras
Knowledge tracing asks a narrow question: given a student's record of right and wrong answers so far, how likely is the next answer to be right? Tutoring software uses the estimate to decide whether to give another exercise on the same skill or move on.
The classic answer is Bayesian knowledge tracing (BKT), described by Albert Corbett and John Anderson in the journal User Modeling and User-Adapted Interaction in 1995. As summarized by Piech and colleagues, BKT treats a learner's state as a set of binary variables, one per concept, each meaning understood or not understood. A hidden Markov model updates the odds after every answer. The original formulation assumed that a skill, once learned, is never forgotten, and later extensions added allowances for guessing and slipping.
The weak point is the map from exercises to concepts. The same authors describe the standard method for building it, cognitive task analysis, as an arduous, iterative process in which domain experts ask learners to talk through their thinking. That is traditional AI in a plain form: the intelligence sits in an expert-built structure.
What the 2015 comparison measured
In 2015 a Stanford and Khan Academy team published Deep Knowledge Tracing, which swapped the hand-built structure for a long short-term memory (LSTM) network, a type of recurrent neural network. The model needs no expert annotation of concepts and can accept any student input that can be turned into a vector. The table reproduces the reported results using AUC, a score where 0.5 is chance and 1.0 is perfect ranking of right above wrong answers.
| Dataset | Students | Marginal baseline | Standard BKT | Deep knowledge tracing |
|---|---|---|---|---|
| Khan Academy, eighth-grade math | 47,495 | 0.63 | 0.68 | 0.85 |
| ASSISTments, grade-school math | 15,931 | 0.62 | 0.67 | 0.86 |
On the ASSISTments data the best earlier result in the literature, with extensions to BKT, was 0.69, which the authors described as a 25 percent gain for their model. The Khan Academy sample covered 1.4 million completed exercises across 69 exercise types and was anonymized and used under an agreement meant to protect student privacy.
The follow-up that narrowed the gap
A paper by Mohammad Khajah, Robert Lindsey, and Michael Mozer, first submitted on March 14, 2016, asked how deep knowledge tracing really is. The authors listed several regularities that standard BKT ignores, including recency effects, the order of trials, similarity between skills, and differences in individual ability. When BKT was extended to model them using existing techniques, it performed about as well as the neural network. They concluded that the neural model's edge did not come from discovering new representations, and that simpler, more interpretable models may be enough for this task.
Piech and colleagues had raised a limit of their own: recurrent networks need large amounts of training data, which suits an online platform but not a small classroom. The practical reading is that neither family wins everywhere.
Questions for judging an educational AI claim
- Is the student model built from expert rules, learned from data, or both?
- How many students produced the training data, and do they resemble the classroom where the tool will run?
- Was it compared against a well-tuned simple model, or only against a basic baseline?
- Can a teacher see why the system believes a student has mastered a skill?
- What student data is collected, and under what agreement?
FAQ
Is traditional AI obsolete?
No. Rule-based components remain common wherever behavior must be predictable, explainable, or auditable, and many systems pair them with learned models.
Is machine learning the same as modern AI?
In everyday use the terms overlap heavily. Machine learning is the method behind most current systems, but AI also covers rule-based and search-based approaches.
Did deep knowledge tracing prove neural networks are better for tutoring?
It showed higher prediction scores on two large datasets. The 2016 follow-up showed that a better-specified Bayesian model closed most of that gap, so the evidence supports a narrower claim.
Why does data size matter for schools?
Neural models learn from examples, and a single class produces few of them. Platforms with tens of thousands of students can train such models, while an individual teacher usually cannot.
Related reading
- How AI and machine learning differ
- AI's place in modern society
- Alternatives to neural networks
- Pattern recognition in machine learning
More in AI in education.
Last verified: October 2026
Sources
- Piech C. et al., Deep Knowledge Tracing, arXiv:1506.05908 and NIPS 2015
- Khajah M., Lindsey R., Mozer M., How deep is knowledge tracing?, arXiv:1604.02416, 2016
- Corbett A., Anderson J., Knowledge tracing: Modeling the acquisition of procedural knowledge, User Modeling and User-Adapted Interaction 4(4), 1995
- McCarthy J. et al., A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955, reprinted in AI Magazine 27(4), 2006