Almost a thousand teenagers, a maths course, two versions of AI. The study by Hamsa Bastani and colleagues, published in the journal PNAS in 2025, is one of the clearest findings we have on AI and learning.
Better in practice, worse in the exam
Some of the teenagers practised with GPT-4 in an interface like ChatGPT. During practice, they did much better than the comparison group without AI. Then came the exam, without AI. Those who had practised freely with AI scored 17 percent lower than the group that never had AI.
A third group practised with an AI tutor. It was based on the same model but had instructions: hints instead of answers. For this group, the disadvantage almost completely disappeared.
The authors describe the AI as a crutch. If you lean on it while practising, you walk worse without it. That does not mean AI harms learning. It means that what it helps with matters.
The question behind the traffic light
Hence the learning traffic light. It does not ask whether AI is allowed. It asks: what is supposed to be learned here?
The learning traffic light
What is supposed to be learned here?
- Red, replaces: the AI does exactly what you are supposed to learn.
- Amber, accompanies: the AI gives hints, questions and feedback. You take the next step.
- Green, relieves: the AI takes over what is not the learning goal.
Thinking model, not a rule from the study
Red means the AI writes the essay when writing is the goal. It solves the problem when the point is to practise calculating. The result looks good. The learning is missing.
Amber is the tutor from the study. The AI gives hints, asks questions back and gives feedback on your draft. The thinking is yours.
Green means relief with everything that is not the learning goal. The AI creates practice questions, explains a technical term in other words or tidies up your notes.
The same activity, two colours
The colour does not depend on the tool or the activity, but on the learning goal. Having a text summarised is green when you only need the content. It is red when summarising is exactly what you are supposed to learn.
For teachers and course leaders, this means: name the learning goal of a task first. Then the colour becomes visible.
What the study does not show
The study covers one subject, maths. The traffic light is my thinking model based on it, not a rule from the researchers.
There are further indications in the same direction, but they are weaker. In 2025, Michael Gerlich found in a survey of 666 people that frequent AI use goes along with weaker critical thinking. That is a correlation, not proof of a cause. A group at MIT led by Nataliya Kosmyna measured less connectivity in the brain when writing with ChatGPT. That was an experiment with 54 people, published as a preprint and not yet peer-reviewed.
AI may take work off your hands. But not the thinking you want to learn.
Take your next task and give it a colour.
Frequently asked questions
Does this apply outside maths?
The study by Bastani and colleagues only looked at maths. The learning traffic light is a thinking model, not a rule from the study. But the question of what is supposed to be learned applies to any learning task.
What did the AI tutor in the study do differently?
Both AI versions were based on GPT-4. The tutor had instructions meant to protect learning: it gave hints instead of finished answers. With it, the disadvantage in the exam without AI almost completely disappeared.
Sources (4)
- Bastani et al.: Generative AI without guardrails can harm learning: Evidence from high school mathematics, PNAS 122 (26) (2025)
- Bastani et al.: Abstract of the study, PubMed 40560616
- Gerlich: AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking, Societies 15 (1) (2025)
- Kosmyna et al.: Your Brain on ChatGPT, arXiv preprint (2025)
