Skip to main content
← Journal

When AI hands over the answer, practice improves but learning doesn't

Research shows when AI supports learning, and when it simply does the work for the student.

Research & DataJune 18, 2026By Taimi8 min readUpdated September 28, 2026

AI can improve performance on a task without the student learning anything. The difference lies in whether it hands over the answer or protects the learner's own thinking.

A 13-year-old takes a photo of a maths problem and sends it to a chatbot. A tidy solution appears in seconds. The homework is done, but did the student learn how to solve the next problem without help?

That is the core question about AI and learning. It is no longer just whether young people use AI. In a Pew Research Center survey in the US, 64 percent of 13- to 17-year-olds said they use AI chatbots, and schoolwork was the most common use. The figures don't describe Finland, but they show how quickly chatbots have become part of where young people learn.

Summary

  • AI can improve performance on a task while independent skill declines.
  • The decisive difference is the teaching design: a good tutor gives hints, asks for reasoning and tests understanding.
  • The best results came from carefully designed tutors in which AI was only one part.
  • AI literacy includes evaluating outputs, spotting errors and knowing what not to outsource, not just using the tool.
  • A rule that works at home is simple: try first, ask for one hint, solve, explain, and test without AI.

Performance is not the same as learning

On a screen, the two look alike. A correct answer, a finished essay or a quickly completed problem set looks like success. Learning only shows later: can the student solve a new problem, explain their reasoning and notice when an answer can't be right?

Bastani and colleagues' field experiment with nearly a thousand Turkish high-school students shows the difference unusually clearly. Students practised maths without AI, with a standard GPT-4 assistant, or with a pedagogically constrained GPT Tutor.

The answer machine improved practice but weakened independent performance

During practice, students with the standard GPT assistant performed 48 percent better than the control group. Students with GPT Tutor performed 127 percent better.

When AI was removed for the exam, the standard GPT group did 17 percent worse than the control group. The tutor group's disadvantage disappeared, but its independent exam result was not significantly better than the control group's.

In the chat logs, standard GPT users more often asked for and copied complete solutions. The system built as a tutor more often steered students to try first and work step by step.

ResearchBastani et al., 2025
AI improved practice but not the exam Difference from the control group that practised without AI in Bastani and colleagues' experiment. Students with standard GPT-4 did 48 percent better in practice but 17 percent worse on the exam without AI. Students with GPT Tutor did 127 percent better in practice, and on the exam they did not differ significantly from the control group. STANDARD GPT-4 Practice with AI +48% Exam without AI −17% GPT TUTOR Practice with AI +127% Exam without AI no significant difference −50 0 +50 +100 +150 DIFFERENCE FROM CONTROL, % 0 = the control group without AI. Source: Bastani et al. 2025, PNAS.
Bars show the difference from the control group that practised without AI. AI was available during practice but not on the exam. The GPT Tutor group's exam result did not differ significantly from the control group, so no bar is drawn for it. Source: Bastani et al. 2025.

The study doesn't mean AI inevitably harms learning. It shows that a good result during practice can hide a weak learning outcome. If the only measure is whether the task got done, an answer machine looks excellent.

A parent might therefore ask less often "did you finish?" and more often "how did you get there?". The second question quickly reveals who did the thinking.

A good AI tutor is built from teaching, not chat

The idea of a personal tutor is not new. Learning is supported by timely feedback, the right level of difficulty, working at one's own pace and the chance to try again. The promise of generative AI is that conversational support could be there at the moment a question arises.

In Kestin and colleagues' randomised trial at Harvard, 194 university students studied physics either with a research-based AI tutor or in an active in-class lesson.

A carefully built AI tutor supported learning

Students using the AI tutor learned more on the study's measures and spent less time doing it. The AI group's median time was 49 minutes, while the active lesson was scheduled for 60.

The result should not be generalised directly to younger children. The trial involved university students, and the tutor was built around expert-written content, active learning, step-by-step support and feedback.

The successful intervention was, in other words, a complete designed learning experience in which AI was only one part.

ResearchKestin et al., 2025

A large meta-analysis of 68 studies found on average a positive effect of generative AI on learning. The variation between studies was very large, which makes the general question "does AI help people learn?" too blunt.

A better question is what the learner does with the AI. Ordering a complete answer and having a Socratic conversation can use the same technology, yet for learning they are close to opposites.

Two 2026 Khanmigo experiments cautiously support that reading. In two NBER working papers (first, second), the benefits of an AI tutor were small on average, and many students used it only sporadically. In the second, an experiment with more than 6,000 middle-school students, AI support helped especially with the next attempt after a mistake. The papers have not yet been peer-reviewed, so they add to the picture rather than settle it.

OpenAI and Anthropic have started describing their study modes in the same terms: they should ask questions, give graduated support and help the user find the solution. These are the companies' own product descriptions, not independent evidence of learning outcomes. They do show that the answer-machine problem is being recognised in product design too.

A five-step routine for using AI

A good AI tutor protects the learner's own work. That can be turned into a five-step routine for home.

  1. Try first. The student writes down their first idea, calculation or question before opening the chatbot. The attempt doesn't have to be right.
  2. Ask for one hint. The AI may open up the next step, but not give the whole solution.
  3. Solve it yourself. The student uses the hint and does the actual work.
  4. Explain it in your own words. Why does the solution work? Which concept or rule did it use?
  5. Test without AI. A new similar problem, a short recap or explaining aloud shows what stuck.

The routine can be given to a chatbot directly:

Don't give me the answer straight away. First ask how I would start solving this myself. Give me one hint at a time if I need it. At the end, ask me to explain the solution in my own words and give me one short check question.

A prompt doesn't guarantee good tutoring. The model can still slip into a full solution, make a mistake or reinforce a wrong starting assumption. The student's own attempt, explanation and final test matter more than any single prompt.

AI literacy is mostly about judgement

AI literacy easily shrinks to writing prompts. A good prompt helps, but it says nothing about whether the student can judge how reliable an answer is, or decide what is worth doing themselves.

Park's systematic review organises AI literacy in schools into five parts: understanding AI, using and applying it, evaluating its outputs, grasping the ethical questions, and a realistic view of the technology's possibilities and limits.

In everyday terms, a young person should be able to ask:

  • What is this answer based on?
  • Could this contain outdated information or an invented source?
  • What is the AI assuming about me or the task?
  • Am I allowed to use AI for this under the school's rules?
  • What information should I not put into the service?
  • Could I explain or defend this result without AI?

In the Finnish National Agency for Education's recommendations, AI does not remove the need to learn reading, arithmetic and each subject's core concepts, and learners need to understand AI's faulty inferences and biases. UNICEF links children's AI skills to safety, privacy, fairness and the child's best interests. Capability and protection are not opposites.

Parents don't need to be AI experts

In Pew's survey, 64 percent of US teens said they use chatbots, but only 51 percent of parents thought their own teen did. About four in ten parents had not talked with their teen about chatbot use.

Closing that gap doesn't require learning every model's settings. A parent's most useful role is to agree on limits, ask about the process, and help spot the moments when AI should be closed.

Checklist

  • Agree on what AI may be used for. The school's and teacher's rules decide whether it's allowed for brainstorming, practice, editing, or not at all.
  • Protect personal data. Sensitive information about the young person, friends or family doesn't go into a chatbot, and identifiable material isn't uploaded without a clear reason and permission.
  • Keep the first attempt with the student. If AI comes in before the student's own idea, it easily becomes the starter of the work rather than a support for thinking.
  • Ask to see the process. A finished answer doesn't show what the student did. The conversation, a draft and an explanation in their own words say more.
  • Check important claims. Numbers, sources, historical facts and anything about health or safety get verified from a reliable original source.
  • Keep an adult responsible for the limits. A chatbot doesn't know the young person's whole situation and doesn't replace a teacher, parent or other trusted adult.

When to close the AI

Sometimes it is worth limiting AI precisely so that the student learns to use it better later.

Close it when the point of the exercise is to recall something without help. Close it when the student can no longer explain what the answer means. Close it in assessments where aids aren't allowed. And close it if the conversation turns to sensitive personal data, health worries, or a situation that needs a real person.

AI can make a kind of personal tutoring far more accessible than before. It will ask again, rephrase, and offer practice exactly when a student gets stuck.

The benefit only shows up next time, though: can the student then do more on their own?

That is easy to check at home. Close the app and ask the student to solve a similar problem. What happens then tells you more than any answer that appeared on the screen.

Sources

  1. Kestin, G. et al. (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, 17458. Open source
  2. Bastani, H. et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. Open source
  3. Han, X., Peng, H. & Liu, M. (2025). The impact of GenAI on learning outcomes: A systematic review and meta-analysis of experimental studies. Educational Research Review, 48, 100714. Open source
  4. Park, J. (2025). A systematic literature review of generative artificial intelligence (GenAI) literacy in schools. Computers and Education: Artificial Intelligence, 9, 100487. Open source
  5. Oreopoulos, P. & Low, N. (2026). One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment. NBER Working Paper 35620. Open source
  6. Oreopoulos, P., Liut, M., Sungu, A. & Low, N. (2026). Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment. NBER Working Paper 35621. Open source
  7. Pew Research Center (2026). How Teens Use and View AI. Open source
  8. Pew Research Center (2026). What Parents Say About Their Teens’ AI Use. Open source
  9. Finnish National Agency for Education & Ministry of Education and Culture (2025). Artificial intelligence in education: legislation and recommendations. Open source
  10. UNICEF Innocenti (2025). Guidance on AI and Children, Version 3.0. Open source
  11. OpenAI (2026). Introducing ChatGPT for Teens. Open source
  12. Anthropic. Claude for Education. Open source

Topics

Continue this learning path

Skills learned through daily life

How learning and money skills grow through small, repeated actions.

  1. When AI hands over the answer, practice improves but learning doesn't

    You are here · 8 min read

A considered note, when there is one to send.

Join Taimi's newsletter for new Journal pieces and progress from the work behind the app. No publishing schedule promises, no noise.