What would a great tutor do?

For the first time in human history, there is an intelligence at our fingertips that knows almost everything. At Aristotle, we’re working to transform that breakthrough into a world-class tutor, available to every student.

Turning raw intelligence into great teaching, however, is its own frontier. Every major model release brings stronger reasoning and higher exam scores, but knowing an answer is very different from helping a learner discover it for themselves.

Today, we’re sharing how we evaluate AI’s ability to teach. We tested Aristotle against the tutoring modes of leading frontier models, measuring both pedagogy and speed across hundreds of pivotal moments from real Aristotle student sessions.

Why assistants make bad tutors

From our earliest days at Aristotle, we ran into a stubborn problem: LLM assistants are consistently bad teachers, even when given strongly worded, detailed pedagogical prompts.

Where an excellent tutor keeps responses focused, LLMs are long-winded, overloading students with information. When an LLM provides an explanation, it assumes the student absorbed every word. Most critically, LLMs don’t push learners to work independently, often solving the problem themselves before the learning can even start.

We believe that these failures are a direct byproduct of model post-training. Frontier models are heavily optimized for coding and agentic tasks, explicitly trained to answer questions, implement fixes, and solve problems from start to finish. Good tutoring requires the exact opposite: leaving a problem open so the student can work through it themselves.

How we measure effective, problem-focused pedagogy

Building an evaluation for real-time teaching starts with understanding how a human tutor works. Great tutoring is both a live, multimodal challenge and a cognitive modeling problem. Rather than just offering paragraph-length explanations, a human tutor coordinates precise spoken guidance with impromptu sketches and simpler sub-problems, all adapted to the student’s own coursework. Meanwhile, the tutor maintains a running mental model of the student’s understanding across phases like setting goals, diagnosing gaps, and delivering lessons.

All of these phases build to the core tutoring loop: guided problem-solving. Here, the student attempts steps, hits roadblocks, and corrects mistakes in real time in order to create mastery. Yet this is the exact moment where assistant-trained LLMs break down, taking over the problem instead of letting the student struggle through it.

Drawing on Ai2’s TutorMoments benchmark (Zhang et al., 2026) and our own session transcripts, we found that guided problem-solving hinges on four recurring inflection points, which we call pivotal turns:

1. First attempt — The student hasn’t tried the problem yet
   A great tutor prompts the student to take the first step. LLMs routinely set up the equations or write out the solution before the student shares any thinking.
2. Correct step — The student executes a step or names what they’ll do next
   A great tutor validates briefly and lets the student keep momentum. LLMs often say "Great job!" but then tell the student what to do next rather than allowing them to continue independently.
3. Wrong answer — The student makes an incorrect step or calculation
   A great tutor isolates the misconception so the student can self-correct. LLMs tend to both write a corrected version of the student’s work and prematurely reveal the next step.
4. Stuck — The student says they don’t know or asks for a walkthrough
   A great tutor offers the smallest possible nudge to get the student moving again. LLMs simply hand over the answer.

To measure how often tutors get these moments right, we sampled 500 live Aristotle sessions, isolated 791 pivotal turns across them, and aligned LLM judges to human expert labels for each of the four turn types. Each judge is tasked with answering: Given a pivotal turn, did the tutor provide the right amount of support?

Within any category, the line between appropriate support and over-scaffolding can be subtle. Our judges detect and penalize tutoring that offers unsolicited shortcuts, overly leading questions, or extended explanations without student input. The judges have access to the full conversation context window, including code representations of all whiteboard content.

Evaluating pedagogical quality: Aristotle vs. frontier baselines

Using these calibrated judges, we evaluated Aristotle against frontier labs’ primary tutoring products: GPT-6.1 Sol running OpenAI’s Study Mode prompt and Gemini 3.8 Flash running Google’s Guided Learning prompt. We also ran Claude Sonnet 5.5 on the same Guided Learning prompt. We ran all candidates on 791 turns, passing in full conversation context with code representations of the whiteboard.

Across all pivotal turns, Aristotle achieves a pass rate of 54%, compared to 34% for Study Mode, 33% for Guided Learning, and 35% for Claude Sonnet 5.5 on the Guided Learning prompt.

Breaking the results down by turn type reveals a clear split in what makes tutoring difficult:

• Holding back when the student has momentum (First attempt & Correct step): Aristotle’s lead is widest when the student is capable of working independently. A system prompt alone struggles to override general assistant post-training, so prompt-only tutors routinely jump in to set up equations or execute the next step for the student.
• Scaffolding when the student falters (Wrong answer & Stuck): Pass rates fall for every system when the student makes an error, and stay low when the student is stuck, since the tutor has to pinpoint the misconception and offer a minimal hint without giving away the answer.

Even with a 19-point lead over frontier tutoring modes, 54% is just the starting point. With aligned judges in place, we now have the signal to systematically hill-climb on the harder moments where AI tutors still fall short.

Achieving frontier results with minimal latency

Strong pedagogy only works in practice if the tutor can deliver it at conversational speed. In a live back-and-forth dialogue, waiting eight or nine seconds for a response after finishing a step breaks a student’s momentum and derails engagement.

Minimizing latency is especially challenging in a full tutoring environment, where the system must coordinate across a shared whiteboard, student uploads, persistent memory, and the live conversation stream. We engineered Aristotle’s multi-agent harness to manage all of that state in real time without stalling the conversation.

To see how our architecture compares against single-model baselines, we compared pedagogical pass rates and median time to first word across a wide range of fast and reasoning-heavy frontier models. Pass rates were evaluated on all 791 pivotal turns; latency was measured on the same 51-turn sample from that set for every model.

Scaling up to larger, slower models adds seconds of latency without solving the core pedagogical problem. Our multi-agent harness sidesteps that bottleneck, delivering a leading 54% pass rate with a median time to first word of 1.6 seconds, more than twice as fast as lightweight models like Gemini 3.8 Flash. Claude Sonnet models reply about as quickly (1.0–1.7 seconds) but pass only 31–36% of turns.

Looking ahead

A great tutor has to know when to hold back, when to push the student forward, and when to change strategy, all while responding at conversational speed. Measuring those decisions during guided problem-solving is only the first step.

We are now expanding our evaluations across the full arc of a tutoring session, capturing how a tutor introduces a concept from scratch, sequences problems to build intuition, and sketches on the whiteboard to make ideas click.

Alongside harness engineering, these evaluations unlock our next research direction: training models directly for pedagogy. By pairing outcome-grounded judges from our product with simulated teaching environments, we see a path to teaching models how to tutor at scale.