Solving the Right Problems

Three headlines from March 2026 capture much of what remains confused in education technology. A private school claims AI helps students learn twice as fast without teachers, citing adaptive assessment gains. A policy brief links the stagnation or decline of NAEP standards to state-level digital adoption. A foundation study reports that teachers using AI save 5.9 hours per week.

Each highlights one small aspect of EdTech while leaving unanswered the question of whether durable learning—simply put, knowledge and skills that are retained for the long term—has improved. None examines retrieval practice, feedback quality, metacognitive monitoring, social reasoning, or transfer—the processes through which durable learning is formed.

This is not a measurement problem. It is a problem-definition problem. Statistical hypothesis testing distinguishes three error types. Type I errors are false positives: concluding something works when short-term proxies improve, while the underlying outcome remains unchanged. Early gains from class size reduction in Kindergarten, for example, showed no lasting benefit over multiple years. Type II errors are false negatives: failing to detect a viable approach because execution was compromised. Poor implementation of remote learning during the pandemic led many to dismiss the medium itself, against almost half a century of evidence to the contrary. Type III errors are more fundamental: solving the wrong problem entirely.

The three headlines illustrate Type III error. The most persistent failures in education technology, and education more broadly, stem from this third category. This confusion arises in part because teaching and learning are not the same phenomenon. Teaching is visible. It can be observed, timed, standardized, and improved in ways that are legible to institutions. Learning is largely latent. It must be inferred from performance over time, across contexts, and under conditions that test whether understanding endures.

Instruction is necessary but insufficient. Learning depends on processes such as retrieval practice, explanation, feedback and error correction, metacognitive monitoring, social reasoning, and transfer. The tools work as designed, the implementations are competent, but the underlying problem was misframed from the start. Technology becomes more capable, claims become more assertive, yet learning outcomes remain largely unchanged because sophistication is being applied to the wrong problem.

As with all education technology, AI fails when it treats instruction as the problem to be automated rather than learning as the system to be improved. Bloom’s famous two-sigma problem illustrates the seduction: individual tutoring produces a two-standard-deviation effect, but it is economically unscalable. The implicit premise of nearly every AI tutoring product is that technology can finally deliver two sigma at scale. This is the wrong target. It automates the tutor—which is automating instruction.

The right problem is strengthening the conditions under which learning occurs: structured opportunities for explanation and elaboration, timely feedback and error correction, deliberate retrieval practice, metacognitive monitoring, and social reasoning.

Consider cooperative learning. The evidence base is strong and replicated. Hattie’s synthesis reports an effect size of 0.59 for cooperative versus competitive learning, well above the 0.40 threshold he identifies as a year’s typical growth. Practitioner acceptance is broad. Yet implementation failure persists not because the approach is ineffective, but because its core conditions (individual accountability, structured interaction, timely feedback, metacognitive reflection) are difficult to sustain when a single teacher must monitor and adjust eight simultaneous small groups in real time. This is the kind of problem AI should solve.

Framed correctly, AI could reduce the burden of implementing proven learning science at scale. One promising role for AI is not as a substitute teacher or an on-demand tutor dispensing explanations, but as a study partner: prompting explanation before answers are given, surfacing misconceptions, monitoring reasoning, and supporting reflection. In that role, the technology strengthens learning processes rather than automating instruction.

The contributors to this issue examine such issues from multiple vantage points: whether AI tutoring produces durable gains, how productive struggle survives optimization for ease, what distinguishes instruction that endures from instruction AI makes obsolete, and what it means to protect powerful thinking when machines can simulate it.

Steve Jobs called the computer a bicycle for the mind. In 2023, we extended the analogy, arguing that if the computer is a bicycle, AI is a rocket ship. But in education, propulsion is not enough. What matters is trajectory. A system can be immensely powerful and still be aimed at the wrong problem.