Psychology
The AI That Aced 160 Psychology Tests — By Not Reading Them
You know the feeling where you've just finished marking a stack of perfectly correct exam papers, and then — in some thought experiment that psychologists love — you imagine asking the student who aced them to simply pick option A on the next question, no matter what. And the student, who is apparently the most capable student you've ever taught, ignores you completely and picks the answer that would score highest on a normal exam. Would you still say they'd understood anything? That's roughly — and in surprisingly precise, published form — what happened to Centaur, the AI model that Nature declared capable of simulating human cognition across 160 behavioural tasks. The declaration was premature, which is quite a lot to get wrong in a flagship journal.
- Centaur, fine-tuned on 160 psychology experiments, matched human behavioral patterns with striking accuracy — enough for a Nature paper claiming it "simulates human cognition."
- Zhejiang University researchers found Centaur ignores explicit instructions about which answer to choose, instead defaulting to its training distribution. The model couldn't read the question; it retrieved the most probable answer.
- Kahneman's System 1 (fast, automatic, pattern-matching) and System 2 (slow, deliberate, instruction-following) give the sharpest frame: Centaur is all System 1, at massive scale — and our benchmarks were never testing for System 2.
- The deeper problem isn't the AI. It's that our standard metrics for "cognitive performance" measure System 1 behavior, and we've been calling that intelligence.
- The mirror effect: humans run on System 1 far more than we admit. Centaur's failure just made that visible at a scale we couldn't ignore.
Nature Said "It Simulates Human Cognition" — 160 Tasks, Near-Perfect Scores, Worldwide Headlines
In July 2025, a model called Centaur arrived in the pages of Nature with a bold claim attached.
The researchers behind Centaur — led by Marcel Binz and colleagues — had done something genuinely unusual. Instead of fine-tuning a large language model on internet text or task accuracy, they'd trained it specifically on behavioural data from 160 cognitive psychology experiments. The range was real: decision-making under uncertainty, executive control, memory retrieval, risk assessment, learning from feedback. Decades of careful human-subjects research, all the established patterns that distinguish systematic thinkers from impulsive ones, everything that maps onto the processes cognitive psychologists have been documenting for generations.
After fine-tuning on that corpus, Centaur's outputs matched human behavioural patterns with a precision that genuinely impressed people who are hard to impress. It didn't just get the right answers — it got things wrong in the same ways humans do. Same response-time distributions, same biases, same characteristic swerves away from pure rationality that researchers have been cataloguing since the seventies. The paper's language was careful: this model, the authors argued, could serve as a "foundation model of human cognition" — a computational stand-in for how real minds actually process the world.
The headlines were not careful. "AI Thinks Like a Human," several outlets reported. For anyone who had spent years watching the slow, incremental grind of cognitive modelling, a model that could replicate human behaviour across 160 varied tasks felt like a qualitative leap. The claim moved fast. It turned out to have travelled much further than the evidence warranted.
Then Researchers Said: "Fine. Now Choose Option A." — And It Didn't.
The test that exposed everything was almost insultingly simple.
Researchers from Zhejiang University, writing in National Science Open in 2026, decided to probe Centaur's claimed understanding with a very straightforward intervention. In the original psychology experiments, participants receive task instructions and then respond accordingly. What if you changed the instructions explicitly? What if you told the model, in plain language, that it must select a specific response — say, option A — regardless of what the task appears to call for?
A human participant, given that instruction, would comply. They might find it strange (I would, honestly, and I'd probably ask why), but they'd follow it. Following an explicit instruction that overrides your default behaviour is one of the clearest markers of what Kahneman calls System 2 operation: deliberate, effortful, instruction-sensitive processing that can override automatic responses. Four- and five-year-olds can do it. It's not a sophisticated cognitive achievement. It's basic instruction comprehension.
Centaur could not do it. When told to choose option A, the model continued selecting whichever response the training distribution indicated was most probable. It wasn't reading the instruction and deciding to ignore it — it was, in the most literal sense, not processing the instruction at all. ScienceDaily's summary captured the finding in a headline that landed hard: "This AI knew the answers but didn't understand the questions."
The Zhejiang team's interpretation was unambiguous. Centaur's remarkable match to human behavioural data was the product of overfitting — the model hadn't learned anything about cognitive processes; it had learned the statistical regularities of a particular training corpus. When the input deviated from those regularities in a way that required actual instruction processing, the whole façade collapsed. "Can Centaur truly simulate human cognition?" the paper asked in its title. The answer it provided: no, not in any meaningful sense.
When explicitly told "you must choose option A," continued selecting whichever response was statistically most probable given its training distribution. Ignored the instruction entirely.
Read the instruction. Recognise that the instruction overrides default behaviour. Select option A. This is routine for any human participant — or any system that actually processes what it's told.
The System 1 / System 2 Divide: Getting the Answer Right Is Not the Same as Understanding the Question
Kahneman's two-system framework was built precisely to make this distinction legible.
In Thinking, Fast and Slow, Daniel Kahneman describes two modes of cognitive operation that are almost like two different characters sharing a single brain. System 1 is the automatic one — it runs constantly, effortlessly, below conscious awareness. It recognises faces, reads emotional expressions, retrieves answers to familiar questions, matches new inputs to stored patterns. System 1 is fast because it never really processes anything fresh; it's a sophisticated retrieval machine that looks at what's in front of it and returns the most statistically likely response from everything it has encountered before. It's very good at this. Most of the time, that's exactly what you need.
System 2 is the effortful, deliberate one. It's slow, expensive, and prone to wandering off. It kicks in when System 1 gets stuck — when the automatic answer feels off, when the task demands explicit rule-following, when instructions need to be held in working memory and applied step by step. The clearest test for System 2, in Kahneman's framework, is exactly the kind of instruction-following that the Zhejiang team used: given an explicit rule that overrides your default, can you apply it? System 2 is the part of you that can be told "this time, do the opposite" and actually do the opposite. System 1 can't. It only does what it always does.
Centaur is a System 1 machine of extraordinary scale. Trained on 160 experiments' worth of human behavioural data, its System 1 is exquisitely calibrated to human behavioural patterns. But it has no System 2 at all — no mechanism for holding an explicit instruction in working memory and applying it to override the probability distribution it's learnt. When Centaur "matched human cognition" across 160 tasks, it was doing something more limited: it was matching the statistical distribution of human responses in those tasks. That's a real and impressive thing to do. It is not the same thing as simulating cognition.
The distinction matters because human cognition routinely requires both. When you take a multiple-choice test, you're using System 1 (rapid pattern-matching to retrieve relevant knowledge) and System 2 (checking your answers, noticing when something feels right for the wrong reasons, following any special instructions about format or timing). Centaur only ever uses the first half.
The Intelligence Metrics We Use Were Built to Measure System 1
This is the structural problem hiding beneath the Centaur debate: our benchmarks were never designed to test understanding.
Think about what most AI cognitive benchmarks actually measure. A model is given a question and evaluated on whether its answer matches the correct answer (or, in the case of psychology tasks, the typical human answer). The evaluation is outcome-based: right output or wrong output. There's no mechanism in the benchmark to distinguish between "this model understood the question and reasoned its way to the correct answer" and "this model retrieved the statistically most likely answer for this type of input."
This isn't an accident of laziness. It reflects a genuine epistemological difficulty: understanding is internal, invisible, and notoriously hard to measure. Outcomes are observable. And for most practical purposes, outcome-based measurement works perfectly well — if a model consistently produces correct outputs, the internal mechanism generating them is a secondary concern. That's why Centaur's performance across 160 psychology tasks looked like such a strong signal. By every measurable criterion in the benchmark, it was indistinguishable from human performance.
But outcome-based metrics are System 1 metrics, which is a real limitation when what you're trying to test is cognition. They measure the end-product of pattern retrieval. They don't test whether the system is sensitive to instructions, rules, or context changes that would require flexible processing. The Zhejiang test was, in effect, inserting a System 2 probe into an evaluation framework built entirely around System 1 outputs — and discovering that what had looked like a full cognitive system was, behind the façade, a very sophisticated autocomplete.
| Benchmark type | What it measures | System tested | Centaur passes? |
|---|---|---|---|
| Behavioral task accuracy (160 psychology experiments) | Match to human response distribution | System 1 | Yes — impressively |
| Response pattern similarity (bias, error distributions) | Statistical shape of answers | System 1 | Yes |
| Explicit instruction override ("choose A regardless") | Instruction comprehension + rule application | System 2 | No — fails entirely |
| Novel rule following (new rule not in training) | Flexible reasoning from stated principles | System 2 | Untested / likely fails |
The implication is uncomfortable for the field at large. If our best cognitive benchmarks are System 1 tests — and they largely are — then an AI that excels at System 1 will ace them regardless of whether it has anything resembling genuine cognitive function. For years we've been measuring the right thing in the wrong way, or the wrong thing entirely. Centaur didn't cause this problem. It just made it undeniable.
The Mirror Effect: What Centaur Exposes About Us, Not Just About AI
Here is the part that requires a certain honesty: Centaur's dominant mode is System 1. So is ours.
Kahneman's most important and most uncomfortable finding is that System 1 runs the show for the overwhelming majority of human mental life. We don't deliberate carefully before most judgements. We retrieve, match, and respond — and then, if pressed, we construct a post-hoc rationale that makes it look as if we'd been reasoning all along. Kahneman calls this "what you see is all there is" (WYSIATI): System 1 forms impressions, makes judgements, and acts on thin evidence without noticing what it doesn't know. System 2 is supposed to check System 1's work, but it's lazy, easily satisfied, and frequently absent. (I find this both very relatable and mildly horrifying, and I spent twelve years teaching this material.)
This means that when Centaur matched human behavioural patterns on 160 psychology tasks, it was matching primarily the System 1 outputs of human participants — outputs that the participants themselves wouldn't necessarily endorse under careful reflection. The cognitive biases Centaur replicated so faithfully — the availability heuristic, the framing effect, loss aversion, anchoring — are exactly the biases that human System 2, when properly engaged, can override. Centaur can't override them because it has no System 2. Humans can override them — but usually don't, because System 2 is expensive and the pressure to take the easy route is constant.
What Centaur's failure reveals, then, is not simply that AI lacks understanding. It reveals that our gold standard for "cognitive performance" — the behavioural data from human psychology experiments — is mostly a record of System 1 outputs. We weren't training Centaur to replicate human cognition in full. We were training it to replicate human cognitive shortcuts. And it succeeded. So when it failed the instruction-following test, it wasn't failing to be human; it was failing to be the deliberately reasoning part of a human — the part we forgot to include in our benchmarks.
What This Means for You: Next Time AI "Understands," Ask If It Can Change Its Mind
A simple diagnostic, derived from Zhejiang's method, that anyone can run.
The instruction-override test isn't a technical curiosity. It's a practical probe you can apply any time you're working with AI and wondering whether the system actually understands what you're asking, or is pattern-matching to a distribution of similar past inputs. The test is simple: after getting an answer, tell the model to change its approach in a way that conflicts with what would typically be the "correct" response. Tell it to argue the opposite position, or to treat its previous conclusion as wrong and find a counter-argument, or to choose the less probable option. If the system can do this coherently — without just slightly rephrasing the same answer in different clothes — it's demonstrating something beyond pure retrieval. If it can't, if the instruction seems to evaporate and the same answer comes back with different words, you're working with a System 1 machine, however impressive its System 1 may be.
This matters in practice because System 1 machines fail in specific, predictable ways. They fail at novel situations that require updating explicit rules. They fail when context changes partway through and the instructions need re-reading. They fail when you ask for advice on something that sits outside their training distribution — because they'll return a confident-sounding answer that is actually the closest match they could find, not the right answer to your specific situation. Knowing you're working with a System 1 tool changes how you use it, which changes nothing if you're not paying attention.
But here is where it turns back towards us, which is the part Kahneman was actually writing about. The moments when System 2 most needs to intervene are exactly the moments when System 1 is most confident — when the answer feels obvious, when the situation feels familiar, when thinking feels effortless. Those are the moments to slow down, check the actual instruction, and ask: am I following the logic here, or am I retrieving the most probable response from everything I've ever seen that looked a bit like this? Centaur can't ask itself that question. The fact that you can — and occasionally do — is the difference that matters.
- When AI produces a confident answer, try the instruction-override test: ask it to take the opposite position or choose differently. Does the instruction actually land?
- Treat fluent, confident AI output as a System 1 retrieval, not as deliberate reasoning — adjust your verification accordingly.
- For your own thinking: notice when cognitive ease (the "this feels obviously right" sensation) is highest — that's when System 2 is most likely standing down.
- Ask, about any important decision: am I following an argument to its conclusion, or matching the situation to a pattern I recognise?
- The question Centaur couldn't answer: "What would you say if the answer you just gave were wrong?" Try asking yourself that, too.
Sources: Binz et al., "Centaur: A Foundation Model of Human Cognition," Nature, July 2025. Zhejiang University researchers, "Can Centaur truly simulate human cognition? The fundamental limitation of instruction understanding," National Science Open, 2026. ScienceDaily (2026-04-29): "This AI knew the answers but didn't understand the questions." Framework: Daniel Kahneman, Thinking, Fast and Slow (System 1 / System 2 distinction, cognitive ease, WYSIATI). This article is a popular-science interpretation applying Kahneman's framework to a reported research controversy; it is a conceptual argument, not a primary research finding. Not professional psychological or AI advice. Centaur's specific test results as described are per published accounts; consult original papers for methodological detail.