ARC-AGI
Ranked 3 of 10 AI benchmarks by recommenders
Puzzles designed to be easy for humans and resistant to training-to-the-test, used to compare AI reasoning. Critics say it predicts little and labs game it.
Puzzles designed to be easy for humans and resistant to training-to-the-test, used to compare AI reasoning. Critics say it predicts little and labs game it.
46 people gave an opinion
- recommend it
- 22
- criticize it
- 24
- approval share
- 48%
Recommended in 8 of 13 months
- November 2025: recommended
- December 2025: recommended
- January 2026: not recommended
- February 2026: recommended
- March 2026: recommended
- April 2026: recommended
- May 2026: recommended
- June 2026: not recommended
- July 2026: recommended
- August 2026: not recommended
- September 2026: recommended
- October 2026: not recommended
- November 2026: not recommended (month in progress)
Why it doesn’t carry the badge
- 22 people recommend itNeeds 10 or more, met.
- 0.9 recommenders for every criticNeeds 2 to 1 or better, not met.
- Recommended in 8 of 13 monthsNeeds 4 or more, met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Good, clever, better designed benchmark
“This is a good and clever benchmark and a worthy successor to the previous two.”
Comment from 25 Mar 2026 in “ARC-AGI-3” “ARC-AGI has thought a fair deal about this and has a good definition here”
Comment from 30 Jul 2026 in “Some thoughts about Anthropic's new cryptanalysis results” “I tend to prefer the ARC-AGI benchmarks for the most part.”
Comment from 12 Apr 2026 in “How We Broke Top AI Agent Benchmarks: And What Comes Next” - Comment from 18 Jul 2026 in “GPT-5.6 used a prompt to close a 30-year gap in convex optimization”
Reasoning-heavy puzzles people enjoy
“I tried out their human simulation, and it feels very reasoning heavy.”
Comment from 26 Apr 2026 in “Why SWE-bench Verified no longer measures frontier coding capabilities” “it feels very reasoning heavy”
Comment from 26 Apr 2026 in “Why SWE-bench Verified no longer measures frontier coding capabilities” “pretty fun to run as a human”
Comment from 25 Jul 2026 in “Claude Opus 5” - Comment from 26 Mar 2026 in “ARC-AGI-3”
Resistant to training-to-the-test
“ARC-AGI was designed specifically for evaluating deeper reasoning in LLMs, including being resistant to LLMs 'training to the test'.”
Comment from 11 Dec 2025 in “GPT-5.2” “ARC-AGI was specifically designed to be resistant to pre-training”
Comment from 12 Feb 2026 in “Gemini 3 Deep Think” Tests tasks humans find easy
“the tests are designed to stress frontier models in tasks that humans are uniquely good at”
Comment from 5 Sep 2026 in “GPT-6 Astra” “ARC-AGI-3 was more of a failure in that regard than -1 or -2”
Comment from 3 Sep 2026 in “GPT-6 Astra”
What people warn about
Each mark is one comment
Doesn't measure real intelligence
“I think it is a nothingburger of a benchmark: lots of grand claims about AGI, but very little predictive power or practical utility.”
Comment from 27 Mar 2026 in “ARC-AGI-3” “ARC-AGI is much more bounded than they seem to think. It correlates with intelligence, but doesn't imply it.”
Comment from 11 Dec 2025 in “GPT-5.2” “It has nothing to do wirh some AGI testing. It is just some kind of useless weird game.”
Comment from 27 Mar 2026 in “ARC-AGI-3” - Comment from 25 Jul 2026 in “ARC-AGI Leaderboard”
Name overstates what it measures
“I'm glad we have the ARC series of tests. But they have "AGI" right in the name of the test.”
Comment from 13 Feb 2026 in “Gemini 3 Deep Think” “It's completely misnamed. It should be called useless visual puzzle benchmark 2.”
Comment from 12 Feb 2026 in “Gemini 3 Deep Think” Benchmark gaming and lab marketing
“OpenAI and Gemini in particular use this as marketing material. You can correlate the benchmark release with stock price increase.”
Comment from 15 May 2026 in “Cut Off: why access to frontier AI will soon be scarce and selective” “benchmarks like this are often flawed because leading model labs frequently participate in 'benchmarkmaxxing'”
Comment from 12 Feb 2026 in “Gemini 3 Deep Think” “every benchmark will be compromised with providers being able tweak things sufficiently to game”
Comment from 3 Sep 2026 in “GPT-6 Astra” Simple puzzles, unsuited to LLMs
“It seems like more of a simple optimization problem (think Sokoban) than anything approaching fluid intelligence.”
Comment from 24 Jul 2026 in “Claude Opus 5” “ARC-AGI is a terrible benchmark for testing LLMs because LLMs are not made, trained, or tuned for playing games.”
Comment from 25 Jul 2026 in “ARC-AGI Leaderboard” “Anyone taking a single look at the ARC-AGI "challenges" can see things a 5 year old could reasonably solve.”
Comment from 4 Sep 2026 in “GPT-6 Astra”
Other AI benchmarks people recommend
The four most recommended other AI benchmarks, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| Artificial Analysis | 62 | 46 | 57% | 11 | — | — |
| LMArena | 39 | 40 | 49% | 11 | — | — |
| ARC-AGIThis page | 22 | 24 | 48% | 8 | — | — |
| OCR Arena | 11 | 4 | 73% | 4 | — | Approved |
| Bullshit Benchmark | 9 | 0 | 100% | 6 | — | — |
What people said
40 of the 52 comments that mention it, newest first. Each links to the original comment.
“the tests are designed to stress frontier models in tasks that humans are uniquely good at”
“Anyone taking a single look at the ARC-AGI "challenges" can see things a 5 year old could reasonably solve.”
“It was always about crafting adversarial tests that humans are good at”
“ARC-AGI-3 was more of a failure in that regard than -1 or -2”
“The ARC-AGI-3 scorecard is extremely misleading”
“every benchmark will be compromised with providers being able tweak things sufficiently to game”
“ARC-AGI has thought a fair deal about this and has a good definition here”
“ARC-AGI3 doesn't seem like a great benchmark to me in the first place”
“ARC-AGI is a terrible benchmark for testing LLMs because LLMs are not made, trained, or tuned for playing games.”
“pretty fun to run as a human”
“I kinda wish I could select level”
“ARC-AGI is a beauty contest for pigs where the pig's owners compete to see who can apply the lipstick most convincingly.”
“It seems like more of a simple optimization problem (think Sokoban) than anything approaching fluid intelligence.”
“ARC AGI 3 is much better designed and harder”
“It's the framework behind ARC-AGI benchmarks.”
“OpenAI and Gemini in particular use this as marketing material. You can correlate the benchmark release with stock price increase.”
“it feels very reasoning heavy”
“I tried out their human simulation, and it feels very reasoning heavy.”
“For some reason they refuse to run this on the private set”
“I tend to prefer the ARC-AGI benchmarks for the most part.”
“reporting 0.25% when the actual reasoning gap is in hypothesis formation”
“I think it is a nothingburger of a benchmark: lots of grand claims about AGI, but very little predictive power or practical utility.”
“It has nothing to do wirh some AGI testing. It is just some kind of useless weird game.”
“I might have found 3-4 bugs with the default harness/API.”
“I could personally not stand any more than the first level.”
“I really like these puzzles.”
“Cool demo, what idiot thought it was ok to have a half second cooldown between inputs?”
“We give humans and AI the same input and measure the results.”
“This is a good and clever benchmark and a worthy successor to the previous two.”
“Simple game. Super annoying though lmao.”
“I'm glad we have the ARC series of tests. But they have "AGI" right in the name of the test.”
“ARC-AGI was specifically designed to be resistant to pre-training”
“It's completely misnamed. It should be called useless visual puzzle benchmark 2.”
“benchmarks like this are often flawed because leading model labs frequently participate in 'benchmarkmaxxing'”
“Really puts into perspective the type of thinking AI is learning!”
“ARC-AGI is much more bounded than they seem to think. It correlates with intelligence, but doesn't imply it.”
“ARC-AGI was designed specifically for evaluating deeper reasoning in LLMs, including being resistant to LLMs 'training to the test'.”
“Looking at the Arc-AGI problems, it seems like visual/spatial reasoning is just about the only thing they are testing.”
“they are still somewhat useful and the likes arc agi even more”
“early attempts at using this kind of definition in practice to improve current systems”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.