LMArena
Ranked 2 of 10 AI benchmarks by recommenders
People recommend it as a leaderboard built on blind human head-to-head votes that is harder to game than benchmarks. Critics say the votes reward style and sycophancy over factual accuracy.
People recommend it as a leaderboard built on blind human head-to-head votes that is harder to game than benchmarks. Critics say the votes reward style and sycophancy over factual accuracy.
79 people gave an opinion
- recommend it
- 39
- criticize it
- 40
- approval share
- 49%
Recommended in 11 of 13 months
- November 2025: recommended
- December 2025: recommended
- January 2026: recommended
- February 2026: recommended
- March 2026: not recommended
- April 2026: recommended
- May 2026: recommended
- June 2026: recommended
- July 2026: recommended
- August 2026: recommended
- September 2026: recommended
- October 2026: recommended
- November 2026: not recommended (month in progress)
Why it doesn’t carry the badge
- 39 people recommend itNeeds 10 or more, met.
- 0.9 recommenders for every criticNeeds 2 to 1 or better, not met.
- Recommended in 11 of 13 monthsNeeds 4 or more, met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Blind human head-to-head comparisons
“On paper it's one of the best because it's meant to be blind comparison of your own prompts.”
Comment from 28 May 2026 in “Claude Opus 4.8” “they use humans to judge a head-to-head comparison between a couple models at a time”
Comment from 17 Aug 2026 in “Qwen3.8 27B scores 52 on Artificial Analysis” “it helps to find a anonymously a model without confirmation bias”
Comment from 18 Jun 2026 in “Local Qwen isn't a worse Opus, it's a different tool” - Comment from 31 May 2026 in “Anthropic surpasses OpenAI to become most valuable AI startup”
- Comment from 4 Jan 2026 in “Total monthly number of StackOverflow questions over time”
Useful overview of many models
“having all these frameworks compared in one place is actually pretty useful”
Comment from 5 Apr 2026 in “Inference Arena – new benchmark of local inference and training” “every comparison like this or at models.dev or arena.ai is good”
Comment from 8 Apr 2026 in “Show HN: We fingerprinted 178 AI models' writing styles and similarity clusters” “I've found this company is a pretty good ranker”
Comment from 28 May 2026 in “Claude Opus 4.8” - Comment from 17 Aug 2026 in “GPT 5.6 Sol is the best "vision" model OpenAI ever released”
Harder to game than benchmarks
“IMO LMArena is the best benchmark that avoids benchmaxxing”
Comment from 9 Jul 2026 in “GPT-5.6” “Lmarena is a bit harder to fake and is human-evaluated.”
Comment from 2 Apr 2026 in “Google releases Gemma 4 open models” “it's the best benchmark that avoids benchmaxxing”
Comment from 10 Jul 2026 in “GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps”
What people warn about
Each mark is one comment
Rewards style and tone over accuracy
“Main issue with it these days are that it kinda measures sycophancy and user preferred tone more than substance.”
Comment from 12 Dec 2025 in “GPT-5.2” “it has been indicative of which LLMs style I like best, not necessarily its factual accuracy”
Comment from 6 Jan 2026 in “LMArena Is a Cancer on AI” “Models that train on Arena converge on helpfulness, not necessarily truthiness”
Comment from 14 May 2026 on Hacker News - Comment from 24 Apr 2026 in “DeepSeek v4”
Shallow side-by-side votes miss complex tasks
“It's biased to small context performance, which is why I don't pay much attention to it as a developer aside from a quick glance.”
Comment from 8 Nov 2025 in “Study identifies weaknesses in how AI systems are evaluated” “biased towards visual aesthetics more than raw coding as the voters are just asked which generated website they prefer”
Comment from 28 May 2026 in “Claude Opus 4.8” “the methodology of letting random people compare outputs side-by-side is a very shallow judgement method in my opinion”
Comment from 21 Jul 2026 in “Nativ: Run frontier open models locally on your Mac” - Comment from 13 May 2026 in “The US Is Winning the AI Race”
Prompt mix unlike real-world use
“the distribution of prompts on arena.ai can be pretty different than real-world usage”
Comment from 27 Oct 2026 in “GPT-6.1 Terra” “it only captures chat, and relies on users spending their time improving it”
Comment from 23 Oct 2026 in “Kimi K3.1 Pro and K3.1 Pro Highspeed” Gaming and benchmaxxing concerns
“It's easy to game and human evaluation data has its trade-offs”
Comment from 2 Apr 2026 in “Google releases Gemma 4 open models” “Arena can definitely be benchmaxxed a bit, if you try.”
Comment from 10 Jul 2026 in “GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps” “There's a ton of arenamaxxing going on, though I don't disagree that it's one of the better actual benchmarks.”
Comment from 10 Jul 2026 in “GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps”
Other AI benchmarks people recommend
The four most recommended other AI benchmarks, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| Artificial Analysis | 62 | 46 | 57% | 11 | — | — |
| LMArenaThis page | 39 | 40 | 49% | 11 | — | — |
| ARC-AGI | 22 | 24 | 48% | 8 | — | — |
| OCR Arena | 11 | 4 | 73% | 4 | — | Approved |
| Bullshit Benchmark | 9 | 0 | 100% | 6 | — | — |
What people said
40 of the 87 comments that mention it, newest first. Each links to the original comment.
“the distribution of prompts on arena.ai can be pretty different than real-world usage”
“it only captures chat, and relies on users spending their time improving it”
“The point of AA and Arena is trying to understand how to think about models.”
“It's not a replacement for actual debate practice but it's a useful tool for studying rhetorical patterns”
“they use humans to judge a head-to-head comparison between a couple models at a time”
“There are many companies now with evals as a core business model: LM Arena, Artificial Analysis, Prompt foo”
“for a nice overview of current models”
“is much clearer, albeit its an arena metric and a bit tricky to find the pareto view”
“the methodology of letting random people compare outputs side-by-side is a very shallow judgement method in my opinion”
“Also very good for free image gen, for what it's worth.”
“We had an incredible run with Zamund, RARBG and Arena”
“Arena can definitely be benchmaxxed a bit, if you try.”
“There's a ton of arenamaxxing going on, though I don't disagree that it's one of the better actual benchmarks.”
“it's the best benchmark that avoids benchmaxxing”
“The results here match up to my real-world experience using these models every day at work”
“IMO LMArena is the best benchmark that avoids benchmaxxing”
“it helps to find a anonymously a model without confirmation bias”
“which I think is blind AB testing”
“biased towards visual aesthetics more than raw coding as the voters are just asked which generated website they prefer”
“On paper it's one of the best because it's meant to be blind comparison of your own prompts.”
“I've found this company is a pretty good ranker”
“Better leaderboard: https://arena.ai/leaderboard/text”
“Models that train on Arena converge on helpfulness, not necessarily truthiness”
“LM Arena uses human side by side voting, which limits its applicability to complex tasks.”
“really only measures the personality and style preferences of the masses at this point”
“I've stopped trusting these "trust me bro" benchmarks and just started going to LM Arena”
“has a nice Pareto distribution of ELO vs price for this”
“every comparison like this or at models.dev or arena.ai is good”
“having all these frameworks compared in one place is actually pretty useful”
“It's easy to game and human evaluation data has its trade-offs”
“Lmarena is a bit harder to fake and is human-evaluated.”
“at least they provide error bars on the ELO and give models the same rank number when they're overlapping”
“I've found it helpful to check the lmarena leaderboards occasionally”
“LMarena,despite its flaws(recaptcha in 2026?) is the only "testing ground" where you”
“it has been indicative of which LLMs style I like best, not necessarily its factual accuracy”
“getting 2 answers at once”
“Main issue with it these days are that it kinda measures sycophancy and user preferred tone more than substance.”
“Obviously neither are good but it is better.”
“the right method would have to be domain expert human reviewers”
“It's biased to small context performance, which is why I don't pay much attention to it as a developer aside from a quick glance.”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.