Terminal-Bench
Ranked 2 of 4 coding benchmarks by recommenders
Recommended as a free, open-source test for comparing AI coding agents on real terminal tasks; critics say it is saturated and its tasks are often ambiguous or overly specific.
Recommended as a free, open-source test for comparing AI coding agents on real terminal tasks; critics say it is saturated and its tasks are often ambiguous or overly specific.
23 people gave an opinion
- recommend it
- 12
- criticize it
- 11
- approval share
- 52%
Recommended in 10 of 13 months
- November 2025: recommended
- December 2025: recommended
- January 2026: not recommended
- February 2026: recommended
- March 2026: recommended
- April 2026: recommended
- May 2026: recommended
- June 2026: recommended
- July 2026: recommended
- August 2026: recommended
- September 2026: recommended
- October 2026: not recommended
- November 2026: not recommended (month in progress)
Why it doesn’t carry the badge
- 12 people recommend itNeeds 10 or more, met.
- 1 recommender for every criticNeeds 2 to 1 or better, not met.
- Recommended in 10 of 13 monthsNeeds 4 or more, met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Best available proxy for coding agents
“I think Terminal-Bench is currently the best proxy for "how good is this harness+agent combo at coding"”
Comment from 16 Dec 2025 in “Letta Code” “that bench has a lot of room to improve, but it is the best we can find now”
Comment from 9 Aug 2026 in “DeepSeek V4 Flash 0731: 82.7% on Terminal-Bench 2.1 with a public harness” Compares agent harnesses on one model
“I think Terminal-Bench is currently the best proxy for "how good is this harness+agent combo at coding"”
Comment from 16 Dec 2025 in “Letta Code” “pretty popular for comparing different harnesses using the same underlying model”
Comment from 25 Aug 2026 in “Headlong: A microharness for persistent agents” Covers longer, multi-step tasks
“I found Terminal-Bench to be the most relevant for me, even for tasks that go far outside the terminal.”
Comment from 5 Nov 2025 on Hacker News “They cover these longer horizon tasks that need more than just writing a bit of code in one file.”
Comment from 27 Mar 2026 in “$500 GPU outperforms Claude Sonnet on coding benchmarks” Shows how far LLMs have come
“The benchmark is actually pretty good. It shows in an extremely comprehensible way how far LLMs have come.”
Comment from 5 Feb 2026 in “Claude Opus 4.6” “This is a great way to become well-versed in benchmarking.”
Comment from 3 May 2026 in “The Terminal Bench 3.0 community is looking for task contributors”
What people warn about
Each mark is one comment
Tasks don't reflect real-world coding
“it's painfully obvious how bad those benchmarks are and why they dont represent real world coding”
Comment from 22 Jul 2026 on Hacker News “I've looked at the Terminal-Bench questions and I just think they're irrelevant.”
Comment from 28 May 2026 in “Claude Opus 4.8” “some of the benchmarks they highlight might not represent the reality”
Comment from 23 Oct 2026 in “Kimi K3.1 Pro and K3.1 Pro Highspeed” Saturated; no longer separates models
“2.1 is saturated - there is no world in which Gemini 3.8 Flash is producing better code”
Comment from 5 Sep 2026 in “Artificial Analysis Intelligence Index v4.2” “I'm certain has been saturated in post-training to no end”
Comment from 11 Jul 2026 in “Introducing Muse Spark 1.1” Ambiguous instructions, overly specific tests
“the instructions are often ambiguous while the test cases are overly specific”
Comment from 8 Jul 2026 in “Separating signal from noise in coding evaluations” “verifiers are overly specific”
Comment from 14 Jul 2026 in “Agents.md – Dumb Human” Scoring flaws and reward hacking
“40 of 83 gold-passing tasks still score 1 when the run also performs a destructive accident”
Comment from 26 Jul 2026 in “Claude Opus 5” “both can be reward-hacked with simple exploits”
Comment from 18 Apr 2026 on Hacker News
Other coding benchmarks people recommend
The three most recommended other coding benchmarks, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| FrontierCode | 14 | 4 | 78% | 5 | — | Approved |
| Terminal-BenchThis page | 12 | 11 | 52% | 10 | — | — |
| SWE-Bench Pro | 5 | 6 | 45% | 4 | — | — |
| SWE-bench | 5 | 17 | 23% | 2 | — | — |
What people said
23 of the 25 comments that mention it, newest first. Each links to the original comment.
“some of the benchmarks they highlight might not represent the reality”
“TerminaBench, DeepSwe sites are out of date.”
“Terminal bench 4 is good largely because it's recent so it hasn't been benchmaxxed yet.”
“2.1 is saturated - there is no world in which Gemini 3.8 Flash is producing better code”
“pretty popular for comparing different harnesses using the same underlying model”
“that bench has a lot of room to improve, but it is the best we can find now”
“40 of 83 gold-passing tasks still score 1 when the run also performs a destructive accident”
“it's painfully obvious how bad those benchmarks are and why they dont represent real world coding”
“verifiers are overly specific”
“I'm certain has been saturated in post-training to no end”
“the instructions are often ambiguous while the test cases are overly specific”
“the great team behind it Ryan/Alex et al, recently cleaned up a lot of dodgy submissions”
“I've looked at the Terminal-Bench questions and I just think they're irrelevant.”
“This is a great way to become well-versed in benchmarking.”
“Terminal Bench is the future”
“both can be reward-hacked with simple exploits”
“Technically unnecessary for pass/fail evaluation”
“They cover these longer horizon tasks that need more than just writing a bit of code in one file.”
“It has almost nothing to do with terminal, but random tools syntax.”
“The benchmark is actually pretty good. It shows in an extremely comprehensible way how far LLMs have come.”
“I think Terminal-Bench is currently the best proxy for "how good is this harness+agent combo at coding"”
“high quality participation equivalent to foundation labs proprietary agentic RL data but it's FOSS”
“I found Terminal-Bench to be the most relevant for me, even for tasks that go far outside the terminal.”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.