ONNX Runtime
Shares rank 11 of 27 inference engines by recommenders
Recommended as a fast, well-optimized engine for running models on CPU and in browsers. Main caveat: converting TensorFlow and PyTorch models often fails.
Recommended as a fast, well-optimized engine for running models on CPU and in browsers. Main caveat: converting TensorFlow and PyTorch models often fails.
24 people gave an opinion
- recommend it
- 11
- criticize it
- 13
- approval share
- 46%
Recommended in 8 of 13 months
- November 2025: recommended
- December 2025: recommended
- January 2026: recommended
- February 2026: recommended
- March 2026: not recommended
- April 2026: not recommended
- May 2026: not recommended
- June 2026: recommended
- July 2026: recommended
- August 2026: recommended
- September 2026: recommended
- October 2026: not recommended
- November 2026: not recommended (month in progress)
Why it doesn’t carry the badge
- 11 people recommend itNeeds 10 or more, met.
- 0.8 recommenders for every criticNeeds 2 to 1 or better, not met.
- Recommended in 8 of 13 monthsNeeds 4 or more, met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Well optimized for CPU inference
“ONNX is my first suggestion to people looking for speed gains on CPU”
Comment from 3 Jul 2026 in “14× faster embeddings: how we rebuilt the ONNX path in Manticore” “ONNX is incredibly well optimized for CPU, best in class even”
Comment from 19 Jul 2026 in “Transcribe.cpp” Runs in browsers and production
“Impressive work on making this run entirely in-browser with ONNX Runtime Web!”
Comment from 31 Jan 2026 in “Show HN: I trained a 9M speech model to fix my Mandarin tones” “We currently serve 5+ million inference requests per day”
Comment from 9 Jun 2026 in “OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision” Engine behind other tools
“everything is enabled by onnx with some cgo (or Rust) plumbing on top”
Comment from 9 Jun 2026 in “OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision” “I know they have a solid ONNX engine”
Comment from 2 Feb 2026 in “Microsoft is walking back Windows 11's AI overload” “ported to plain C from ONNX Runtime”
Comment from 3 Aug 2026 in “Why we write our own C and C++ inference engines”
What people warn about
Each mark is one comment
Large size and heavy dependencies
“The ONNX Runtimes are also huge (150mb), JNI, and drag gargantuan dependencies!”
Comment from 12 Apr 2026 in “Ask HN: What Are You Working On? (April 2026)” “the primary onnxruntime jar a boat anchor; weighing is 90mb+ just by itself”
Comment from 13 Apr 2026 in “Ask HN: What Are You Working On? (April 2026)” “In my case I found that ORT alone was 3.4MB over the wire”
Comment from 19 Jul 2026 in “Speech Recognition and TTS in less than 500kb” - Comment from 30 May 2026 in “Show HN: Open-source private home security camera system (end-to-end encryption)”
Model conversion often fails or is painful
“Conversions from models written with Tensorflow and Pytorch often fail.”
Comment from 25 Dec 2025 in “ONNX Runtime and CoreML May Silently Convert Your Model to FP16” “Exporting models to work on ONNX is a pain in the ass.”
Comment from 9 Jan 2026 in “Dell's CES 2026 chat was the most pleasingly un-AI briefing I've had in 5 years” “an incredible inconvenience masked as the easy way of doing things”
Comment from 9 Jun 2026 in “OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision” Wrong outputs or slow quantization
“That onnx model run using onnxruntime with cuda ep is a different model than the one run with TRT ep.”
Comment from 23 May 2026 in “Making Deep Learning Go Brrrr from First Principles” “never got it produce correct outputs with my quantized models”
Comment from 17 Sep 2026 in “Nvidia announces native GPU programming in Rust” “ORT quantizer made it slower than FP32”
Comment from 28 May 2026 in “Kenosis – Quantize ONNX models without breaking kernel fusion” Crashes and opaque behavior
“unable to fix it. latest dev ort, dispoing tensors, restarting the session/worker. nothing, crashes every time.”
Comment from 26 Jul 2026 in “Inflect-Micro-v2: complete voice in 9.36M parameters” “They are incredible for production, but they are "Blackboxes."”
Comment from 29 Jan 2026 in “Show HN: Kandle – A WebGPU-based ML library written from scratch in JavaScript”
Other inference engines people recommend
The four most recommended other inference engines, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| vLLM | 112 | 56 | 67% | 12 | — | Approved |
| Strata | 31 | 15 | 67% | 5 | — | Approved |
| DwarfStar 4 | 26 | 2 | 93% | 6 | — | Approved |
| SGLang | 19 | 13 | 59% | 8 | — | — |
| ONNX RuntimeThis page | 11 | 13 | 46% | 8 | — | — |
What people said
23 of the 25 comments that mention it, newest first. Each links to the original comment.
“never got it produce correct outputs with my quantized models”
“ported to plain C from ONNX Runtime”
“unable to fix it. latest dev ort, dispoing tensors, restarting the session/worker. nothing, crashes every time.”
“In my case I found that ORT alone was 3.4MB over the wire”
“ONNX is incredibly well optimized for CPU, best in class even”
“I've used ONNX runtime to run models on Nvidia GPUs”
“ONNX is my first suggestion to people looking for speed gains on CPU”
“an incredible inconvenience masked as the easy way of doing things”
“We currently serve 5+ million inference requests per day”
“everything is enabled by onnx with some cgo (or Rust) plumbing on top”
“No one uses ONNXRuntime (nor the new engine in OpenCV 5) in production.”
“ONNX Runtime drags in things such as python”
“ORT quantizer made it slower than FP32”
“That onnx model run using onnxruntime with cuda ep is a different model than the one run with TRT ep.”
“ONNX web leaves a lot to be desired”
“the primary onnxruntime jar a boat anchor; weighing is 90mb+ just by itself”
“The ONNX Runtimes are also huge (150mb), JNI, and drag gargantuan dependencies!”
“I know they have a solid ONNX engine”
“Impressive work on making this run entirely in-browser with ONNX Runtime Web!”
“They are incredible for production, but they are "Blackboxes."”
“Exporting models to work on ONNX is a pain in the ass.”
“the process especially on MacOS is a bit messy”
“Conversions from models written with Tensorflow and Pytorch often fail.”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.