The best inference engines, according to hackers
27 inference engines were recommended on Hacker News between November 2025 and November 2026. We rank them by how many separate people recommend each one, and 8 earn the Hacker Approved badge.
Top picks
The three most recommended inference engines with the badge. Each figure is 10 people. RecommendsCriticizes One comment
Continuous batching and concurrency
Recommended for fast, scalable LLM serving with continuous batching, multi-GPU tensor parallelism and concurrent requests; critics cite fiddly setup, version juggling, weak AMD support and bugs.
Runs big models on modest hardware
Runs large local LLMs fast on consumer hardware. Caveats: not open source, and quantized models may make more mistakes.
Runs big models on consumer hardware
Lets people run large DeepSeek-class models locally on consumer hardware, supporting few models well. Critics say speeds are often too slow for comfort.
All 27 inference engines, ranked
Select a column heading to sort. Approval share is recommenders out of everyone who gave an opinion; the tick on each bar marks the 2 to 1 line the badge needs. Months counts the months, out of 13, in which someone recommended it.
| Rank | |||||||
|---|---|---|---|---|---|---|---|
| 1 | vLLM | 112 | 56 | 67% | 12 | — | Approved |
| 2 | Strata | 31 | 15 | 67% | 5 | — | Approved |
| 3 | DwarfStar 4 | 26 | 2 | 93% | 6 | — | Approved |
| 4 | SGLang | 19 | 13 | 59% | 8 | — | — |
| 5 | ninfer | 16 | 2 | 89% | 3 | — | — |
| Tied 6 | exo | 15 | 3 | 83% | 9 | — | Approved |
| Tied 6 | MTPLX | 15 | 2 | 88% | 5 | — | Approved |
| Tied 6 | Gemma 4 26B inference engine for Mac | 15 | 2 | 88% | 1 | — | — |
| Tied 9 | ik_llama.cpp | 12 | 1 | 92% | 9 | — | Approved |
| Tied 9 | Mesh LLM | 12 | 2 | 86% | 2 | — | — |
| Tied 11 | ONNX Runtime | 11 | 13 | 46% | 8 | — | — |
| Tied 11 | Tiny-vLLM | 11 | 1 | 92% | 4 | — | Approved |
| Tied 11 | Flash-Moe | 11 | 3 | 79% | 3 | — | — |
| Tied 11 | Magnitude inference engine | 11 | 9 | 55% | 2 | — | — |
| 15 | OpenVINO | 10 | 2 | 83% | 7 | — | Approved |
| Tied 16 | ONNX | 9 | 6 | 60% | 7 | — | — |
| Tied 16 | Timber | 9 | 2 | 82% | 3 | — | — |
| Tied 16 | RunAnywhere SDK | 9 | 13 | 41% | 2 | — | — |
| Tied 19 | WebLLM | 8 | 4 | 67% | 4 | — | — |
| Tied 19 | AirLLM | 8 | 3 | 73% | 2 | — | — |
| Tied 19 | h3.c MiniMax H3 inference engine | 8 | 0 | 100% | 2 | — | — |
| 22 | DeepSeek 4 Flash local inference engine for Metal | 7 | 1 | 88% | 1 | — | — |
| Tied 23 | LiteRT-LM | 6 | 0 | 100% | 4 | — | — |
| Tied 23 | FastFlowLM | 6 | 3 | 67% | 4 | — | — |
| Tied 23 | mlx-audio | 6 | 0 | 100% | 3 | — | — |
| Tied 26 | mlx-lm | 5 | 1 | 83% | 4 | — | — |
| Tied 26 | mlx-vlm | 5 | 2 | 71% | 3 | — | — |
How we rank
Rank is the number of separate people who recommend a product in a Hacker News comment over the past year. Someone who praises the same inference engine in five threads counts once.
The badge goes to inference engines that at least 10 people recommend, with recommenders outnumbering critics at least 2 to 1, recommended in at least 4 of the 13 months.
Rankings are never for sale. We may earn a commission when you buy through our links. It never changes the order.