# ninfer

16 Hacker News commenters recommended ninfer and 2 criticized it, November 2025 to November 2026. It was recommended in 3 of 13 months.

**Verdict:** Free local inference engine that people report running Qwen models at high token speeds on single consumer GPUs, often faster than llama.cpp. Critics note one benchmark slower than vLLM.

## Why people recommend it

- Very fast token generation on GPUs (10 comments): "Folks over on /r/localllama have been reporting wild prefill/token gen speeds with ninfer" ([source](https://news.ycombinator.com/item?id=49333341)) "I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast" ([source](https://news.ycombinator.com/item?id=49555292)) "gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup" ([source](https://news.ycombinator.com/item?id=49305938))
- Works as advertised (2 comments): "really cool and works as advertised" ([source](https://news.ycombinator.com/item?id=49492903)) "which it does outstandingly well" ([source](https://news.ycombinator.com/item?id=49961417))
- Faster than llama.cpp (2 comments): "it is impressive. The place that it's doing better at than llama-cpp seems to really be late in the context window." ([source](https://news.ycombinator.com/item?id=49559769)) "gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup" ([source](https://news.ycombinator.com/item?id=49305938))
- Good for single-GPU setups (2 comments): "Definitely worth looking at if you have only a single 5090 is ninfer" ([source](https://news.ycombinator.com/item?id=49944121)) "ninfer-3090 is worth checking out if you use just one GPU." ([source](https://news.ycombinator.com/item?id=50244756))

## Alternatives hackers recommend

- [vLLM](https://hackerapproved.com/p/vllm.md): 112 recommend
- [Strata](https://hackerapproved.com/p/strata.md): 31 recommend
- [DwarfStar 4](https://hackerapproved.com/p/dwarfstar.md): 26 recommend
- [SGLang](https://hackerapproved.com/p/sglang.md): 19 recommend
- [exo](https://hackerapproved.com/p/exo.md): 15 recommend

## Cite as

Hacker Approved, "ninfer", https://hackerapproved.com/p/ninfer/ (counts from 1,795,640 Hacker News comments, November 2025 to November 2026).