# Magnitude inference engine

11 Hacker News commenters recommended Magnitude inference engine and 9 criticized it, November 2025 to November 2026. It was recommended in 2 of 13 months.

**Verdict:** Recommended for fast token generation, lower KV memory use and an autotuner. Common caveat: it often fails to detect hardware or load models.

## Why people recommend it

- Faster decode and token generation (3 comments): "Magnitude will continue to have an edge, especially for decode kernels." ([source](https://news.ycombinator.com/item?id=49919080)) "This drops KV memory usage by over half and also speeds up decode." ([source](https://news.ycombinator.com/item?id=49912522)) "Was indeed faster on token generation then on oMLX" ([source](https://news.ycombinator.com/item?id=49919823))
- Lower KV memory and bandwidth use (2 comments): "our quantized KV and associated decode kernels allow us to reduce the effective bandwidth needed" ([source](https://news.ycombinator.com/item?id=49915949)) "This drops KV memory usage by over half and also speeds up decode." ([source](https://news.ycombinator.com/item?id=49912522))

## What people warn about

- Few models load or fit (3 comments): "there are no small models that can fit my 16GB RAM or GTX1650 4GB GPU" ([source](https://news.ycombinator.com/item?id=49919610)) "On a 64gb Ram and 5080 machine the biggest model suggested was Qwen 9B" ([source](https://news.ycombinator.com/item?id=49917287)) "does not load a single model on my M1 16 GB" ([source](https://news.ycombinator.com/item?id=49930793))
- Hardware not detected correctly (2 comments): "it detects them each twice (says I have 4 GPUs)" ([source](https://news.ycombinator.com/item?id=49912533)) "It couldn't detect my hardware at all" ([source](https://news.ycombinator.com/item?id=49920389))
- Setup bugs and required assessment (2 comments): "It's not quite clear why all need to be assessed before I can download the first model to try." ([source](https://news.ycombinator.com/item?id=49913203)) "Trying to run this but keep hitting bugs" ([source](https://news.ycombinator.com/item?id=49912900))

## Alternatives hackers recommend

- [vLLM](https://hackerapproved.com/p/vllm.md): 112 recommend
- [Strata](https://hackerapproved.com/p/strata.md): 31 recommend
- [DwarfStar 4](https://hackerapproved.com/p/dwarfstar.md): 26 recommend
- [SGLang](https://hackerapproved.com/p/sglang.md): 19 recommend
- [ninfer](https://hackerapproved.com/p/ninfer.md): 16 recommend

## Cite as

Hacker Approved, "Magnitude inference engine", https://hackerapproved.com/p/magnitude/ (counts from 1,795,640 Hacker News comments, November 2025 to November 2026).