# llama.cpp

471 Hacker News commenters recommended llama.cpp and 109 criticized it, November 2025 to November 2026. It was recommended in 13 of 13 months; owners who said how long report a median of 1 years (4 owners). It meets the Hacker Approved badge rule.

**Verdict:** A fast, frequently updated engine for running open-weight models locally, with flexible GPU/CPU offloading and an OpenAI-compatible API server, though support for new models is often buggy at first.

## Why people recommend it

- Flexible GPU and CPU offloading (4 comments): "llama.cpp is designed for partial offloading, the most important part of the model will be loaded into the GPU and the rest on system ram" ([source](https://news.ycombinator.com/item?id=47193061)) "You can try to offload the experts on CPU with llama.cpp (--cpu-moe) and that should give you quite the extra context space" ([source](https://news.ycombinator.com/item?id=47793277)) "That allows you to force a certain number of experts to be kept in system memory while everything else is kept in VRAM." ([source](https://news.ycombinator.com/item?id=46232011))
- Easy to build and run (4 comments): "I just run cmake and build with no special options. Same thing with llama-server, I just specify the model and use the built-in web UI." ([source](https://news.ycombinator.com/item?id=49260016)) "It's not as bad as you might think to compile llama.cpp for your target architecture and spin up an OpenAI compatible API endpoint." ([source](https://news.ycombinator.com/item?id=47369899)) "compiled for CUDA backend, but there are also prebuilt binaries and instructions for other backends in the README" ([source](https://news.ycombinator.com/item?id=46682547))
- Fast with frequent updates (4 comments): "ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements." ([source](https://news.ycombinator.com/item?id=49268500)) "I get in the habit of building llama.cpp from upstream head and checking for updated quantizations right before I start using it" ([source](https://news.ycombinator.com/item?id=47627363)) "just dropping down to llama-cpp's GGUF model support and asking for default settings has provided faster inference speeds" ([source](https://news.ycombinator.com/item?id=48223766))
- OpenAI/Anthropic-compatible API server (3 comments): "Since Llama.cpp/llama-server recently added support for the Anthropic messages API, running Claude Code with open-weight local models is now very easy." ([source](https://news.ycombinator.com/item?id=46892439)) "It's not as bad as you might think to compile llama.cpp for your target architecture and spin up an OpenAI compatible API endpoint." ([source](https://news.ycombinator.com/item?id=47369899)) "it's not difficult to make llama-server available to your laptop over just about any form of internet connection and VPN" ([source](https://news.ycombinator.com/item?id=47923292))

## What people warn about

- Buggy early support for new models (4 comments): "the llama.cpp implementation for this model is still quite half-baked due to missing the support for DSA sparse attention mechanism" ([source](https://news.ycombinator.com/item?id=48640496)) "the bottleneck is the 48 hours of guessing llama.cpp flags and chat template bugs before the ecosystem catches up" ([source](https://news.ycombinator.com/item?id=47866740)) "There's also a regression in llama.cpp wrt. Step Flash, where quantisation is getting worse KLD and Perplexity" ([source](https://news.ycombinator.com/item?id=48548045))
- Confusing setup and configuration (3 comments): "llama.cpp having 2 different settings.json (and Google lying about where 1 goes) made it far too difficulty to figure out." ([source](https://news.ycombinator.com/item?id=49407702)) "finding out how to download and install models was a pain in Llama.cpp and it was also _much_ slower than Ollama." ([source](https://news.ycombinator.com/item?id=47792862)) "the bottleneck is the 48 hours of guessing llama.cpp flags and chat template bugs before the ecosystem catches up" ([source](https://news.ycombinator.com/item?id=47866740))
- Crashes, OOM and slow performance (3 comments): "The official llama.cpp ROCm build detected the card correctly, but kept crashing with 0xC0000005 while loading the model, so I switched to the Windows Vulkan build of llama.cpp instead, which worked reliably." ([source](https://news.ycombinator.com/item?id=49616131)) "I'm hitting OOM fairly often right now, it turns out that the sliding window attention context is huge and llama.cpp wants to keep lots of context snapshots." ([source](https://news.ycombinator.com/item?id=47798645)) "the experience is not pleasant, low pcie (gpu ram) utilization, low gpu utilization, and really low tokens per second" ([source](https://news.ycombinator.com/item?id=47484838))
- Architectural limits on optimization (3 comments): "llama.cpp has backed itself into a bit of a corner architecturally by trying to support all models on all possible backends" ([source](https://news.ycombinator.com/item?id=49295989)) "It is unlikely that llama.cpp will include any time soon code capable of reading SSDs at the speed limited by hardware." ([source](https://news.ycombinator.com/item?id=50083647)) "A big problem with the existing engines like llama or sd is that they don’t support optimal graph compilation." ([source](https://news.ycombinator.com/item?id=49153179))

## Alternatives hackers recommend

- [Ollama](https://hackerapproved.com/p/ollama.md): 350 recommend
- [LM Studio](https://hackerapproved.com/p/lm-studio.md): 217 recommend
- [DS4 by antirez](https://hackerapproved.com/p/ds4.md): 109 recommend
- [Unsloth Studio](https://hackerapproved.com/p/unsloth.md): 94 recommend
- [oMLX](https://hackerapproved.com/p/omlx.md): 62 recommend

## Cite as

Hacker Approved, "llama.cpp", https://hackerapproved.com/p/llama-cpp/ (counts from 1,795,640 Hacker News comments, November 2025 to November 2026).