The best local llm runners, according to hackers

27 local llm runners recommended in Hacker News comments over the past year, ranked by how many separate people recommend each one.

All local llm runners

llama.cppA fast, frequently updated engine for running open-weight models locally, with flexible GPU/CPU offloading and an OpenAI-compatible API server, though support for new models is often buggy at first.471recommend
OllamaRecommended for running open models locally with one command, and for its cheap, generous Ollama Cloud plan; critics cite poor defaults, slow new-model support, and locked-in model storage.350recommend
LM StudioAn easy way to download and run local LLMs, with a model browser, hardware-fit hints and a headless API server, though critics object to it being closed source.217recommend
DS4 by antirezRuns DeepSeek V4 Flash on home hardware like Macs, with well-made quants and quality some find close to Claude, though it's slow and its reasoning can derail.109recommend
Unsloth StudioAn open-source local AI app with web search and tool calling, from the makers of fast-arriving, high-quality quantized models, though installation and Python setup are hard, especially on macOS.94recommend
oMLXRecommended for fast local models on Apple Silicon, with KV caching that keeps long agent sessions quick. Critics find it slower than llama.cpp in some tests and sometimes unstable.62recommend
ApfelFree, fully local command-line wrapper for Apple's on-device model, useful for shell scripts without cloud calls. Critics note a 4K context window, weak guardrails and an overhyped landing page.41recommend
Lemonade Server by AMDFree AMD-backed server running local models on AMD GPUs and NPUs, handling backends and OpenAI-compatible endpoints. Main caveat: proprietary NPU kernels.30recommend
Google AI Edge GalleryFree Google app that runs small AI models directly on phones and Macs, working offline and easy to set up.24recommend
Survival LLMPeople recommend this free offline AI assistant for its survival and off-grid knowledge, grounded in vetted sources. The most common caveat is that it gives wrong answers.21recommend
ColibriRecommended for running large models from disk on modest hardware and as a Discord-like chat with nice UX; critics say its privacy falls short of its marketing and that local inference is very slow.20recommend
JanOpen-source desktop app for chatting with local AI models, with a simple interface and built-in model downloads. Critics say workspace features are thin and it needs internet and has telemetry.18recommend
llamafile by Mozilla AIPackages a language model and its runtime into one file that runs locally on most operating systems, so it is simple to start with. Critics say updates have stalled.15recommend
Off GridFree, MIT-licensed app that runs AI models offline on phones and laptops, valued for privacy and easy setup. Caveat: image generation sometimes needs network access.15recommend
AnythingLLMPeople recommend it as a plug-and-play front end for chatting with documents and local open source models. The most common caveat is its memory use, with critics noting it takes about 2GB of RAM even when idle.11recommend
Ente Local LLMRuns a small language model locally on your phone with fast responses and no API round trip, from the App Store and Play Store. Critics call it a generic wrapper.11recommend
HarborFree tool that sets up a local LLM stack in a few commands, with an easy interface and simple navigation.8recommend
GerbilFree desktop app for running text, image and video AI models locally, with a simple interface. One critic reported a hard lock-up after the splash screen.7recommend
Gemma 4 iPhone appFree app that runs Google's Gemma 4 model on iPhone with easy setup and no sign-in. Critics report crashes and slowdowns in long chats.7recommend
NativA Mac app for running open models locally, written mostly in Swift with MLX support. Critics say the website reads as slop and the server often fails to start or connect.7recommend
OsaurusA free, open-source, local-first macOS app for running AI models and agents, recommended for being small, fast and provider-agnostic.6recommend
LocalAIFree local server for running models behind OpenAI-style APIs, handy for CI and tests. Critics say it does not fully solve the problem it targets.6recommend
KoboldCppFree local runner for text and image models in one executable, valued for minimal dependencies and easy use.5recommend
vMLXFree app built for performance on Apple Silicon Macs, using unified memory and NVMe caching. Main caveat: limited support for some models, such as gpt-oss.5recommend
Locally AI by LM StudioFree, uncluttered iPhone app with a wide range of local AI models and a link to desktop LM Studio. Some models are hidden in settings.5recommend
HobA free, on-device AI assistant recommended as a better alternative to big companies' tools. Critics say its models are weak and that it is not fully open source.5recommend
Samosa ChatPeople recommend Samosa Chat for running local models such as Qwen3.6-35B-A3B on a Mac at a usable speed without overheating it.5recommend