DS4 by antirez

“the value of human ingenuity coupled with a model like DS4.1 is technically enough to get most people to where they want to go”

“The ds4 q4 quant performs a lot better than other similar sizes that I've seen.”

“It worked, the change I had to do was also minimal for the compilation of the .cu unit.”

“I immediately knew what it was, what it required, and it provided the download links right there.”

“It was like 50GB instead of 5GB for context, and it wasn't fixed for weeks”

“Getting DS4 to run at a reasonable speed was pretty tricky”

“I wasn't always a fan of what it came up with when asked to plan something.”

“the capabilities of models like DS4 Flash are those of frontier models from months ago”

“it works perfectly fine as a replacement for Claude in a large variety of cases”

“It will literally have the right approach and reason itself away from it.”

“it's _good_ at some tasks, but it really just spins its wheels when it hits the limit of what it can reason through”

“There are more focused frameworks like DS4 and Colibri which have better support for streaming weights from disk”

“indistinguishable for me from Opus for my personal daily use/assistant benchmarks”

“his 4 bit quant does have decent accuracy, but is a bit pointless by being bigger than the full precision MXFP4 quant”

“I've been running Deepseek V4 Flash locally on my Macbook Pro for weeks using Open WebUI + DS4.”

“the model literally talks itself out of giving the cached wrong answer with a latent only thinking budget”

“The DS4 repo is a really interesting place to watch folks who heavily code with agents collaborate together in a way that seems pretty effective.”

“It definitely seems to be the leader on 'general intelligence' on this hardware from my casual usage”

“it often builds a completely wrong mental model and charges off down the wrong path”

“DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.”

“I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache”

“DS4 (a 1-bit quantization of DeepSeek V4 Flash) runs at something like 9-13 tokens/second”

“the quality of the 2bit quants antirez has provided have been validated by folks like ggernov of llama.cpp project”

“it tops out at something like 10 or 11 with a long time to first response”

“antirez DS4 build that made that actually runnable on MacBooks and Mac Studios”

“there are local inference configurations that allow with that cost to run sufficiently powerful models with huge contexts”

“it loops and over thinks even more, burning massive walk clock time”

“A little slower than frontier models but my $200/mo weekly usage exhausted with 3 days left on the week”

“This is probably as close to it gets to state-of-the-art local LLM + agentic coding.”

“you'll have a context limit of about 64k, which is also prohibitive for serious work”

“allowing for SOTA inference (DeepSeek V4 Flash and Pro!) on RAM-limited machines”

“one can use just one system to a very, very decent speed and ttft for chat inference”

“Even at 2-bit quantization, DS4 is probably on par with a 2024 frontier model.”

“it was behaving quite poorly after a few agentic turns with opencode”

“the ux of ds4 is fantastic too -- it's dead-easy to get a known-good model, great quant”

“DS4 is hosting the DeepSeek v4 Flash model with good performance on home hardware”

“It’s shocking how close this feels to claude, obviously it's much slower, but I don’t know that it’s significantly dumber.”

“it's the only way I've actually managed to run DeepSeek flash on my local hardware”

“31t/s prefill is basically unusable in an agentic situation”

“pretty painless, model runs in about 80GB of RAM and it seemed to be very capable at writing code”