Apple M5 Max

“In 2.4 seconds it gave me 4 examples that work with different sorting configurations”

“m5 max really fixed pp with the better matmul support”

“My biggest problem with running local LLMs on my M4 Max/128GB RAM is the prefill latency.”

“If you can settle for a M5 Max base model that would be a $49/month lease.”

“My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory.”

“the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance”

“that chip just doesn't really have the memory bandwidth to run this (or nearly any) model at acceptable speeds”

“my only concern has been the very meager memory bandwidth of these chips”

“Buy an M5 max for $4k and you have a portable Deepseek 0731 for life.”

“the acoustic engineers at Apple have done a very good job of making it be pretty much not noticeable”

“one of the best laptop compute GPUs for blender rendering”

“the M5 Max has a 50% longer TTFT on Qwen's 27b dense model at only 16k of context”

“Benchmarking that I've seen shows that the M5 Max outperforms the DGX Spark”

“I do not see Apple as a competitive solution, as their price is much, much higher”

“an M5 Max 128GB was a bargain if you want to run local LLMs”

“It's feasible to run 256k token workflows that would take the better half of an hour for TTFT on the M5.”

“the M5 Max still doesn't have the compute to make it practical, at least from an interactive usage standpoint”

“real-world training workloads aren't getting anywhere near Apple Silicon”

“I do think Apple has a real opportunity here but there offerings aren't quite there yet.”

“They're honestly not competitive for inference, it's why datacenters largely ignore Apple Silicon.”

“M5 max is 3x stronger and 50% more power efficient”

“a desktop-grade M5 Max that is bandwidth-constrained in practice”

“the new neural engines in those GPU cores have made a big difference in a good way here”

“best quality price ratio, future proof, reliable, resellable and flexible workloads”

“runs llms surprisingly well without quantization, at least the moe models”

“I’ve seen 95-100 tokens per second on M5 Max 128GB running Gemma 4 31B.”

“The situation on pre-fill is not nearly as bad as in the M4 generation.”

“at 16k pre-fill, the M5 Max is 3.6 times faster than the M4 Max”

“Apple Silicon parts have unusually high memory bandwidth for general purpose compute chips”

“M5 Max is a major improvement for local inference due to the added matmul accelerators, but the RAM capacity and bandwidth bottleneck is huge.”

“can almost match it in the CPU-bound workload I care about while sipping some 45W”

“I'm trying to run inference on my home hardware that I also want to use for other things.”

“which isn't very "enthusiast-compatible", it's about 3x the minimum salary in my country!”

“You can already do this. Get an M5 Max with 129GB model and run a Qwen model.”

“CPU and GPU don't have to throttle for each other under sustained load anymore.”

“Apple CPU holds the fastest single core CPU benchmarks”

“in Blender, the M4 Max is on par with the top of Nvidia's (mobile) 5xxx line, except with much more VRAM”

“it will fix the biggest issue with the current lineup, prompt processing, in addition to a bandwidth bump”

“It works well enough for most mid sized LLMs.”

“I'd recommend holding off until the M5 Max comes out early in the early as the M5 has vastly superior performance”