Nvidia RTX Spark

“The rtx has decent prefill speed and enough memory for both context and using a dense model”

“I see this new chip/setup as being an obvious technological upgrade and more cost efficient”

“If the price was lower, I'd be in the market for one of these.”

“I’m struggling to find a way to find a worse value buying something, anything that isn’t an RTX Spark.”

“Nvidia designed these to perform very well for local inference”

“The limitation (beyond total size of the memory) with the Spark is DDR5.”

“the article goes on to show that they're quite similar in performance”

“We have to see what RTX Spark systems sell for”

“it's very interesting for some application areas, less so for others”

“The Nvidia boxes have only slightly more memory bandwidth, so I wouldn't expect them to be notably faster.”

“It's going to be dog-slow unusable for large models, as tok/sec is basically bandwidth divided by active weights.”

“Models this small and this capable bode really well for the usefulness of a PC like the RTX Spark”

“given how terrible nvidias software has been with the DXG et el, I would not put much faith into this”

“an incredible machine (without getting into competitive comparisons/minutiae)”

“neither are really worth it for llm use compared to just building a desktop”

“Single-core CPU performance is going to be fully 20% slower than Snapdragon X2 Elite Extreme.”

“It seems the work with vLLM and training models could make the Spark worth the price difference”

“If you can get desktop RTX 5070 performance, oodles of (v)RAM, and minimal power usage out of a thin and light mobile device it's a win.”

“This has the potential to be a worthy competitor for the Windows ecosystem”

“it's pretty cool how Nvidia has the clout to convince a bunch of game publishers and creative apps to release Arm versions”

“It's a copy of Apple's SoC design philosophy, same as AMD's Strix Halo”

“it has lower memory bandwidth and probably loses a token/s fight for many LLMs”

“There are still a *lot* of sharp edges with the Spark: compatibility, overstated performance, power consumption/heat generation”

“I was rather unimpressed with the out-of-box experience for an "ai" computer”

“Kinda underwhelming. I was hoping to see that they improved their memory bandwidth”

“I'm just pleased to have a Linux on Aarch64 machine that is well supported for a change.”

“I really hope these take off and succeed and they support Linux.”

“This is much better value than 5090, you can run much bigger models.”

“There Spark setup was at 3k, way to expensive for normal people.”

“I guess it's pretty cool, I'm a big supporter of local LLMs/open weight models so seems enticing to me”

“I was disappointed to see that the RTX Spark has the ARM cores from the DGX Spark.”

“It's an ARM64 CPU with a nVidia GPU rather than garbage Adreno”

“the top model will have up to 6144 FP32 execution units compared to 2560 of Strix Halo”

“After nvidia's many years of neglecting Linux, paired with direct Microsoft's involvement?”

“still ~300GB/s memory bandwidth. That will be slower than the M5 max, by a wide margin for LLM inference.”

“Around 2-3K USD something with a good GPU + CPU + 128GB of integrated RAM is just going to be an awesome experience.”

“This may finally be the chip family ARM on Windows has always needed.”

“This will crush the M5 Max going by the numbers.”

“Higher developer productivity and higher chances of a substantial audience.”

“the 3x faster GPU is quite nice”