AMD ROCm

“The build uses my machine's ROCM and I just link it instead of embedded all that in an image”

“Support has gotten much better in just the last couple months.”

“Been able to run all models on day 0 at comparable speeds to Nvidia with similar memory bandwidth.”

“ROCm nightlies do actually support these old cards, just not the ROCm stable releases.”

“ROCm runs most things just fine without any ceremony or difficulty.”

“ROCM can be installed from the regular repo, and most AI software has ROCM builds by now.”

“there are frequent/constant regressions on the main branch, especially for gfx1151 (Strix Halo)”

“Yes, ROCm can be used to run frontier models and is being used by OpenAI, Anthropic, and Meta.”

“HIP and ROCm have actually made amazing strides in making CUDA developers' porting work easy”

“lack of a PTX analogue for ROCm was so painful that it set back adoption of AMD GPUs”

“The "ROCm" situation with Strix Halo was pretty bad for a while. I think it finally stabilized late last year.”

“ROCm stack is not for people though who aren't willing to dig in and patch things themselves.”

“ROCm works great too, the only issue i have had is that my machine froze a couple of times”

“around 7.2 it became reliable and as fast as or faster than ROCm 6.4.”

“it is way less reliable; any upgrade in kernel version or AMD driver and suddenly everything is broken”

“I've found 0 models or tasks I haven't been able to use with ROCm.”

“Current ROCm, 7.2.1, works quite well on the Strix Halo.”

“It is now faster than ROCm - both tg and pp over 16k ctx - on Strix Halo”

“Debian build their ROCm with support for all possible devices.”

“I have been trying to use Jax on AMD/ROCM since 2023 and still can't because of the countless ROCM bugs and terrible response time of their team.”

“Right now the entire source base is literally throw a bunch of crap into the rocm brand and hope it builds together”

“I got plenty of AI stuff working on my 6750XT years ago.”

“I guess it counts if you can find the information from one of the many conflicting wikis out there and then figure out how to hack support for your card”

“I can take a new, officially unsupported version of ROCm and just use it with my 7900 XT”

“The day ROCm supports EVERY AMD card on release, just like CUDA does, is the day I will actually believe this marketing hype.”

“how AMD and Intel keep failing to understand what CUDA, the whole ecosystem, is actually about”

“Supporting only Server grade hardware and ignoring laptop/consumer grade GPU/APU for ROCm was a terrible strategical mistake.”

“ROCm usually only supports two generations of consumer GPUs, and sometimes the latest generation is slow to gain support.”

“It has been a bit of a nightmare and had to package like 30+ deps”

“It's okay for inference, and okay for training if you don't stray too far beyond plain torch.”

“ROCm is up and coming, but support ... is still a work in progress, especially for newer chips.”

“ROCM finally shaking off its reputation for poor support”

“ROCm is supported by the minority of AMD GPUs, and is accelerated inconsistently across GPU models.”

“It's a LOT less bad than it used to be, amd deserves serious credit.”

“it's first party supported by AMD (ROCm LLVM, with a lot upstreamed as well) where it's fairly widely used in production”

“AMD's drivers actually come with a recent version of torch that 'just works' on windows”

“imo it's been a buggy mess, and I'm talking bugs that make your entire system lock up until you hard reboot”

“if vllm is not provided and built against ROCm dependencies it's going to be time consuming”

“they're actually trying to invest in ROCm and their Instinct ecosystem”

“ROCm didn't have such a team until hyperscalers lost their shit over our visibly poor development practices.”