Groq API

“they have never been able to keep up with Groq/Cerebras-level inference speeds”

“GROQ / gpt-os-120b does a pretty good job to generate pipelines for it”

“Groq has always been more efficient than Cerebras at running very small models”

“I agree that Groq with multilayer hybrid bonding could be a good idea.”

“Default (embedded) provider is Groq, because when I start the project, was the best free API service for AI.”

“response times would vary so wildly that it just about erased any benefit in their throughput”

“they were still extremely fast, which is still incredibly useful”

“they are quantizing the models underneath. And they dont even disclose it.”

“Their website doesn't list them as offering any models that were released recently.”

“Groq is more performant for the growing categories of inference-based tasks”

“My company had a really terrible experience trying to use Groq, and I would NOT recommend anyone use their service if you need reliability.”

“I love Groq, but across every single line break in your post there is a glaring issue”

“They're running much larger models (orders of magnitude) and still reaching competitive speeds.”

“Groq was the preview of the broadband era of LLMs for me.”

“Currently the best performance I can get is the gpt-oss models on Groq or Cerebras”

“Really liked Groq due to its speed but it seems like after Nvidia bought it it has been discontinued”

“I have to say I was impressed by the simplicity, but afterwards less so by their controversial Kushner/Saudi investment”

“Groq is insanely fast if you want pure performance from an LLM”

“run GPT-OSS 120B (a 6X larger model) at 360 tok/s (30X faster) on Groq”

“We already have Groq, Celebras and AWS Bedrock and others in the inference of open models space”

“TTFT with Groq and colocation are both real wins that don't get talked about enough”

“Groq is very fast, but the latency wasn't always consistent and I saw some very strange responses on a few calls”

“I use them heavily in production because OSS 120B on Groq/Cerebras is one of the fastest models available”

“When you use Cereberas (or Groq) to develop an API, the turn around speed of iterating on jobs is so much faster”

“The problem with groq was they only allowed LORA on llama 8b and 70b”

“FPGAs have really low density so that would be ridiculously inefficient”

“It is certainly fast, but I think there might be some caching issues somewhere.”

“they gave predictable latency and behavior while I was validating the UX and reasoning flow”

“to lock labs in before they loose another to objectively superior inference.”

“using the free Kimi K2 model via Groq over CLI, which is much faster”

“inference is finally fast enough to be in the system loop”

“they offer something different to most other providers (cheaper, faster, and until recently "we don't store your data by default")”

“We use Groq with gpt-oss-20b, which gives great results and only adds ~250ms to the processing pipeline.”

“Its inferencing speeds are faster than those of H200s, and it has the lowest costs in the industry.”

“I run my small apps on groq. I get good enough summarization and simple agents”

“Many hundreds of tokens per second of output is the norm, in my experience.”

“I could swear they were doing some problematic quantization”

“K2 hosted on groq is pretty crazy for intellgence/second”

“That's definitely much cheaper and much faster than anthropic models”

“Fastest inference (cost-optimized), best for batch operations”