ChatJimmy

“It's several orders of magnitude faster at inference than anything else”

“Chat jimmy launched in february, and nothing has come of it, not even GA let alone improvements with intelligence.”

“"speed of thought" is wildly underselling how fast chatjimmy is”

“There is an emergent space for super-fast-models esp finetuned or guardrailed to solve very specific latency sensitive tasks.”

“I missed the fact that it had responded because I was still hitting the enter key”

“it's based on a 2+ year old tiny model. It's very much an alpha proof of concept”

“Compaction, busting the cache, and other issues like that will lose significance when you're running at 15k tokens per second like chatjimmy.”

“I just tested out chatjimmy and a bit and it did some misinterpretation of my prompts”

“Its a cool demo, but its gpt-3.5 level stupid, or worse.”

“it's incredible. Both reasoning and tool use generation scale with TPS”

“training material is stopped at around July 2022, a little too outdated”

“This is the coolest LLM thing I've seen since the original ChatGPT announcement”

“It's great at knowledge and basic stuff but the output is complete junk for anything concerning new facts”

“the model used for Chat Jimmy is not very smart, but the world where it is smart is very interesting”

“Answers like 8Bish model it appears, but instant response”

“it is fast, but on the once good llama3.1-8b but now pretty useless”

“Hallucinates on the first question I ask, as 90% of these models that try to take shortcuts.”

“things like chatjimmy, I sometimes wonder if we might have built in chips with specialized models”

“this seems like a better choice to use a tool like this so you don't have your agent burn tons of tokens”

“I remember looking at this earlier this year and being that fast just feels fake.”

“It's eye opening: generated an AVX-512 optimized Mersenne Twister in C in 0.076s, 13,706 tok/s.”

“Wow that incredibly fast. I like this outcome more than centralized datacenters.”

“the combination of small context window, no tool calls and just small model, made me understand, that it's not going to work”

“it is fast but unfortunately its accuracy is not great”

“chatjimmy.ai can do that and output 17000 tokens/sec on cheaper silicon”

“That's really impressive from a speed point of view but it hallucinates like it's on drugs.”

“It's unbelievably (1000x) fast, but not flexible of course”

“Millisecond inference even on much older models will change the whole game.”

“it's running against Taalas' "hardcore" silicon model, ie a dedicated, ASIC-like chip.”

“this leads to super-fast speeds but the model they're using is an old small Llama model so the quality is pretty bad”

“It's quite impressive what purpose build inference can/will do”

“which worked well and was so fast every time I've hit it this past week”

“It is to be fair running a pretty trash model, a quantized Llama 3.1 8B from ~18 months ago”

“That thing is near instantaneous, it's all the proof you need that this is real.”

“This think takes under 30ms to process a small query - about 10 times faster.”

“I asked it some basic questions and it fudged it like it was chatgpt 1.0”

“a lot of LLM use cases don't need smart, they need fast and cheap”

“Is super fast but also super inaccurate, I would say not even gpt-3 levels.”

“even simple queries I tried got very inaccurate answers”

“it's impressive and feels like a completely different technology”