microgpt by Andrej Karpathy

“does an awesome job explaining the very foundations of LLMs making it super understandable”

“worthy of genuine applaud for being a good starting point for further work”

“I found it very helpful to look at karpathy's microgpt, a barebones LLM in 200 lines of python.”

“Definitely Andrej Kapathy's 250 line code of Micro GPT.”

“is the best in this genre in a long time”

“Microgpt is really good, if you want to understand exactly what happens.”

“I thought Karpathy’s microgpt explain how LLMs work”

“It's so small and so fast to train, you can really dig deep into the optimization landscape.”

“made it fast enough that you can also throw it around and learn its behavior by feel”

“this gpt just outputs random shit that looks like names”

“Now you can train it in under a second on one CPU core with no dependencies”

“The attached website is a fully ai-generated "visualization" based on the original blog post with little added.”

“the website is pretty neat if you are learning LLM internals”

“I feel its wrong to call it microgpt, since its smaller than nanogpt”

“As helpful again to an ML novice like me as Karpathy's original.”

“This guy is so amazing! With his video and the code base”

“together with the code it makes it so clear!”

“projects like this can spark creativity and critical thinking”

“it's great for exactly that purpose”

“the code is more or less a standard assignment in all Deep Learning courses”

“Great stuff! I wrote an interactive blogpost that walks through the code and visualizes it”

“incredibly useful for learning in depth the core of LLM's”

“That is really beautiful literate program.”

“What I find most valuable about this kind of project is how it forces you to understand the entire pipeline end-to-end.”

“It's like the same thing over and over again.”

“Andrej's code is really quite poetic, I love how much it packs into such a concise program”

“This is beautiful and highly readable”

“Karapthy with another gem !”

“when you see the memory layout spelled out in raw mallocs instead of hidden behind PyTorch abstractions”

“things you thought were hard in fact fit in a screen”

“it's a great learning tool and it shows it can be done concisely”

“a project like this can be invaluable for education”

“I love the visualizations, especially the next token probabilities”

“this is just a low-quality post with mostly empty words”

“Really nicely presented, well done!”

“on my desktop it works great!”

“The linked inspiration has a good blog post of microgpt implemented in python.”

“I can actually run the training loops, which makes sense given the simplicity of the original microgpt”

“Wtok and Wpos should be 26-dim along one of the axis but it shows a 16x16 matrix be default”

“this site assumed a bit too much knowledge for me to get much from it”