Tiny-vLLM
Hacker ApprovedShares rank 11 of 27 inference engines by recommenders
A small, readable C++ and CUDA LLM inference engine that people use to learn how inference works and to teach it. Some find the documentation dense in places.
A small, readable C++ and CUDA LLM inference engine that people use to learn how inference works and to teach it. Some find the documentation dense in places.
12 people gave an opinion
- recommend it
- 11
- criticize it
- 1
- approval share
- 92%
Recommended in 4 of 13 months
- November 2025: not recommended
- December 2025: not recommended
- January 2026: not recommended
- February 2026: not recommended
- March 2026: not recommended
- April 2026: recommended
- May 2026: recommended
- June 2026: recommended
- July 2026: not recommended
- August 2026: recommended
- September 2026: not recommended
- October 2026: not recommended
- November 2026: not recommended (month in progress)
Why it carries the badge
- 11 people recommend itNeeds 10 or more, met.
- 11 recommenders for every criticNeeds 2 to 1 or better, met.
- Recommended in 4 of 13 monthsNeeds 4 or more, met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Clear, well-documented walkthrough of inference
“Breaking down LLM inference into digestible steps makes the codebase approachable”
Comment from 30 May 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA” “Looks interesting, it reminds me of the first llama.cpp, but better documented.”
Comment from 29 May 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA” “Genuinely one of the clearest walkthrough of inference internals.”
Comment from 1 Jun 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA” Amazing documentation in lesson format
“The documentation is amazing. I'm using it as a resource for a university course I'm teaching.”
Comment from 8 Aug 2026 in “vLLM: Anatomy of a High-Throughput LLM Inference System” “I love the documentation formatted in lessons.”
Comment from 29 May 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA” “The documentation is amazing.”
Comment from 7 Aug 2026 in “vLLM: Anatomy of a High-Throughput LLM Inference System” Teaches inference better than alternatives
“I feel like I learned twice as much in 10 minutes reading this than I did reading LLM for Dummies”
Comment from 30 May 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA” “much shorter and not that deep as this one, for anyone interested in topic you might like it too”
Comment from 7 Apr 2026 in “Floating point from scratch: Hard Mode” “I wrote it to help others build useful mental model to be able to recreate the project yourself”
Comment from 29 May 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA”
What people warn about
Each mark is one comment
Documentation dense for some readers
“it's a bit dense though, some schemas will help i think”
Comment from 1 Jun 2026 in “Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA”
Other inference engines people recommend
The four most recommended other inference engines, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| vLLM | 112 | 56 | 67% | 12 | — | Approved |
| Strata | 31 | 15 | 67% | 5 | — | Approved |
| DwarfStar 4 | 26 | 2 | 93% | 6 | — | Approved |
| SGLang | 19 | 13 | 59% | 8 | — | — |
| Tiny-vLLMThis page | 11 | 1 | 92% | 4 | — | Approved |
What people said
All 15 comments that mention it, newest first. Each links to the original comment.
“The documentation is amazing. I'm using it as a resource for a university course I'm teaching.”
“The documentation is amazing.”
“If you prefer C++ and CUDA, then there's also tiny-vllm of mine”
“Genuinely one of the clearest walkthrough of inference internals.”
“it's a bit dense though, some schemas will help i think”
“I read solely for the README and enjoyed it supremely”
“I feel like I learned twice as much in 10 minutes reading this than I did reading LLM for Dummies”
“I'm sure I'll be referencing this quite a bit going forward.”
“Breaking down LLM inference into digestible steps makes the codebase approachable”
“Really practical teaching approach.”
“checking the return values of CUDA API calls is not "tiny" enough”
“Looks interesting, it reminds me of the first llama.cpp, but better documented.”
“I love the documentation formatted in lessons.”
“I wrote it to help others build useful mental model to be able to recreate the project yourself”
“much shorter and not that deep as this one, for anyone interested in topic you might like it too”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.