Flash-Moe
Shares rank 11 of 27 inference engines by recommenders
Open-source engine that runs huge mixture-of-experts models on a Mac by streaming weights from SSD; praised as clever engineering, but throughput is low for everyday use.
Open-source engine that runs huge mixture-of-experts models on a Mac by streaming weights from SSD; praised as clever engineering, but throughput is low for everyday use.
14 people gave an opinion
- recommend it
- 11
- criticize it
- 3
- approval share
- 79%
Recommended in 3 of 13 months
- November 2025: not recommended
- December 2025: not recommended
- January 2026: not recommended
- February 2026: not recommended
- March 2026: recommended
- April 2026: recommended
- May 2026: not recommended
- June 2026: not recommended
- July 2026: recommended
- August 2026: not recommended
- September 2026: not recommended
- October 2026: not recommended
- November 2026: not recommended (month in progress)
Why it doesn’t carry the badge
- 11 people recommend itNeeds 10 or more, met.
- 3.7 recommenders for every criticNeeds 2 to 1 or better, met.
- Recommended in 3 of 13 monthsNeeds 4 or more, not met.
The same three rules apply to every product, and the badge can’t be bought. How the badge works
What people say about it
The reasons people give, with the number of comments that give each one. Open a reason to read what they said.
Why people recommend it
Each mark is one comment
Runs huge models on a Mac
“Flash-MoE is basically turning “impossible” into a weekend project.”
Comment from 23 Mar 2026 in “Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM” “over 200B params with flash-moe occasionally”
Comment from 23 Apr 2026 in “Local LLM for Private Companies” Clever engineering and write-up
“The technical write up is great, but Mac users should not get too excited just yet on running 300B+ parameter models locally as the TPS isn't that good.”
Comment from 22 Mar 2026 in “Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM” “This project shows an interesting automated search for engineering problems”
Comment from 23 Mar 2026 in “Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM” Inspired other projects and adaptations
“I'm sure people will adapt the Flash-MoE integration to be even better soon as well”
Comment from 27 Apr 2026 in “GitHub Copilot is moving to usage-based billing” “Huge respect to them, it was a big inspiration for this project!”
Comment from 29 Jul 2026 in “Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac”
What people warn about
Each mark is one comment
Too slow for daily use
“6t/s is unbearable, if you used it with OpenCode you would be waiting 20+ minutes per turn.”
Comment from 23 Mar 2026 in “Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM” “they are doing similar things with bigger models, but tok/s is not high”
Comment from 29 Jul 2026 in “Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac” “it's not really usable for non-niche purposes due to the low throughput”
Comment from 31 Mar 2026 in “OpenAI raises $122B” - Comment from 22 Mar 2026 in “Flash-Moe: Running a 397B Parameter Model on a Mac with 48GB RAM”
Other inference engines people recommend
The four most recommended other inference engines, with the same counts for each, so you can compare like for like. The tick on each bar marks the 2 to 1 line the badge needs.
| Product | Recommend | Criticize | Approval | Months | Owned | Badge |
|---|---|---|---|---|---|---|
| vLLM | 112 | 56 | 67% | 12 | — | Approved |
| Strata | 31 | 15 | 67% | 5 | — | Approved |
| DwarfStar 4 | 26 | 2 | 93% | 6 | — | Approved |
| SGLang | 19 | 13 | 59% | 8 | — | — |
| Flash-MoeThis page | 11 | 3 | 79% | 3 | — | — |
What people said
All 16 comments that mention it, newest first. Each links to the original comment.
“they are doing similar things with bigger models, but tok/s is not high”
“Huge respect to them, it was a big inspiration for this project!”
“it gets over 5 tokens/sec on an M3 Max”
“I'm sure people will adapt the Flash-MoE integration to be even better soon as well”
“over 200B params with flash-moe occasionally”
“That's what seems to make the new Flash-MoE work so well.”
“it's not really usable for non-niche purposes due to the low throughput”
“It's early stages though. Compute is an issue too; a lot of code is PoC level.”
“6t/s is unbearable, if you used it with OpenCode you would be waiting 20+ minutes per turn.”
“Flash-MoE is basically turning “impossible” into a weekend project.”
“This project shows an interesting automated search for engineering problems”
“4tok/s is kinda unusable though”
“This project couldn't even get useful JSON out of the model because it can't produce the right token for quotes”
“applying such extreme measures to LLMs to get them to run severely degrades quality”
“The technical write up is great, but Mac users should not get too excited just yet on running 300B+ parameter models locally as the TPS isn't that good.”
“seems promising , this is the way”
Counts come from 1,795,640 Hacker News comments, November 2025 to November 2026. How we count. Report a mistake.