The best quantized language models, according to hackers
4 quantized language models recommended in Hacker News comments over the past year, ranked by how many separate people recommend each one.
All quantized language models
- Microsoft BitNetMicrosoft's open 1-bit model and bitnet.cpp framework run efficiently on ordinary CPUs and cheap hardware. Critics say output quality and benchmark evidence are weak.16recommend
- Ternary BonsaiCompact 1.58-bit model people praise for accuracy per byte and fast on-device use. Critics say it hallucinates often and is weak at coding and logic.14recommend
- Maple-PreviewPeople recommend it as a fast, capable small ternary model that runs on an iPhone at about 120 tokens per second. Critics say it hallucinates facts and gives confident wrong answers.6recommend
- BitNet b1.58 2B4TFast, memory-efficient model using about 400 MB of RAM and less training data; critics say factors left out of the paper make it less practical.5recommend