The best inference providers, according to hackers
14 inference providers recommended in Hacker News comments over the past year, ranked by how many separate people recommend each one.
All inference providers
- Fireworks AIRecommended for fast, reliable hosting of open-weight models with strict data retention and US compliance, though quality and billing draw criticism.67recommend
- DeepInfraRecommended for cheap, fast access to many open-source models through an OpenAI-compatible API. Critics say it quantizes some models and sometimes runs them worse.36recommend
- Together AIPeople recommend it for fast, cheap hosting of open-weight models with data-confidentiality terms. The most common caveat is that it can be pricier than alternatives and sometimes slower.20recommend
- fal.aiPeople recommend fal for fast, pay-as-you-go access to image and video models, with ready-to-use APIs. Critics most often cite pricing and markup.19recommend
- NeuralwattPeople recommend Neuralwatt for cheap, energy-metered access to open-weight models, with per-call energy reporting. Main caveat: it raised its prices after a cheap start.18recommend
- BasetenHosted inference service for open-weight models, praised for fast throughput, low prices and responsive support. Critics point to weak security practices and poor handling of a security incident report.15recommend
- Novita AIPeople recommend Novita as a cheap, fast, hosted API for open-weight models, with strict data retention and good cache hit rates. Critics say it serves quantized versions and doesn't test its models carefully.13recommend
- ReplicateRecommended for running open-source image and language models through a simple pay-per-request API and for quick prototyping. Main caveat: unclear pricing and billing errors, plus doubts about the Cloudflare acquisition's direction.13recommend
- Azure OpenAIGives access to OpenAI models under Microsoft's contract terms, with Azure identity access and no license fee; the most common caveat is outages and latency.9recommend
- IonRouterRecommended for inference on open-source and fine-tuned models, with a $10 minimum load and claimed higher throughput and lower prices. Critics say it still costs more than some rivals and does not clearly differ from other providers.8recommend
- NVIDIA NIMPeople recommend NVIDIA NIM for free access to top AI models at no cost to start. Critics mention spotty aarch64 coverage, slow speeds and doubts about staying power.7recommend
- Cloudflare Workers AIPeople recommend it as fast, cheap and simple to set up; the common caveat is slowness for interactive use.7recommend
- Chutes AIA subscription service for running open-weight AI models on GPUs, with TEE instances that keep prompts private from the GPU owners.6recommend
- Telnyx Inference APIRecommended for low per-token prices from a provider that runs its own GPU infrastructure in several regions. One critic doubts that providing inference is a professional move.5recommend