The best distributed computing frameworks, according to hackers
3 distributed computing frameworks recommended in Hacker News comments over the past year, ranked by how many separate people recommend each one.
- Most recommendedApache Spark18 people recommend it
- Least regretRay distributed computing framework10 recommend, 7 criticize
All distributed computing frameworks
- Apache SparkFree open-source engine for big data processing that people say just works. Critics say it is overkill for small datasets and hard to run yourself.18recommend
- Ray distributed computing frameworkPeople recommend Ray for scaling Python workloads across machines, especially ML infrastructure, with built-in autoscaling. Critics say clusters add heavy overhead and cost, and that it is slow.10recommend
- Apache HadoopRecommended as a known quantity that really works for large-scale work; critics say it is overkill for smaller scale, with a large upfront cost.8recommend