Ceph

“Reliability problems have been exclusively caused by us penny-pinching and trying to have as few drives and servers as possible”

“many of those at the very cheap end pretty much just ran a Ceph/RADOS cluster and then wondered why they were having lots of problems with reliability”

“Ceph is pretty annoying to operate but if you do, it works fine.”

“It's a bit painful to set up and operate, but it seems to work pretty well.”

“If you want your own replicated storage cluster at home, you can use Ceph.”

“Very complex to manage, though, but that might just be necessary complexity for a distributed storage system.”

“we built our own storage system rather than relying on Ceph. The reason is scalability to many volumes”

“their data is not stored on any one system (local XFS, Ceph, Lustre), a recipe for disaster”

“The built-in ceph is decently configured and more performant than FC if you have 100GBS mellanox NICs”

“Ceph scales great if you have loads of large files where you're read/writing in parallel.”

“Done correctly, Ceph is extremely reliable, resilient, and fast.”

“where I'd use Ceph anyway (or build on-top of an existing solid distributed database) if I needed it”

“Deploying and configuring ceph was a nightmare due to the mountain of options”

“If you setup ceph correctly (multiple failure domains, correct replication rules across failure domains, monitors spread across failure domain, osds are not force purged) it is actually pretty hard to break it.”

“it is a truly collaborative open source project developed by multiple companies with a long track record”

“It's been pretty bulletproof; took some time to set up and familiarize with Ceph but now it's simple to operate.”

“Ceph solves the distributed consistent block storage problem very well.”

“Ceph is too difficult to use; I wouldn't dare deploy it without a deep understanding of the source code.”

“Ceph is a non-starter for me because you cannot have an existing filesystem on the disk.”

“Ceph has been pretty good to us for ~1PB scalable backup storage”

“I successfully migrated from MinIO to Ceph, which I highly recommend.”

“We have some clusters spanning 20 racks in a network fabric that has over 100 racks.”

“Ceph can scale to pretty large numbers for both storage, writes and reads.”

“Ceph's storage and consensus layer is battle-tested and a much more solid foundation for serious use”

“Ceph works well, but requires a lot more hardware and dedicated cluster monitoring”

“Community-governed projects under foundations (Ceph under Linux Foundation, for example) tend to be more durable”

“What's neat is that ceph allows expanding clusters, just throw more storage servers at it”

“I love the flexibility of it... but I don't think ceph truly makes sense until you have multiple racks”

“Scales to Petabytes and has lots of failure protection options.”

“but with that complexity you get much cheaper S3, block storage, and cephfs.”

“I do not recommend ceph unless you are okay with not using shared filesystems”

“Ceph clusters when mistreated can get into death spirals that only an experienced practitioner can avert”

“Ceph is quite expensive in terms of resource usage, but it is robust and battle-tested.”

“I simply migrated their elderly VMware based cluster to a smart new set of HPEs with flash on board - Ceph cluster.”

“the only properly engineered open source choices are Ceph + radosgw (single-region, though)”

“operators for Ceph and databases are worth the up-front trouble for ease of management and consumption”

“ceph can do all that but setup is more complex and RAM requirements far, far higher”

“Ceph is awesome for software defined storage where you have multiple storage nodes and multiple storage devices on each. It's way too heavy and resource intensive for a single machine”

“My cluster went from 6 to 7, 7 to 8 and recently 8 to 9, along with Ceph - all without a single problem.”

“This tells you everything how hard it is to keep Ceph humming along in production.”