Ceph Minimum Hardware Requirements: Nodes, CPU, RAM, Network

Ceph’s minimum requirements for a cluster that survives losing a host are three nodes with three monitors and, per OSD, a drive of at least 1 TiB, 4 GiB of RAM and one CPU thread (HDD) or four (NVMe). Networking needs 1 Gb/s at minimum and 10 Gb/s by recommendation.

Ceph is designed for commodity hardware, but a cluster built below these published figures tends to fail months later, during a recovery, when the unbudgeted margin is what would have kept it serving reads.

Figures here follow the project documentation as checked in October 2026, with Tentacle 20.2.4 the current stable release. For complete builds, see Ceph hardware recommendations for small clusters.

Ceph requirements per daemon

Ceph sets its requirements per daemon, so a host needs the sum of what it runs. The OSD, monitor and MDS rows below come from the minimum hardware per daemon page; the manager and boot-drive row draws on the CPU and memory sizing and storage devices pages.

The OSD, monitor and MDS rows each list 1 Gb/s as the minimum network and 10 Gb/s as recommended. OSD figures are per OSD, so a host with twelve NVMe OSDs needs 48 threads at minimum and 72 by recommendation before it runs anything else.

Minimum nodes: one for a lab, three for production

Ceph needs three hosts for a cluster that survives losing one, because the default replicated pool keeps three copies (osd_pool_default_size is 3) and a host-level failure domain needs three distinct hosts to place them. One host works for testing; two is the awkward case.

The documentation’s simplest erasure-coded pool likewise requires at least three hosts. Monitors want an odd count so a majority can always be formed; the monitor reference says quorum requires a majority of monitors running, so three monitors tolerate one loss and five tolerate two.

The single-host flag sets osd_crush_chooseleaf_type = 0, osd_pool_default_size = 2 and mgr_standby_modules = False, per the cephadm install guide. Two hosts fail on arithmetic: a majority of two monitors is both, so even stretch mode adds a tiebreaker monitor in a third location.

If you are still choosing a storage platform, compare Ceph vs Gluster for a small cluster before committing to a node count and hardware budget.

Four hosts is materially better than three, because at three the cluster has nowhere to re-replicate to when a host dies. It stays degraded until you fix the hardware. That distinction, not raw capacity, is usually what separates a homelab cluster from something that can be left alone over a weekend. The way copies are distributed across those hosts is decided by the CRUSH rule, which is covered in how Ceph places data across CRUSH, pools, and placement groups.

Erasure coding raises the count. Most EC deployments need at least k+m failure domains, usually hosts, with operational advantages at k+m+1. The default k=2, m=2 profile with a host failure domain needs four hosts; a 4+2 profile needs six and is easier to run on seven.

CPU: the metric changed

Older Ceph guidance was expressed as cores per OSD. The Tentacle hardware page says plainly that this metric is no longer as useful as cycles per I/O and IOPS per OSD, and advises selecting for IOPS per core. The reason is that a spinning disk saturates long before a modern core does, while an NVMe device can absorb far more work than one core can generate. The documentation notes that on real clusters Ceph can easily use five or six cores for a single NVMe OSD, and up to roughly fourteen cores for a single OSD in isolation.

The minimums are counted in threads. The Tentacle hardware page says a core means a thread when hyperthreading is enabled, and states that “hyperthreading is usually beneficial for Ceph servers.” The minimum hardware page adds that ARM CPUs may need more cores and should be benchmarked.

Two qualifications matter. First, those OSD figures are stated as being before replication, and erasure coding and compression push them up, because parity computation is real CPU work on every write and on every degraded read. If the plan is an erasure-coded pool, budget above the recommended figure rather than at it; the tradeoff is laid out in erasure coding versus replication. Second, the metadata server is single-threaded and benefits from clock rate, so a high-core low-clock part is the wrong shape of CPU for CephFS metadata.

RAM: the number that people underestimate

BlueStore does not lean on the operating system page cache. It manages its own memory, governed by osd_memory_target, which defaults to 4 GiB per OSD daemon. The documentation is direct about the shape of that curve:

• Below 2 GB is not recommended, and extremely slow performance is described as likely.
• Between 2 GB and 4 GB typically works but may degrade performance, because metadata has to be read from disk during I/O unless the working set is small.
• 4 GB is the default, chosen to balance RAM cost against OSD performance.
• Above 4 GB helps when there are many small objects or large per-OSD data sets, especially on fast NVMe.
• An effective target of at least 6 GiB is described as helping to mitigate slow requests on HDD OSDs.

The sizing rule for a whole server is the part worth writing down: total server RAM greater than the number of OSDs multiplied by osd_memory_target multiplied by two. The doubling covers the operating system, other Ceph daemons, and the fact that memory use rises during recovery and rebalancing rather than during the calm period you sized against. The documentation’s own worked example is a 1U server with eight to ten OSDs being well provisioned with 128 GB.

The CPU and memory sizing page also sets a floor of at least 20 percent RAM above the sum of osd_memory_target values, because OSD memory management is best-effort: the daemon may unmap memory without the kernel reclaiming it promptly. Swap is discouraged outright on Ceph hosts, since a crashed daemon recovers faster than one that is swapping.

Applied to the 4 GiB default, for the OSDs alone:

Cephadm sets osd_memory_target_autotune to true by default, so on cephadm clusters each OSD’s target is a share of host memory rather than a flat 4 GiB. The docs warn this is usually not appropriate for converged nodes that also run compute. To pin one OSD instead:

ceph config set osd.123 osd_memory_target_autotune false
ceph config set osd.123 osd_memory_target 16G

Monitor and manager memory scales with cluster size: 32 GiB suffices for very small clusters, 64 GiB for clusters up to roughly 300 OSDs, and 128 GiB beyond that. Metadata servers want at least 8 GiB per daemon, with cache size set by mds_cache_memory_limit, which defaults to 4 GiB. Placement group count feeds into this too, since OSD memory consumption is related to the number of placement groups a daemon serves, so an unmanaged PG count is a memory problem as well as a balance problem.

Drives: one OSD per device, and a floor on size

The default assumption is one storage drive per OSD. The documentation recommends a minimum OSD size of 1 TiB, on the grounds that much smaller devices spend a significant fraction of their capacity on metadata, and the storage devices page adds that drives under 100 GiB are not effective at all. At the other end, PCIe Gen 4 and newer SSDs larger than 30 TB may benefit from being split into two or more OSDs, because a single OSD daemon becomes the bottleneck before the device does. Cephadm only treats a device as available if it is at least 5 GB, unmounted, and free of partitions, LVM state and file systems.

To expand later, the guide to add an OSD to Ceph covers checking the device, previewing a cephadm specification and verifying the rebalance.

For hybrid nodes, the DB/WAL offload ratios are specific: one SSD partition per HDD OSD, four to five HDD OSDs per SATA SSD carrying DB/WAL, and no more than fifteen HDD OSDs per NVMe SSD. Exceeding those ratios turns the offload device into a shared bottleneck and a shared failure domain, since losing it takes down every OSD that depends on it. Size each block.db at no less than 2.5 percent of the HDD behind it, per the BlueStore reference, which is 250 GB for a 10 TB drive.

Monitors want about 100 GB per daemon, and the documentation strongly urges SSD for that, because monitor stores are latency-sensitive and a slow monitor store affects the whole cluster. Where a node has a single drive, keep the OS on a separate partition at minimum and on a separate drive by preference. Enterprise-grade media is called out specifically for production workloads, which is a polite way of saying consumer SSDs without power-loss protection are a poor fit for a system that fsyncs constantly.

Network: 10 Gb/s is the starting line

The recommendation is at least 10 Gb/s between Ceph hosts and between clients and the cluster, 25 Gb/s for substantial workloads, and 100 Gb/s links for dense nodes. Active/active bonding across separate switches is strongly recommended, both for throughput and to survive a switch failure or a maintenance window, with attention to whether the bonding hash policy actually spreads traffic.

The documentation frames the cost of getting this wrong as recovery time rather than throughput. Replicating 1 TiB across a 1 Gb/s network takes about three hours; 10 TiB takes about thirty. On a 10 Gb/s network the same transfers take about twenty minutes and about three hours. Recovery speed is a durability property, not a convenience one: the faster a placement group returns to active and clean, the smaller the window in which a second overlapping failure can cost data. That is why the 1 Gb/s minimum in the per-daemon guidance is a lab figure.

The Tentacle hardware page notes that a 40 Gb/s link is effectively four 10 Gb/s channels and a 100 Gb/s link four 25 Gb/s channels, so a single packet on 25 Gb/s can have slightly lower latency than on 40 Gb/s. Also budget a cheap separate 1 Gb/s path for out-of-band management, since baseboard management controllers rarely offer more than 1 Gb/s.

Ports and the optional cluster network

Only a public network is required, per the network configuration reference. Monitors listen on ports 3300 and 6789, other daemons bind within 6800 to 7568, and the docs say to open that whole range on each Ceph network.

Operating system and software requirements

Ceph’s software requirements are a supported Linux distribution plus the cephadm prerequisites on every host: Python 3, systemd, Podman or Docker, LVM2, and time synchronisation through chrony or the legacy ntpd.

For Tentacle 20.2, the OS recommendations list CentOS Stream 9 and Ubuntu 22.04 as packaged and fully tested, Rocky Linux 10 the same from 20.2.2, Debian 12 as packaged but untested, and Ubuntu 24.04 as upcoming. Squid 19.2 has an estimated end of life of 31 October 2026, so a new cluster belongs on Tentacle. Under cephadm the daemons run in containers, and the compatibility notes report no known issues with Podman 3.0 or newer from Quincy onward. Kernel RBD clients want kernel 5.3 or Enterprise Linux 8.2, and 4.19 long-term at the very least.

Clock discipline is not optional in a quorum-based system. Monitors raise a health warning once their clocks drift more than 0.05 seconds apart, and skewed monitor clocks are one of the recurring health warnings covered in diagnosing and clearing Ceph HEALTH_WARN.

Turning specifications into a bill of materials

Two numbers decide the shape of a first cluster: how much usable capacity is needed, and which durability scheme delivers it. Raw capacity is not usable capacity, and the gap is large. A three-replica pool has a space amplification factor of 3.0, so a third of raw capacity is usable. A 4+2 erasure-coded pool has a factor of 1.5, so two thirds is usable, at a real cost in write latency and CPU.

The Ceph erasure coding calculator takes an OSD count, a drive size and a redundancy profile and returns raw capacity, theoretical usable capacity and the sum of the default OSD memory targets. Use it to compare a proposed parts list with a capacity target before allowing for metadata and recovery headroom.

The mistakes that cost the most

Treating exactly three hosts as fault tolerant, when they only tolerate a failure rather than recover from one. Sizing RAM without the doubling rule. Replicating over 1 Gb/s, so recovery from a disk failure takes a day. Putting more than fifteen HDD OSDs behind one NVMe DB/WAL device. Buying consumer SSDs on price. Sharing one drive between the OS and the OSDs, which the storage documentation names as a common cause of slow OSD problems. And sizing capacity from raw terabytes rather than from the durability scheme actually configured.

FAQ

can you run ceph on a single node

Yes, for testing. Add --single-host-defaults to cephadm bootstrap and Ceph places copies on separate OSDs instead of separate hosts, with pools defaulting to two copies. That survives a failed drive but never a failed host, so use a one-node cluster to learn commands, not to keep data.

can ceph run on two nodes

Not safely on its own. Monitor quorum needs a majority, and a majority of two monitors is both, so losing either host loses quorum. Two hosts also cannot give the default three replicas a host each. Add a third machine running at least a monitor, which is what stretch mode does with its tiebreaker.

how much ram does a ceph node need

More than the number of OSDs times 4 GiB times two, plus the other daemons on the host. A twelve-OSD node therefore needs over 96 GiB before monitors, managers or metadata servers, which add at least 5 GB per monitor and 8 GiB per metadata server.

does ceph need a separate cluster network

No. Ceph’s network reference says it functions fine with only a public network in many deployments, especially on 25 Gb/s or faster links. A separate cluster network carries replication and recovery traffic. If you build one, keep it unreachable from the public network and open the OSD port range on both.