From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 186BD1FF179 for ; Wed, 29 Oct 2025 10:26:57 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4F45A46C0; Wed, 29 Oct 2025 10:27:30 +0100 (CET) To: pve-devel@lists.proxmox.com Date: Wed, 29 Oct 2025 10:18:09 +0100 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alwin Antreich via pve-devel Precedence: list Cc: Alwin Antreich X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH docs v2 1/3] pveceph: update OSD memory considerations Content-Type: multipart/mixed; boundary="===============3242105296436955985==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============3242105296436955985== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7BC2FD6229 for ; Wed, 29 Oct 2025 10:27:28 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5E78A45CA for ; Wed, 29 Oct 2025 10:26:58 +0100 (CET) Received: from mx.antreich.com (mx.antreich.com [173.249.42.230]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 29 Oct 2025 10:26:57 +0100 (CET) Received: from mx.antreich.com (localhost [127.0.0.1]) by mx.antreich.com (Proxmox) with ESMTP id 0D872760F55 for ; Wed, 29 Oct 2025 10:18:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=antreich.com; h= cc:content-transfer-encoding:date:from:from:message-id :mime-version:reply-to:subject:subject:to:to; s=2025; bh=VhCE0GH M8CCrskzshw+WGJr6qppZjFHNXnkwvl6hY24=; b=oJ1Ns7w/cfPyiWj0ylqySoS 9qa/qpVwPUYR0pnotLt272hSHZ89Kg4qDlVfdaescq81oKpnmbrqX2T+pk05At6z HD+0JevdVcTEi+mFzbdvxJFzvFH8KrRZE4S40I+QXqMs8Csr+JDKkiMSAqP8cwXm 4ULGFGpRTqcKxd4axMQTNFRyohwGmLwXKN56WXjtC3YOiC1d2BpA8ezpl8b8Txfn vUpfmbzZ2VQp+FeBmjPogmdcf2CZPwz51ao8zGt/qkdLkF42USqvgsOXfM6ZFJ/u SE4FB7q6jpl8ykVJX254BQbXwK+MAqH8+WL8GZbuwcujRtEDP7yFinlDglMrD9g= = From: Alwin Antreich To: pve-devel@lists.proxmox.com Subject: [PATCH docs v2 1/3] pveceph: update OSD memory considerations Date: Wed, 29 Oct 2025 10:18:09 +0100 Message-Id: <20251029091811.2800592-1-alwin@antreich.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.075 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Since bluestore, OSDs adhere to the osd_memory_target and the recommended amount of memory was increased. See: https://docs.ceph.com/en/reef/start/hardware-recommendations/#ram Signed-off-by: Alwin Antreich --- pveceph.adoc | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/pveceph.adoc b/pveceph.adoc index 17efa4d..1ed6666 100644 --- a/pveceph.adoc +++ b/pveceph.adoc @@ -131,14 +131,17 @@ carefully planned out and monitored. In addition to the predicted memory usage of virtual machines and containers, you must also account for having enough memory available for Ceph to provide excellent and stable performance. -As a rule of thumb, for roughly **1 TiB of data, 1 GiB of memory** will be used -by an OSD. While the usage might be less under normal conditions, it will use -most during critical operations like recovery, re-balancing or backfilling. -That means that you should avoid maxing out your available memory already on -normal operation, but rather leave some headroom to cope with outages. - -The OSD service itself will use additional memory. The Ceph BlueStore backend of -the daemon requires by default **3-5 GiB of memory** (adjustable). +While usage may be less under normal conditions, it will consume more memory +during critical operations, such as recovery, rebalancing, or backfilling. That +means you should avoid maxing out your available memory already on regular +operation, but rather leave some headroom to cope with outages. + +The current recommendation is to configure OSDs with at least **8 GiB of +memory** for good performance. The OSD daemon requires 4 GiB by default. + +NOTE: Total memory consumption is primarily determined by the number of OSD +daemons. However, for critical operations, all Ceph services may require +additional memory. [[pve_ceph_recommendation_network]] .Network -- 2.39.5 --===============3242105296436955985== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3242105296436955985==--