From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-user-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id EC6B81FF16F
for <inbox@lore.proxmox.com>; Thu, 27 Mar 2025 10:22:46 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id B413C82DE;
Thu, 27 Mar 2025 10:22:38 +0100 (CET)
Date: Thu, 27 Mar 2025 10:22:27 +0100
To: Proxmox VE user list <pve-user@lists.proxmox.com>
MIME-Version: 1.0
Message-ID: <mailman.257.1743067357.359.pve-user@lists.proxmox.com>
List-Id: Proxmox VE user list <pve-user.lists.proxmox.com>
List-Post: <mailto:pve-user@lists.proxmox.com>
From: Eneko Lacunza via pve-user <pve-user@lists.proxmox.com>
Precedence: list
Cc: Eneko Lacunza <elacunza@binovo.es>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-user@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>,
<mailto:pve-user-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>,
<mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-user/>
Reply-To: Proxmox VE user list <pve-user@lists.proxmox.com>
List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help>
Subject: [PVE-User] Advice request for new cluster
Content-Type: multipart/mixed; boundary="===============6586035639958470795=="
Errors-To: pve-user-bounces@lists.proxmox.com
Sender: "pve-user" <pve-user-bounces@lists.proxmox.com>
--===============6586035639958470795==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <elacunza@binovo.es>
X-Original-To: pve-user@lists.proxmox.com
Delivered-To: pve-user@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id DE3E4CB46F
for <pve-user@lists.proxmox.com>; Thu, 27 Mar 2025 10:22:36 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id C87AC81F5
for <pve-user@lists.proxmox.com>; Thu, 27 Mar 2025 10:22:36 +0100 (CET)
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-user@lists.proxmox.com>; Thu, 27 Mar 2025 10:22:36 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-43cfba466b2so6720115e9.3
for <pve-user@lists.proxmox.com>; Thu, 27 Mar 2025 02:22:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=binovo.es; s=google; t=1743067349; x=1743672149; darn=lists.proxmox.com;
h=content-transfer-encoding:subject:content-language:from:to
:user-agent:mime-version:date:message-id:from:to:cc:subject:date
:message-id:reply-to;
bh=HeSreVOkk3NHBA4/fLEzsgqRqIGjEh1xB55kX0zMBA0=;
b=VsilgfQzCVNwM4DLxkQhc8F3a5cxYvUG7FyQsRgWCljeEUNAT8ACdJ0UsNYauOyzVp
BBqH0cM39POaVRAAEVwo3X2hhQzL3sHvc5/L08De5jMF7vxEhBx4E5wzNvOL/rE1o66J
MgIu5AQ4t76UDmrAVRJjaSAhRDQjdeFBGQbkQVxhvdXSv4wbiL26xHkVzL/Sv1rw9rEn
Z17otFul7lbWgH53PZz1vP9Pa3ULQ3htvDstBcjLtu2Zg7S3RfKG/0cl+LVeugtRxYE5
hzrBaZhchPAFVmoJe1JKKGML68sAzX2X5SoyIOl5uC/KLhZrXeQcdKQn98p7bKivBpfP
DR/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1743067349; x=1743672149;
h=content-transfer-encoding:subject:content-language:from:to
:user-agent:mime-version:date:message-id:x-gm-message-state:from:to
:cc:subject:date:message-id:reply-to;
bh=HeSreVOkk3NHBA4/fLEzsgqRqIGjEh1xB55kX0zMBA0=;
b=lTGqbom2NYiS5RQQWGHgmDbHcDKBO/ww0KLMtfl29Y5XsOPGhaAN/tCIl39n6yziBJ
QyFu8643q/b78Mvayn/Ol6/Kj15t50mal+0yCDXEt0WgqAJFn+fGSbLQ0PLDNfiXAXXg
9nXKeQcQzTO6fvU/iXEa2dtZ8R7r7wqFV2JRfan4mblXKvSkuavEpaJC9aTLcE4uCd5s
+5VeSqccPOF4R3ttKrnoQzFMok6wNV8Roflr/afxrxm/banHP62Z+5ap2cu+pAdjSEVH
IwIyWyDs0/sNPmdztzxehQ9qishl8Pd7qwrCC4zYDvYEQer2OVDUIcJRjOKVUe0CYWfv
WcSQ==
X-Gm-Message-State: AOJu0YwOyLJgB0oL6pOjsQizET85I4uZ0Yd4mqS6cuQvckvy/GRjUTmz
VCpLzTCs3P+kbiiuR5ixLc5Dg5hCsfnAHXyyKCWK3frwJ9qt/3yhRGZkAErLvr9PbHXEeBAdH2c
=
X-Gm-Gg: ASbGnctN7PHVdeqr4Jh4yeAmy/gVY8arPIJZ7APr+2w/GhPoiv2UDv4OEgXWOGQJ5RB
zfTLfsB5pb7APfJ9SogmHa0yF/njge1KVSnruyFAjpUpPwuW+wgCxbd7ZLY3RNIqGDjer17OVpP
rremFN8L0pttcHFjwf057yKD0a4TvX76PMqzj1ANZIfq1pmF9OIS1hlIseZTQOUo3bAszFFkdV+
j2Fqiytptgpm4XCUTWDyLsORkSZwaO5iEJOaUiDBKG4JMaNW8oM+MK33urKcLRjQz33J5Cwz3wS
LEhh7EPB8uK/z5qujkGESfyQEzb1KL7NwbD2H6emaN7Ra8Hdb5yYRltiLo54RD66VsjTBgu3bZo
asjxXPYsLCh1HEq2LVQ==
X-Google-Smtp-Source: AGHT+IHxK0orvh4+Vx0CzQuKP2+Kl0o6CaTa4MOJ/atIVGQJzE+LoBFKpOIzmAK+rk/4kqQ0Mf31kQ==
X-Received: by 2002:a05:600c:1d9e:b0:43c:f75a:eb54 with SMTP id 5b1f17b1804b1-43d8582c4ddmr22629085e9.13.1743067349084;
Thu, 27 Mar 2025 02:22:29 -0700 (PDT)
Received: from [172.29.37.147] (249.red-213-96-132.staticip.rima-tde.net. [213.96.132.249])
by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d82f1bb80sm31323235e9.28.2025.03.27.02.22.28
for <pve-user@lists.proxmox.com>
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Thu, 27 Mar 2025 02:22:28 -0700 (PDT)
Message-ID: <1adce0d3-b1cb-4cf3-85da-4f37c6a31ca1@binovo.es>
Date: Thu, 27 Mar 2025 10:22:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Proxmox VE user list <pve-user@lists.proxmox.com>
From: Eneko Lacunza <elacunza@binovo.es>
Content-Language: es-CO
Subject: Advice request for new cluster
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.130 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
DMARC_PASS -0.1 DMARC pass policy
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Hi all,
We're in the process of designing and procuring a new PVE HCI cluster.
We will service about 6,000 tenants, each running two docker containers,
one with PostgreSQL and the other a python web application.
Containers will be hosted in VMs, 250 tenants per VM. VMs will have
512GB RAM and 16-32 vcores. 4 VMs per server.
Based on current data, we expect to be memory constrained; CPU,
networking and disk (Ceph) not being our main concern.
We're looking to optimize our cost per GB of RAM, so our current plan is
to deploy 6 servers with 2,3TB RAM and 2 EPYC CPU sockets, each with 48
cores, 4x25 Gbps network and 3x7,68T disks for Ceph. The more expensive
alternative would be to deploy 12x 1,15TB RAM 1 EPYC servers (+30%
adquisition and about +40% running costs)
I have been reading reports about NUMA performance issues on Proxmox
mailing lists and elsewhere. Memory bandwith issues for example.
Based on what I understood, and seeing that we'll have more than 2,000
not very demanding containers in each server, I think those NUMA issues
shouldn't be a problem in our use case.
I'll be very grateful on any suggestion or comment about this NUMA issue
and our cluster design.
Cheers
Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project
Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun
https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/
--===============6586035639958470795==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
--===============6586035639958470795==--