[PVE-User] Max limits questions

[PVE-User] Max limits questions

[Anton Shevtsov]

Hi,

Where i can find information about below questions

VM

1. Max vCPU count per VM ?


Node host

1. How many maximum CPU cores are supported per node?

2. Max RAM are supported per node? (i think - unlimited, but..)



p.s. pve 7.0.6


--

Anton Shevtsov

Re: [PVE-User] Max limits questions

[Thomas Lamprecht]

Hi,

Am 16/11/2022 um 07:30 schrieb Anton Shevtsov:
> Where i can find information about below questions
> 
> VM
> 
> 1. Max vCPU count per VM ?

Proxmox VE places no limit on that, so basically how many the PVE host CPU has.
With more the KVM acceleration cannot work any more, so it gets rather slow and unpracticable.
> 
> Node host
> 
> 1. How many maximum CPU cores are supported per node?

Kernel dependent. For the current supported Proxmox VE 7.2's 5.15 kernel it's
8192 per CPU as you can find out by checking the kernel compile config:

grep NR_CPUS /boot/config-5.15.*-pve

Multi-Socket systems normally stop at 4 sockets, but we know a setup with 8 sockets
that uses Proxmox VE. So I guess 8 * 8192 = 65536

> 
> 2. Max RAM are supported per node? (i think - unlimited, but..)

No not unlimited, Linux currently supports 48- and 57-bit virtual addresses.
So, depending on your CPU/Mainboard/... it's something between 2^48 = 256 TiB
and 2^57 = 128 PiB.

In practice the limit will be currently in the range of 6 TiB for modern (last gen)
of Server CPUs. For example, with DDR5 you can get 512 GiB per DIMM and Epyc Genoa
CPUs support 12 channels, so 12 * 512 GiB = 6 TiB.

cheers,
Thomas

Re: [PVE-User] Max limits questions

[Anton Shevtsov]

Thomas, thanks for answers.

I've got one more question.

We plan to use approximately 80-100 nodes in pve cluster.

Will the cluster work be stable? Or is it unknown?


18.11.2022 16:16, Thomas Lamprecht пишет:
> Hi,
>
> Am 16/11/2022 um 07:30 schrieb Anton Shevtsov:
>> Where i can find information about below questions
>>
>> VM
>>
>> 1. Max vCPU count per VM ?
> Proxmox VE places no limit on that, so basically how many the PVE host CPU has.
> With more the KVM acceleration cannot work any more, so it gets rather slow and unpracticable.
>> Node host
>>
>> 1. How many maximum CPU cores are supported per node?
> Kernel dependent. For the current supported Proxmox VE 7.2's 5.15 kernel it's
> 8192 per CPU as you can find out by checking the kernel compile config:
>
> grep NR_CPUS /boot/config-5.15.*-pve
>
> Multi-Socket systems normally stop at 4 sockets, but we know a setup with 8 sockets
> that uses Proxmox VE. So I guess 8 * 8192 = 65536
>
>> 2. Max RAM are supported per node? (i think - unlimited, but..)
> No not unlimited, Linux currently supports 48- and 57-bit virtual addresses.
> So, depending on your CPU/Mainboard/... it's something between 2^48 = 256 TiB
> and 2^57 = 128 PiB.
>
> In practice the limit will be currently in the range of 6 TiB for modern (last gen)
> of Server CPUs. For example, with DDR5 you can get 512 GiB per DIMM and Epyc Genoa
> CPUs support 12 channels, so 12 * 512 GiB = 6 TiB.
>
> cheers,
> Thomas
>
-- 
basealt logo *Шевцов Антон Юрьевич*
/Отдел технического пресейла/
ООО Базальт СПО
мобильный : +79222651692
telegram : @anton_shevtsov
From tonci@suma-informatika.hr  Fri Nov 18 21:24:02 2022
Return-Path: <tonci@suma-informatika.hr>
X-Original-To: pve-user@pve.proxmox.com
Delivered-To: pve-user@pve.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id D60ED9EC6
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:24:02 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 838D87153
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:23:32 +0100 (CET)
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [IPv6:2a00:1450:4864:20::52b])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:23:27 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id x2so8713564edd.2
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 12:23:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=suma-informatika.hr; s=google;
 h=in-reply-to:references:to:from:content-language:subject:user-agent
 :mime-version:date:message-id:from:to:cc:subject:date:message-id
 :reply-to; bh=BpFOhL/pTQAVC2e1QA6G3gua1k530GNJj0kF/BfXLrw=;
 b=NAWZrMiC/L3OKtmXSZ5lPTGatUez9LdWPJQgr7XSXyPDIx+PY08tLhIF0ewA7OS/PD
 RV22XtbMoI+WSwvkGI3Jw+XTFOwWrKQFOzhBL5a1vhPZh9kQVC6d4d50YJi8sQZztBzd
 4sfmPC/Syz6p7uE2tw/9Z5y9b/Gw7f360yjUOH+WYKsMetZIUIvZXAVBGGIgvgaIJa3O
 DCKPJOLixZh1LEUPgytjJV17TZuxooufTO2zZW1es9i1KaHvsKouiUcHu5fyX8NoqoGS
 KvUWo//fRMaxDeXVA/K8FLaIM79E9U3qGHVGjxlecQxOofHUrSQyG5Fvl59qz7doSsfm
 VJrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=in-reply-to:references:to:from:content-language:subject:user-agent
 :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject
 :date:message-id:reply-to;
 bh=BpFOhL/pTQAVC2e1QA6G3gua1k530GNJj0kF/BfXLrw=;
 b=iVeahsL3OmKRouFgRR2t8FC+9U27iD8KTj/PKnk4X5LwUq1hyD7ogfz68cpg5v/7iO
 da4cJgBna1ck3YcBuSNB6VoV2rZbfeZEPRHL9rcoi0aQV9zDdcgkuXJfefTsNAdsryBu
 BwTeB0C00eDSJtTOxSrdkWDRyZLUekkEpTLMBZhzzdddLZsZTk7ixJ0FDv7q7CLEJknj
 ZXZMc2J5R9QtYQuLU9sipLPXZ5dFsM2ayxYJ8bsgoWi0T+X1MjhutZN51YzrKT4PCgVI
 bioqy0zg4XHbLugDOXsMYkSHYG10+JMqC+lv2vZrYzz2k3Y4H3a6yBPzs8LxQr3LkgMu
 0/Dw==
X-Gm-Message-State: ANoB5pmNoibo49yawr4dww5RKzOT91ptRXvcWKlUzaNfKWqk3D5cUmYY
 R8t3rIvh/I9Px5g9VX0r3rLP8FmwtgMxHA==
X-Google-Smtp-Source: AA0mqf4M44RPrQfrHgCFCl4FcsUvuOsDwksiHMp9LIo7wg9ORRtccSkBGrUXG8bPlAcWdCnF+q9X0A==
X-Received: by 2002:a05:6402:3226:b0:461:37c2:e85c with SMTP id
 g38-20020a056402322600b0046137c2e85cmr7349118eda.74.1668802999762; 
 Fri, 18 Nov 2022 12:23:19 -0800 (PST)
Received: from [10.10.21.51] (93-137-31-52.adsl.net.t-com.hr. [93.137.31.52])
 by smtp.gmail.com with ESMTPSA id
 fj9-20020a0564022b8900b004589da5e5cesm2171872edb.41.2022.11.18.12.23.18
 for <pve-user@pve.proxmox.com>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 18 Nov 2022 12:23:19 -0800 (PST)
Message-ID: <000c2ab6-ea94-1979-57af-c2f20ba47e2a@suma-informatika.hr>
Date: Fri, 18 Nov 2022 21:23:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Content-Language: en-US
From: =?UTF-8?B?VG9uxI1pIFN0aXBpxI1ldmnEhw==?= <tonci@suma-informatika.hr>
To: PVE User List <pve-user@pve.proxmox.com>
References: <8678ddf7-98ef-f87b-bb16-09ad4953804e@suma-informatika.hr>
 <20190708201918.00ca0f22@rosa.proxmox.com>
 <649afeba-9626-6688-7967-3ee604eea2bb@suma-informatika.hr>
 <ce32bc10-50c5-6d45-ad71-39d3e259ccb0@suma-informatika.hr>
In-Reply-To: <ce32bc10-50c5-6d45-ad71-39d3e259ccb0@suma-informatika.hr>
X-SPAM-LEVEL: Spam detection results:  0
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
 DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
 domain HTML_MESSAGE            0.001 HTML included in message
 RCVD_IN_DNSWL_NONE     -0.0001 Sender listed at https://www.dnswl.org/,
 no trust
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [tools.pm, suma-informatika.hr]
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
Subject: [PVE-User] PVE 7.2.-14 LXC boot problem
X-BeenThere: pve-user@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE user list <pve-user.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-user/>
List-Post: <mailto:pve-user@lists.proxmox.com>
List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Fri, 18 Nov 2022 20:24:02 -0000

Hello to all,
after today's upgrade from 7.2-11 to 7.2.14 LXC's won't boot

run_buffer: 321 Script exited with status 2
lxc_init: 847 Failed to run lxc.hook.pre-start for container "109"
__lxc_start: 2008 Failed to initialize container "109"
TASK ERROR: startup for container '109' failed

lxc-start 109 20221118200745.382 INFO confile - 
../src/lxc/confile.c:set_config_idmaps:2267 - Read uid map: type u nsid 
0 hostid 100000 range 65536
lxc-start 109 20221118200745.382 INFO confile - 
../src/lxc/confile.c:set_config_idmaps:2267 - Read uid map: type g nsid 
0 hostid 100000 range 65536
lxc-start 109 20221118200745.383 INFO lsm - 
../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security 
driver AppArmor
lxc-start 109 20221118200745.383 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "109", config 
section "lxc"
lxc-start 109 20221118200747.295 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-prestart-hook 109 lxc pre-start produced 
output: objdump on /lib/systemd/systemd failed: open3: exec of objdump 
-p /var/lib/lxc/109/rootfs/lib/systemd/systemd failed: No such file or 
directory at /usr/share/perl5/PVE/Tools.pm line 455.

lxc-start 109 20221118200747.321 ERROR conf - 
../src/lxc/conf.c:run_buffer:321 - Script exited with status 2
lxc-start 109 20221118200747.321 ERROR start - 
../src/lxc/start.c:lxc_init:847 - Failed to run lxc.hook.pre-start for 
container "109"
lxc-start 109 20221118200747.321 ERROR start - 
../src/lxc/start.c:__lxc_start:2008 - Failed to initialize container "109"
lxc-start 109 20221118200747.321 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxcfs/lxc.reboot.hook" for container "109", config section "lxc"
lxc-start 109 20221118200747.827 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "109", config 
section "lxc"
lxc-start 109 20221118200749.635 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-poststop-hook 109 lxc post-stop produced 
output: umount: /var/lib/lxc/.pve-staged-mounts/mp0: not mounted.

lxc-start 109 20221118200749.636 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-poststop-hook 109 lxc post-stop produced 
output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp0' failed: 
exit code 32

lxc-start 109 20221118200749.737 ERROR lxc_start - 
../src/lxc/tools/lxc_start.c:main:306 - The container failed to start
lxc-start 109 20221118200749.738 ERROR lxc_start - 
../src/lxc/tools/lxc_start.c:main:311 - Additional information can be 
obtained by setting the --logfile and --logpriority options


Host restart, restore from bck , ... nothing helps

so take caution ....

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr