Re: [PVE-User] Max limits questions

[Anton Shevtsov <shevtsovay@basealt.ru>]

Thomas, thanks for answers.

I've got one more question.

We plan to use approximately 80-100 nodes in pve cluster.

Will the cluster work be stable? Or is it unknown?


18.11.2022 16:16, Thomas Lamprecht пишет:
> Hi,
>
> Am 16/11/2022 um 07:30 schrieb Anton Shevtsov:
>> Where i can find information about below questions
>>
>> VM
>>
>> 1. Max vCPU count per VM ?
> Proxmox VE places no limit on that, so basically how many the PVE host CPU has.
> With more the KVM acceleration cannot work any more, so it gets rather slow and unpracticable.
>> Node host
>>
>> 1. How many maximum CPU cores are supported per node?
> Kernel dependent. For the current supported Proxmox VE 7.2's 5.15 kernel it's
> 8192 per CPU as you can find out by checking the kernel compile config:
>
> grep NR_CPUS /boot/config-5.15.*-pve
>
> Multi-Socket systems normally stop at 4 sockets, but we know a setup with 8 sockets
> that uses Proxmox VE. So I guess 8 * 8192 = 65536
>
>> 2. Max RAM are supported per node? (i think - unlimited, but..)
> No not unlimited, Linux currently supports 48- and 57-bit virtual addresses.
> So, depending on your CPU/Mainboard/... it's something between 2^48 = 256 TiB
> and 2^57 = 128 PiB.
>
> In practice the limit will be currently in the range of 6 TiB for modern (last gen)
> of Server CPUs. For example, with DDR5 you can get 512 GiB per DIMM and Epyc Genoa
> CPUs support 12 channels, so 12 * 512 GiB = 6 TiB.
>
> cheers,
> Thomas
>
-- 
basealt logo *Шевцов Антон Юрьевич*
/Отдел технического пресейла/
ООО Базальт СПО
мобильный : +79222651692
telegram : @anton_shevtsov
From tonci@suma-informatika.hr  Fri Nov 18 21:24:02 2022
Return-Path: <tonci@suma-informatika.hr>
X-Original-To: pve-user@pve.proxmox.com
Delivered-To: pve-user@pve.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by lists.proxmox.com (Postfix) with ESMTPS id D60ED9EC6
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:24:02 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
 by firstgate.proxmox.com (Proxmox) with ESMTP id 838D87153
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:23:32 +0100 (CET)
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [IPv6:2a00:1450:4864:20::52b])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by firstgate.proxmox.com (Proxmox) with ESMTPS
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 21:23:27 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id x2so8713564edd.2
 for <pve-user@pve.proxmox.com>; Fri, 18 Nov 2022 12:23:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=suma-informatika.hr; s=google;
 h=in-reply-to:references:to:from:content-language:subject:user-agent
 :mime-version:date:message-id:from:to:cc:subject:date:message-id
 :reply-to; bh=BpFOhL/pTQAVC2e1QA6G3gua1k530GNJj0kF/BfXLrw=;
 b=NAWZrMiC/L3OKtmXSZ5lPTGatUez9LdWPJQgr7XSXyPDIx+PY08tLhIF0ewA7OS/PD
 RV22XtbMoI+WSwvkGI3Jw+XTFOwWrKQFOzhBL5a1vhPZh9kQVC6d4d50YJi8sQZztBzd
 4sfmPC/Syz6p7uE2tw/9Z5y9b/Gw7f360yjUOH+WYKsMetZIUIvZXAVBGGIgvgaIJa3O
 DCKPJOLixZh1LEUPgytjJV17TZuxooufTO2zZW1es9i1KaHvsKouiUcHu5fyX8NoqoGS
 KvUWo//fRMaxDeXVA/K8FLaIM79E9U3qGHVGjxlecQxOofHUrSQyG5Fvl59qz7doSsfm
 VJrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=in-reply-to:references:to:from:content-language:subject:user-agent
 :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject
 :date:message-id:reply-to;
 bh=BpFOhL/pTQAVC2e1QA6G3gua1k530GNJj0kF/BfXLrw=;
 b=iVeahsL3OmKRouFgRR2t8FC+9U27iD8KTj/PKnk4X5LwUq1hyD7ogfz68cpg5v/7iO
 da4cJgBna1ck3YcBuSNB6VoV2rZbfeZEPRHL9rcoi0aQV9zDdcgkuXJfefTsNAdsryBu
 BwTeB0C00eDSJtTOxSrdkWDRyZLUekkEpTLMBZhzzdddLZsZTk7ixJ0FDv7q7CLEJknj
 ZXZMc2J5R9QtYQuLU9sipLPXZ5dFsM2ayxYJ8bsgoWi0T+X1MjhutZN51YzrKT4PCgVI
 bioqy0zg4XHbLugDOXsMYkSHYG10+JMqC+lv2vZrYzz2k3Y4H3a6yBPzs8LxQr3LkgMu
 0/Dw==
X-Gm-Message-State: ANoB5pmNoibo49yawr4dww5RKzOT91ptRXvcWKlUzaNfKWqk3D5cUmYY
 R8t3rIvh/I9Px5g9VX0r3rLP8FmwtgMxHA==
X-Google-Smtp-Source: AA0mqf4M44RPrQfrHgCFCl4FcsUvuOsDwksiHMp9LIo7wg9ORRtccSkBGrUXG8bPlAcWdCnF+q9X0A==
X-Received: by 2002:a05:6402:3226:b0:461:37c2:e85c with SMTP id
 g38-20020a056402322600b0046137c2e85cmr7349118eda.74.1668802999762; 
 Fri, 18 Nov 2022 12:23:19 -0800 (PST)
Received: from [10.10.21.51] (93-137-31-52.adsl.net.t-com.hr. [93.137.31.52])
 by smtp.gmail.com with ESMTPSA id
 fj9-20020a0564022b8900b004589da5e5cesm2171872edb.41.2022.11.18.12.23.18
 for <pve-user@pve.proxmox.com>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 18 Nov 2022 12:23:19 -0800 (PST)
Message-ID: <000c2ab6-ea94-1979-57af-c2f20ba47e2a@suma-informatika.hr>
Date: Fri, 18 Nov 2022 21:23:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Content-Language: en-US
From: =?UTF-8?B?VG9uxI1pIFN0aXBpxI1ldmnEhw==?= <tonci@suma-informatika.hr>
To: PVE User List <pve-user@pve.proxmox.com>
References: <8678ddf7-98ef-f87b-bb16-09ad4953804e@suma-informatika.hr>
 <20190708201918.00ca0f22@rosa.proxmox.com>
 <649afeba-9626-6688-7967-3ee604eea2bb@suma-informatika.hr>
 <ce32bc10-50c5-6d45-ad71-39d3e259ccb0@suma-informatika.hr>
In-Reply-To: <ce32bc10-50c5-6d45-ad71-39d3e259ccb0@suma-informatika.hr>
X-SPAM-LEVEL: Spam detection results:  0
 BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
 DKIM_SIGNED               0.1 Message has a DKIM or DK signature,
 not necessarily valid
 DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
 DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's
 domain
 DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from
 domain HTML_MESSAGE            0.001 HTML included in message
 RCVD_IN_DNSWL_NONE     -0.0001 Sender listed at https://www.dnswl.org/,
 no trust
 SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
 SPF_PASS               -0.001 SPF: sender matches SPF record
 URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
 http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
 information. [tools.pm, suma-informatika.hr]
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
Subject: [PVE-User] PVE 7.2.-14 LXC boot problem
X-BeenThere: pve-user@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE user list <pve-user.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-user/>
List-Post: <mailto:pve-user@lists.proxmox.com>
List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, 
 <mailto:pve-user-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Fri, 18 Nov 2022 20:24:02 -0000

Hello to all,
after today's upgrade from 7.2-11 to 7.2.14 LXC's won't boot

run_buffer: 321 Script exited with status 2
lxc_init: 847 Failed to run lxc.hook.pre-start for container "109"
__lxc_start: 2008 Failed to initialize container "109"
TASK ERROR: startup for container '109' failed

lxc-start 109 20221118200745.382 INFO confile - 
../src/lxc/confile.c:set_config_idmaps:2267 - Read uid map: type u nsid 
0 hostid 100000 range 65536
lxc-start 109 20221118200745.382 INFO confile - 
../src/lxc/confile.c:set_config_idmaps:2267 - Read uid map: type g nsid 
0 hostid 100000 range 65536
lxc-start 109 20221118200745.383 INFO lsm - 
../src/lxc/lsm/lsm.c:lsm_init_static:38 - Initialized LSM security 
driver AppArmor
lxc-start 109 20221118200745.383 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "109", config 
section "lxc"
lxc-start 109 20221118200747.295 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-prestart-hook 109 lxc pre-start produced 
output: objdump on /lib/systemd/systemd failed: open3: exec of objdump 
-p /var/lib/lxc/109/rootfs/lib/systemd/systemd failed: No such file or 
directory at /usr/share/perl5/PVE/Tools.pm line 455.

lxc-start 109 20221118200747.321 ERROR conf - 
../src/lxc/conf.c:run_buffer:321 - Script exited with status 2
lxc-start 109 20221118200747.321 ERROR start - 
../src/lxc/start.c:lxc_init:847 - Failed to run lxc.hook.pre-start for 
container "109"
lxc-start 109 20221118200747.321 ERROR start - 
../src/lxc/start.c:__lxc_start:2008 - Failed to initialize container "109"
lxc-start 109 20221118200747.321 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxcfs/lxc.reboot.hook" for container "109", config section "lxc"
lxc-start 109 20221118200747.827 INFO conf - 
../src/lxc/conf.c:run_script_argv:337 - Executing script 
"/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "109", config 
section "lxc"
lxc-start 109 20221118200749.635 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-poststop-hook 109 lxc post-stop produced 
output: umount: /var/lib/lxc/.pve-staged-mounts/mp0: not mounted.

lxc-start 109 20221118200749.636 DEBUG conf - 
../src/lxc/conf.c:run_buffer:310 - Script exec 
/usr/share/lxc/hooks/lxc-pve-poststop-hook 109 lxc post-stop produced 
output: command 'umount -- /var/lib/lxc/.pve-staged-mounts/mp0' failed: 
exit code 32

lxc-start 109 20221118200749.737 ERROR lxc_start - 
../src/lxc/tools/lxc_start.c:main:306 - The container failed to start
lxc-start 109 20221118200749.738 ERROR lxc_start - 
../src/lxc/tools/lxc_start.c:main:311 - Additional information can be 
obtained by setting the --logfile and --logpriority options


Host restart, restore from bck , ... nothing helps

so take caution ....

srdačan pozdrav / best regards

Tonči Stipičević, dipl. ing. elektr.
direktor / manager

SUMA Informatika d.o.o., Badalićeva 27, OIB 93926415263

Podrška / Upravljanje IT sustavima za male i srednje tvrtke
Small & Medium Business IT Support / Management

mob: 091 1234003
www.suma-informatika.hr