From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 3E1BB91766 for ; Thu, 2 Feb 2023 12:03:58 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 24A70B055 for ; Thu, 2 Feb 2023 12:03:58 +0100 (CET) Received: from bastionodiso.odiso.net (bastionodiso.odiso.net [IPv6:2a0a:1580:2000::2d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 2 Feb 2023 12:03:56 +0100 (CET) Received: from kvmformation3.odiso.net (formationkvm3.odiso.net [10.3.94.12]) by bastionodiso.odiso.net (Postfix) with ESMTP id 04C258103; Thu, 2 Feb 2023 12:03:50 +0100 (CET) Received: by kvmformation3.odiso.net (Postfix, from userid 0) id E761622A5A2; Thu, 2 Feb 2023 12:03:49 +0100 (CET) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Date: Thu, 2 Feb 2023 12:03:31 +0100 Message-Id: <20230202110344.840195-1-aderumier@odiso.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.028 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% HEADER_FROM_DIFFERENT_DOMAINS 0.25 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods NO_DNS_FOR_FROM 0.001 Envelope sender has no MX or A DNS records SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [qemuconfig.pm, helpers.pm, qemuserver.pm, qemumigrate.pm, pci.pm, qemu.pm, memory.pm] Subject: [pve-devel] [PATCH v3 qemu-server 00/13] rework memory hotplug + virtiomem X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2023 11:03:58 -0000 This patch series rework the current memory hotplug + virtiomem. memory option now have extra options: memory: [[current=]] [,max=] [,virtio=<1|0>] ex: memory: current=1024,max=131072,virtio=1 patches 1-4: add a memory parser patches 5-10: add the max option with 64 static dimm hotplug for classic memory hotplug, when maxmemory is defined, we use 64 fixed size dimm. The max option is a multiple of 64GB. patches 11-13: add virtio-mem The virtio option enable new virtio-mem support, instead of plugging dimm, it's add/removed block inside big dimm. virtio-mem can use 32000 blocks, the blocksize is compute from max memory. Changelog v2: update differents patches based on Fiona comments. (I have send 2 others mails with comments not yet addressed) Biggest change is on virtio-mem, instead of trying to have same amount of memory on each virtiomem (some block could be unmovable and break unplug), we try to balance/dispatch remaining block on other available virtiomems. Also, the minimum blocksize supported by linux guest os is 4MB currently, even if virtiomem can use 2MB on qemu side. Patch10 with hotplug fix has be merged in others patches. Changelog v3: - cleanup from last Fiona comments - classic hotplug : fix memory unplug - virtio-mem: split retry logic in a second optional patch (feel free to apply it later if needed) Alexandre Derumier (13): memory: extract some code to their own sub for mocking tests: add memory tests qemu_memory_hotplug: remove unused $opt arg add memory parser memory: add get_static_mem && remove parse_hotplug_features config: memory: add 'max' option memory: get_max_mem: use config memory max memory: don't use foreach_reversedimm for unplug memory: use 64 slots && static dimm size when max is defined test: add memory-max tests memory: add virtio-mem support memory: virtio-mem : implement redispatch retry. tests: add virtio-mem tests PVE/API2/Qemu.pm | 45 +- PVE/QemuConfig.pm | 4 +- PVE/QemuMigrate.pm | 6 +- PVE/QemuServer.pm | 39 +- PVE/QemuServer/Helpers.pm | 3 +- PVE/QemuServer/Memory.pm | 508 ++++++++++++++---- PVE/QemuServer/PCI.pm | 8 + test/cfg2cmd/memory-hotplug-hugepages.conf | 12 + .../cfg2cmd/memory-hotplug-hugepages.conf.cmd | 62 +++ test/cfg2cmd/memory-hotplug.conf | 11 + test/cfg2cmd/memory-hotplug.conf.cmd | 174 ++++++ test/cfg2cmd/memory-hugepages-1g.conf | 11 + test/cfg2cmd/memory-hugepages-1g.conf.cmd | 30 ++ test/cfg2cmd/memory-hugepages-2m.conf | 11 + test/cfg2cmd/memory-hugepages-2m.conf.cmd | 30 ++ test/cfg2cmd/memory-max-128G.conf | 11 + test/cfg2cmd/memory-max-128G.conf.cmd | 86 +++ test/cfg2cmd/memory-max-512G.conf | 11 + test/cfg2cmd/memory-max-512G.conf.cmd | 58 ++ test/cfg2cmd/memory-virtio-hugepages-1G.conf | 12 + .../memory-virtio-hugepages-1G.conf.cmd | 35 ++ test/cfg2cmd/memory-virtio-max.conf | 11 + test/cfg2cmd/memory-virtio-max.conf.cmd | 35 ++ test/cfg2cmd/memory-virtio.conf | 11 + test/cfg2cmd/memory-virtio.conf.cmd | 35 ++ test/run_config2command_tests.pl | 15 + 26 files changed, 1125 insertions(+), 149 deletions(-) create mode 100644 test/cfg2cmd/memory-hotplug-hugepages.conf create mode 100644 test/cfg2cmd/memory-hotplug-hugepages.conf.cmd create mode 100644 test/cfg2cmd/memory-hotplug.conf create mode 100644 test/cfg2cmd/memory-hotplug.conf.cmd create mode 100644 test/cfg2cmd/memory-hugepages-1g.conf create mode 100644 test/cfg2cmd/memory-hugepages-1g.conf.cmd create mode 100644 test/cfg2cmd/memory-hugepages-2m.conf create mode 100644 test/cfg2cmd/memory-hugepages-2m.conf.cmd create mode 100644 test/cfg2cmd/memory-max-128G.conf create mode 100644 test/cfg2cmd/memory-max-128G.conf.cmd create mode 100644 test/cfg2cmd/memory-max-512G.conf create mode 100644 test/cfg2cmd/memory-max-512G.conf.cmd create mode 100644 test/cfg2cmd/memory-virtio-hugepages-1G.conf create mode 100644 test/cfg2cmd/memory-virtio-hugepages-1G.conf.cmd create mode 100644 test/cfg2cmd/memory-virtio-max.conf create mode 100644 test/cfg2cmd/memory-virtio-max.conf.cmd create mode 100644 test/cfg2cmd/memory-virtio.conf create mode 100644 test/cfg2cmd/memory-virtio.conf.cmd -- 2.30.2