From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id ED3201FF0AA for ; Tue, 22 Sep 2026 13:06:08 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id A09CF21533; Tue, 22 Sep 2026 13:06:06 +0200 (CEST) Message-ID: Date: Tue, 22 Sep 2026 13:06:01 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [RFC proxmox-perl-rs/qemu-server/qemu-server-rs 0/9] pci-handling rewrite (part 1) From: Dominik Csapak To: pve-devel@lists.proxmox.com References: <20260922105550.2084078-1-d.csapak@proxmox.com> Content-Language: en-US In-Reply-To: <20260922105550.2084078-1-d.csapak@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1790075161388 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.461 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 3S7N6H5XB3VKZJTW6FMZVT2VACR4SE66 X-Message-ID-Hash: 3S7N6H5XB3VKZJTW6FMZVT2VACR4SE66 X-MailFrom: d.csapak@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: meh, accidentally included a fixup commit, should work regardless^^ On 9/22/26 12:56 PM, Dominik Csapak wrote: > # Motivation > > We want to extend the PCI layout mechanism to be easily extendable, especially > for NUMA awareness and replicating host-toplogy (e.g. see bug #7283 [0]). > > This is necessary for some performance gains in guests, especially with > high-end hardware where users pay extra for higher performance. > > # The Problem > > PCI addresses (and hardware layout in general) must be fixed for our qemu > guests, since this is part of the machine state and must be stable for > live-migration suspend/resume, etc. > > Currently, these addresses are hardcoded in perl hashes, with a long history of > devs (me included) piling onto new addresses on there, using whatever free > address there exists. > > This lead to a very convoluted and scattered assignment list, without any > underlying structure, which makes it harder the more new devices are added. > (Using a wrong address only surfaces when running some tests or starting a > guest with it) > > # My Proposed Solution > > So to fix the current situation I propose a plan in multiple phases: > > ## Phase 1 - untangling perl code to rust (this series) > > To extend/switch between multiple PCI layouts, we first must untangle the > current situation and make it easier to see what is actually used. > > I did this in rust, by using the type system to represent the PCI layout, and > inferring the addresses from the layout, not the other way around. > > That makes it easier to see where there is actually free space and where we can > add new things. > > It also highlights how messy the current layout is. (See the legacy layout in > the `pve-qemu-server-pci` crate) > > I also introduced here a generic `Machine` struct that's intended to hold > information useful to more call sites in contrast to passing a lot of arguments > everywhere, similar to the Cfg2Cmd class, but usable in the rust codebase. > > ## Phase 2 - extending pci layouts (partially this series) > > This would introduce a new layout ("v2") that can be used in an opt-in manner, > and is vastly more logically constructed, which makes it easy to extend and > argue about. See the "preview" patch (last of pve-qemu-server-rs) for how this > layout can look like. > > This part would also entail restructuring the addr calls in qemu-server's perl > code, so to only have a single entry point ("address" for example) instead of > splitting between 'print_pci_addr' and 'print_pcie_addr'. > > In this phase we could also emit the commandline for devices that are currently > in the q35 config files read with readconfig. > > ## Phase 3 - more perl to rust code port > > To better work with the layout, we'd need to pull more code from qemu-server > into rust, for this plan mostly from PCI.pm and USB.pm but maybe some other > helpers too (version comparsion comes to mind). > > Here we could start using the schema from pve-api-types to parse e.g. the > config rust-side, but the schema still has to live perl side (ofc). > > ## Phase 4 - Host and NUMA layouts > > With all the pieces in place, we can then easily extend the new layout by PCI > switches that are needed for assigning to NUMA nodes and replicating the host > toplogy for passed through devices. > > # Notes > > * pve-qemu-server-rs is a new git repo, but the crate in it could > easily be integrated somewhere else. Having this as a separate repo > could replace 'qemu-server' at one point though. > * This is an RFC, so comments about the general design/plan are desired. > * There are still some rough edges (e.g. hardcoding the perl config max values > in rust again), but it should work as intended and it passes our qemu-server > regression tests. > * Names and crate placement are not fixed, I know that some names I chose are > not optimal, if you do have better names/places, please suggest them. > * I based this on my recent qemu-server hotplug series [1], so keep that in > mind when testing/applying > * Opted to do this in rust for now, since I think that is the best way forward. > If the consensus is to to the layouting changes in perl, or keep some parts in > perl, that's fine with me too and I'll change my efforts accordingly > > 0: https://bugzilla.proxmox.com/show_bug.cgi?id=7283 > 1: https://lore.proxmox.com/pve-devel/20260914085725.1299009-1-d.csapak@proxmox.com/ > > > pve-qemu-server-rs: > > Dominik Csapak (4): > add pve-qemu-server-pci crate for guest PCI address generation > pci: add machine abstraction and PCI bridge generation > pci: layout: add v2 PCI and PCIe layouts > fixup! add pve-qemu-server-pci crate for guest PCI address generation > > > proxmox-perl-rs: > > Dominik Csapak (2): > pve: add bindings for `pve-qemu-server-pci` crate > pve: pci bindings: add bindings for the `Machine` struct > > pve-rs/Cargo.toml | 1 + > pve-rs/Makefile | 2 ++ > pve-rs/src/bindings/mod.rs | 3 ++ > pve-rs/src/bindings/pci/machine.rs | 45 ++++++++++++++++++++++++++++ > pve-rs/src/bindings/pci/mod.rs | 48 ++++++++++++++++++++++++++++++ > 5 files changed, 99 insertions(+) > create mode 100644 pve-rs/src/bindings/pci/machine.rs > create mode 100644 pve-rs/src/bindings/pci/mod.rs > > > qemu-server: > > Dominik Csapak (3): > pci: use PVE::RS::PCI bindings > helpers: factor out the version parts parsing > pci: bridges: use the rust `Machine` struct to pass parameters > > src/PVE/QemuServer.pm | 4 +- > src/PVE/QemuServer/Helpers.pm | 22 +- > src/PVE/QemuServer/PCI.pm | 296 +++-------------------- > src/test/Makefile | 5 +- > src/test/TestsCommon/CommandLineMocks.pm | 3 + > src/test/run_pci_addr_checks.pl | 141 ----------- > 6 files changed, 56 insertions(+), 415 deletions(-) > delete mode 100755 src/test/run_pci_addr_checks.pl > > > Summary over all repositories: > 11 files changed, 155 insertions(+), 415 deletions(-) >