From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id EC7901FF0A7 for ; Wed, 02 Sep 2026 13:29:27 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id BE2382141F; Wed, 02 Sep 2026 13:29:27 +0200 (CEST) Message-ID: <29570235-53c4-409e-82b6-36425c5ac092@proxmox.com> Date: Wed, 2 Sep 2026 13:29:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Stefan Hanreich Subject: Re: [RFC] VMware/NSX to Proxmox migration: policy transformation and staged network handoff To: LIONNETON Vincent , "pve-devel@lists.proxmox.com" References: Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.946 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: VIJ47MGW7DXECVKD4IR6X2GINUHFURAT X-Message-ID-Hash: VIJ47MGW7DXECVKD4IR6X2GINUHFURAT X-MailFrom: s.hanreich@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: "pdm-devel@lists.proxmox.com" X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Thanks for your inquiry, I tried to keep the answers relatively short without diving too much into technical details. On 8/31/26 9:11 AM, LIONNETON Vincent wrote: [snip] > Before going further, particularly with same-address coexistence and network handoff, I would appreciate Proxmox's guidance on the following points: > > > 1. Is NSX-aware migration already planned or being worked on within Proxmox, publicly or internally? > > In particular, are there plans around NSX DFW/security-policy migration, NSX network preservation, or staged migration from NSX networking to PVE SDN? There are currently no plans for extending the importer functionality to migrating NSX objects / entities, so efforts on developing something like that would be welcome. > 2. Do you see this functionality primarily as an external migration/transformation tool using PVE/PDM APIs, or do some of these functions belong in PVE or PDM themselves? > > We explicitly do not want to develop a competing permanent SDN/firewall control plane if PDM is intended to provide that functionality. Optimally, the functionality is implemented natively in PVE / PDM and then only managed by the external tool. For your purposes, the most interesting things that are still in development are VRF support and microsegmentation, which both have a RFC on the mailing list. Looking at those should give you a good glimpse into our development workflow and the current codebase. > 3. For an external migration tool, what would you consider the preferred and sufficiently stable API integration points? > > In particular for VM migration, SDN/VNet provisioning, EVPN/BGP configuration, Security Groups/IP sets, guest/VNet firewall policies and multi-cluster orchestration through PDM. The whole REST API, as described in the documentation [1], is considered to be stable and does not introduce breaking changes during a major version. Our policy on what we deem a breaking API change and how we treat evolving the API can be found in our Wiki [2]. > 4. Would effective NSX-policy translation and policy-parity validation be useful functionality from Proxmox's perspective? > > For example, resolving dynamic NSX group memberships and the effective DFW policy for a VM, compiling that intent into native PVE firewall objects, and comparing the resulting effective policy before cutover. For that particular purpose, the ongoing work on microsegmentation is likely of interest to you [3] as that should provide the functionality required. As it's in a stage of development, feel free to chime in and test the functionality - we always value feedback and in this early stage it's a good opportunity for you to check out the implementation to see if it fits the use-case. > 5. Does the proposed same-address migration model fit the intended PVE SDN architecture? > > The idea is to temporarily extend the existing NSX L2 network to PVE, migrate the workloads while keeping IP/gateway addressing unchanged, and only later hand the gateway/routing function over from NSX to PVE EVPN/BGP on a network-by-network basis. That should work generally and I know of some customers that have done a similar setup, but I personally have zero practical experience with this - so it's hard for me to talk about details. Since standard routing protocols are involved the integration should work, but I can imagine needing something like a translation layer, i.e. when using a route reflector for both PVE / NSX-T the routes might need to get transformed by the reflector to match what the respective stacks expect. Potentially even a router in front of the PVE cluster, that handles NSX-T / Proxmox VE specifics during the migration. With the recently introduced route maps feature, this might be possible on the PVE side via the SDN stack solely. There's very likely a few pitfalls that'd need to be worked out. Ir's very likely that some setups are currently impossible to model with the SDN stack alone and an external router or custom FRR configuration is needed - although we're working hard on addressing those deficiencies. I'd start with attempting to migrate relatively simple setups (e.g. single-tenant) and see how everything works out for simpler setups. > 6. Is low-downtime/incremental VMware migration already in scope for the existing importer? > > We are particularly interested in whether a VDDK/CBT-style precopy/delta mechanism, or another warm-migration mechanism from ESXi, is already planned. There has been some work done internally on improving the importer by utilizing NFC/CBT, but it still needs a bit of work to drag it over the finish line, I cannot provide an ETA at this point. > 7. Are there generic PVE/PDM capabilities that would be useful to implement upstream for this use case? > > Possible examples are effective-firewall-policy inspection, policy diff/validation, dry-run APIs, structured external-migration prechecks, or bulk firewall/IP-set operations. Yes, I would see all of those capabilities as something that should be provided by the Proxmox VE API and then consumed by your tool. The firewall currently does not support staging changes, as the configuration is applied continously by the firewall daemon. SDN itself already has a dry-run API that has been recently introduced [4] and the related endpoints all support querying the current / pending configuration via the pending / running GET parameters. I'm currently working on improving firewall configuration handling together with a colleague, but the general mechanism stay the same. For the use-cases you mentioned that would require implementing some sort of staging mechanism for firewall changes. I'm not sure what you mean with bulk firewall / IP-set operations exactly? > 8. If some of these pieces would be appropriate upstream, where would you prefer them to live and what development interfaces/conventions should we target? > > For example PVE vs PDM, the appropriate repositories and APIs, Rust vs existing PVE components, expected test coverage and contribution workflow. Development of Proxmox VE (and other products) happens out here in the open on the respective mailing lists. We have a short introductory guide to our workflow in the Wiki [5]. Discussion of specific features usually happens in our bug tracking instance [6], rather than here on the mailing list directly. For the SDN stack in particular, new features are implemented in Rust and only exposed via our perlmod module, which allows calling into Rust code. The new firewall daemon is implemented in Rust as well, but the API still lives on the Perl side. That might change in the future though, as we plan on implementing more and more new features in Rust while also porting existing functionality over where it makes sense. In the short to mid-term we also want to work on eliminating the need for Perl to act as a middle man for Rust. > intention would be to keep all VMware/NSX-specific discovery and transformation logic in the migration tooling, while contributing only genuinely generic Proxmox capabilities upstream if that is of interest. That would be the preferred way, in my opinion. > Once the migration is complete, the resulting environment should consist solely of native PVE/PDM SDN and firewall configuration and should not depend on the migration tool for normal operation. Yes, I think there are two phases to this? First, the network topology / structure would need to get set up on PVE (and hooked up with its NSX-T counterparts) - then the migration of the VMs and their network configuration, which maps the guest configuration from VMWare onto the SDN entities. The tool would need to maintain some form of mapping in that case / or it'd need to be stored in the SDN stack directly - so the tool can consume that (which is preferable imo). > I would be happy to share a more detailed architecture or the results of the current proof of concept if useful. Yes, I'd be interested in that! [1] https://pve.proxmox.com/pve-docs/api-viewer/ [2] https://pve.proxmox.com/wiki/Proxmox_VE_API#API_Stability_&_Breakage [3] https://lore.proxmox.com/all/DKRAX8JGPAP2.19JLE3T7U6DS1@proxmox.com/ [4] https://pve.proxmox.com/pve-docs/api-viewer/#/cluster/sdn/dry-run [5] https://pve.proxmox.com/wiki/Developer_Documentation [6] https://bugzilla.proxmox.com/