From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 368851FF09F for ; Thu, 03 Sep 2026 14:46:54 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id D9448215E9; Thu, 03 Sep 2026 14:46:49 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Pkx7sPH58Hsok0B/mbMFqcmo6tOmV3jb2QapsA3F2gOBt2fMEmsQ0eTRxlY8ovvnUWYKE5enFN3S/809JLlH65BVwZzorkwvW9CBsLuVSk5vp6Ob2ZV3oYPD1mNDyB6lJd/8tsZb8DSEi26ksNylGK0GgZYPsADatiwCzUsHLtl23l0FwnOu6qq96ZIbS3yPDGDKzZimeliONNm/Ac1le74ViPyIgap8qF5YOdCXvenNMZTPEi5LSofZ3gxRn3IrKWtOh7C808aIwtQtUVG6u9VmevYM8Pjfxd1d0KC0kCpLpbwvYEcnTGrEpaTg/uoEBtDWw+TGo3N+JpsoxrLVLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1VCDKun5EWh/4p3GOhVluSEw37zl+fBR95U3rygqjQ4=; b=lHYRo3tcCvjmSQlVlahIva03vszCOvOnKtBR6GjWW+4xuAC+rSg97Q3HXyUpPia+1RIW2WGViBL6+6cjNmujcz0FiSwIEuAD3/ac2jm44T3YKwhJgfnVQ5m5NPNm0nvlmPTrEqmNxM5EwN3Zxxkl04+rqqBLahUTmR86XG7xDYvwWlX2JN9BNh396Hvswdse4kKpEIaL4fw1bv9ZSwweyg1hvq015GHuaHSn4DcAII5kvS7WlOzJoaMEeXeQ6GNq6sVs0xN8UL2TBLN5SnI/pLErYWr5jo3wriBuWfX6o1/Jadj06X/S0hZlV7DC1kdMPNjG17KOX3NNXLCjzsBKAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=sword-group.com; dmarc=pass action=none header.from=sword-group.com; dkim=pass header.d=sword-group.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sword-group.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1VCDKun5EWh/4p3GOhVluSEw37zl+fBR95U3rygqjQ4=; b=VuZxUR/iDuLdJ9ZxULsWEITPo1jQyb6bRvK0D6dZUqQIRpAymrZYZWg/HAnm9VqYEkRhTZtrzHODj327cwTzUeHkj6ASkj1WYeUjQeeW/haLj7A1U5UnO43pqL1rrpPQyuVoNRRc8tpQezoCZcoyDfq3JAL4I+EdKv05J5s01KVNB7GNbuJzRjsJSMrctrztQQA7PUa7yAu6csGbf4hrjgrV3DdowF/hkzwjDHJHMWtJdbx6LKEeuHKQSk6tESxIEvV8AJo7uTWXNgSPMehkV0nj5otwgR3+g+luQ0ZM37FzBu7VpM/6++jOHXY2pCRo2HD/+E497VnFVLeLYBd6cg== From: LIONNETON Vincent To: Stefan Hanreich , "pve-devel@lists.proxmox.com" Subject: Re: [RFC] VMware/NSX to Proxmox migration: policy transformation and staged network handoff Thread-Topic: [RFC] VMware/NSX to Proxmox migration: policy transformation and staged network handoff Thread-Index: AQHdORXHyt2nRfnSlUqZNK7+3ZBZY7a7KmIAgAGjV9A= Date: Thu, 3 Sep 2026 12:31:09 +0000 Message-ID: References: <29570235-53c4-409e-82b6-36425c5ac092@proxmox.com> In-Reply-To: <29570235-53c4-409e-82b6-36425c5ac092@proxmox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=sword-group.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DU5PR08MB10400:EE_|AS8PR08MB10143:EE_ x-ms-office365-filtering-correlation-id: a20f9871-88e2-4d45-627a-08df09b73bb3 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|366016|376014|1800799024|23010399003|8096899003|38070700021|10067099003|11063799006|56012099006|3023799007|4143699003|13003099007|6133799003|22082099003|18002099003; x-microsoft-antispam-message-info: oXcABoB4vjVRzzZ7ovwcouhZRpHE41hTrBJHf6rNm0ehY+EbQC5XBfnZe9QGdCn0Y9o25x1Aev8373OJOrRYqkkOCP3HxzI/IG2wACg1qcQa6HoIpL6ocx7NsAcqSBIbNgPhJCsS535yD8foUG4+6ZS4MQv/ktMeD574dtSQ/WYbehhSj/egmvr3nZJdXEV8eUcF8Db0GViexea4FGfiOW2Db+Lr5CMbkIRHztm0Ud0T+rQA+vPESd38RTLN0aJuyDBd7Jj1k49Z8dYq9sQK2fyr/SXSkL7MwvSWML/9IQ5NejW2tdu81DszQBA6rWYNg5agyLwGSLsa/Oo3cv0BhqDUqZBrpVU01F/yI3XSJX0SxYGaUIjVOyfDSDeMy28HdZcsCdsN8DmXJKzWKU1wuWnHtsTIMbD/CwFqUY0ABoyD+QHPkpBaR2SYfnmA/V1qErJK6N/HFMBSfnZMWF6xT9NVNRstb+NfG12ax0zlpvUi0RZJ0YTO7jLQoDi2h0mZyvuYFruaiKj0d38G+bjkRa6Lbr59Da0ZpkN20GPHqKn7Wu8r7uuhRCDevWmHBv/o8ei/4peu650kCn5ruhxIysEPnaGjQu3Z1W+yTBjhlJKk1iLnAedJZ98eM5nhH86rdoEk2BL83lu591/39a6U+HZb6wupBJ8hNNVtWW/mvYU= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU5PR08MB10400.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(23010399003)(8096899003)(38070700021)(10067099003)(11063799006)(56012099006)(3023799007)(4143699003)(13003099007)(6133799003)(22082099003)(18002099003);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?/KsxU8fX3vPRcQ/1czqz6mVEGUSZ0ZDAmUrTbRtYQhrSiQJDG9LUPJKVGW?= =?iso-8859-1?Q?Xd5q40oPGUHprF6U+0OA5vyxypiFZKTf3m6LVRlqfiTUMGzU2kApFqC6os?= =?iso-8859-1?Q?vl5r9TERHo+CTOYXBTY+oTfhBBVFL8pS/8mz8dkhtT3WJMHOydaasOznbh?= =?iso-8859-1?Q?yXxHG80DOCvBrr/Zx2Auj0FTqWIprdOALKGZ03xN/N69PpSrmGEzrTWpuN?= =?iso-8859-1?Q?ZGCQEGFVHTA6efFBLcfOjfcrFKaxEawHQkkGaC+CZuW3OfL0d0r8vFs6i7?= =?iso-8859-1?Q?U8GmWPLPq+Wx4ifftRgag4Rn8lvhWwMh3l5GLDszDtaTHyEIiPMXNRbtzV?= =?iso-8859-1?Q?MqfhlIjYQewClIIAIAHyffN5Qa3PtNxWZ0FesIjoze6Hm4y1AGa0LT01h+?= =?iso-8859-1?Q?r3MVV5mA0sjzccTczJ9X3FzBgYm2GjbZcnnPBRLl3ffbX/FeqoCI+tAMPI?= =?iso-8859-1?Q?RyqFWGrSjqyE3MQtSc0IxEhr274EP9lcuK0Id8TgSyR5UrtBrnmUytq9XL?= =?iso-8859-1?Q?WVdtkg0hu1q6xJpMO1EDKKwVGyihFfvqmIlJXn/RWKorAevN5Yz+Bz8SWb?= =?iso-8859-1?Q?1U2uPkpRg/LSKB4MEMUlXrZjBCB+Vt5IoUaWdNFAQPYzbgWiiNawOc1vVH?= =?iso-8859-1?Q?LIKFJFEdFD0uorFSBClLsis9aakT2V4C29s/6xps/PieiXSpV4PsKdzSCP?= =?iso-8859-1?Q?aOp1icpVUIeqtMuaApzkVj45tYkxFNRQisrGU8xxtBU/PzKPqPbTB5cPeq?= =?iso-8859-1?Q?5D53kimQb1MOUhEuKpzT/gvndcGzsKwjVwgBVw6PzKPBlau9eKJAOIz19v?= =?iso-8859-1?Q?EyVtv3PyX2qpp+uGSr3LUn/SG/FcQcE0wj3a0KLOcgZaPU1w/LnPbMwxzE?= =?iso-8859-1?Q?aqmjj9/M1LXQiGebVwuD8KluOnLn5oxI4sEZ7H6AirprpEFlRCYMtrTkPJ?= =?iso-8859-1?Q?efe9GFjfEikdIKNYL+TVi3XYxBDqttftLrNX0G/EKtq9cTkdXRyzR4UOwo?= =?iso-8859-1?Q?k3VgbhxopGBzyfmtP8Ww8u33N1Dj7rey2zkmeyu9AexMZ25/oSJMJzBXNc?= =?iso-8859-1?Q?Orqn0mSRdS0DMucMwVQu9sPR7r88Lwd/sf4ImapF4gICHVgNhqYrgPrpH0?= =?iso-8859-1?Q?1A5/zJEUH9K6hr0h5Ucidl5GEn3xsCyetvSh9bwNjFv7aHxCzhcWQ6f2wA?= =?iso-8859-1?Q?NfJ6CJ/1OQ/JzUbT7NebhJfQ50B70bUkdjoQJo+QDAUvJ+ZGZgqjpLto33?= =?iso-8859-1?Q?XSWsfYA04cYcT4l2hnnbZvv0orugcHTo2Z3LJRIc3RipWZLJEZDVdjPtZX?= =?iso-8859-1?Q?JZqwDBcht+f6JkkfM9KLjemL3rroTmVHVqBb/BGPUKTEzFCvj9djPHxyXC?= =?iso-8859-1?Q?h0ZoV+++FVV/xPff/+qCzap1fu4SqyTKkNK7OpNAhfm0ZOyZUhQg4B7rHJ?= =?iso-8859-1?Q?pQcaL2yS2swcz2tUptZbHC7lgYjBgbOVUCATHQ7j4HNvB6ZchI6lI+ipb4?= =?iso-8859-1?Q?DgnMmofqsnptPBCfTpIigD2scLtbR/lopcEsxpVZJooJHFVfQprGrGzAkc?= =?iso-8859-1?Q?6+mr4s4P3tNz66wHZ9J+typSpV4u+oVNMrpzNR/htQ17gpKgzeOn5T4QJ9?= =?iso-8859-1?Q?5NcaEYEgQqSDigxBYhm5uUDlsMEvaloRTnSbS5kwhqAtP7EMREM+jrmsCw?= =?iso-8859-1?Q?to8T+U+spCGmr3Qq/7ZLYhUxPBZU5hxNRht494rjDkYwZdpofbQwigzB9F?= =?iso-8859-1?Q?QmJkUGw5CB3BtJ8eaWg/i/zh35kc/Ks/BDi4qq46tiBbJSq/Uh+FQ687Pw?= =?iso-8859-1?Q?Nf8/Bv30I4owvSdCc46PNl843PoRkvU=3D?= MIME-Version: 1.0 X-OriginatorOrg: sword-group.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DU5PR08MB10400.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a20f9871-88e2-4d45-627a-08df09b73bb3 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Sep 2026 12:31:09.4574 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 6adf23d8-eabe-44c8-b68a-0b8fb7aacef9 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zI408pA8eeA+zfkYT9Dj7XQbkbkWY+SST7fRMGrFJB5AUy2tTToyC1O/vk2MSesMKjTqk5mMeIzoJQfTGEk0WT5Cy1erqNm54BKRnJX0xinwGR5DX6by4dcifDxCkJLo X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB10143 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.083 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy HTML_MESSAGE 0.001 HTML included in message RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: GRQY3EBOFVNGXO7KDAZPSB5GW5CFSIH4 X-Message-ID-Hash: GRQY3EBOFVNGXO7KDAZPSB5GW5CFSIH4 X-MailFrom: vincent.lionneton@sword-group.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-Content-Filtered-By: Mailman/MimeDel 3.3.10 CC: "pdm-devel@lists.proxmox.com" X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hello, Thank you again for the detailed response. It confirms that the most approp= riate boundary would be to keep VMware/NSX-specific discovery and migration= logic in an external tool, while relying on native PVE/PDM capabilities fo= r the resulting networking and security configuration. After reviewing the ongoing microsegmentation and VRF work, I think it woul= d be premature for us to implement the Proxmox target side of the migration= tool against the current SDN and firewall models. Both areas are still evolving significantly, and we would prefer to wait un= til the relevant data models and APIs are closer to their intended released= form rather than building against the current implementation and having to= redesign the integration shortly afterwards. In the meantime, we will continue documenting the NSX migration requirement= s and concrete policy examples, particularly around: * dynamic group membership based on multiple VM tags; * stateful and service-specific Layer 3/Layer 4 rules; * workloads external to the PVE identity domain during mixed NSX/PVE co= existence; * atomic staging and validation of large policy changes; * same-address workload migration and subsequent gateway handoff. Once the relevant functionality is available through the supported PVE/PDM = APIs, we can then build the migration layer against the native target model= rather than an intermediate implementation. Thank you as well for confirming that NSX object migration is not currently= planned as part of the importer. This gives us a clear and complementary s= cope to revisit once the target SDN and security interfaces have stabilized. Best regards, Vincent Lionneton ________________________________ From: Stefan Hanreich Sent: Wednesday, September 2, 2026 1:29 PM To: LIONNETON Vincent ; pve-devel@lists.= proxmox.com Cc: pdm-devel@lists.proxmox.com Subject: Re: [RFC] VMware/NSX to Proxmox migration: policy transformation a= nd staged network handoff [You don't often get email from s.hanreich@proxmox.com. Learn why this is i= mportant at https://aka.ms/LearnAboutSenderIdentification ] Thanks for your inquiry, I tried to keep the answers relatively short witho= ut diving too much into technical details. On 8/31/26 9:11 AM, LIONNETON Vincent wrote: [snip] > Before going further, particularly with same-address coexistence and netw= ork handoff, I would appreciate Proxmox's guidance on the following points: > > > 1. Is NSX-aware migration already planned or being worked on within Proxm= ox, publicly or internally? > > In particular, are there plans around NSX DFW/security-policy migration, = NSX network preservation, or staged migration from NSX networking to PVE SD= N? There are currently no plans for extending the importer functionality to mi= grating NSX objects / entities, so efforts on developing something like that would be w= elcome. > 2. Do you see this functionality primarily as an external migration/trans= formation tool using PVE/PDM APIs, or do some of these functions belong in = PVE or PDM themselves? > > We explicitly do not want to develop a competing permanent SDN/firewall c= ontrol plane if PDM is intended to provide that functionality. Optimally, the functionality is implemented natively in PVE / PDM and then = only managed by the external tool. For your purposes, the most interesting things that a= re still in development are VRF support and microsegmentation, which both have a RFC on= the mailing list. Looking at those should give you a good glimpse into our development = workflow and the current codebase. > 3. For an external migration tool, what would you consider the preferred = and sufficiently stable API integration points? > > In particular for VM migration, SDN/VNet provisioning, EVPN/BGP configura= tion, Security Groups/IP sets, guest/VNet firewall policies and multi-clust= er orchestration through PDM. The whole REST API, as described in the documentation [1], is considered to= be stable and does not introduce breaking changes during a major version. Our policy on w= hat we deem a breaking API change and how we treat evolving the API can be found in our W= iki [2]. > 4. Would effective NSX-policy translation and policy-parity validation be= useful functionality from Proxmox's perspective? > > For example, resolving dynamic NSX group memberships and the effective DF= W policy for a VM, compiling that intent into native PVE firewall objects, = and comparing the resulting effective policy before cutover. For that particular purpose, the ongoing work on microsegmentation is likel= y of interest to you [3] as that should provide the functionality required. As i= t's in a stage of development, feel free to chime in and test the functionality - = we always value feedback and in this early stage it's a good opportunity for you to c= heck out the implementation to see if it fits the use-case. > 5. Does the proposed same-address migration model fit the intended PVE SD= N architecture? > > The idea is to temporarily extend the existing NSX L2 network to PVE, mig= rate the workloads while keeping IP/gateway addressing unchanged, and only = later hand the gateway/routing function over from NSX to PVE EVPN/BGP on a = network-by-network basis. That should work generally and I know of some customers that have done a si= milar setup, but I personally have zero practical experience with this - so it's hard for me= to talk about details. Since standard routing protocols are involved the integration shou= ld work, but I can imagine needing something like a translation layer, i.e. when using a r= oute reflector for both PVE / NSX-T the routes might need to get transformed by the reflec= tor to match what the respective stacks expect. Potentially even a router in front of the PVE= cluster, that handles NSX-T / Proxmox VE specifics during the migration. With the recentl= y introduced route maps feature, this might be possible on the PVE side via the SDN stac= k solely. There's very likely a few pitfalls that'd need to be worked out. Ir's very = likely that some setups are currently impossible to model with the SDN stack alone and an ex= ternal router or custom FRR configuration is needed - although we're working hard on address= ing those deficiencies. I'd start with attempting to migrate relatively simple setups= (e.g. single-tenant) and see how everything works out for simpler setups. > 6. Is low-downtime/incremental VMware migration already in scope for the = existing importer? > > We are particularly interested in whether a VDDK/CBT-style precopy/delta = mechanism, or another warm-migration mechanism from ESXi, is already planne= d. There has been some work done internally on improving the importer by utili= zing NFC/CBT, but it still needs a bit of work to drag it over the finish line, I cannot = provide an ETA at this point. > 7. Are there generic PVE/PDM capabilities that would be useful to impleme= nt upstream for this use case? > > Possible examples are effective-firewall-policy inspection, policy diff/v= alidation, dry-run APIs, structured external-migration prechecks, or bulk f= irewall/IP-set operations. Yes, I would see all of those capabilities as something that should be prov= ided by the Proxmox VE API and then consumed by your tool. The firewall currently d= oes not support staging changes, as the configuration is applied continously by the= firewall daemon. SDN itself already has a dry-run API that has been recently introdu= ced [4] and the related endpoints all support querying the current / pending config= uration via the pending / running GET parameters. I'm currently working on improving firewall configuration handling together= with a colleague, but the general mechanism stay the same. For the use-cases you= mentioned that would require implementing some sort of staging mechanism for firewall= changes. I'm not sure what you mean with bulk firewall / IP-set operations exactly? > 8. If some of these pieces would be appropriate upstream, where would you= prefer them to live and what development interfaces/conventions should we = target? > > For example PVE vs PDM, the appropriate repositories and APIs, Rust vs ex= isting PVE components, expected test coverage and contribution workflow. Development of Proxmox VE (and other products) happens out here in the open= on the respective mailing lists. We have a short introductory guide to our workflo= w in the Wiki [5]. Discussion of specific features usually happens in our bug tracki= ng instance [6], rather than here on the mailing list directly. For the SDN stack in particular, new features are implemented in Rust and o= nly exposed via our perlmod module, which allows calling into Rust code. The new firewall daemon is implemented in Rust as well, but the API still lives on = the Perl side. That might change in the future though, as we plan on implementi= ng more and more new features in Rust while also porting existing functionality over where it makes sense. In the short to mid-term we also want to work on eliminating the need for Perl to act as a middle man for Rust. > intention would be to keep all VMware/NSX-specific discovery and transfor= mation logic in the migration tooling, while contributing only genuinely ge= neric Proxmox capabilities upstream if that is of interest. That would be the preferred way, in my opinion. > Once the migration is complete, the resulting environment should consist = solely of native PVE/PDM SDN and firewall configuration and should not depe= nd on the migration tool for normal operation. Yes, I think there are two phases to this? First, the network topology / st= ructure would need to get set up on PVE (and hooked up with its NSX-T counterparts)= - then the migration of the VMs and their network configuration, which maps the gu= est configuration from VMWare onto the SDN entities. The tool would need to mai= ntain some form of mapping in that case / or it'd need to be stored in the SDN st= ack directly - so the tool can consume that (which is preferable imo). > I would be happy to share a more detailed architecture or the results of = the current proof of concept if useful. Yes, I'd be interested in that! [1] https://pve.proxmox.com/pve-docs/api-viewer/ [2] https://pve.proxmox.com/wiki/Proxmox_VE_API#API_Stability_&_Breakage [3] https://lore.proxmox.com/all/DKRAX8JGPAP2.19JLE3T7U6DS1@proxmox.com/ [4] https://pve.proxmox.com/pve-docs/api-viewer/#/cluster/sdn/dry-run [5] https://pve.proxmox.com/wiki/Developer_Documentation [6] https://bugzilla.proxmox.com/