From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 90CE560726 for ; Thu, 24 Sep 2020 10:58:43 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 7DF9610007 for ; Thu, 24 Sep 2020 10:58:13 +0200 (CEST) Received: from mailpro.odiso.net (mailpro.odiso.net [89.248.211.110]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 4ECA7FFFD for ; Thu, 24 Sep 2020 10:58:11 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mailpro.odiso.net (Postfix) with ESMTP id 094381BAD5F0; Thu, 24 Sep 2020 10:58:04 +0200 (CEST) Received: from mailpro.odiso.net ([127.0.0.1]) by localhost (mailpro.odiso.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ACnzZBcR7jAk; Thu, 24 Sep 2020 10:58:03 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mailpro.odiso.net (Postfix) with ESMTP id E131E1BAD5F1; Thu, 24 Sep 2020 10:58:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at mailpro.odiso.com Received: from mailpro.odiso.net ([127.0.0.1]) by localhost (mailpro.odiso.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7JIWz33tGhaI; Thu, 24 Sep 2020 10:58:03 +0200 (CEST) Received: from mailpro.odiso.net (mailpro.odiso.net [10.1.31.111]) by mailpro.odiso.net (Postfix) with ESMTP id C450D1BAD5F0; Thu, 24 Sep 2020 10:58:03 +0200 (CEST) Date: Thu, 24 Sep 2020 10:58:03 +0200 (CEST) From: Alexandre DERUMIER To: Thomas Lamprecht Cc: Proxmox VE development discussion Message-ID: <1797569343.1221144.1600937883591.JavaMail.zimbra@odiso.com> In-Reply-To: <376471267.594430.1599791248991.JavaMail.zimbra@odiso.com> References: <20200824164923.12652-1-aderumier@odiso.com> <1480692740.435706.1599537124012.JavaMail.zimbra@odiso.com> <65ed6f59-a12b-c331-4b65-a9e95afda377@proxmox.com> <1179667870.443488.1599555533057.JavaMail.zimbra@odiso.com> <376471267.594430.1599791248991.JavaMail.zimbra@odiso.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Zimbra 8.8.12_GA_3866 (ZimbraWebClient - GC83 (Linux)/8.8.12_GA_3844) Thread-Topic: POC : add/del/update ip from vnet-subnet-ipam Thread-Index: YoBLmfwHJJhLEw2MQBgGYDDb0UyxiLUjWdbxqAEiGSw= X-SPAM-LEVEL: Spam detection results: 0 AWL 0.012 Adjusted score from AWL reputation of From: address KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pve-devel] [PATCH v2 pve-container] POC : add/del/update ip from vnet-subnet-ipam X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2020 08:58:43 -0000 Hi Thomas, I have sent new patches series. I have rework the gui, to have subnets in a vnet split panel (like ipset). = (also move ipam/dns to advanced section in subnet form) I have also move controllers, ipams, dns to a new "options" section, in 3 = differents box I have also change the subnet api in pve-network and default ipam to pve. I think it should be more easy to setup for user like this. I'm going to make some unit tests soon. (I was a bit busy with corosync deb= ug this last week) About: >>Could it make sense to have subnets and vnets at least in the same sectio= n config,=20 >>with different types? (a bit like storage.cfg) So that we reduce the conf= iguration=20 >>file amount a bit.=20 I'm not sure how to handle this correctly (sharing the same cfg, with 2 dif= ferents kind of objects, where the code is really different) ----- Mail original ----- De: "aderumier" =C3=80: "Proxmox VE development discussion" Cc: "Thomas Lamprecht" Envoy=C3=A9: Vendredi 11 Septembre 2020 04:27:28 Objet: Re: [pve-devel] [PATCH v2 pve-container] POC : add/del/update ip fro= m vnet-subnet-ipam Hi Thomas,=20 I have begin to work on new subnet panel (like ipset),=20 it's working fine.=20 I have also changed the api endpoint to /cluster/sdn/vnets//subnets= /=20 I'll try to send patch next week.=20 About your problem with ipam, and the gateway.=20 I have found bug, if you create first the subnet + gateway, then add the ip= am option later,=20 the ip of the gateway was not registered.=20 Do you think it could be great to make ipam option mandatory ? (and default= ing to internal pve ipam ?)=20 without the need to declare pve ipam in ipams.cfg.=20 (Like this, ipams.cfg is only for external ipams)=20 I think also than ipam driver should not be changed/removed from subnet cre= ation, or it'll be possible=20 to have conflict/duplicated ips.=20 ----- Mail original -----=20 De: "aderumier" =20 =C3=80: "Thomas Lamprecht" =20 Cc: "Proxmox VE development discussion" =20 Envoy=C3=A9: Mardi 8 Septembre 2020 10:58:53=20 Objet: Re: [pve-devel] [PATCH v2 pve-container] POC : add/del/update ip fro= m vnet-subnet-ipam=20 >>I'd like to have stuff in priv/ folder prefixed with a directory=20 >>namespace, maybe "sdn" here.=20 Ok,I'll add the extra /sdn/=20 (for your problem, maybe look if the gateway ip is present in the ipam.db)= =20 >>Besides that, how big can this get on huge setups? We only can have 512k = files=20 >>for now.=20 currently, it's just a perl hash converted to json with ip list. (so 32bits= by ipv4, and 128bits by ipv6).=20 So I think it could be ok for common cluster ?=20 >>Could it make sense to have subnets and vnets at least in the same sectio= n config,=20 >>with different types? (a bit like storage.cfg) So that we reduce the conf= iguration=20 >>file amount a bit.=20 yes, I think it's possible. I'll look to see how to implement that.=20 do you want to have 1 api endpoint for both subnets && vnets ?=20 >>Maybe we could also visualize this in the gui a bit "easier".=20 >>We could merge VNet and SubNet management into one panel, maybe with spli= t view=20 >>like FW ipsets=20 That's a really good idea ! Like it :)=20 I'll look at FW ipset extjs implementation.=20 >>you won't have the same subnet in different VNets after all,or ?=20 Yes, currently I manage only 1subnet-1vnet.=20 I don't think users will used same subnet multiple times, until they used v= rf for routed vnet or isolated bridge.=20 >>We could also merge DNS and IP management into one panel, maybe with spli= t view=20 >>like HA or vertical like FW ipsets.=20 >>=20 >>But this is not too relevant for now, can always be fine tuned once the A= PI/backend=20 >>stuff is in.=20 Ok, no problem.=20 >>On another note, are there some unit/regressions tests for this stuff?=20 >>Would give a bit more confidence with this.=20 No sorry. I have done tests manually currently.=20 I don't know too much how to implement the unit tests, but I'll try too see= what can be done.=20 >>skimming through the code currently, seems mostly OK for now, need to thi= nk a bit=20 >>about how the general concepts are implemented and if that fits all OK.= =20 Thanks for the review && comments !=20 ----- Mail original -----=20 De: "Thomas Lamprecht" =20 =C3=80: "Proxmox VE development discussion" , = "aderumier" =20 Envoy=C3=A9: Mardi 8 Septembre 2020 09:44:19=20 Objet: Re: [pve-devel] [PATCH v2 pve-container] POC : add/del/update ip fro= m vnet-subnet-ipam=20 On 08.09.20 05:52, Alexandre DERUMIER wrote:=20 >>> When trying this I got the gateway IP returned for both, as CT IP and g= ateway IP.=20 >>> Did not checked this patch closer, but I figured that this behavior is = caused by=20 >>> the SDN code.=20 >=20 > mmm, that's strange.=20 >=20 > When you create or update the subnet, the gateway ip you define on the su= bnet should be registered in the ipam.=20 > (you have enable an ipam right ?)=20 Yes, the built-in "PVE" one=20 >=20 >=20 > Then, when you create CT, without any ip, it'll try to find first availab= le ip in ipam.=20 I did it on an existing CT, changing from a normal bridge to that vnet.=20 > (So if the gateway was not registered in ipam (bug maybe), that could exp= lain why you have it both).=20 >=20 > for internal ipam, i'm writing ipam database in /etc/pve/priv/ipam.db. (B= TW,I'm not sure that it's the best path location)=20 I'd like to have stuff in priv/ folder prefixed with a directory=20 namespace, maybe "sdn" here.=20 Besides that, how big can this get on huge setups? We only can have 512k fi= les=20 for now.=20 >=20 >>> On another node, do you think it makes sense to have vnets, subnets, IP= am, DNS completely=20 >>> split and separated from each other? I mean, it is flexible, but a user= needs to do a lot=20 >>> of, almost boilerplate-like, work to get this started.=20 >>> Advanced users may profit from this, maybe we just need a "simple wizar= d" for the easiest=20 >>> beginner case..=20 >=20 > Well for subnet, you can assign multiple subnets by vnet, so yes, it's re= ally need to by separated.=20 > (Somebody at hertzner for example, buying subnets or /32 failovers ips, a= nd want to add them to a vnet)=20 > IPAM/DNS, are more reusable configurations. (like api url,key,....). So I= think you'll define 1 or 2 of them max.=20 >=20 > I think subnet+ipam+dns are ip features.=20 > zones,vnets,controller are physical network features=20 Could it make sense to have subnets and vnets at least in the same section = config,=20 with different types? (a bit like storage.cfg) So that we reduce the config= uration=20 file amount a bit.=20 Maybe we could also visualize this in the gui a bit "easier".=20 We could merge VNet and SubNet management into one panel, maybe with split = view=20 like FW ipsets, you won't have the same subnet in different VNets after all= , or?=20 We could also merge DNS and IP management into one panel, maybe with split = view=20 like HA or vertical like FW ipsets.=20 But this is not too relevant for now, can always be fine tuned once the API= /backend=20 stuff is in.=20 On another note, are there some unit/regressions tests for this stuff?=20 Would give a bit more confidence with this.=20 skimming through the code currently, seems mostly OK for now, need to think= a bit=20 about how the general concepts are implemented and if that fits all OK.=20 _______________________________________________=20 pve-devel mailing list=20 pve-devel@lists.proxmox.com=20 https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel=20 _______________________________________________=20 pve-devel mailing list=20 pve-devel@lists.proxmox.com=20 https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel=20