From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id D68E274C1B for ; Tue, 20 Apr 2021 09:51:39 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id CB5831FC67 for ; Tue, 20 Apr 2021 09:51:39 +0200 (CEST) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 453BE1FC51 for ; Tue, 20 Apr 2021 09:51:36 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id x7so36527282wrw.10 for ; Tue, 20 Apr 2021 00:51:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=odiso-com.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=uKTatouDZHLB/2gZT3M5qrml55cS8DGiHsxr/2kB4wM=; b=U6b/+4WZErbjTp9kfgOlwWnm/Vv2toBJUIZ06uK2RMUjuyNoVg8SRr+39xLOkbwNxS qSzmtCI969VnEikXtKVhef3tje/3XNWRzHLInrn4lGPKTAHufU+q+Qb/Jd/XhH4GI+Y7 j/dsWevp2HQAprOUnXDNDbJP9M/meOPdPhxtb09AvtJCgB4NvKXNQxg6B6b4hLL01TW2 I+TG9n+WSZdMUH+oQUIhYEU3tSs1yIr/un96/vPLEhvj/cuX0HEN07yiIIPBo3rNTMO8 /lmsxj809Bs5eUtusCgCbFst/4PN6XrAhoV591vbtMHSG1sTbIl/UWk43ljgdq235KKJ EW5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=uKTatouDZHLB/2gZT3M5qrml55cS8DGiHsxr/2kB4wM=; b=RJqA3SuppvZH71yVXUZglZIhLftIFCQlYz8XyZhF8RFwCCU7XFiC4DdslWVFy2gZVT slE4247DG7zD1y4D0dgv+h0ZwVBaufEr2cj2sjJmGhv/4Y3EbN46/G9tKcthZZ5YaS1J d1XaFeHBElxNUAypOdkQOBAOV/gg223iM6UYi1gizu915sopKaCu6POt39mM3gSSj/AZ IDpbiNttkRXUfpaF2X76bBJ/J2jQpqNiTTmYz5zAhUS0JO322ofHx98hJhjQVTGumuXe SP2d6SIjh1LlIOimJfb79oJlllXwHnzJfDjcVisWoIQWBezWGhK8KlPe0O46CMqyJdMZ K0OA== X-Gm-Message-State: AOAM531eMuF1JVaKCijHRaIz8wNb6vm7WGls8ujRSXIr6hI4mU5p18dn nOmXPAIDWtBzf6XkQibH35m4eQ== X-Google-Smtp-Source: ABdhPJzaCr7Fn/we8UakJ++9Q9wIcw6oHGpcSnlBLn5l349KGS/BhHJ8LCuV+eLG1qBpXzZE4PxwwQ== X-Received: by 2002:a5d:590d:: with SMTP id v13mr18724589wrd.85.1618905090653; Tue, 20 Apr 2021 00:51:30 -0700 (PDT) Received: from [10.59.100.76] (globalOdiso.M6Lille.odiso.net. [89.248.211.242]) by smtp.gmail.com with ESMTPSA id o17sm7053931wrg.80.2021.04.20.00.51.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Apr 2021 00:51:30 -0700 (PDT) To: Thomas Lamprecht , Proxmox VE development discussion Cc: pve-devel References: <8ed4f090-2021-bbe1-4b7b-ceef71d7f4ff@proxmox.com> From: alexandre derumier Message-ID: <19d18da4-2ae4-b8fd-b574-318f645287ec@odiso.com> Date: Tue, 20 Apr 2021 09:51:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.9.1 MIME-Version: 1.0 In-Reply-To: <8ed4f090-2021-bbe1-4b7b-ceef71d7f4ff@proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-SPAM-LEVEL: Spam detection results: 0 AWL 0.851 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com, mail-archive.com] Subject: Re: [pve-devel] sdn: looking to unify .cfg files, need opinions about config format X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2021 07:51:39 -0000 Hi Thomas! On 18/04/2021 19:01, Thomas Lamprecht wrote: > On 12.01.21 10:19, aderumier@odiso.com wrote: >> Hi, >> >> I'm looking to unify sdn .cfg files with only 1 file, >> with something different than section config format. >> >> We have relationship like zones->vnets->subnets, >> so I was thinking about something like this: >> >> >> >> [zone myzone] >> type: vxlan >> option1: xxx >> option2: xxx >> [[vnet myvnet]] >> option1: xxx >> option2: xxx >> [[[subnet 10.0.0.0/8]]] >> option1: xxx >> option2: xxx >> >> >> [controller mycontroller] >> type: evpn >> option1: xxx >> option2: xxx >> >> [dns mydns] >> type: powerdns >> option1: xxx >> option2: xxx >> >> >> What do you think about this ? > That looks like section config, just spelled differently? > > But yes, the way section config does schema and types are not ideal when combined > with quite different things. > > Maybe we should really just go the simple way and keep it separated for now. > > For zones it works good this way, there exist different types and we can use that as > section config type. Subnets and vnets could be combined as vnets are really not that > specially I guess? I think that maybe the only thing that could be improve is indeed subnets/vnets. currently, we can have same subnet range defined on zones. but they are really different object, as gateway or other subnet option can be different, that's why I'm doing concatenate zone+subnet to have a unique subnetid, something like subnet: zone1-192.168.0.0-24         vnet vnet1         gateway 192.168.0.1 subnet: zone2-192.168.0.0-24         vnet vnet2         gateway 192.168.0.254 It's not bad, but maybe it could be better, with defining subnet somewhere inside the vnet directly. Not sure how the config format should be to handle this ? > > We had a mail about what would be Ok to merge, but I do not remember/find it anymore... Small reminder of other related patches: pve-network: [pve-devel] [PATCH pve-network 0/2] evpn && bgp improvements https://www.mail-archive.com/pve-devel@lists.proxmox.com/msg03265.html (2 small patches) pve-manager: [PATCH V11 pve-manager 1/1] sdn: add subnet/ipam/sdn management https://www.mail-archive.com/pve-devel@lists.proxmox.com/msg02746.html (I had merged and rebased the differents patches from previous series) pve-cluster: [PATCH V5 pve-cluster 0/5] sdn : add subnets management https://lists.proxmox.com/pipermail/pve-devel/2020-September/045284.html pve-common: INotify: add support for dummy interfaces type (this is a small patch for ebgp loopback/dummy interface support) https://www.mail-archive.com/pve-devel@lists.proxmox.com/msg01755.html pve-container: (maybe we could wait a little bit to finish qemu support too) [PATCH pve-container] add ipam support https://lists.proxmox.com/pipermail/pve-devel/2021-January/046609.html >> Another way could be a simple yaml config file. (but I think it's not >> really matching currents proxmox configs formats) >> > I do not like yaml to much, it looks simple first but can do way to much (turing > complete, IIRC) and we do not really use it anywhere atm., so that would mean lots > of new tooling/work to do to handle it sanely and as first-class citizen in PVE > stack... > > My goal would be to do a pve-network bump at end of next week, and for that we > need pve-cluster bump. > > Currently there we get three new configs: > > 1. ipams, different management plugins (types), so OK to be own section config > 2. dns, different, APIs/DNS servers (types), so OK to be own section config > 3. subnets, only one type, or? subnet only 1 type indeed > hmm, rethinking this now it could be OK to keep as is... While subnets could > possibly be merged into vnets, there's a mediocre benefit there, API could > maybe even get more complicated? not sure about api, but if current config format with subnetid is ok for you, it's ok for me ;) > If we'd bump now the biggest thing missing is applying an IP to a VM and CT. > > For a CT we can quite easily do it. yes, I have already send patches, maybe it need more testing. > > For a VM we could even need to support different ways? > > * DHCP (?) for dhcp, It'll be more difficult for bridged setup, as we need an 1 by subnet. for routed setup, it's more easy. I think we should see that later, I have an idea about managing some kind of gateways edges vms/appliance feature like vmware nsx edge gateway. https://bugzilla.proxmox.com/show_bug.cgi?id=3382 where you could manage this kind of central service (dhcp, vpn, nat 1:1, balancing,...). (a lot of users use pfsense for exemple or other gateway applicance, my idea is to manage this kind of appliance through api, or maybe manage our own appliance) This should works with any kind of network, bridged/routed, or any zone (vlan/vxlan/...) But It's a big thing, so later ;) > * cloudinit yes, this is my current plan. offline, it's easy; online, it's more difficult. That's why I was working on cloudinit too recently, with pending features,... I need to do more test with ipam + ipconfig in pending state (if user try to rollback,...) Also, cloudinit is a little bit tricky to apply changes online, but it's possible at least on linux with some udev rules. That's why I'm also playing with opennebula context agent https://github.com/OpenNebula/addon-context-linux https://github.com/OpenNebula/addon-context-windows This is doing the same thing than cloudinit (I have added configdrive format recently), but with simply bash scripts + udev rules for online change. It's working very well. I wonder if at the end, we shouldn't have similar proxmox daemon (or maybe fork it, https://github.com/aderumier/addon-context-linux/commits/proxmox ;) to have true online changes. (adding udev rules for hotplug mem/cpu ,extend disk/lvm partition, do network config changes,...) Cloud-init is really done to bootstrap the vm, but online changes are still tricky. > * would the qemu-guest-agent work too? This is a good question. I don't known if we can write config file with qemu-guest-agent. > > IMO that's the biggest thing left, or what do you think? > yes, qemu is the biggest thing. I think that the sdn part itself (zone/vnets/subnets/...) are working fine.