From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 504B192494 for ; Wed, 28 Dec 2022 10:51:54 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 2A03723127 for ; Wed, 28 Dec 2022 10:51:24 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 28 Dec 2022 10:51:22 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B16BA45242 for ; Wed, 28 Dec 2022 10:51:22 +0100 (CET) Date: Wed, 28 Dec 2022 10:51:21 +0100 From: Stoiko Ivanov To: Christoph Heiss Cc: pmg-devel@lists.proxmox.com Message-ID: <20221228105121.7e3ff1f1@rosa.proxmox.com> In-Reply-To: <20221228093152.r2f5kbhnutaxrszc@maui.proxmox.com> References: <20221227122915.218159-1-c.heiss@proxmox.com> <20221227192135.7f7bdf4f@rosa.proxmox.com> <20221228093152.r2f5kbhnutaxrszc@maui.proxmox.com> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.24; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.158 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pmg-devel] [PATCH pmg-api v2] fix #4410: Remove non-null host-bits from CIDR when reading `mynetworks` X-BeenThere: pmg-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Mail Gateway development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2022 09:51:54 -0000 On Wed, 28 Dec 2022 10:31:52 +0100 Christoph Heiss wrote: > On Tue, Dec 27, 2022 at 07:21:35PM +0100, Stoiko Ivanov wrote: > > On Tue, 27 Dec 2022 13:29:15 +0100 > > Christoph Heiss wrote: > > > > > This will simply drop non-null host bits when reading the config file, > > > thus preserving backwards-compatibility. > > > When creating new entries, invalid CIDRs are now rejected to prevent > > > creation of such entries in the future. > > > > > > In the GUI, the entries are displayed as the user entered them (as > > > suggested my Stoiko). This is done by considering /etc/pmg/mynetworks > > > as the "source of truth" - all entries are saved there verbatim, but > > > when writing the postfix config the right ones are picked. > > This sounds good! > > Currently the code breaks the GET,PUT,DELETE api calls for mynetworks > > (mismatch between the key (which is the 'computed/long string' and the > > provided parameter (which is the the data from the file) > > > > (tested with an ipv6 prefix - creating works, getting/setting/deleting > > does not work) > Weird, I though I tested at least deleting extensively enough. Well, > back to the drawing board. > > > > > maybe keep the user-entered value as key - and add the host-bit-clean cidr > > (address/mask string) as additional field - then get this field when > > serializing the data in get_template_vars > Had that same thought too already, but then a new problem arises - > duplicate (IPv6) entries can be created, since the check then just > compare the user-entered value. E.g. both 2001:db8::/32 and > 2001:db8::0/32 could be created with issue. That's why I chose to use > the normalized prefixes as keys. > But this was also possible before .. I'll see what I can come up with. One option that came to my mind was ... just keeping the API part and the values in /etc/pmg/mynetworks as they are .. only normalize the prefixes in get_template_vars for use in the postfix config It has the downside of users getting different values there than they entered - but I personally never understood why programs forced me to clear out the host-bits for their use - so I'm not sure if there's any upside to requiring our users to clear that up? > > > > > > > > [..] > > > > > > diff --git a/src/PMG/Config.pm b/src/PMG/Config.pm > > > index 9ba5c76..a29060b 100755 > > > --- a/src/PMG/Config.pm > > > +++ b/src/PMG/Config.pm > > > @@ -730,6 +730,7 @@ use PVE::SafeSyslog; > > > use PVE::Tools qw($IPV4RE $IPV6RE); > > > use PVE::INotify; > > > use PVE::JSONSchema; > > > +use PVE::Network; > > > > > > use PMG::Cluster; > > > use PMG::Utils; > > > @@ -1008,13 +1009,13 @@ sub read_pmg_mynetworks { > > > while (defined(my $line = <$fh>)) { > > > chomp $line; > > > next if $line =~ m/^\s*$/; > > > - if ($line =~ m!^((?:$IPV4RE|$IPV6RE))/(\d+)\s*(?:#(.*)\s*)?$!) { > > > - my ($network, $prefix_size, $comment) = ($1, $2, $3); > > > - my $cidr = "$network/${prefix_size}"; > > > - $mynetworks->{$cidr} = { > > > + if ($line =~ m!^((?:$IPV4RE|$IPV6RE)/\d+)\s*(?:#(.*)\s*)?$!) { > > > + my ($cidr, $comment) = ($1, $2); > > > + my $ip = PVE::Network::IP_from_cidr($cidr); > > > + $mynetworks->{$ip->prefix()} = { > > > cidr => $cidr, > > > - network_address => $network, > > > - prefix_size => $prefix_size, > > > + network_address => $ip->short(), > > the short() method yields IPv4 addresses in a somewhat uncommon format (at > > least for me -> 10.2.2.0/24 -> '10.2.2') - however at a quick glance it > > seems that the 'network_address' field is not really used anywhere - so we > > should probably just drop it. > Yeah, ->short() also abbreviates IPv4 addresses. At first I had a check > differentiating between v4 and v6 and only ->short()'ening v6 addresses. > > I'll investigate if and where this field is actually used (and > prefix_size too, while at it). > > Removing it would change the API though - is stability / backwards > compatibility something we have to be wary of or can I just drop it if > it's really not used anywhere? good point! - we do return it from the API - so let's keep it for now (a 'FIXME: drop network_address and prefix_size with PMG 8.0' comment might make sense though - then we can clear it up with the next major release) > > > > > > > > > > + prefix_size => $ip->prefixlen(), > > > comment => $comment // '', > > > }; > > > } else { > > > @@ -1032,7 +1033,7 @@ sub write_pmg_mynetworks { > > > foreach my $cidr (sort keys %$mynetworks) { > > > my $data = $mynetworks->{$cidr}; > > > my $comment = $data->{comment} // '*'; > > > - PVE::Tools::safe_print($filename, $fh, "$cidr #$comment\n"); > > > + PVE::Tools::safe_print($filename, $fh, "$data->{cidr} #$comment\n"); > > > } > > > } > > > > > > -- > > > 2.30.2 > > > > > > > > > > > > _______________________________________________ > > > pmg-devel mailing list > > > pmg-devel@lists.proxmox.com > > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel > > > > > > > >