From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 09D171FF139 for ; Tue, 24 Feb 2026 11:13:52 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 39D9D362B; Tue, 24 Feb 2026 11:14:44 +0100 (CET) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Tue, 24 Feb 2026 11:14:06 +0100 Message-Id: Subject: Re: [pve-devel] [PATCH cluster 1/3] datacenter config: add setting for HTTP{, S} proxies From: "Daniel Kral" To: "Proxmox VE development discussion" X-Mailer: aerc 0.21.0-38-g7088c3642f2c-dirty References: <20251021100332.251697-1-m.sandoval@proxmox.com> <20251021100332.251697-2-m.sandoval@proxmox.com> In-Reply-To: <20251021100332.251697-2-m.sandoval@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1771928031626 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.047 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 1.179 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.717 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.236 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 7MIFQBTBNBSM3MIBOJ4D4GDUSGNXSMND X-Message-ID-Hash: 7MIFQBTBNBSM3MIBOJ4D4GDUSGNXSMND X-MailFrom: d.kral@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue Oct 21, 2025 at 12:03 PM CEST, Maximiliano Sandoval wrote: > Adds a 'proxy' setting which is meant to replace 'http_proxy'. This new > setting allows to specify different HTTP and HTTPS proxies for different > pieces of the stack. AFAICT setting http_proxy and https_proxy - at least for curl - only specifies the proxy to use for HTTP and HTTPS requests respectively, so the terms 'HTTP proxy' and 'HTTPS proxy' in itself could be confusing here and below as one can use many different proxy protocols for either HTTP or HTTPS requests. > > In the UI each option would set both the HTTP and HTTPS proxies together > to the same value to avoid configuration mistakes, e.g. if only one > proxy is set. This should be ideally split into one part, where the ability to add an HTTPS proxy (additionally to the HTTP proxy) is added, and then adding the ability to specify different proxies for different use cases (or vice versa). > > The use-case this option intends to cover is a proxy which allows to > proxy HTTP(S) requests to the outside but will reject any connection to > resources which are already in the internal network, for this cases the > 'none' option would declare that no proxy should be used. > > The {proxy}->{global} default key of the property string acts as a > drop-in replacement for the {http_proxy} setting. However, we document > that this will be used both as a HTTP and a HTTPS proxy which was not > done always for the 'http_proxy' setting. > > Individual proxy configurations accept a 'none' value that allows to say > that no proxy should be used for this use-case, this takes precedence > over both the new global proxy and the 'http_proxy'. > > Subscriptions only need HTTPS proxies and thus we do not offer the > option to setup a HTTP proxy here. > > Signed-off-by: Maximiliano Sandoval > --- > src/PVE/DataCenterConfig.pm | 60 +++++++++++++++++++++++++++++++++++++ > 1 file changed, 60 insertions(+) > > diff --git a/src/PVE/DataCenterConfig.pm b/src/PVE/DataCenterConfig.pm > index c6d56c1..57c5c1c 100644 > --- a/src/PVE/DataCenterConfig.pm > +++ b/src/PVE/DataCenterConfig.pm > @@ -120,6 +120,52 @@ my $notification_format =3D { > }, > }; > =20 > +my $proxy_format =3D { > + 'global' =3D> { > + default_key =3D> 1, > + optional =3D> 1, > + type =3D> 'string', > + description =3D> "Proxy used as a fallback. It will be used when= the respective component does not have a proxy defined. Will be used both = as a HTTP and HTTPS proxies.", > + pattern =3D> "http://.*", > + format_description =3D> 'URL', > + }, It might also make sense to have a distinction between a global fallback for the HTTP and HTTPS protocol, as is done for curl [0]. For example, this `global` setting here could be the equivalent to `all_proxy` and then there are fallbacks for both `http_proxy` and `https_proxy`... [0] https://curl.se/docs/manpage.html#ENVIRONMENT ...But it really depends on how much options we want to expose to users here. Another idea here could also be to use figure out the type of proxy ourselves as the `--proxy` option in curl [1] does instead of allowing to set both the proxy. If we go for that route, we could introduce a proxy format and then each use case would only be e.g. download =3D> get_standard_option('pve-proxy', { description =3D> '...', }), Then we would only need to check whether a use case overwrites the global proxy and otherwise fallback, and afterwards figure out which type of proxy protocol it is instead of checking both at the same time. [1] https://curl.se/docs/manpage.html#--proxy > + 'http-download' =3D> { > + optional =3D> 1, > + type =3D> 'string', > + description =3D> "HTTP proxy used for downloading ISOs and conta= iner templates. When set to 'none' no proxy will be used.", > + pattern =3D> "(http://.*|none)", The pattern for the proxies here and below should probably at least also allow `https://.*` as an option. This could be extended in the future to also allow socks5, ..., if all the underlying programs where we pass these values to can handle these proxy types. In general, at least curl fallbacks to `http://` if there's no protocol prefix at all [2] [3], so that could also be an option, but not necessary for an initial implementation. [2] https://github.com/curl/curl/blob/master/lib/url.c#L2059 [3] https://github.com/curl/curl/blob/master/docs/libcurl/curl_url_set.md#c= urlu_guess_scheme > + format_description =3D> 'URL', > + }, > + 'https-download' =3D> { > + optional =3D> 1, > + description =3D> "HTTPS proxy used for downloading ISOs and cont= ainer templates. When set to 'none' no proxy will be used.", > + type =3D> 'string', > + pattern =3D> "(http://.*|none)", > + format_description =3D> 'URL', > + }, > + 'https-subscription' =3D> { > + optional =3D> 1, > + description =3D> "HTTPS proxy used for subscription related task= s. When set to 'none' no proxy will be used.", > + type =3D> 'string', > + pattern =3D> "(http://.*|none)", > + format_description =3D> 'URL', > + }, > + 'http-apt' =3D> { > + optional =3D> 1, > + description =3D> "HTTP proxy used for APT. When set to 'none' no= proxy will be used.", > + type =3D> 'string', > + pattern =3D> "(http://.*|none)", > + format_description =3D> 'URL', > + }, > + 'https-apt' =3D> { > + optional =3D> 1, > + description =3D> "HTTPS proxy used for APT. When set to 'none' n= o proxy will be used.", > + type =3D> 'string', > + pattern =3D> "(http://.*|none)", > + format_description =3D> 'URL', > + }, > +}; I'm also not certain whether having all use cases in one property string in the datacenter config... Maybe it would be better to have a proxy standard option for these, that are used in their own use-case-specific configs and then helpers which allow falling back to the global proxy or none at all. This would also allow future 'cascading', e.g., to specify a proxy only for a specific storage. > + > register_standard_option( > 'pve-ha-shutdown-policy', > { > @@ -352,6 +398,12 @@ my $datacenter_schema =3D { > "Specify external http proxy which is used for downloads= (example: 'http://username:password\@host:port/')", > pattern =3D> "http://.*", > }, > + proxy =3D> { > + optional =3D> 1, > + type =3D> 'string', > + description =3D> "Settings for declaring HTTP and HTTPS prox= ies for individual components. When a specific proxy is not specied 'http_p= roxy' will be used instead.", s/specied/specified/ > + format =3D> $proxy_format, > + }, > # FIXME: remove with 8.0 (add check to pve7to8!), merged into "m= igration" since 4.3 > migration_unsecure =3D> { > optional =3D> 1, > @@ -536,6 +588,10 @@ sub parse_datacenter_config { > $res->{replication} =3D parse_property_string($replication_forma= t, $replication); > } > =20 > + if (my $proxy =3D $res->{proxy}) { > + $res->{proxy} =3D parse_property_string($proxy_format, $proxy); > + } > + > if (my $next_id =3D $res->{'next-id'}) { > $res->{'next-id'} =3D parse_property_string($next_id_format, $ne= xt_id); > } > @@ -619,6 +675,10 @@ sub write_datacenter_config { > $cfg->{replication} =3D PVE::JSONSchema::print_property_string($= replication, $replication_format); > } > =20 > + if (ref(my $proxy =3D $cfg->{proxy})) { > + $cfg->{proxy} =3D PVE::JSONSchema::print_property_string($proxy,= $proxy_format); > + } > + > if (defined(my $next_id =3D $cfg->{'next-id'})) { > $next_id =3D parse_property_string($next_id_format, $next_id) if= !ref($next_id); > =20