Re: [pve-devel] [PATCH cluster 1/3] datacenter config: add setting for HTTP{, S} proxies

["Daniel Kral" <d.kral@proxmox.com>]

On Tue Oct 21, 2025 at 12:03 PM CEST, Maximiliano Sandoval wrote:
> Adds a 'proxy' setting which is meant to replace 'http_proxy'. This new
> setting allows to specify different HTTP and HTTPS proxies for different
> pieces of the stack.

AFAICT setting http_proxy and https_proxy - at least for curl - only
specifies the proxy to use for HTTP and HTTPS requests respectively, so
the terms 'HTTP proxy' and 'HTTPS proxy' in itself could be confusing
here and below as one can use many different proxy protocols for either
HTTP or HTTPS requests.

>
> In the UI each option would set both the HTTP and HTTPS proxies together
> to the same value to avoid configuration mistakes, e.g. if only one
> proxy is set.

This should be ideally split into one part, where the ability to add an
HTTPS proxy (additionally to the HTTP proxy) is added, and then adding
the ability to specify different proxies for different use cases (or
vice versa).

>
> The use-case this option intends to cover is a proxy which allows to
> proxy HTTP(S) requests to the outside but will reject any connection to
> resources which are already in the internal network, for this cases the
> 'none' option would declare that no proxy should be used.
>
> The {proxy}->{global} default key of the property string acts as a
> drop-in replacement for the {http_proxy} setting. However, we document
> that this will be used both as a HTTP and a HTTPS proxy which was not
> done always for the 'http_proxy' setting.
>
> Individual proxy configurations accept a 'none' value that allows to say
> that no proxy should be used for this use-case, this takes precedence
> over both the new global proxy and the 'http_proxy'.
>
> Subscriptions only need HTTPS proxies and thus we do not offer the
> option to setup a HTTP proxy here.
>
> Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> ---
>  src/PVE/DataCenterConfig.pm | 60 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 60 insertions(+)
>
> diff --git a/src/PVE/DataCenterConfig.pm b/src/PVE/DataCenterConfig.pm
> index c6d56c1..57c5c1c 100644
> --- a/src/PVE/DataCenterConfig.pm
> +++ b/src/PVE/DataCenterConfig.pm
> @@ -120,6 +120,52 @@ my $notification_format = {
>      },
>  };
>  
> +my $proxy_format = {
> +    'global' => {
> +        default_key => 1,
> +        optional => 1,
> +        type => 'string',
> +        description => "Proxy used as a fallback. It will be used when the respective component does not have a proxy defined. Will be used both as a HTTP and HTTPS proxies.",
> +        pattern => "http://.*",
> +        format_description => 'URL',
> +    },

It might also make sense to have a distinction between a global fallback
for the HTTP and HTTPS protocol, as is done for curl [0]. For example,
this `global` setting here could be the equivalent to `all_proxy` and
then there are fallbacks for both `http_proxy` and `https_proxy`...

[0] https://curl.se/docs/manpage.html#ENVIRONMENT


...But it really depends on how much options we want to expose to users
here. Another idea here could also be to use figure out the type of
proxy ourselves as the `--proxy` option in curl [1] does instead of
allowing to set both the proxy. If we go for that route, we could
introduce a proxy format and then each use case would only be e.g.

    download => get_standard_option('pve-proxy', {
        description => '...',
    }),

Then we would only need to check whether a use case overwrites the
global proxy and otherwise fallback, and afterwards figure out which
type of proxy protocol it is instead of checking both at the same time.

[1] https://curl.se/docs/manpage.html#--proxy

> +    'http-download' => {
> +        optional => 1,
> +        type => 'string',
> +        description => "HTTP proxy used for downloading ISOs and container templates. When set to 'none' no proxy will be used.",
> +        pattern => "(http://.*|none)",

The pattern for the proxies here and below should probably at least also
allow `https://.*` as an option. This could be extended in the future to
also allow socks5, ..., if all the underlying programs where we pass
these values to can handle these proxy types.

In general, at least curl fallbacks to `http://` if there's no protocol
prefix at all [2] [3], so that could also be an option, but not
necessary for an initial implementation.

[2] https://github.com/curl/curl/blob/master/lib/url.c#L2059
[3] https://github.com/curl/curl/blob/master/docs/libcurl/curl_url_set.md#curlu_guess_scheme

> +        format_description => 'URL',
> +    },
> +    'https-download' => {
> +        optional => 1,
> +        description => "HTTPS proxy used for downloading ISOs and container templates. When set to 'none' no proxy will be used.",
> +        type => 'string',
> +        pattern => "(http://.*|none)",
> +        format_description => 'URL',
> +    },
> +    'https-subscription' => {
> +        optional => 1,
> +        description => "HTTPS proxy used for subscription related tasks. When set to 'none' no proxy will be used.",
> +        type => 'string',
> +        pattern => "(http://.*|none)",
> +        format_description => 'URL',
> +    },
> +    'http-apt' => {
> +        optional => 1,
> +        description => "HTTP proxy used for APT. When set to 'none' no proxy will be used.",
> +        type => 'string',
> +        pattern => "(http://.*|none)",
> +        format_description => 'URL',
> +    },
> +    'https-apt' => {
> +        optional => 1,
> +        description => "HTTPS proxy used for APT. When set to 'none' no proxy will be used.",
> +        type => 'string',
> +        pattern => "(http://.*|none)",
> +        format_description => 'URL',
> +    },
> +};

I'm also not certain whether having all use cases in one property string
in the datacenter config... Maybe it would be better to have a proxy
standard option for these, that are used in their own use-case-specific
configs and then helpers which allow falling back to the global proxy or
none at all. This would also allow future 'cascading', e.g., to specify
a proxy only for a specific storage.

> +
>  register_standard_option(
>      'pve-ha-shutdown-policy',
>      {
> @@ -352,6 +398,12 @@ my $datacenter_schema = {
>                  "Specify external http proxy which is used for downloads (example: 'http://username:password\@host:port/')",
>              pattern => "http://.*",
>          },
> +        proxy => {
> +            optional => 1,
> +            type => 'string',
> +            description => "Settings for declaring HTTP and HTTPS proxies for individual components. When a specific proxy is not specied 'http_proxy' will be used instead.",

s/specied/specified/

> +            format => $proxy_format,
> +        },
>          # FIXME: remove with 8.0 (add check to pve7to8!), merged into "migration" since 4.3
>          migration_unsecure => {
>              optional => 1,
> @@ -536,6 +588,10 @@ sub parse_datacenter_config {
>          $res->{replication} = parse_property_string($replication_format, $replication);
>      }
>  
> +    if (my $proxy = $res->{proxy}) {
> +        $res->{proxy} = parse_property_string($proxy_format, $proxy);
> +    }
> +
>      if (my $next_id = $res->{'next-id'}) {
>          $res->{'next-id'} = parse_property_string($next_id_format, $next_id);
>      }
> @@ -619,6 +675,10 @@ sub write_datacenter_config {
>          $cfg->{replication} = PVE::JSONSchema::print_property_string($replication, $replication_format);
>      }
>  
> +    if (ref(my $proxy = $cfg->{proxy})) {
> +        $cfg->{proxy} = PVE::JSONSchema::print_property_string($proxy, $proxy_format);
> +    }
> +
>      if (defined(my $next_id = $cfg->{'next-id'})) {
>          $next_id = parse_property_string($next_id_format, $next_id) if !ref($next_id);
>