Re: [PATCH http-server 0/1] fix pveproxy OOM during PDM cross-cluster migration to LVM-thin

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

Quoting Kefu Chai (2026-04-12 13:12:08)
> pveproxy on the destination host can be OOM-killed during PDM offline
> cross-cluster migration when the target storage (e.g. LVM-thin) writes more
> slowly than data arrives over the network [1].
> 
> the websocket proxy already had wbuf_max set on the backend handle, but it
> turns out not to help. AnyEvent::Handle only checks it inside the
> `if (!$self->{_ww})` guard in _drain_wbuf:
> 
>     if (!$self->{_ww} && length $self->{wbuf}) {
>         ...
>         if (defined $self->{wbuf_max} && $self->{wbuf_max} < ...) {
>             $self->_error(Errno::ENOSPC, 1);
>         }
>     }
> 
> once the first EAGAIN installs the write watcher (_ww), all subsequent
> push_write calls return immediately without reaching the check, so wbuf
> grows without bound.
> 
> to fix this, follow the same approach response_stream() already takes: stop
> reading from the source handle when the backend write buffer exceeds the
> limit, and resume via on_drain once it empties.
> 
> handle_spice_proxy_request() has the same issue and even carries a
> "# todo: use stop_read/start_read" comment acknowledging it, but is not
> addressed here as SPICE carries interactive VM console traffic rather than
> bulk data.

note that in regular PVE (and also PDM), the websocket proxy is used for
proxying console sessions for nodes and guests. so in principle, the same
applies for both - but it is probably harder to trigger in practice via such
connections ;)

spice does potentially handle bulk streams of data as well though - e.g., it
supports passing through USB devices from the client to the spice session..

> tested with a synthetic AnyEvent script that drives a fast writer through a
> proxy into a slow reader. without backpressure, the proxy write buffer grows
> to ~1.4 GB in 5 seconds (2254x the limit). with the fix, it stays bounded at
> just over the 640 KB limit, as expected.
> 
> [1] https://bugzilla.proxmox.com/show_bug.cgi?id=7483
> 
> Kefu Chai (1):
>   fix #7483: apiserver: add backpressure to websocket proxy
> 
>  src/PVE/APIServer/AnyEvent.pm | 45 +++++++++++++++++++++++++++++------
>  1 file changed, 38 insertions(+), 7 deletions(-)
> 
> -- 
> 2.47.3
> 
> 
> 
> 
>