From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id B28141FF0AF for ; Thu, 08 Oct 2026 09:36:07 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id B029D212F7; Thu, 08 Oct 2026 09:36:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1791382222; x=1791987022; darn=lists.proxmox.com; h=disposition-notification-to:thread-index:in-reply-to:content-type :mime-version:message-id:date:subject:cc:to:from:return-receipt-to :from:to:cc:subject:date:message-id:reply-to:content-type; bh=I4e4OMObbc5urmf8VetJB/KuZfPMjsHhDS3ITe8hSUw=; b=q0pAA7ciMcfOZEuG5k8o6uHyfA/b6VTprjiN7hUQ/Bl/PmhO2TPAIQ7vBHO1GVRBp/ pWbgdkusQ1/ZP5CUs3BD1rdjm96yS08VkTUpnicGUQ4zG/M5Mxfb38VCf+mK3xxIa48n fzfwTjYnmugXMvVM3sXFbAdnjPcF114eF7FnFSUhtPOltR4auscW0trXXEIOOtEklAXP 6wn8iaslTqSZZm7Tz93oHP6NEdzIGQHsrTJJ1KMntGC4O+nGG4AU/vu2Mw6GjFMEOwro Y84aH3nsrfqNB+UdsE681iWXlLLMUbQ385iCYsjAtDN7M3LhIFSgoncjDvQHBIImTZTX mLag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791382222; x=1791987022; h=disposition-notification-to:thread-index:in-reply-to:content-type :mime-version:message-id:date:subject:cc:to:from:return-receipt-to :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to:content-type; bh=I4e4OMObbc5urmf8VetJB/KuZfPMjsHhDS3ITe8hSUw=; b=inpHDyPJBMvOybiCVKnIEgZH6GuT3IV9f0aAAU+hnmnnb/1ej3KP2LtsGUAWJfSkLl j+nRlhPT3+YyP5A8Hn7v7LynF07yQyV9/AzrnDbEBslzytDfg/zyP9XXgscUjwJhdOQ+ NrXkct+hiz07Q3/lNSJYVdNo78ym/zcdEkRLfF82suljNX8gZA+J1Q197P0G2vvbDoQT xm5x6Q052NHJZpgJO9swihAwyZM2KGjoP8d5PxvrGNqA+PzHeuiJNs3P07pgkVrS0rvF Sue60U+KaoR19pLVwKwnJx/vY4JTb1qXMq3cI6rv3h3RefGxs9Md7TJ6COuQI3rY4V97 Vleg== X-Gm-Message-State: AFq9FYJ7qesY03/frBGQcYQ09KPeyjGxgzZQcKLLbMkO63FyeZbsMbbG V4invTHvihM4+DBZT7CL+vRjmtbvwB2vSDoTLT9GWv04ny4/LIuk59RwLpEgKYn4 X-Gm-Gg: AYBFou3uRqvQFPebo0K9DDJJydRAwY9KenSuci1oEcLlJAx7crlfAxq0D0QM1YYTkaG +4PPhw+IrTaZgT/XDWufOnzZS9E8cOTYxjTI9Sc+Vu/l65w0+xFiaQ7vF1DH+5iU6rLGNft8Qfm AvOD0rggOTwlqvCMk3OrlzFagqV5S49p0lfTBdUdDr7jBz+ivxlUUQ2UtTZ+egr2C+Mt1LTMIcz XZgotoY+oPZ+y0EhgP4PvTI7wRq6c2OecPVQL6CU0nbxBjfz+szbDtPCJKdgoa4ISZtd9Rqq3nt dBtMjSlObjuEwkCpOnzUFOakDWmKatC3Jju10/f5/mskmBXoo0nnQ1h6RL1rzvxpPAZ6aF4Lhf5 4plHwkk0MTuAePdGDSp+H1Gpvrn1m1vE6UvTOCN67xuLVXT3WtJiCYmHfHrSWkF1jfqfzOiizF5 TQHadrN9AuIo9c77RzNDkvU7avQy8a6oiWCIweYaOdOxLyZHhYnz2inkOLL9CU+okC40NJryqD/ 29s5g+u3y1w8WR1ow== X-Received: by 2002:a05:6402:a25b:10b0:6ad:795a:f94e with SMTP id 4fb4d7f45d1cf-6afd841af12mr4363786a12.16.1791382220940; Wed, 07 Oct 2026 07:10:20 -0700 (PDT) From: "Dmitry R" To: subject: SPAM: RE: [RFC] WebRTC console for VMs via QEMU's D-Bus display Date: Wed, 7 Oct 2026 17:10:17 +0300 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook, Build 11.0.6353 In-Reply-To: Thread-Index: Ad1Rt5q3D0eMibptQTe+h8//ek1mBAErK6kA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-SPAM-LEVEL: Spam detection results: 3 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_ENVFROM_END_DIGIT 1 Envelope-from freemail username ends in digit FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider GB_FREEMAIL_DISPTO 0.499 Disposition-Notification-To/From or Disposition-Notification-To/body contain different freemails GB_FREEMAIL_NUM 0.75 Freemail spammy address HTML_MESSAGE 0.001 HTML included in message MANY_SPAN_IN_TEXT 1 Many tags embedded within text POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record X-MailFrom: rdmitry0911@gmail.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation Message-ID-Hash: OEWG6KWAON72K5FIJAC5XYILMLURA5WW X-Message-ID-Hash: OEWG6KWAON72K5FIJAC5XYILMLURA5WW X-Mailman-Approved-At: Thu, 08 Oct 2026 09:35:37 +0200 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 3.3.10 X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi all, hi Alexandre, only after sending this RFC I noticed that Alexandre posted "add rdp && kyber consoles for qemu over D-Bus display" in August [1]. Both proposals need the same foundation in qemu-server -- QEMU's D-Bus display -- and differ in what consumes it. Rather than two competing ways to enable it, I would suggest one shared base with the consoles on top. What the two series have in common: * qemu-server enables '-display dbus' for a VM, and a separate package attaches to it and streams the guest to the browser (RDP and Kyber daemons there, a WebRTC media service here); * with GL, QEMU refuses VNC and SPICE next to the D-Bus display, so both have to deal with a VM that has no VNC console. Where they differ, and why I think Alexandre's base is the better one: * His series runs a private dbus-daemon per VM; mine uses p2p=yes and attaches every console with getfd + add_client. * With p2p=yes, each add_client replaces the previous connection, so only one consumer can be attached at a time. On a bus, RDP, Kyber and WebRTC can attach to the same VM side by side. * With p2p=yes, QEMU 11.0 did not deliver audio in my tests: the AudioOutListener gets the Init/SetEnabled calls made while RegisterOutListener runs, then nothing (the same with a plain GDBus client, so it does not look like a problem of my listener). On a private bus, which my add-on uses today, both playback and the microphone (AudioInListener) work. A possible split: 1. qemu-server: the per-VM D-Bus display (Alexandre's DBusDisplay.pm), the option that enables it, 'rendernode' for virtio-gl (bug #4771) and a 'dbus' driver for audio0. 2. One package per console, each attaching as a client on that bus: RDP and Kyber (Alexandre), WebRTC (mine, bug #8106), with their API calls and UI entries in qemu-server and pve-manager. I am happy to rebase the WebRTC part onto Alexandre's DBusDisplay.pm and drop my p2p attach and the separate 'dbus' VGA flag. Alexandre wrote that the DMA-BUF path is untested on NVIDIA, so I tried it on an RTX 3080 (driver 595.91, nvidia-vaapi-driver 0.0.16, QEMU 11.0, a virtio-vga-gl guest on a private per-VM bus; kyber-qemu-server and txproto built from the v2 patch against Debian trixie's FFmpeg 7.1, encoding to a file): - --dmabuf (h264_vaapi): QEMU hands over the scanout (XB24/AB24 with the NVIDIA block-linear modifier 0x300000000e08014), but the VAAPI frames context fails on the first frame -- "Failed to create surface: 14 (the requested RT Format is not supported)" -- and nothing is encoded. With --encoder h264_nvenc it ends the same way, since the path goes through VAAPI whatever the encoder. - The cause is the driver: on NVIDIA, VAAPI means nvidia-vaapi-driver, which only decodes (VAEntrypointVLD): no encoding, no video processing, no RGB surfaces. The mmap fallback would not help either, as a block-linear buffer does not read as linear memory. - Without --dmabuf, the listener received no scanout at all from the GL display within 15 s (libx264 and h264_nvenc alike). - Control on the same VM: my worker imports the same DMA-BUF with EGL (eglCreateImageKHR, EGL_LINUX_DMA_BUF_EXT with the modifier), reads it back with glReadPixels and encodes it with NVENC -- that works. So with a virtio-gl guest on NVIDIA, the Kyber console shows no picture today. A fix, tested [2]: a patch on top of v2 adds --dmabuf-readback to kyber-qemu-server (358 lines, mostly one new file, LGPL like dmabuf.c). It still requests DMA-BUF scanouts, imports each buffer with EGL (GBM platform, surfaceless context, the modifier passed), reads it back with glReadPixels and feeds your existing software path, so any --encoder works -- h264_nvenc on NVIDIA. A buffer is imported once and read only after QEMU reports damage. Same setup, a terminal running 'top -d 0.1' for 20 s: 1226 frames encoded with h264_nvenc, 0 dropped, the picture correct; about 24% of one CPU core (36% with libx264). The --dmabuf path is unchanged; kycontroller could choose '--dmabuf-readback --encoder h264_nvenc' when the render node belongs to the nvidia driver. To be fair about the trade-off: the readback copies every changed frame into system memory, so on AMD and Intel your zero-copy VAAPI path is likely the more efficient one. The best of both would be zero-copy where VAAPI can import and encode, and the EGL readback everywhere else -- the same split would serve the WebRTC console. Maintainers: would one shared base series like this work for you, and is there a preferred name and place for the option that enables the D-Bus display? [1] https://lore.proxmox.com/pve-devel/20260826074347.1256659-1-alexandre.derumi er@groupe-cyllene.com/ [2] https://github.com/rdmitry0911/qsm-rd/blob/main/qsm-direct/upstream/kyber-nv idia/0001-kyber-qemu-server-read-DMA-BUF-scanouts-back-through.patch Thanks, Dmitry _____ From: Dmitry R [mailto:rdmitry0911@gmail.com] Sent: Thursday, October 01, 2026 6:11 PM To: pve-devel@lists.proxmox.com Subject: [RFC] WebRTC console for VMs via QEMU's D-Bus display Hi all, I would like to ask whether a hardware-encoded WebRTC console for VMs would be welcome upstream, and if so, in which shape. It runs today as an add-on for PVE 9.2, and I have prepared the qemu-server and pve-manager parts as patch series against current master (tested on PVE 9.2 with an RTX 3080, see below) -- but I'd like your opinion on the design before sending them. Feature request: bug #8106 https://bugzilla.proxmox.com/show_bug.cgi?id=8106 Motivation ---------- noVNC is the right console for firmware, installers and text consoles. For time spent inside a running desktop it transfers changed rectangles instead of video, has no audio, and a VirGL (virtio-gl) guest reaches it through egl-headless readback. Measured with one guest image and the same workloads (method and raw numbers in the repository below): WebRTC, virtio-gl noVNC, std VGA 720p video, pictures/s 30 (source rate) 7.4 scrolling, pictures/s 36 (52 on NVENC) 12.0 animated UI, pictures/s 38 (45 on NVENC) 11.0 bandwidth, 720p video 18.8 Mbit/s 120 Mbit/s single pointer event 88 ms 52 ms console open -> 1st picture 3.4 s 1.0 s The last two rows are where noVNC stays better: a single isolated event is one small rectangle for VNC, but a full encode/decode round trip for video (the 88 ms were measured with software libx264 on a CPU-only node). With NVENC (RTX 3080) the encoder used 4% while streaming the animated scene. What QEMU already provides -------------------------- QEMU's D-Bus display (-display dbus) exports the guest scanout -- the GL/DMA-BUF scanout of virtio-gl included -- and takes keyboard, mouse, multi-touch, clipboard and audio. With p2p=yes there is no bus daemon: a client is attached per connection through QMP (getfd + add_client), which PVE::QMPClient can already do (it passes the fd for getfd). Proposed series --------------- 1. qemu-server: 'rendernode' for the VGA property vga: virtio-gl,rendernode=/dev/dri/renderD129 so a virtio-gl guest can be placed on a chosen GPU instead of the first render node (fixes #4771). Small and useful on its own. 2. qemu-server: opt-in 'dbus' for the VGA property vga: virtio-gl,dbus=1 adds '-display dbus,p2p=yes' (gl=on with the render node for virtio-gl, gl=off for the other types). One constraint: with GL, QEMU refuses VNC and SPICE next to it ("Display vnc is incompatible with the GL context"), so a virtio-gl VM with dbus=1 has neither and offers the WebRTC console instead; non-GL types keep VNC as well. The clipboard reaches the guest through qemu-vdagent (as for the VNC clipboard) and the guest's spice-vdagent. Audio is not part of the series yet: the add-on plays it through a bus-mode D-Bus display (a 'dbus' audiodev named in -display), but with p2p=yes our listener got no audio from QEMU 11.0 -- only the Init/SetEnabled calls made while RegisterOutListener runs, nothing afterwards. A 'dbus' audio0 driver is ready once that is resolved. 3. qemu-server: API call to attach a console POST /nodes/{node}/qemu/{vmid}/webrtcproxy (VM.Console) takes the browser's SDP offer, creates a socketpair, attaches one end to QEMU (getfd + add_client) and hands the other end with the offer to a local media service over a root-owned unix socket (SCM_RIGHTS); returns the SDP answer. Authentication and ACL stay exactly those of vncproxy/termproxy -- no extra port, no own auth. The status API reports 'dbus' and 'vnc' so the UI knows which consoles a VM offers. 4. pve-manager: WebRTC console type next to noVNC/SPICE/xterm.js in the console button and the VM's Console panel, offered when the VM's display has dbus=1; the display editor gets the two options. Like noVNC and xterm.js, the viewer itself is a page from a separate package that pveproxy serves ('?console=kvm&webrtc=1'). 5. A separate package with the media service: capture from the D-Bus display (the virtio-gl DMA-BUF scanout is imported through EGL and read back; a zero-copy path to the encoder is future work), encode (NVENC, VA-API, QSV; libx264 fallback; H.264, HEVC where the browser decodes it), Opus audio, WebRTC. Today it is C++ (FFmpeg) plus a Python/aiortc WebRTC bridge; I realise Python/aiortc is unusual in your stack. Questions --------- - Is this something you would consider upstream at all? - Property naming: 'rendernode' and 'dbus' inside 'vga', or a separate display property? - Would you rather keep the media service out of tree (users install it, the API call returns an error when it is absent), or should it eventually live in a Proxmox-maintained package, and would you then want it in Rust? - Is 'webrtcproxy' next to 'vncproxy'/'termproxy' the right place? Not part of this proposal: the prototype also has a graphical console for LXC containers (a headless compositor in the container with its own login screen, one console per tty like the terminal console). It needs software inside the guest, so I plan to keep it as a separate add-on package. Tested end to end on PVE 9.2 (qemu-server 9.2.7 and pve-manager 9.2.11 plus the series): a Kubuntu 26.04 guest with 'vga: virtio-gl,dbus=1' streams 1280x800 at 60 fps (NVENC H.264) through webrtcproxy, in a popup and in the VM's Console panel, for a user holding only PVEVMUser on the VM (403 without VM.Console). Code and measurements: https://github.com/rdmitry0911/qsm-rd (qsm-direct/, docs/COMPARISON.md for the numbers; the prepared series are in qsm-direct/upstream/patches/). Transparency: I developed the prototype with the help of an AI coding assistant (Claude). I have reviewed and tested the code and will sign the Harmony CLA; please tell me if that is a problem for you. Thanks, Dmitry