From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 377641FF0AE for ; Tue, 27 Oct 2026 07:44:58 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 231F2213A6; Tue, 27 Oct 2026 07:44:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=taktcycles.com; s=takt1; t=1793083480; bh=u+YNnN4rmLREKRu2iGYPpdn5j/9dFyp0z6DJMgVI8IQ=; h=From:Reply-To:To:Cc:Subject:In-Reply-To:References:Date:From; b=SnYBpXk2zzmy8pgcyjYOO/pUGn2IXaUS5a72l+qcRK/6M8XGLr2Xm1SgLDDlJsyoG 8dTbVzYp89So/lklkTXBzr1EoQU6ceQcBumAhoicjR0PSvoO3KAdfMaGuNhgW34KBv DB2r3u8eRN13UQ2CjAaW4zOY2+pMbiZT8k1d8Gdcv9D3KpT28XOxQ+eWAZNPbjKGB3 EyWBT5mdbfzcIepoMC9yRu1hTWVvS2fZ5aIlGh6497URn/ZBFZx0qbXecdPkaf3Cx5 xofT+mYKX+JoCdnrEJTuKKRleNBpei+9QGpLHdKmlyJamuwnHe6Z8YTjs6VIiEr8JZ oXdmL6cUhI9uw== From: Dmitry R To: pve-devel@lists.proxmox.com Subject: Re: [RFC] WebRTC console for VMs via QEMU's D-Bus display In-Reply-To: =?utf-8?q?=3C179293981837=2E3090704=2E6601261792618612901=2Eqsm?= =?utf-8?q?-stock4=2E13-update=40taktcycles=2Ecom=3E?= References: =?utf-8?q?=3C179293981837=2E3090704=2E6601261792618612901=2Eqsm-stock4=2E13?= =?utf-8?q?-update=40taktcycles=2Ecom=3E?= Date: Tue, 27 Oct 2026 06:41:00 +0000 Message-ID: <179308326004.3823774.17621657443116900285.qsm-exact-ui-exp6-update@taktcycles.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 2 AWL 0.387 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_FORGED_REPLYTO 1 Freemail in Reply-To, but not From FREEMAIL_REPLYTO_END_DIGIT 0.25 Reply-To freemail username ends in digit FROM_FMBLA_NEWDOM28 0.799 From domain was registered in last 14-28 days SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: CAS37XTTER24NHGXJWUOGL5YYGJD25LL X-Message-ID-Hash: CAS37XTTER24NHGXJWUOGL5YYGJD25LL X-MailFrom: rdmitry@taktcycles.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: Dmitry R List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi Alexandre, hi all, Following the stock4.13 update, I have an experimental mixed-content path in the QSM consumer. It uses the same capture surface and does not change the proposed shared per-VM D-Bus foundation or require new PVE patches. The existing full-desktop video remains. An optional lossless pixel-tile layer carries compressible UI regions over a separate reliable WebRTC data channel. Frequently changing tiles stay in video; settled regions can be refined. There is no OCR, replacement font rendering or application text extraction, and this stage needs no additional guest-package functionality. The implementation bounds geometry, tile payloads, decompression, shared cache and each viewer's pending state/SCTP queue. The worker and cached replay reserve a small share of the current bandwidth target; the worker's UI bytes also debit its video budget. New changes take priority over idle refinement. Gaps/heartbeat expiry clear stale cache state and request a snapshot. Epochs and serials invalidate late browser decodes after clears, geometry changes or disable. Old/new worker and bridge combinations were checked for video/input compatibility. I specifically tested a separate text-input window overlapping a playing clip in a real Wayland LXC/Firefox session, with H.264/NVENC and the existing TURN/TCP 8006 gateway. In one controlled run, median key-to-visible-marker response changed from 157 to 59 ms at a 20 Mbit/s downlink, and 176 to 75 ms at 2 Mbit/s, with 15 keys per case and LAN RTT. The guest checked the full input string. A 512x128 text crop matched all 65,536 source pixels. Moving, scrolling, resizing and closing the overlapping window passed the recorded checks; the uncovered clip continued producing different pictures. These are small laboratory results, not a comparison with RDP or an Internet latency guarantee. Large-window refinement still took about 4.5 s at the narrow target. This stage still video-encodes the whole desktop and has no CopyRect/scroll protocol, compositor window hints or cropped video substream. Tile/video presentation is not frame-synchronous. Equivalent live VM, HEVC, QSV and VA-API mixed-content runs remain outstanding. The experimental package set is 0.4.0+stock4.14~exp6. It is off by default; the viewer has an Experimental exact UI checkbox and there is an explicit node-default override. An explicit browser choice takes precedence. The installed stock-PVE service path was also checked with H.264/libx264. The new layer uses DTLS and adds no public port in the existing 8006 gateway. Source, matching debs, design/installation notes and qualification: https://github.com/rdmitry0911/qsm-rd/releases/tag/v0.4.0-qsm-direct-20261027= -exact-ui-exp6 27 CTest gates and 83 bridge tests pass. The normal signed APT preview remains stock4.13; this experimental snapshot is distributed separately. The Debian review source and pending native-integration proposal are not superseded. Feedback on region classification, occlusion and integrating future compositor hints with the shared display consumer would be useful. Project sponsor: Takt, https://taktcycles.com/ Thanks, Dmitry