From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 612A61FF13E for ; Fri, 03 Apr 2026 18:54:26 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 511F978F3; Fri, 3 Apr 2026 18:54:56 +0200 (CEST) From: Christoph Heiss To: pdm-devel@lists.proxmox.com Subject: [PATCH proxmox/yew-pwt/datacenter-manager/installer v3 00/38] add auto-installer integration Date: Fri, 3 Apr 2026 18:53:32 +0200 Message-ID: <20260403165437.2166551-1-c.heiss@proxmox.com> X-Mailer: git-send-email 2.53.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1775235226071 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.590 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLACK 3 Contains an URL listed in the URIBL blacklist [types.rs] Message-ID-Hash: UGHV6L7XR54FEML6JDPMOUIKN3RCYFTV X-Message-ID-Hash: UGHV6L7XR54FEML6JDPMOUIKN3RCYFTV X-MailFrom: c.heiss@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This series adds integration with our automated installer [0] for all our products. With this, Proxmox Datacenter Manager can be used for serving answer files via HTTPS in an automated fashion. It provides three panels under the "Automated Installations" tab in the "Remotes" menu: - Automated Installations: List all past and currently ongoing installations. - Prepared Answers: Enables users to do the whole CRUD cycle for prepared answers, i.e. create new answers (from scratch or based on existing ones), edit and delete. - Authentication Tokens: Simple token for the /answer endpoint. What tokens can be used to obtain which answer can be set when editing a prepared answer under the "Authentication" tab. Permission-wise, everything is currently scoped under /system/auto-installation. User interface ============== Happy about feedback regarding the UI, especially the editing dialog! By the nature of it, the auto-installer has a lot more options than the standard GUI/TUI installer, and it's kind of hard to not fully cramp up the UI while still providing all options. The viewing of the system information and post-installation hook notification data is currently maybe a bit bare-bones. For the first one, displaying it as a tree could work, if we don't want to simply display raw JSON documents to the user. Answer endpoint =============== The endpoint for serving answer files to the automated installer lives under `/api2/json/auto-install/answer`. This makes it (unfortunately) incompatible with older ISOs, as currently they only support answer files formatted as TOML. See also the discussion in [1]. We could serve TOML (as done in previous revisision), but the introduction of authentication tokens [2] for the /answer endpoint breaks backwards-compatibility anyway, rendering the whole endeavour moot - so I removed TOML support entirely. Installer changes ================= Most of the changes to pve-installer are churn from switching to the moved types in proxmox-installer-types. They were heavily intertwined between the different crates in pve-installer, so a) untwisting that, b) doing some long-overdue quality-of-life improvements and c) making them #[api] ready created a lot of churn, although nothing in the actual TOML/JSON interface changed for any of these types. Other possible future work ========================== - Target filter selection by dropdown/tree. Based on the system information schema, a list of all possible JSON pointers, or even a tree, could be generated and displayed to the user, in a form consisting of the key, a text input for the value/glob to match and a set of add/remove buttons for each filter. - Display/saving the installation progress/log. The progress log generated by the auto-installer can be sent to PDM, showing it there similar to tasks. - Implement a "wait for answer" system. Core idea here is that the installer waits for PDM to provide in answer by implemented e.g. an retry system. The use case is for to be able to effectively install systems on demand, without the need for preparations. - Providing the option for automatically adding the installed node to PDM (for PVE/PBS) - Maybe mTLS authentication for the answer endpoint, as suggested by Lukas/Thomas? [0] https://pve.proxmox.com/wiki/Automated_Installation [1] https://lore.proxmox.com/pdm-devel/11b27b4a-7e1f-4af9-8384-12d54d72ef17@proxmox.com/ [2] https://lore.proxmox.com/pdm-devel/DETMUXY1Q877.32G593TWC52WW@proxmox.com/#:~:text=%20I%20think%20this%20is%20dangerous History ======= v2: https://lore.proxmox.com/pdm-devel/20251205112528.373387-1-c.heiss@proxmox.com/ v1: https://lore.proxmox.com/pdm-devel/20251204125122.945961-1-c.heiss@proxmox.com/ Notable Changes v2 -> v3: * moved "Automated Installations" panels to a tab under "Remotes" * added authentication to answer endpoint * answers are now served as JSON unconditionally * moved installations state file to /var/lib/proxmox-datacenter-manager/ * added required pve-installer changes * ui: use `DataTable`s instead of text areas for filter entries * ui: use `PdmClient` for interacting with API instead of sending requests directly Notable Changes v1 -> v2: * add documentation patch * fixed compilation due to leftover type Diffstat ======== proxmox: Christoph Heiss (12): api-macro: allow $ in identifier name schema: oneOf: allow single string variant schema: implement UpdaterType for HashMap and BTreeMap network-types: move `Fqdn` type from proxmox-installer-common network-types: implement api type for Fqdn network-types: add api wrapper type for std::net::IpAddr network-types: cidr: implement generic `IpAddr::new` constructor network-types: fqdn: implement standard library Error for Fqdn node-status: make KernelVersionInformation Clone + PartialEq installer-types: add common types used by the installer installer-types: add types used by the auto-installer installer-types: implement api type for all externally-used types Cargo.toml | 2 + proxmox-api-macro/src/util.rs | 2 +- proxmox-installer-types/Cargo.toml | 28 + proxmox-installer-types/debian/changelog | 5 + proxmox-installer-types/debian/control | 67 + proxmox-installer-types/debian/debcargo.toml | 7 + proxmox-installer-types/src/answer.rs | 1199 ++++++++++++++++++ proxmox-installer-types/src/lib.rs | 182 +++ proxmox-installer-types/src/post_hook.rs | 215 ++++ proxmox-network-types/Cargo.toml | 3 +- proxmox-network-types/debian/control | 8 +- proxmox-network-types/src/fqdn.rs | 257 ++++ proxmox-network-types/src/ip_address.rs | 73 +- proxmox-network-types/src/lib.rs | 1 + proxmox-node-status/src/types.rs | 2 +- proxmox-schema/src/schema.rs | 78 +- proxmox-schema/tests/schema.rs | 91 +- 17 files changed, 2207 insertions(+), 13 deletions(-) create mode 100644 proxmox-installer-types/Cargo.toml create mode 100644 proxmox-installer-types/debian/changelog create mode 100644 proxmox-installer-types/debian/control create mode 100644 proxmox-installer-types/debian/debcargo.toml create mode 100644 proxmox-installer-types/src/answer.rs create mode 100644 proxmox-installer-types/src/lib.rs create mode 100644 proxmox-installer-types/src/post_hook.rs create mode 100644 proxmox-network-types/src/fqdn.rs proxmox-yew-widget-toolkit: Christoph Heiss (1): widget: kvlist: add widget for user-modifiable data tables src/widget/key_value_list.rs | 429 +++++++++++++++++++++++++++++++++++ src/widget/mod.rs | 3 + 2 files changed, 432 insertions(+) create mode 100644 src/widget/key_value_list.rs proxmox-datacenter-manager: Christoph Heiss (11): api-types, cli: use ReturnType::new() instead of constructing it manually api-types: add api types for auto-installer integration config: add auto-installer configuration module acl: wire up new /system/auto-installation acl path server: api: add auto-installer integration module server: api: auto-installer: add access token management endpoints client: add bindings for auto-installer endpoints ui: auto-installer: add installations overview panel ui: auto-installer: add prepared answer configuration panel ui: auto-installer: add access token configuration panel docs: add documentation for auto-installer integration Cargo.toml | 5 + cli/client/src/pbs.rs | 10 +- cli/client/src/pve.rs | 15 +- cli/client/src/remotes.rs | 5 +- debian/control | 5 + docs/automated-installations.rst | 124 ++ docs/index.rst | 1 + lib/pdm-api-types/Cargo.toml | 3 + lib/pdm-api-types/src/acl.rs | 4 +- lib/pdm-api-types/src/auto_installer.rs | 415 ++++++ lib/pdm-api-types/src/lib.rs | 10 +- lib/pdm-buildcfg/src/lib.rs | 10 + lib/pdm-client/src/lib.rs | 232 ++++ lib/pdm-config/Cargo.toml | 3 + lib/pdm-config/src/auto_install.rs | 559 ++++++++ lib/pdm-config/src/lib.rs | 1 + lib/pdm-config/src/setup.rs | 7 + server/Cargo.toml | 4 + server/src/api/auto_installer/mod.rs | 1218 +++++++++++++++++ server/src/api/mod.rs | 2 + ui/Cargo.toml | 2 + .../configuration/permission_path_selector.rs | 1 + .../auto_installer/installations_panel.rs | 305 +++++ ui/src/remotes/auto_installer/mod.rs | 86 ++ .../prepared_answer_add_wizard.rs | 192 +++ .../prepared_answer_edit_window.rs | 187 +++ .../auto_installer/prepared_answer_form.rs | 875 ++++++++++++ .../auto_installer/prepared_answers_panel.rs | 248 ++++ ui/src/remotes/auto_installer/token_panel.rs | 476 +++++++ .../remotes/auto_installer/token_selector.rs | 137 ++ ui/src/remotes/mod.rs | 10 + 31 files changed, 5122 insertions(+), 30 deletions(-) create mode 100644 docs/automated-installations.rst create mode 100644 lib/pdm-api-types/src/auto_installer.rs create mode 100644 lib/pdm-config/src/auto_install.rs create mode 100644 server/src/api/auto_installer/mod.rs create mode 100644 ui/src/remotes/auto_installer/installations_panel.rs create mode 100644 ui/src/remotes/auto_installer/mod.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_add_wizard.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_edit_window.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_form.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answers_panel.rs create mode 100644 ui/src/remotes/auto_installer/token_panel.rs create mode 100644 ui/src/remotes/auto_installer/token_selector.rs pve-installer: Christoph Heiss (14): install: iso env: use JSON boolean literals for product config common: http: allow passing custom headers to post() common: options: move regex construction out of loop assistant: support adding an authorization token for HTTP-based answers tree-wide: used moved `Fqdn` type to proxmox-network-types tree-wide: use `Cidr` type from proxmox-network-types tree-wide: switch to filesystem types from proxmox-installer-types post-hook: switch to types in proxmox-installer-types auto: sysinfo: switch to types from proxmox-installer-types fetch-answer: switch to types from proxmox-installer-types fetch-answer: http: prefer json over toml for answer format fetch-answer: send auto-installer HTTP authorization token if set tree-wide: switch out `Answer` -> `AutoInstallerConfig` types auto: drop now-dead answer file definitions Cargo.toml | 8 + Proxmox/Install/ISOEnv.pm | 16 +- proxmox-auto-install-assistant/Cargo.toml | 1 + proxmox-auto-install-assistant/src/main.rs | 35 +- proxmox-auto-installer/Cargo.toml | 2 + proxmox-auto-installer/src/answer.rs | 499 ------------- .../src/bin/proxmox-auto-installer.rs | 20 +- proxmox-auto-installer/src/lib.rs | 2 - proxmox-auto-installer/src/sysinfo.rs | 91 +-- proxmox-auto-installer/src/udevinfo.rs | 11 - proxmox-auto-installer/src/utils.rs | 154 ++-- proxmox-auto-installer/tests/parse-answer.rs | 6 +- .../tests/resources/iso-info.json | 4 +- .../ipv4_and_subnet_mask_33.json | 2 +- ...rface_pinning_overlong_interface_name.json | 2 +- proxmox-chroot/Cargo.toml | 1 + proxmox-chroot/src/main.rs | 60 +- proxmox-fetch-answer/Cargo.toml | 2 +- .../src/fetch_plugins/http.rs | 116 ++- proxmox-fetch-answer/src/main.rs | 21 +- proxmox-installer-common/Cargo.toml | 2 + proxmox-installer-common/src/disk_checks.rs | 5 +- proxmox-installer-common/src/dmi.rs | 43 ++ proxmox-installer-common/src/http.rs | 40 +- proxmox-installer-common/src/lib.rs | 6 +- proxmox-installer-common/src/options.rs | 365 +++------ proxmox-installer-common/src/setup.rs | 103 +-- proxmox-installer-common/src/sysinfo.rs | 52 -- proxmox-installer-common/src/utils.rs | 382 ---------- proxmox-post-hook/Cargo.toml | 4 +- proxmox-post-hook/src/main.rs | 691 +++++++----------- proxmox-tui-installer/Cargo.toml | 2 + proxmox-tui-installer/src/main.rs | 12 +- proxmox-tui-installer/src/options.rs | 23 +- proxmox-tui-installer/src/setup.rs | 5 +- proxmox-tui-installer/src/views/bootdisk.rs | 44 +- proxmox-tui-installer/src/views/mod.rs | 21 +- proxmox-tui-installer/src/views/network.rs | 15 +- 38 files changed, 779 insertions(+), 2089 deletions(-) delete mode 100644 proxmox-auto-installer/src/answer.rs delete mode 100644 proxmox-auto-installer/src/udevinfo.rs create mode 100644 proxmox-installer-common/src/dmi.rs delete mode 100644 proxmox-installer-common/src/sysinfo.rs delete mode 100644 proxmox-installer-common/src/utils.rs Summary over all repositories: 88 files changed, 8540 insertions(+), 2132 deletions(-) -- Generated by murpp 0.11.0+ch1