From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 22D2D1FF13C for ; Thu, 30 Apr 2026 14:47:32 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D62256469; Thu, 30 Apr 2026 14:47:31 +0200 (CEST) From: Christoph Heiss To: pdm-devel@lists.proxmox.com Subject: [PATCH datacenter-manager/installer/proxmox/yew-comp v4 00/40] add auto-installer integration Date: Thu, 30 Apr 2026 14:46:29 +0200 Message-ID: <20260430124712.1614305-1-c.heiss@proxmox.com> X-Mailer: git-send-email 2.53.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1777553141427 X-SPAM-LEVEL: Spam detection results: 0 AWL -1.580 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLACK 3 Contains an URL listed in the URIBL blacklist [types.rs] URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [mod.rs] Message-ID-Hash: DYAR7RHFMT2DP5QIGNDWZ6SVAQ3ZURCG X-Message-ID-Hash: DYAR7RHFMT2DP5QIGNDWZ6SVAQ3ZURCG X-MailFrom: c.heiss@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Datacenter Manager development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This series adds integration with our automated installer [0] for all our products. With this, Proxmox Datacenter Manager can be used for serving answer files via HTTPS in an automated fashion. It provides three panels under the "Automated Installations" tab in the "Remotes" menu: - Automated Installations: List all past and currently ongoing installations. - Prepared Answers: Enables users to do the whole CRUD cycle for prepared answers, i.e. create new answers (from scratch or based on existing ones), edit and delete. - Authentication Tokens: Simple token for the /answer endpoint. What tokens can be used to obtain which answer can be set when editing a prepared answer under the "Authentication" tab. When a new token was created, either through the dedicated panel or directly while creating a new prepared answer, the secret itself as well as the command line for preparing an installation ISO is shown in the UI. Permission-wise, everything is currently scoped under /system/auto-installation. I can provide pre-built packages for testing, if needed - just holler at me. User interface ============== By the nature of it, the auto-installer has a lot more options than the standard GUI/TUI installer, and it's kind of hard to not fully cramp up the UI while still providing all options. The viewing of the system information and post-installation hook notification data could be maybe considered a bit bare-bones. For the first one, displaying it as a tree could work, if we don't want to simply display raw JSON documents to the user. Answer endpoint =============== The endpoint for serving answer files to the automated installer lives under `/api2/json/auto-install/answer`. This makes it (unfortunately) incompatible with older ISOs, as currently they only support answer files formatted as TOML. See also the discussion in [1]. Installer changes ================= Most of the changes to pve-installer are churn from switching to the moved types in proxmox-installer-types. They were heavily intertwined between the different crates in pve-installer, so a) untwisting that, b) doing some long-overdue quality-of-life improvements and c) making them #[api] ready created a lot of churn, although nothing in the actual TOML/JSON interface changed for any of these types. Other possible future work ========================== - Target filter selection by dropdown/tree. Based on the system information schema, a list of all possible JSON pointers, or even a tree, could be generated and displayed to the user, in a form consisting of the key, a text input for the value/glob to match and a set of add/remove buttons for each filter. - Display/saving the installation progress/log. The progress log generated by the auto-installer can be sent to PDM, showing it there similar to tasks. - Implement a "wait for answer" system. Core idea here is that the installer waits for PDM to provide in answer by implemented e.g. an retry system. The use case is for to be able to effectively install systems on demand, without the need for preparations. - Providing the option for automatically adding the installed node to PDM (for PVE/PBS) - Maybe mTLS authentication for the answer endpoint, as suggested by Lukas/Thomas? [0] https://pve.proxmox.com/wiki/Automated_Installation [1] https://lore.proxmox.com/pdm-devel/11b27b4a-7e1f-4af9-8384-12d54d72ef17@proxmox.com/ [2] https://lore.proxmox.com/pdm-devel/DETMUXY1Q877.32G593TWC52WW@proxmox.com/#:~:text=%20I%20think%20this%20is%20dangerous History ======= v3: https://lore.proxmox.com/pdm-devel/20260403165437.2166551-1-c.heiss@proxmox.com/ v2: https://lore.proxmox.com/pdm-devel/20251205112528.373387-1-c.heiss@proxmox.com/ v1: https://lore.proxmox.com/pdm-devel/20251204125122.945961-1-c.heiss@proxmox.com/ Notable Changes v3 -> v4: * answer tokens are now automatically created when preparing a new answer, if no existing one is selected * use minijinja instead of handlebars as templating engine * add some more validation to some ui form fields * .. and udev filter/template counters in the backend * fix root password on prepared answer creation * add "Authorized tokens" column to prepared answers panel * use `Bearer` instead of `ProxmoxInstallerToken` in `Authorization` HTTP header Notable Changes v2 -> v3: * moved "Automated Installations" panels to a tab under "Remotes" * added authentication to answer endpoint * answers are now served as JSON unconditionally * moved installations state file to /var/lib/proxmox-datacenter-manager/ * added required pve-installer changes * ui: use `DataTable`s instead of text areas for filter entries * ui: use `PdmClient` for interacting with API instead of sending requests directly Notable Changes v1 -> v2: * add documentation patch * fixed compilation due to leftover type Diffstat ======== proxmox: Christoph Heiss (12): api-macro: allow $ in identifier name schema: oneOf: allow single string variant schema: implement UpdaterType for HashMap and BTreeMap network-types: move `Fqdn` type from proxmox-installer-common network-types: implement api type for Fqdn network-types: add api wrapper type for std::net::IpAddr network-types: cidr: implement generic `IpAddr::new` constructor network-types: fqdn: implement standard library Error for Fqdn node-status: make KernelVersionInformation Clone + PartialEq installer-types: add common types used by the installer installer-types: add types used by the auto-installer installer-types: implement api type for all externally-used types Cargo.toml | 2 + proxmox-api-macro/src/util.rs | 2 +- proxmox-installer-types/Cargo.toml | 28 + proxmox-installer-types/debian/changelog | 5 + proxmox-installer-types/debian/control | 67 + proxmox-installer-types/debian/debcargo.toml | 7 + proxmox-installer-types/src/answer.rs | 1199 ++++++++++++++++++ proxmox-installer-types/src/lib.rs | 182 +++ proxmox-installer-types/src/post_hook.rs | 215 ++++ proxmox-network-types/Cargo.toml | 3 +- proxmox-network-types/debian/control | 8 +- proxmox-network-types/src/fqdn.rs | 257 ++++ proxmox-network-types/src/ip_address.rs | 73 +- proxmox-network-types/src/lib.rs | 1 + proxmox-node-status/src/types.rs | 2 +- proxmox-schema/src/schema.rs | 78 +- proxmox-schema/tests/schema.rs | 91 +- 17 files changed, 2207 insertions(+), 13 deletions(-) create mode 100644 proxmox-installer-types/Cargo.toml create mode 100644 proxmox-installer-types/debian/changelog create mode 100644 proxmox-installer-types/debian/control create mode 100644 proxmox-installer-types/debian/debcargo.toml create mode 100644 proxmox-installer-types/src/answer.rs create mode 100644 proxmox-installer-types/src/lib.rs create mode 100644 proxmox-installer-types/src/post_hook.rs create mode 100644 proxmox-network-types/src/fqdn.rs proxmox-yew-comp: Christoph Heiss (1): widget: kvlist: add widget for user-modifiable data tables src/key_value_list.rs | 348 ++++++++++++++++++++++++++++++++++++++++++ src/lib.rs | 3 + 2 files changed, 351 insertions(+) create mode 100644 src/key_value_list.rs proxmox-datacenter-manager: Christoph Heiss (11): api-types, cli: use ReturnType::new() instead of constructing it manually api-types: add api types for auto-installer integration config: add auto-installer configuration module acl: wire up new /system/auto-installation acl path server: api: add auto-installer integration module server: api: auto-installer: add access token management endpoints client: add bindings for auto-installer endpoints ui: auto-installer: add installations overview panel ui: auto-installer: add prepared answer configuration panel ui: auto-installer: add access token configuration panel docs: add documentation for auto-installer integration Cargo.toml | 6 + cli/client/src/pbs.rs | 10 +- cli/client/src/pve.rs | 15 +- cli/client/src/remotes.rs | 5 +- debian/control | 5 + docs/automated-installations.rst | 146 ++ docs/index.rst | 1 + lib/pdm-api-types/Cargo.toml | 3 + lib/pdm-api-types/src/acl.rs | 4 +- lib/pdm-api-types/src/auto_installer.rs | 458 ++++++ lib/pdm-api-types/src/lib.rs | 10 +- lib/pdm-buildcfg/src/lib.rs | 10 + lib/pdm-client/src/lib.rs | 245 ++++ lib/pdm-config/Cargo.toml | 3 + lib/pdm-config/src/auto_install.rs | 575 ++++++++ lib/pdm-config/src/lib.rs | 1 + lib/pdm-config/src/setup.rs | 7 + server/Cargo.toml | 5 + server/src/api/auto_installer/mod.rs | 1259 +++++++++++++++++ server/src/api/mod.rs | 2 + ui/Cargo.toml | 2 + .../configuration/permission_path_selector.rs | 1 + .../auto_installer/installations_panel.rs | 303 ++++ ui/src/remotes/auto_installer/mod.rs | 86 ++ .../prepared_answer_add_wizard.rs | 219 +++ .../prepared_answer_edit_window.rs | 217 +++ .../auto_installer/prepared_answer_form.rs | 1089 ++++++++++++++ .../auto_installer/prepared_answers_panel.rs | 311 ++++ ui/src/remotes/auto_installer/token_panel.rs | 420 ++++++ .../remotes/auto_installer/token_selector.rs | 159 +++ ui/src/remotes/mod.rs | 10 + 31 files changed, 5557 insertions(+), 30 deletions(-) create mode 100644 docs/automated-installations.rst create mode 100644 lib/pdm-api-types/src/auto_installer.rs create mode 100644 lib/pdm-config/src/auto_install.rs create mode 100644 server/src/api/auto_installer/mod.rs create mode 100644 ui/src/remotes/auto_installer/installations_panel.rs create mode 100644 ui/src/remotes/auto_installer/mod.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_add_wizard.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_edit_window.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answer_form.rs create mode 100644 ui/src/remotes/auto_installer/prepared_answers_panel.rs create mode 100644 ui/src/remotes/auto_installer/token_panel.rs create mode 100644 ui/src/remotes/auto_installer/token_selector.rs pve-installer: Christoph Heiss (16): install: iso env: use JSON boolean literals for product config common: http: allow passing custom headers to post() common: http: retrieve error message from body on post() common: options: move regex construction out of loop assistant: support adding an authorization token for HTTP-based answers post-hook: run cargo fmt tree-wide: used moved `Fqdn` type to proxmox-network-types tree-wide: use `Cidr` type from proxmox-network-types tree-wide: switch to filesystem types from proxmox-installer-types auto: sysinfo: switch to types from proxmox-installer-types fetch-answer: switch to types from proxmox-installer-types fetch-answer: http: prefer json over toml for answer format fetch-answer: send auto-installer HTTP authorization token if set fetch-answer: print full error messages when fetching failed tree-wide: switch out `Answer` -> `AutoInstallerConfig` types auto: drop now-dead answer file definitions Cargo.toml | 8 + Proxmox/Install/ISOEnv.pm | 16 +- proxmox-auto-install-assistant/Cargo.toml | 1 + proxmox-auto-install-assistant/src/main.rs | 35 +- proxmox-auto-installer/Cargo.toml | 2 + proxmox-auto-installer/src/answer.rs | 499 ------------- .../src/bin/proxmox-auto-installer.rs | 20 +- proxmox-auto-installer/src/lib.rs | 2 - proxmox-auto-installer/src/sysinfo.rs | 91 +-- proxmox-auto-installer/src/udevinfo.rs | 11 - proxmox-auto-installer/src/utils.rs | 154 ++-- proxmox-auto-installer/tests/parse-answer.rs | 6 +- .../tests/resources/iso-info.json | 4 +- .../ipv4_and_subnet_mask_33.json | 2 +- ...rface_pinning_overlong_interface_name.json | 2 +- proxmox-chroot/Cargo.toml | 1 + proxmox-chroot/src/main.rs | 60 +- proxmox-fetch-answer/Cargo.toml | 2 +- .../src/fetch_plugins/http.rs | 116 ++- proxmox-fetch-answer/src/main.rs | 21 +- proxmox-installer-common/Cargo.toml | 2 + proxmox-installer-common/src/disk_checks.rs | 5 +- proxmox-installer-common/src/dmi.rs | 43 ++ proxmox-installer-common/src/http.rs | 85 ++- proxmox-installer-common/src/lib.rs | 6 +- proxmox-installer-common/src/options.rs | 365 +++------ proxmox-installer-common/src/setup.rs | 103 +-- proxmox-installer-common/src/sysinfo.rs | 52 -- proxmox-installer-common/src/utils.rs | 382 ---------- proxmox-post-hook/Cargo.toml | 4 +- proxmox-post-hook/src/main.rs | 691 +++++++----------- proxmox-tui-installer/Cargo.toml | 2 + proxmox-tui-installer/src/main.rs | 12 +- proxmox-tui-installer/src/options.rs | 23 +- proxmox-tui-installer/src/setup.rs | 5 +- proxmox-tui-installer/src/views/bootdisk.rs | 44 +- proxmox-tui-installer/src/views/mod.rs | 21 +- proxmox-tui-installer/src/views/network.rs | 15 +- 38 files changed, 824 insertions(+), 2089 deletions(-) delete mode 100644 proxmox-auto-installer/src/answer.rs delete mode 100644 proxmox-auto-installer/src/udevinfo.rs create mode 100644 proxmox-installer-common/src/dmi.rs delete mode 100644 proxmox-installer-common/src/sysinfo.rs delete mode 100644 proxmox-installer-common/src/utils.rs Summary over all repositories: 88 files changed, 8939 insertions(+), 2132 deletions(-) --