Re: [PATCH proxmox/yew-pwt/datacenter-manager/installer v3 00/38] add auto-installer integration

[Dominik Csapak <d.csapak@proxmox.com>]

also two things i forgot:

i think we could make the endpoint available without authentication?
at least that's what i would have expected, since the current
http endpoints also had to be public?

second things is about the tokens + acl, I'd probably
use the existing token acl mechanism, so instead of
having custom tokens + a token list per answer,

use the standard pdm tokens + an acl path (e.g. 
/system/auto-installation/answers/<id> ) maybe a separate privilege
could make sense here, so the token does not have access
to anything else?

On 4/17/26 10:41 AM, Dominik Csapak wrote:
> I'll answer on the individual ui patches when looking in detail,
> but wanted to give a high level feedback from using it a bit:
> 
> (note that i didn't actually test the install part due to
> not yet building a new installer helper, but i'm on it)
> 
> Main layout:
> 
> * not super convinced this 3 part layout makes the most sense for
>    the following reasons:
>    - the flow is basically reversed: first add a token (if wanted)
>      then add an answer, then see the installations, but the
>      reading flow is in reverse.
>      maybe having (sub) tabs here could make sense, e.g. a
>      config and a status tab would make it much clearer
>      we could also split the config and status part (e.g. put
>      the config part into the 'config' menu instead of remotes.
>      but that would split the view a bit ofc.
> * there is something wrong with the flex-layout + grids, when
>    i make the window bigger, the left side is growing, but it's
>    not shrinking again
> * neither of the grids on the right is scollable, if the viewport
>    is too narrow, they overflow the complete page.
>    IMO it's preferrable to have each grid do the scrolling itself
> 
> Prepared answers:
> 
> * the 'copy' button is always enabled, should only be enabled
>    when one is selected
> * there is something wrong with setting the password:
>    the root-password is never sent to the backend (it's not even in the
>    struct) but we don't calculate the root_password_hashed
> * i'd like for it maybe two 'password' fields that get compared?
> * we do have a timezone selector on the webui already,
>    not sure if/how we can integrate that, but a simple text field
>    is imo not that good of an experience.
> * the email adress is not checked, neither in the frontend
>    or the backend. I'd expect the same checks as in the installer
>    or at least check for an '@'
> * not defining root-keys are broken atm
> * disk setup layout could use an improvement:
>    - instead of having a drop down that enables/disables the fields
>      we could add a radio button that changes the mode
>      (here is also where i noticed the missing disabled for the add
>      button)
>    - i find it a bit confusing that the advanced options for the disks
>      are only available in the edit window, and not in the initial
>      add wizard. any special reason why?
>    - AFAICS the udef filter grid does not have a maximum height.
>      Could use that + scrolling enabled
> * having an invalid state for the target filter makes no sense imho
>    if it's not usable (neither default nor any rules defined)
>    we shouldn't allow the user to continue?
> 
>    target filters syntax: a bit unclear what to enter here, i guess
>    this is documented somewhere what can be chosen, but either a bit
>    more helptext or a direct link to the explanation what we can choose
>    here (instead of a link to the json pointer rfc) would make more
>    sense (i couldn't find how/what i can choose here easily...)
> * same with the templating: how and for what can i use this?
>    it's fine to document these things, but such a wizard gui should
>    nonetheless be a bit more self-explanatory, or alternatively
>    point to the docs where this is explained at the very beginning.
>    if it's only an advanced feature, putting it behind an advanced
>    checkbox, or having it as a seperate edit button in the grid
>    could also be fine
> * authorized token: if it can't work without any token selected,
>    this page should not be submittable if none is selected.
> 
> On 4/3/26 6:54 PM, Christoph Heiss wrote:
>> This series adds integration with our automated installer [0] for all
>> our products. With this, Proxmox Datacenter Manager can be used for
>> serving answer files via HTTPS in an automated fashion.
>>
>> It provides three panels under the "Automated Installations" tab in the
>> "Remotes" menu:
>>
>> - Automated Installations: List all past and currently ongoing
>>    installations.
>> - Prepared Answers: Enables users to do the whole CRUD cycle for
>>    prepared answers, i.e. create new answers (from scratch or based on
>>    existing ones), edit and delete.
>> - Authentication Tokens: Simple token for the /answer endpoint. What
>>    tokens can be used to obtain which answer can be set when editing a
>>    prepared answer under the "Authentication" tab.
>>
>> Permission-wise, everything is currently scoped under
>> /system/auto-installation.
>>
>> User interface
>> ==============
>>
>> Happy about feedback regarding the UI, especially the editing dialog! By
>> the nature of it, the auto-installer has a lot more options than the
>> standard GUI/TUI installer, and it's kind of hard to not fully cramp up
>> the UI while still providing all options.
>>
>> The viewing of the system information and post-installation hook
>> notification data is currently maybe a bit bare-bones. For the first
>> one, displaying it as a tree could work, if we don't want to simply
>> display raw JSON documents to the user.
>>
>> Answer endpoint
>> ===============
>>
>> The endpoint for serving answer files to the automated installer lives
>> under `/api2/json/auto-install/answer`. This makes it (unfortunately)
>> incompatible with older ISOs, as currently they only support answer
>> files formatted as TOML. See also the discussion in [1].
>>
>> We could serve TOML (as done in previous revisision), but the
>> introduction of authentication tokens [2] for the /answer endpoint
>> breaks backwards-compatibility anyway, rendering the whole endeavour
>> moot - so I removed TOML support entirely.
>>
>> Installer changes
>> =================
>>
>> Most of the changes to pve-installer are churn from switching to the
>> moved types in proxmox-installer-types. They were heavily intertwined
>> between the different crates in pve-installer, so
>> a) untwisting that, b) doing some long-overdue quality-of-life
>> improvements and c) making them #[api] ready created a lot of churn,
>> although nothing in the actual TOML/JSON interface changed for any of
>> these types.
>>
>> Other possible future work
>> ==========================
>>
>> - Target filter selection by dropdown/tree. Based on the system
>>    information schema, a list of all possible JSON pointers, or even a
>>    tree, could be generated and displayed to the user, in a form
>>    consisting of the key, a text input for the value/glob to match and a
>>    set of add/remove buttons for each filter.
>>
>> - Display/saving the installation progress/log. The progress log
>>    generated by the auto-installer can be sent to PDM, showing it there
>>    similar to tasks.
>>
>> - Implement a "wait for answer" system. Core idea here is that the
>>    installer waits for PDM to provide in answer by implemented e.g. an
>>    retry system. The use case is for to be able to effectively install
>>    systems on demand, without the need for preparations.
>>
>> - Providing the option for automatically adding the installed node to
>>    PDM (for PVE/PBS)
>>
>> - Maybe mTLS authentication for the answer endpoint, as suggested by
>>    Lukas/Thomas?
>>
>> [0] https://pve.proxmox.com/wiki/Automated_Installation
>> [1] https://lore.proxmox.com/pdm- 
>> devel/11b27b4a-7e1f-4af9-8384-12d54d72ef17@proxmox.com/
>> [2] https://lore.proxmox.com/pdm-devel/ 
>> DETMUXY1Q877.32G593TWC52WW@proxmox.com/ 
>> #:~:text=%20I%20think%20this%20is%20dangerous
>>
>> History
>> =======
>>
>> v2: https://lore.proxmox.com/pdm-devel/20251205112528.373387-1- 
>> c.heiss@proxmox.com/
>> v1: https://lore.proxmox.com/pdm-devel/20251204125122.945961-1- 
>> c.heiss@proxmox.com/
>>
>> Notable Changes v2 -> v3:
>>    * moved "Automated Installations" panels to a tab under "Remotes"
>>    * added authentication to answer endpoint
>>    * answers are now served as JSON unconditionally
>>    * moved installations state file to
>>      /var/lib/proxmox-datacenter-manager/
>>    * added required pve-installer changes
>>    * ui: use `DataTable`s instead of text areas for filter entries
>>    * ui: use `PdmClient` for interacting with API instead of sending
>>      requests directly
>>
>> Notable Changes v1 -> v2:
>>    * add documentation patch
>>    * fixed compilation due to leftover type
>>
>> Diffstat
>> ========
>>
>>
>> proxmox:
>>
>> Christoph Heiss (12):
>>    api-macro: allow $ in identifier name
>>    schema: oneOf: allow single string variant
>>    schema: implement UpdaterType for HashMap and BTreeMap
>>    network-types: move `Fqdn` type from proxmox-installer-common
>>    network-types: implement api type for Fqdn
>>    network-types: add api wrapper type for std::net::IpAddr
>>    network-types: cidr: implement generic `IpAddr::new` constructor
>>    network-types: fqdn: implement standard library Error for Fqdn
>>    node-status: make KernelVersionInformation Clone + PartialEq
>>    installer-types: add common types used by the installer
>>    installer-types: add types used by the auto-installer
>>    installer-types: implement api type for all externally-used types
>>
>>   Cargo.toml                                   |    2 +
>>   proxmox-api-macro/src/util.rs                |    2 +-
>>   proxmox-installer-types/Cargo.toml           |   28 +
>>   proxmox-installer-types/debian/changelog     |    5 +
>>   proxmox-installer-types/debian/control       |   67 +
>>   proxmox-installer-types/debian/debcargo.toml |    7 +
>>   proxmox-installer-types/src/answer.rs        | 1199 ++++++++++++++++++
>>   proxmox-installer-types/src/lib.rs           |  182 +++
>>   proxmox-installer-types/src/post_hook.rs     |  215 ++++
>>   proxmox-network-types/Cargo.toml             |    3 +-
>>   proxmox-network-types/debian/control         |    8 +-
>>   proxmox-network-types/src/fqdn.rs            |  257 ++++
>>   proxmox-network-types/src/ip_address.rs      |   73 +-
>>   proxmox-network-types/src/lib.rs             |    1 +
>>   proxmox-node-status/src/types.rs             |    2 +-
>>   proxmox-schema/src/schema.rs                 |   78 +-
>>   proxmox-schema/tests/schema.rs               |   91 +-
>>   17 files changed, 2207 insertions(+), 13 deletions(-)
>>   create mode 100644 proxmox-installer-types/Cargo.toml
>>   create mode 100644 proxmox-installer-types/debian/changelog
>>   create mode 100644 proxmox-installer-types/debian/control
>>   create mode 100644 proxmox-installer-types/debian/debcargo.toml
>>   create mode 100644 proxmox-installer-types/src/answer.rs
>>   create mode 100644 proxmox-installer-types/src/lib.rs
>>   create mode 100644 proxmox-installer-types/src/post_hook.rs
>>   create mode 100644 proxmox-network-types/src/fqdn.rs
>>
>>
>> proxmox-yew-widget-toolkit:
>>
>> Christoph Heiss (1):
>>    widget: kvlist: add widget for user-modifiable data tables
>>
>>   src/widget/key_value_list.rs | 429 +++++++++++++++++++++++++++++++++++
>>   src/widget/mod.rs            |   3 +
>>   2 files changed, 432 insertions(+)
>>   create mode 100644 src/widget/key_value_list.rs
>>
>>
>> proxmox-datacenter-manager:
>>
>> Christoph Heiss (11):
>>    api-types, cli: use ReturnType::new() instead of constructing it
>>      manually
>>    api-types: add api types for auto-installer integration
>>    config: add auto-installer configuration module
>>    acl: wire up new /system/auto-installation acl path
>>    server: api: add auto-installer integration module
>>    server: api: auto-installer: add access token management endpoints
>>    client: add bindings for auto-installer endpoints
>>    ui: auto-installer: add installations overview panel
>>    ui: auto-installer: add prepared answer configuration panel
>>    ui: auto-installer: add access token configuration panel
>>    docs: add documentation for auto-installer integration
>>
>>   Cargo.toml                                    |    5 +
>>   cli/client/src/pbs.rs                         |   10 +-
>>   cli/client/src/pve.rs                         |   15 +-
>>   cli/client/src/remotes.rs                     |    5 +-
>>   debian/control                                |    5 +
>>   docs/automated-installations.rst              |  124 ++
>>   docs/index.rst                                |    1 +
>>   lib/pdm-api-types/Cargo.toml                  |    3 +
>>   lib/pdm-api-types/src/acl.rs                  |    4 +-
>>   lib/pdm-api-types/src/auto_installer.rs       |  415 ++++++
>>   lib/pdm-api-types/src/lib.rs                  |   10 +-
>>   lib/pdm-buildcfg/src/lib.rs                   |   10 +
>>   lib/pdm-client/src/lib.rs                     |  232 ++++
>>   lib/pdm-config/Cargo.toml                     |    3 +
>>   lib/pdm-config/src/auto_install.rs            |  559 ++++++++
>>   lib/pdm-config/src/lib.rs                     |    1 +
>>   lib/pdm-config/src/setup.rs                   |    7 +
>>   server/Cargo.toml                             |    4 +
>>   server/src/api/auto_installer/mod.rs          | 1218 +++++++++++++++++
>>   server/src/api/mod.rs                         |    2 +
>>   ui/Cargo.toml                                 |    2 +
>>   .../configuration/permission_path_selector.rs |    1 +
>>   .../auto_installer/installations_panel.rs     |  305 +++++
>>   ui/src/remotes/auto_installer/mod.rs          |   86 ++
>>   .../prepared_answer_add_wizard.rs             |  192 +++
>>   .../prepared_answer_edit_window.rs            |  187 +++
>>   .../auto_installer/prepared_answer_form.rs    |  875 ++++++++++++
>>   .../auto_installer/prepared_answers_panel.rs  |  248 ++++
>>   ui/src/remotes/auto_installer/token_panel.rs  |  476 +++++++
>>   .../remotes/auto_installer/token_selector.rs  |  137 ++
>>   ui/src/remotes/mod.rs                         |   10 +
>>   31 files changed, 5122 insertions(+), 30 deletions(-)
>>   create mode 100644 docs/automated-installations.rst
>>   create mode 100644 lib/pdm-api-types/src/auto_installer.rs
>>   create mode 100644 lib/pdm-config/src/auto_install.rs
>>   create mode 100644 server/src/api/auto_installer/mod.rs
>>   create mode 100644 ui/src/remotes/auto_installer/installations_panel.rs
>>   create mode 100644 ui/src/remotes/auto_installer/mod.rs
>>   create mode 100644 ui/src/remotes/auto_installer/ 
>> prepared_answer_add_wizard.rs
>>   create mode 100644 ui/src/remotes/auto_installer/ 
>> prepared_answer_edit_window.rs
>>   create mode 100644 ui/src/remotes/auto_installer/ 
>> prepared_answer_form.rs
>>   create mode 100644 ui/src/remotes/auto_installer/ 
>> prepared_answers_panel.rs
>>   create mode 100644 ui/src/remotes/auto_installer/token_panel.rs
>>   create mode 100644 ui/src/remotes/auto_installer/token_selector.rs
>>
>>
>> pve-installer:
>>
>> Christoph Heiss (14):
>>    install: iso env: use JSON boolean literals for product config
>>    common: http: allow passing custom headers to post()
>>    common: options: move regex construction out of loop
>>    assistant: support adding an authorization token for HTTP-based
>>      answers
>>    tree-wide: used moved `Fqdn` type to proxmox-network-types
>>    tree-wide: use `Cidr` type from proxmox-network-types
>>    tree-wide: switch to filesystem types from proxmox-installer-types
>>    post-hook: switch to types in proxmox-installer-types
>>    auto: sysinfo: switch to types from proxmox-installer-types
>>    fetch-answer: switch to types from proxmox-installer-types
>>    fetch-answer: http: prefer json over toml for answer format
>>    fetch-answer: send auto-installer HTTP authorization token if set
>>    tree-wide: switch out `Answer` -> `AutoInstallerConfig` types
>>    auto: drop now-dead answer file definitions
>>
>>   Cargo.toml                                    |   8 +
>>   Proxmox/Install/ISOEnv.pm                     |  16 +-
>>   proxmox-auto-install-assistant/Cargo.toml     |   1 +
>>   proxmox-auto-install-assistant/src/main.rs    |  35 +-
>>   proxmox-auto-installer/Cargo.toml             |   2 +
>>   proxmox-auto-installer/src/answer.rs          | 499 -------------
>>   .../src/bin/proxmox-auto-installer.rs         |  20 +-
>>   proxmox-auto-installer/src/lib.rs             |   2 -
>>   proxmox-auto-installer/src/sysinfo.rs         |  91 +--
>>   proxmox-auto-installer/src/udevinfo.rs        |  11 -
>>   proxmox-auto-installer/src/utils.rs           | 154 ++--
>>   proxmox-auto-installer/tests/parse-answer.rs  |   6 +-
>>   .../tests/resources/iso-info.json             |   4 +-
>>   .../ipv4_and_subnet_mask_33.json              |   2 +-
>>   ...rface_pinning_overlong_interface_name.json |   2 +-
>>   proxmox-chroot/Cargo.toml                     |   1 +
>>   proxmox-chroot/src/main.rs                    |  60 +-
>>   proxmox-fetch-answer/Cargo.toml               |   2 +-
>>   .../src/fetch_plugins/http.rs                 | 116 ++-
>>   proxmox-fetch-answer/src/main.rs              |  21 +-
>>   proxmox-installer-common/Cargo.toml           |   2 +
>>   proxmox-installer-common/src/disk_checks.rs   |   5 +-
>>   proxmox-installer-common/src/dmi.rs           |  43 ++
>>   proxmox-installer-common/src/http.rs          |  40 +-
>>   proxmox-installer-common/src/lib.rs           |   6 +-
>>   proxmox-installer-common/src/options.rs       | 365 +++------
>>   proxmox-installer-common/src/setup.rs         | 103 +--
>>   proxmox-installer-common/src/sysinfo.rs       |  52 --
>>   proxmox-installer-common/src/utils.rs         | 382 ----------
>>   proxmox-post-hook/Cargo.toml                  |   4 +-
>>   proxmox-post-hook/src/main.rs                 | 691 +++++++-----------
>>   proxmox-tui-installer/Cargo.toml              |   2 +
>>   proxmox-tui-installer/src/main.rs             |  12 +-
>>   proxmox-tui-installer/src/options.rs          |  23 +-
>>   proxmox-tui-installer/src/setup.rs            |   5 +-
>>   proxmox-tui-installer/src/views/bootdisk.rs   |  44 +-
>>   proxmox-tui-installer/src/views/mod.rs        |  21 +-
>>   proxmox-tui-installer/src/views/network.rs    |  15 +-
>>   38 files changed, 779 insertions(+), 2089 deletions(-)
>>   delete mode 100644 proxmox-auto-installer/src/answer.rs
>>   delete mode 100644 proxmox-auto-installer/src/udevinfo.rs
>>   create mode 100644 proxmox-installer-common/src/dmi.rs
>>   delete mode 100644 proxmox-installer-common/src/sysinfo.rs
>>   delete mode 100644 proxmox-installer-common/src/utils.rs
>>
>>
>> Summary over all repositories:
>>    88 files changed, 8540 insertions(+), 2132 deletions(-)
>>
> 
> 
> 
> 
>