Re: [PATCH proxmox/yew-pwt/datacenter-manager/installer v3 00/38] add auto-installer integration

[Dominik Csapak <d.csapak@proxmox.com>]

I'll answer on the individual ui patches when looking in detail,
but wanted to give a high level feedback from using it a bit:

(note that i didn't actually test the install part due to
not yet building a new installer helper, but i'm on it)

Main layout:

* not super convinced this 3 part layout makes the most sense for
   the following reasons:
   - the flow is basically reversed: first add a token (if wanted)
     then add an answer, then see the installations, but the
     reading flow is in reverse.
     maybe having (sub) tabs here could make sense, e.g. a
     config and a status tab would make it much clearer
     we could also split the config and status part (e.g. put
     the config part into the 'config' menu instead of remotes.
     but that would split the view a bit ofc.
* there is something wrong with the flex-layout + grids, when
   i make the window bigger, the left side is growing, but it's
   not shrinking again
* neither of the grids on the right is scollable, if the viewport
   is too narrow, they overflow the complete page.
   IMO it's preferrable to have each grid do the scrolling itself

Prepared answers:

* the 'copy' button is always enabled, should only be enabled
   when one is selected
* there is something wrong with setting the password:
   the root-password is never sent to the backend (it's not even in the
   struct) but we don't calculate the root_password_hashed
* i'd like for it maybe two 'password' fields that get compared?
* we do have a timezone selector on the webui already,
   not sure if/how we can integrate that, but a simple text field
   is imo not that good of an experience.
* the email adress is not checked, neither in the frontend
   or the backend. I'd expect the same checks as in the installer
   or at least check for an '@'
* not defining root-keys are broken atm
* disk setup layout could use an improvement:
   - instead of having a drop down that enables/disables the fields
     we could add a radio button that changes the mode
     (here is also where i noticed the missing disabled for the add
     button)
   - i find it a bit confusing that the advanced options for the disks
     are only available in the edit window, and not in the initial
     add wizard. any special reason why?
   - AFAICS the udef filter grid does not have a maximum height.
     Could use that + scrolling enabled
* having an invalid state for the target filter makes no sense imho
   if it's not usable (neither default nor any rules defined)
   we shouldn't allow the user to continue?

   target filters syntax: a bit unclear what to enter here, i guess
   this is documented somewhere what can be chosen, but either a bit
   more helptext or a direct link to the explanation what we can choose
   here (instead of a link to the json pointer rfc) would make more
   sense (i couldn't find how/what i can choose here easily...)
* same with the templating: how and for what can i use this?
   it's fine to document these things, but such a wizard gui should
   nonetheless be a bit more self-explanatory, or alternatively
   point to the docs where this is explained at the very beginning.
   if it's only an advanced feature, putting it behind an advanced
   checkbox, or having it as a seperate edit button in the grid
   could also be fine
* authorized token: if it can't work without any token selected,
   this page should not be submittable if none is selected.

On 4/3/26 6:54 PM, Christoph Heiss wrote:
> This series adds integration with our automated installer [0] for all
> our products. With this, Proxmox Datacenter Manager can be used for
> serving answer files via HTTPS in an automated fashion.
> 
> It provides three panels under the "Automated Installations" tab in the
> "Remotes" menu:
> 
> - Automated Installations: List all past and currently ongoing
>    installations.
> - Prepared Answers: Enables users to do the whole CRUD cycle for
>    prepared answers, i.e. create new answers (from scratch or based on
>    existing ones), edit and delete.
> - Authentication Tokens: Simple token for the /answer endpoint. What
>    tokens can be used to obtain which answer can be set when editing a
>    prepared answer under the "Authentication" tab.
> 
> Permission-wise, everything is currently scoped under
> /system/auto-installation.
> 
> User interface
> ==============
> 
> Happy about feedback regarding the UI, especially the editing dialog! By
> the nature of it, the auto-installer has a lot more options than the
> standard GUI/TUI installer, and it's kind of hard to not fully cramp up
> the UI while still providing all options.
> 
> The viewing of the system information and post-installation hook
> notification data is currently maybe a bit bare-bones. For the first
> one, displaying it as a tree could work, if we don't want to simply
> display raw JSON documents to the user.
> 
> Answer endpoint
> ===============
> 
> The endpoint for serving answer files to the automated installer lives
> under `/api2/json/auto-install/answer`. This makes it (unfortunately)
> incompatible with older ISOs, as currently they only support answer
> files formatted as TOML. See also the discussion in [1].
> 
> We could serve TOML (as done in previous revisision), but the
> introduction of authentication tokens [2] for the /answer endpoint
> breaks backwards-compatibility anyway, rendering the whole endeavour
> moot - so I removed TOML support entirely.
> 
> Installer changes
> =================
> 
> Most of the changes to pve-installer are churn from switching to the
> moved types in proxmox-installer-types. They were heavily intertwined
> between the different crates in pve-installer, so
> a) untwisting that, b) doing some long-overdue quality-of-life
> improvements and c) making them #[api] ready created a lot of churn,
> although nothing in the actual TOML/JSON interface changed for any of
> these types.
> 
> Other possible future work
> ==========================
> 
> - Target filter selection by dropdown/tree. Based on the system
>    information schema, a list of all possible JSON pointers, or even a
>    tree, could be generated and displayed to the user, in a form
>    consisting of the key, a text input for the value/glob to match and a
>    set of add/remove buttons for each filter.
> 
> - Display/saving the installation progress/log. The progress log
>    generated by the auto-installer can be sent to PDM, showing it there
>    similar to tasks.
> 
> - Implement a "wait for answer" system. Core idea here is that the
>    installer waits for PDM to provide in answer by implemented e.g. an
>    retry system. The use case is for to be able to effectively install
>    systems on demand, without the need for preparations.
> 
> - Providing the option for automatically adding the installed node to
>    PDM (for PVE/PBS)
> 
> - Maybe mTLS authentication for the answer endpoint, as suggested by
>    Lukas/Thomas?
> 
> [0] https://pve.proxmox.com/wiki/Automated_Installation
> [1] https://lore.proxmox.com/pdm-devel/11b27b4a-7e1f-4af9-8384-12d54d72ef17@proxmox.com/
> [2] https://lore.proxmox.com/pdm-devel/DETMUXY1Q877.32G593TWC52WW@proxmox.com/#:~:text=%20I%20think%20this%20is%20dangerous
> 
> History
> =======
> 
> v2: https://lore.proxmox.com/pdm-devel/20251205112528.373387-1-c.heiss@proxmox.com/
> v1: https://lore.proxmox.com/pdm-devel/20251204125122.945961-1-c.heiss@proxmox.com/
> 
> Notable Changes v2 -> v3:
>    * moved "Automated Installations" panels to a tab under "Remotes"
>    * added authentication to answer endpoint
>    * answers are now served as JSON unconditionally
>    * moved installations state file to
>      /var/lib/proxmox-datacenter-manager/
>    * added required pve-installer changes
>    * ui: use `DataTable`s instead of text areas for filter entries
>    * ui: use `PdmClient` for interacting with API instead of sending
>      requests directly
> 
> Notable Changes v1 -> v2:
>    * add documentation patch
>    * fixed compilation due to leftover type
> 
> Diffstat
> ========
> 
> 
> proxmox:
> 
> Christoph Heiss (12):
>    api-macro: allow $ in identifier name
>    schema: oneOf: allow single string variant
>    schema: implement UpdaterType for HashMap and BTreeMap
>    network-types: move `Fqdn` type from proxmox-installer-common
>    network-types: implement api type for Fqdn
>    network-types: add api wrapper type for std::net::IpAddr
>    network-types: cidr: implement generic `IpAddr::new` constructor
>    network-types: fqdn: implement standard library Error for Fqdn
>    node-status: make KernelVersionInformation Clone + PartialEq
>    installer-types: add common types used by the installer
>    installer-types: add types used by the auto-installer
>    installer-types: implement api type for all externally-used types
> 
>   Cargo.toml                                   |    2 +
>   proxmox-api-macro/src/util.rs                |    2 +-
>   proxmox-installer-types/Cargo.toml           |   28 +
>   proxmox-installer-types/debian/changelog     |    5 +
>   proxmox-installer-types/debian/control       |   67 +
>   proxmox-installer-types/debian/debcargo.toml |    7 +
>   proxmox-installer-types/src/answer.rs        | 1199 ++++++++++++++++++
>   proxmox-installer-types/src/lib.rs           |  182 +++
>   proxmox-installer-types/src/post_hook.rs     |  215 ++++
>   proxmox-network-types/Cargo.toml             |    3 +-
>   proxmox-network-types/debian/control         |    8 +-
>   proxmox-network-types/src/fqdn.rs            |  257 ++++
>   proxmox-network-types/src/ip_address.rs      |   73 +-
>   proxmox-network-types/src/lib.rs             |    1 +
>   proxmox-node-status/src/types.rs             |    2 +-
>   proxmox-schema/src/schema.rs                 |   78 +-
>   proxmox-schema/tests/schema.rs               |   91 +-
>   17 files changed, 2207 insertions(+), 13 deletions(-)
>   create mode 100644 proxmox-installer-types/Cargo.toml
>   create mode 100644 proxmox-installer-types/debian/changelog
>   create mode 100644 proxmox-installer-types/debian/control
>   create mode 100644 proxmox-installer-types/debian/debcargo.toml
>   create mode 100644 proxmox-installer-types/src/answer.rs
>   create mode 100644 proxmox-installer-types/src/lib.rs
>   create mode 100644 proxmox-installer-types/src/post_hook.rs
>   create mode 100644 proxmox-network-types/src/fqdn.rs
> 
> 
> proxmox-yew-widget-toolkit:
> 
> Christoph Heiss (1):
>    widget: kvlist: add widget for user-modifiable data tables
> 
>   src/widget/key_value_list.rs | 429 +++++++++++++++++++++++++++++++++++
>   src/widget/mod.rs            |   3 +
>   2 files changed, 432 insertions(+)
>   create mode 100644 src/widget/key_value_list.rs
> 
> 
> proxmox-datacenter-manager:
> 
> Christoph Heiss (11):
>    api-types, cli: use ReturnType::new() instead of constructing it
>      manually
>    api-types: add api types for auto-installer integration
>    config: add auto-installer configuration module
>    acl: wire up new /system/auto-installation acl path
>    server: api: add auto-installer integration module
>    server: api: auto-installer: add access token management endpoints
>    client: add bindings for auto-installer endpoints
>    ui: auto-installer: add installations overview panel
>    ui: auto-installer: add prepared answer configuration panel
>    ui: auto-installer: add access token configuration panel
>    docs: add documentation for auto-installer integration
> 
>   Cargo.toml                                    |    5 +
>   cli/client/src/pbs.rs                         |   10 +-
>   cli/client/src/pve.rs                         |   15 +-
>   cli/client/src/remotes.rs                     |    5 +-
>   debian/control                                |    5 +
>   docs/automated-installations.rst              |  124 ++
>   docs/index.rst                                |    1 +
>   lib/pdm-api-types/Cargo.toml                  |    3 +
>   lib/pdm-api-types/src/acl.rs                  |    4 +-
>   lib/pdm-api-types/src/auto_installer.rs       |  415 ++++++
>   lib/pdm-api-types/src/lib.rs                  |   10 +-
>   lib/pdm-buildcfg/src/lib.rs                   |   10 +
>   lib/pdm-client/src/lib.rs                     |  232 ++++
>   lib/pdm-config/Cargo.toml                     |    3 +
>   lib/pdm-config/src/auto_install.rs            |  559 ++++++++
>   lib/pdm-config/src/lib.rs                     |    1 +
>   lib/pdm-config/src/setup.rs                   |    7 +
>   server/Cargo.toml                             |    4 +
>   server/src/api/auto_installer/mod.rs          | 1218 +++++++++++++++++
>   server/src/api/mod.rs                         |    2 +
>   ui/Cargo.toml                                 |    2 +
>   .../configuration/permission_path_selector.rs |    1 +
>   .../auto_installer/installations_panel.rs     |  305 +++++
>   ui/src/remotes/auto_installer/mod.rs          |   86 ++
>   .../prepared_answer_add_wizard.rs             |  192 +++
>   .../prepared_answer_edit_window.rs            |  187 +++
>   .../auto_installer/prepared_answer_form.rs    |  875 ++++++++++++
>   .../auto_installer/prepared_answers_panel.rs  |  248 ++++
>   ui/src/remotes/auto_installer/token_panel.rs  |  476 +++++++
>   .../remotes/auto_installer/token_selector.rs  |  137 ++
>   ui/src/remotes/mod.rs                         |   10 +
>   31 files changed, 5122 insertions(+), 30 deletions(-)
>   create mode 100644 docs/automated-installations.rst
>   create mode 100644 lib/pdm-api-types/src/auto_installer.rs
>   create mode 100644 lib/pdm-config/src/auto_install.rs
>   create mode 100644 server/src/api/auto_installer/mod.rs
>   create mode 100644 ui/src/remotes/auto_installer/installations_panel.rs
>   create mode 100644 ui/src/remotes/auto_installer/mod.rs
>   create mode 100644 ui/src/remotes/auto_installer/prepared_answer_add_wizard.rs
>   create mode 100644 ui/src/remotes/auto_installer/prepared_answer_edit_window.rs
>   create mode 100644 ui/src/remotes/auto_installer/prepared_answer_form.rs
>   create mode 100644 ui/src/remotes/auto_installer/prepared_answers_panel.rs
>   create mode 100644 ui/src/remotes/auto_installer/token_panel.rs
>   create mode 100644 ui/src/remotes/auto_installer/token_selector.rs
> 
> 
> pve-installer:
> 
> Christoph Heiss (14):
>    install: iso env: use JSON boolean literals for product config
>    common: http: allow passing custom headers to post()
>    common: options: move regex construction out of loop
>    assistant: support adding an authorization token for HTTP-based
>      answers
>    tree-wide: used moved `Fqdn` type to proxmox-network-types
>    tree-wide: use `Cidr` type from proxmox-network-types
>    tree-wide: switch to filesystem types from proxmox-installer-types
>    post-hook: switch to types in proxmox-installer-types
>    auto: sysinfo: switch to types from proxmox-installer-types
>    fetch-answer: switch to types from proxmox-installer-types
>    fetch-answer: http: prefer json over toml for answer format
>    fetch-answer: send auto-installer HTTP authorization token if set
>    tree-wide: switch out `Answer` -> `AutoInstallerConfig` types
>    auto: drop now-dead answer file definitions
> 
>   Cargo.toml                                    |   8 +
>   Proxmox/Install/ISOEnv.pm                     |  16 +-
>   proxmox-auto-install-assistant/Cargo.toml     |   1 +
>   proxmox-auto-install-assistant/src/main.rs    |  35 +-
>   proxmox-auto-installer/Cargo.toml             |   2 +
>   proxmox-auto-installer/src/answer.rs          | 499 -------------
>   .../src/bin/proxmox-auto-installer.rs         |  20 +-
>   proxmox-auto-installer/src/lib.rs             |   2 -
>   proxmox-auto-installer/src/sysinfo.rs         |  91 +--
>   proxmox-auto-installer/src/udevinfo.rs        |  11 -
>   proxmox-auto-installer/src/utils.rs           | 154 ++--
>   proxmox-auto-installer/tests/parse-answer.rs  |   6 +-
>   .../tests/resources/iso-info.json             |   4 +-
>   .../ipv4_and_subnet_mask_33.json              |   2 +-
>   ...rface_pinning_overlong_interface_name.json |   2 +-
>   proxmox-chroot/Cargo.toml                     |   1 +
>   proxmox-chroot/src/main.rs                    |  60 +-
>   proxmox-fetch-answer/Cargo.toml               |   2 +-
>   .../src/fetch_plugins/http.rs                 | 116 ++-
>   proxmox-fetch-answer/src/main.rs              |  21 +-
>   proxmox-installer-common/Cargo.toml           |   2 +
>   proxmox-installer-common/src/disk_checks.rs   |   5 +-
>   proxmox-installer-common/src/dmi.rs           |  43 ++
>   proxmox-installer-common/src/http.rs          |  40 +-
>   proxmox-installer-common/src/lib.rs           |   6 +-
>   proxmox-installer-common/src/options.rs       | 365 +++------
>   proxmox-installer-common/src/setup.rs         | 103 +--
>   proxmox-installer-common/src/sysinfo.rs       |  52 --
>   proxmox-installer-common/src/utils.rs         | 382 ----------
>   proxmox-post-hook/Cargo.toml                  |   4 +-
>   proxmox-post-hook/src/main.rs                 | 691 +++++++-----------
>   proxmox-tui-installer/Cargo.toml              |   2 +
>   proxmox-tui-installer/src/main.rs             |  12 +-
>   proxmox-tui-installer/src/options.rs          |  23 +-
>   proxmox-tui-installer/src/setup.rs            |   5 +-
>   proxmox-tui-installer/src/views/bootdisk.rs   |  44 +-
>   proxmox-tui-installer/src/views/mod.rs        |  21 +-
>   proxmox-tui-installer/src/views/network.rs    |  15 +-
>   38 files changed, 779 insertions(+), 2089 deletions(-)
>   delete mode 100644 proxmox-auto-installer/src/answer.rs
>   delete mode 100644 proxmox-auto-installer/src/udevinfo.rs
>   create mode 100644 proxmox-installer-common/src/dmi.rs
>   delete mode 100644 proxmox-installer-common/src/sysinfo.rs
>   delete mode 100644 proxmox-installer-common/src/utils.rs
> 
> 
> Summary over all repositories:
>    88 files changed, 8540 insertions(+), 2132 deletions(-)
>