public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup v9 00/26] add removable datastores
@ 2024-04-23  8:29 Hannes Laimer
  2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 01/26] tools: add disks utility functions Hannes Laimer
                   ` (25 more replies)
  0 siblings, 26 replies; 47+ messages in thread
From: Hannes Laimer @ 2024-04-23  8:29 UTC (permalink / raw)
  To: pbs-devel

These patches add support for removable datastores. All removable
datastores have a backing-device(a UUID) associated with them. Removable
datastores work like normal ones, just that they can be unplugged. It is
possible to create a removable datastore, sync backups onto it, unplug
it and use it on a different PBS.

The datastore path specified is relative to `/mnt/removable_datastore/<name>`.

When a removable datastore is deleted and 'destroy-data' is set, the
device has to be mounted in. If 'destroy-data' is not set the datastore
can be deleted even if the device is not present. Removable datastores
are automatically mounted when plugged in. 

 
v9:
 * change mount point to `/mnt/datastore/<name>`
 * update "Directory" list UI
 * add `absolute_path()` from Dietmar's RFC
 * update docs

v8:
 * still depends on [1]
 * paths for removable datastores are now relative to 
    `/mnt/removable_datastore/<UUID>`
 * add support for creation of removable datastore through the 
    "create directory" endpoint (last 3 patches)
 * update datastore creation UI
 * update docs

v7:
 * depends on [1]
 * improve logging when waiting for tasks
 * drop `update-datatore-cache` refactoring
 * fix some commit messages

[1] https://lists.proxmox.com/pipermail/pbs-devel/2024-April/008739.html

v6:
 * remove 'drop' flag in datastore cache
 * use maintenance-mode 'unmount' for unmounting process, only for the
    unmounting not for being unmounted
 * rename/simplify update-datastore-cache command
 * ui: integrate new unmounting maintenance mode
 * basically a mix of v3 and v4

v5: thanks @Dietmar and @Christian
 * drop --force for unmount since it'll always fail if tasks are still running, and if
    there are not normal unount will work
 * improve several commit messages
 * improve error message wording
 * add removable datastore section to docs
 * add documentation for is_datastore_available

v4: thanks a lot @Dietmar and @Christian
 * make check if mounted wayyy faster
 * don't keep track of mounting state
 * drop Unplugged maintenance mode
 * use UUID_FORMAT for uuid field
 * a lot of small things, like use of bail!, inline format!, ...
 * include improvement to cache handling

v3:
 * remove lazy unmounting (since 9cba51ac782d04085c0af55128f32178e5132358 is applied)
 * fix CLI (un)mount command, thanks @Gabriel
 * add removable datastore CLI autocomplete helper
 * rebase onto master
 * move ui patches to the end

thanks @Lukas and @Thomas for the feedback
v2:
 * fix datastore 'add' button in the UI
 * some format!("{}", a) -> format!("{a}")
 * replace `const` with `let` in js code
 * change icon `fa-usb` -> `fa-plug`
 * add some docs
 * add JDoc for parseMaintenanceMode
 * proxmox-schema dep bump

Dietmar Maurer (2):
  config: factor out method to get the absolute datastore path
  maintenance: add 'Unmount' maintenance type

Hannes Laimer (24):
  tools: add disks utility functions
  pbs-api-types: add backing-device to DataStoreConfig
  disks: add UUID to partition info
  datastore: add helper for checking if a removable datastore is
    available
  api: admin: add (un)mount endpoint for removable datastores
  api: removable datastore creation
  api: disks list: add exclude-used flag
  pbs-api-types: add removable/is-available flag to DataStoreListItem
  bin: manager: add (un)mount command
  add auto-mounting for removable datastores
  datastore: handle deletion of removable datastore properly
  docs: add removable datastores section
  ui: add partition selector form
  ui: add removable datastore creation support
  ui: add (un)mount button to summary
  ui: tree: render unmounted datastores correctly
  ui: utils: make parseMaintenanceMode more robust
  ui: add datastore status mask for unmounted removable datastores
  ui: maintenance: fix disable msg field if no type is selected
  ui: render 'unmount' maintenance mode correctly
  api: node: allow creation of removable datastore through directory
    endpoint
  api: node: include removable datastores in directory list
  node: disks: replace BASE_MOUNT_DIR with DATASTORE_MOUNT_DIR
  ui: support create removable datastore through directory creation

 debian/proxmox-backup-server.install        |   1 +
 debian/proxmox-backup-server.udev           |   3 +
 docs/storage.rst                            |  29 +++
 etc/Makefile                                |   3 +-
 etc/removable-device-attach@.service.in     |   8 +
 pbs-api-types/src/datastore.rs              |  42 +++-
 pbs-api-types/src/maintenance.rs            |  11 +-
 pbs-config/src/datastore.rs                 |  14 ++
 pbs-datastore/src/datastore.rs              |  65 ++++++-
 pbs-datastore/src/lib.rs                    |   2 +-
 src/api2/admin/datastore.rs                 | 204 ++++++++++++++++++--
 src/api2/config/datastore.rs                |  79 +++++++-
 src/api2/node/disks/directory.rs            | 104 ++++++++--
 src/api2/node/disks/mod.rs                  |   8 +
 src/api2/status.rs                          |  18 +-
 src/bin/proxmox-backup-proxy.rs             |   4 +-
 src/bin/proxmox_backup_manager/datastore.rs | 130 ++++++++++++-
 src/tools/disks/mod.rs                      |  90 ++++++++-
 www/DirectoryList.js                        |  13 ++
 www/Makefile                                |   1 +
 www/NavigationTree.js                       |  15 +-
 www/Utils.js                                |  33 +++-
 www/css/ext6-pbs.css                        |  20 ++
 www/datastore/DataStoreListSummary.js       |   1 +
 www/datastore/Summary.js                    | 113 ++++++++++-
 www/form/PartitionSelector.js               |  74 +++++++
 www/window/CreateDirectory.js               |  14 ++
 www/window/DataStoreEdit.js                 |  37 ++++
 www/window/MaintenanceOptions.js            |  17 +-
 29 files changed, 1068 insertions(+), 85 deletions(-)
 create mode 100644 etc/removable-device-attach@.service.in
 create mode 100644 www/form/PartitionSelector.js

-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 47+ messages in thread

end of thread, other threads:[~2024-04-25  8:58 UTC | newest]

Thread overview: 47+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-23  8:29 [pbs-devel] [PATCH proxmox-backup v9 00/26] add removable datastores Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 01/26] tools: add disks utility functions Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 02/26] config: factor out method to get the absolute datastore path Hannes Laimer
2024-04-24 13:26   ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 03/26] pbs-api-types: add backing-device to DataStoreConfig Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 04/26] maintenance: add 'Unmount' maintenance type Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 05/26] disks: add UUID to partition info Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 06/26] datastore: add helper for checking if a removable datastore is available Hannes Laimer
2024-04-24 11:16   ` Gabriel Goller
2024-04-24 13:30   ` Wolfgang Bumiller
2024-04-25  6:05     ` Hannes Laimer
2024-04-25  7:19       ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 07/26] api: admin: add (un)mount endpoint for removable datastores Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 08/26] api: removable datastore creation Hannes Laimer
2024-04-24 13:04   ` Wolfgang Bumiller
2024-04-25  8:52     ` Wolfgang Bumiller
2024-04-25  8:58       ` Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 09/26] api: disks list: add exclude-used flag Hannes Laimer
2024-04-24 13:09   ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 10/26] pbs-api-types: add removable/is-available flag to DataStoreListItem Hannes Laimer
2024-04-24 13:13   ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 11/26] bin: manager: add (un)mount command Hannes Laimer
2024-04-24 11:25   ` Gabriel Goller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 12/26] add auto-mounting for removable datastores Hannes Laimer
2024-04-24 13:18   ` Wolfgang Bumiller
2024-04-25  5:32     ` Hannes Laimer
2024-04-25  5:34       ` Hannes Laimer
2024-04-24 13:35   ` Wolfgang Bumiller
2024-04-25  4:59     ` Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 13/26] datastore: handle deletion of removable datastore properly Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 14/26] docs: add removable datastores section Hannes Laimer
2024-04-24 11:53   ` Gabriel Goller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 15/26] ui: add partition selector form Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 16/26] ui: add removable datastore creation support Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 17/26] ui: add (un)mount button to summary Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 18/26] ui: tree: render unmounted datastores correctly Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 19/26] ui: utils: make parseMaintenanceMode more robust Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 20/26] ui: add datastore status mask for unmounted removable datastores Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 21/26] ui: maintenance: fix disable msg field if no type is selected Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 22/26] ui: render 'unmount' maintenance mode correctly Hannes Laimer
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 23/26] api: node: allow creation of removable datastore through directory endpoint Hannes Laimer
2024-04-24 11:56   ` Gabriel Goller
2024-04-24 13:21   ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 24/26] api: node: include removable datastores in directory list Hannes Laimer
2024-04-24 13:06   ` Wolfgang Bumiller
2024-04-23  8:29 ` [pbs-devel] [PATCH proxmox-backup v9 25/26] node: disks: replace BASE_MOUNT_DIR with DATASTORE_MOUNT_DIR Hannes Laimer
2024-04-23  8:30 ` [pbs-devel] [PATCH proxmox-backup v9 26/26] ui: support create removable datastore through directory creation Hannes Laimer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal