From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 27BA11FF37F for ; Thu, 18 Apr 2024 16:29:45 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 14FE51E81C; Thu, 18 Apr 2024 16:29:44 +0200 (CEST) From: Hannes Laimer To: pbs-devel@lists.proxmox.com Date: Thu, 18 Apr 2024 16:29:08 +0200 Message-Id: <20240418142931.122618-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.001 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [maintenance.rs, etc.com, status.rs, lib.rs, mod.rs, proxmox-backup-proxy.rs, datastore.rs] Subject: [pbs-devel] [PATCH proxmox-backup v6 00/23] add removable datastores X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" These patches add support for removable datastores. All removable datastores have a backing-device(a UUID) associated with them. Removable datastores work like normal ones, just that they can be unplugged. It is possible to create a removable datastore, sync backups onto it, unplug it and use it on a different PBS. The datastore path is also the mountpoint for the removable datastore. By default when creating one through the web UI it will be `/mnt/removable-datastores/`, using the CLI it is possible to specify something else. Since a removable datastore is associated with the UUID of a partition, it is technically possible to have two removable datastores on the same device, but I don't think there is a use-case that couldn't also be done using namespaces. When a removable datastore is deleted and 'destroy-data' is set, the device has to be plugged in. If 'destroy-data' is not set the datastore can be deleted even if the device is not present. Removable datastores are automatically mounted when plugged in. At the API service start all removable datastores are marked as 'unplugged', unless they are already mounted. v6: * remove 'drop' flag in datastore cache * use maintenance-mode 'unmount' for unmounting process, only for the unmounting not for being unmounted * rename/simplify update-datastore-cache command * ui: integrate new unmounting maintenance mode * basically a mix of v3 and v4 v5: thanks @Dietmar and @Christian * drop --force for unmount since it'll always fail if tasks are still running, and if there are not normal unount will work * improve several commit messages * improve error message wording * add removable datastore section to docs * add documentation for is_datastore_available v4: thanks a lot @Dietmar and @Christian * make check if mounted wayyy faster * don't keep track of mounting state * drop Unplugged maintenance mode * use UUID_FORMAT for uuid field * a lot of small things, like use of bail!, inline format!, ... * include improvement to cache handling v3: * remove lazy unmounting (since 9cba51ac782d04085c0af55128f32178e5132358 is applied) * fix CLI (un)mount command, thanks @Gabriel * add removable datastore CLI autocomplete helper * rebase onto master * move ui patches to the end thanks @Lukas and @Thomas for the feedback v2: * fix datastore 'add' button in the UI * some format!("{}", a) -> format!("{a}") * replace `const` with `let` in js code * change icon `fa-usb` -> `fa-plug` * add some docs * add JDoc for parseMaintenanceMode * proxmox-schema dep bump Dietmar Maurer (2): pbs-api-types: new MaintenanceType::Unmount, implement and use set_maintenance_mode pbs-api-types: use SchemaDeserializer for maintenance mode Hannes Laimer (21): datastore: rename and simplify update-datastore-cache socket command tools: add disks utility functions pbs-api-types: add backing-device to DataStoreConfig disks: add UUID to partition info add helper for checking if a removable datastore is available api2: admin: add (un)mount endpoint for removable datastores api2: removable datastore creation api2: disks list: add only-unused flag pbs-api-types: add removable/is-available flag to DataStoreListItem bin: manager: add (un)mount command add auto-mounting for removable datastores datastore: handle deletion of removable datastore properly docs: add removable datastores section ui: add partition selector form ui: add removable datastore creation support ui: add (un)mount button to summary ui: tree: render unmounted datastores correctly ui: utils: make parseMaintenanceMode more robust ui: add datastore status mask for unmounted removable datastores ui: maintenance: fix disable msg field if no type is selected ui: render 'unmount' maintenance mode correctly debian/proxmox-backup-server.install | 1 + debian/proxmox-backup-server.udev | 3 + docs/storage.rst | 23 +++ etc/Makefile | 3 +- etc/removable-device-attach@.service.in | 8 + pbs-api-types/src/datastore.rs | 71 ++++++- pbs-api-types/src/maintenance.rs | 17 +- pbs-config/src/datastore.rs | 14 ++ pbs-datastore/src/datastore.rs | 90 ++++++--- pbs-datastore/src/lib.rs | 2 +- src/api2/admin/datastore.rs | 202 ++++++++++++++++++-- src/api2/config/datastore.rs | 87 ++++++++- src/api2/node/disks/mod.rs | 8 + src/api2/status.rs | 18 +- src/bin/proxmox-backup-proxy.rs | 8 +- src/bin/proxmox_backup_manager/datastore.rs | 130 ++++++++++++- src/tools/disks/mod.rs | 90 ++++++++- www/Makefile | 1 + www/NavigationTree.js | 15 +- www/Utils.js | 33 +++- www/css/ext6-pbs.css | 20 ++ www/datastore/DataStoreListSummary.js | 1 + www/datastore/Summary.js | 113 ++++++++++- www/form/PartitionSelector.js | 74 +++++++ www/window/DataStoreEdit.js | 51 +++++ www/window/MaintenanceOptions.js | 17 +- 26 files changed, 995 insertions(+), 105 deletions(-) create mode 100644 etc/removable-device-attach@.service.in create mode 100644 www/form/PartitionSelector.js -- 2.39.2 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel