From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 712AB97025 for ; Tue, 16 Apr 2024 17:24:59 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 40DD61EAA9 for ; Tue, 16 Apr 2024 17:24:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 16 Apr 2024 17:24:27 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id C1D3245A3B for ; Tue, 16 Apr 2024 17:24:26 +0200 (CEST) From: Hannes Laimer To: pbs-devel@lists.proxmox.com Date: Tue, 16 Apr 2024 17:23:54 +0200 Message-Id: <20240416152416.96561-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.008 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH proxmox-backup v4 00/22] add removable datastores X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2024 15:24:59 -0000 These patches add support for removable datastores. All removable datastores have a backing-device(a UUID) associated with them. Removable datastores work like normal ones, just that they can be unplugged. It is possible to create a removable datastore, sync backups onto it, unplug it and use it on a different PBS. The datastore path is also the mountpoint for the removable datastore. By default when creating one through the web UI it will be `/mnt/removable-datastores/`, using the CLI it is possible to specify something else. Since a removable datastore is associated with the UUID of a partition, it is technically possible to have two removable datastores on the same device, but I don't think there is a use-case that couldn't also be done using namespaces. When a removable datastore is deleted and 'destroy-data' is set, the device has to be plugged in. If 'destroy-data' is not set the datastore can be deleted even if the device is not present. Removable datastores are automatically mounted when plugged in. At the API service start all Patche 20/22 is not strictly needed, but they made sense in this context, so I kept them in this series. v4: thanks a lot @Dietmar and @Christian * make check if mounted wayyy faster, checking takes ~0.1ms, only happens once per operation and only on removable datastores so that seems fast enough to just do it instead of keeping track of the state * don't keep track of mounting state * drop Unplugged maintenance mode * use UUID_FORMAT for uuid field * a lot of small things, like use of bail!, inline format!, ... * include improvement to cache handling v3: * remove lazy unmounting (since 9cba51ac782d04085c0af55128f32178e5132358 is applied) * fix CLI (un)mount command, thanks @Gabriel * add removable datastore CLI autocomplete helper * rebase onto master * move ui patches to the end thanks @Lukas and @Thomas for the feedback v2: * fix datastore 'add' button in the UI * some format!("{}", a) -> format!("{a}") * replace `const` with `let` in js code * change icon `fa-usb` -> `fa-plug` * add some docs * add JDoc for parseMaintenanceMode * proxmox-schema dep bump Hannes Laimer (22): tools: add disks utility functions datastore: make dropping from cache more efficient pbs-api-types: add backing-device to DataStoreConfig disks: add UUID to partition info add helper for checking if a removable datastore is available api2: admin: add (un)mount endpoint for removable datastores api2: removable datastore creation api2: disks list: add only-unused flag pbs-api-types: datastore: use new proxmox_schema::de for deserialization pbs-api-types: add removable/is-available flag to DataStoreListItem pb-manager: add (un)mount command add auto-mounting for removable datastores datastore: handle deletion of removable datastore properly docs: mention maintenance mode reset when removable datastore is unplugged ui: add partition selector form ui: add removable datastore creation support ui: add (un)mount button to summary ui: display removable datastores in list ui: utils: render unplugged datastores correctly ui: utils: make parseMaintenanceMode more robust ui: add datastore status mask for unplugged removable datastores ui: maintenance: fix disable msg field if no type is selected debian/proxmox-backup-server.install | 1 + debian/proxmox-backup-server.udev | 3 + docs/maintenance.rst | 2 + etc/Makefile | 3 +- etc/removable-device-attach@.service.in | 8 + pbs-api-types/src/datastore.rs | 32 +++- pbs-api-types/src/maintenance.rs | 4 + pbs-config/src/datastore.rs | 14 ++ pbs-datastore/src/datastore.rs | 119 ++++++++---- pbs-datastore/src/lib.rs | 2 +- src/api2/admin/datastore.rs | 192 ++++++++++++++++++-- src/api2/config/datastore.rs | 75 +++++++- src/api2/node/disks/mod.rs | 8 + src/api2/status.rs | 18 +- src/bin/proxmox_backup_manager/datastore.rs | 130 ++++++++++++- src/tools/disks/mod.rs | 90 ++++++++- www/Makefile | 1 + www/NavigationTree.js | 15 +- www/Utils.js | 31 +++- www/css/ext6-pbs.css | 20 ++ www/datastore/DataStoreListSummary.js | 1 + www/datastore/Summary.js | 103 ++++++++++- www/form/PartitionSelector.js | 74 ++++++++ www/window/DataStoreEdit.js | 51 ++++++ www/window/MaintenanceOptions.js | 7 +- 25 files changed, 903 insertions(+), 101 deletions(-) create mode 100644 etc/removable-device-attach@.service.in create mode 100644 www/form/PartitionSelector.js -- 2.39.2