From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DB29E7A8B3 for ; Tue, 5 Jul 2022 15:08:48 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B71E62F3AE for ; Tue, 5 Jul 2022 15:08:48 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 5 Jul 2022 15:08:46 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9DC4243B91 for ; Tue, 5 Jul 2022 15:08:46 +0200 (CEST) From: Hannes Laimer To: pbs-devel@lists.proxmox.com Date: Tue, 5 Jul 2022 13:08:06 +0000 Message-Id: <20220705130834.14285-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.038 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [schema.rs, mod.rs, proxmox-backup-api.rs, maintenance.rs, lib.rs, proxmox-backup-manager.rs, datastore.rs] Subject: [pbs-devel] [SERIES proxmox-backup/proxmox/widget-toolkit 00/28] add removable datastore support X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jul 2022 13:08:48 -0000 Adds removable-device which has * _unique_ 'name' * _unique_ 'uuid' * 'initialized': true if the chunkstore is already created on the device, if false, it will be created the next time it is mounted * 'store': the datastore the device is associated with, the path of the datastore will be used as a mountpoint for this removable-device When a new datastore is created it has to be makred as 'removable', if it is marked as removable the creation of the chunkstore is skipped. After the creation a removable datastore has no remoavbale-devices associated with with. When creating a new removable-device a name, uuid and store have to be provided, by default a removable device is not initialized. If a device already contains a chunkstore (possibly created on a different PBS) initialized has to be set to true. Unmounting a removable-device puts the associated datastore in 'unplugged' maintenance mode, but it is only really unplugged after all already started reading and writing operations to the datastore are finished. The 'unplugged' maintenance mode prevents the start of reading and writing operations to the datastore, the only way to change the maintenance mode from 'unplugged' to something else is by mounting(not mount, but through PBS) a removable-device associated with the datastore. Mounting a removable-device device 1. if nothing else is already mounted there -> mount otherwise bail! 2. if removable device has to be initialized -> create chunckstore 3. maintenance mode is changed from 'unplugged' to none. UI-wise this series adds icons specific to datastores that are marked as removable, removable-devices behave very similar to sync/prune jobs in the UI. One grid with all the removable-devices and a grid per datastore only containing the removable-devices associated to the current datastore. Since the maintenance mode has to be changed 'in code' and not just through the API, the print_property_string function is add to schema. WIP: * list of jobs(per removable-device) that should be triggert when the removable-device is mounted mounted. The patches for proxmox-backup depends on the changes in proxmox and in proxmox-widget-toolkit. Version bump needed. (patch 19 needs the changes in proxmox-widget-toolkit) (patches >=6 need the changes in proxmox) Since the maintenance mode has to be changed 'in code' and not just through the API, the print_property_string function is add to schema. * proxmox Hannes Laimer (1): schema: add print_property_string function proxmox-schema/src/schema.rs | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) * proxmox-widget-toolkit Hannes Laimer (1): proxy: allow setting of reader config src/data/ProxmoxProxy.js | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) * proxmox-backup Hannes Laimer (26): api-types: add RemovableDeviceConfig config: make RemovableDeviceConfig savable to config file api-types: add "removable" field to DataStoreConfig api2: add config endpoints for RemovableDeviceConfig api-types: add "unplugged" maintenance type api-types: add set_maintenance_mode function to DataStoreConfig api2: datastore create: don't init chunkstore if removable tools: disks: add mount and unmount helper api2: admin: add unmount-device endpoint to datastore api2: admin: add mount-device and list endpoint for RemavableDevices tools: disks: add uuid to PrtitionInfo api-types: add "removable" to DataStoreListItem ui: utils: remove parseMaintenanceMode function ui: maintenance edit: disable description in maintenance edit if no type is selected ui: config: add RemovableDeviceView ui: add "no removable device present" mask to summary ui: add removable datastore specific icons to list ui: window: add RemovableDeviceEdit ui: form: add PartitionSelector ui: add "removable" checkbox to datastore edit ui: disable maintenance update while removable datastore is unplugged ui: datstore selector: add icon for removable datastores api2: admin: add mount-device endpoint that just takes an UUID cli: manager: add removable-device commands debian/etc: add udev rules and simple service for automounting api: mark all removable datastores as 'unplugged' after restart debian/proxmox-backup-server.install | 1 + debian/proxmox-backup-server.udev | 3 + etc/Makefile | 3 +- etc/removable-device-attach@.service.in | 6 + pbs-api-types/src/datastore.rs | 26 ++ pbs-api-types/src/lib.rs | 3 + pbs-api-types/src/maintenance.rs | 16 +- pbs-api-types/src/removable_device.rs | 79 +++++ pbs-config/src/lib.rs | 1 + pbs-config/src/removable_device.rs | 63 ++++ pbs-datastore/src/datastore.rs | 2 +- src/api2/admin/datastore.rs | 98 +++++- src/api2/admin/mod.rs | 6 + src/api2/admin/removable_device.rs | 283 +++++++++++++++++ src/api2/config/datastore.rs | 52 +++- src/api2/config/mod.rs | 2 + src/api2/config/removable_device.rs | 284 ++++++++++++++++++ src/bin/proxmox-backup-api.rs | 15 + src/bin/proxmox-backup-manager.rs | 1 + src/bin/proxmox_backup_manager/mod.rs | 2 + .../removable_device.rs | 209 +++++++++++++ src/tools/disks/mod.rs | 27 +- www/Makefile | 3 + www/NavigationTree.js | 19 +- www/Utils.js | 27 +- www/config/RemovableDeviceView.js | 211 +++++++++++++ www/css/ext6-pbs.css | 29 ++ www/datastore/DataStoreList.js | 5 + www/datastore/Panel.js | 9 + www/datastore/Summary.js | 17 +- www/form/DataStoreSelector.js | 3 + www/form/PartitionSelector.js | 61 ++++ www/window/DataStoreEdit.js | 10 + www/window/MaintenanceOptions.js | 23 +- www/window/RemovableDeviceEdit.js | 76 +++++ 35 files changed, 1614 insertions(+), 61 deletions(-) create mode 100644 etc/removable-device-attach@.service.in create mode 100644 pbs-api-types/src/removable_device.rs create mode 100644 pbs-config/src/removable_device.rs create mode 100644 src/api2/admin/removable_device.rs create mode 100644 src/api2/config/removable_device.rs create mode 100644 src/bin/proxmox_backup_manager/removable_device.rs create mode 100644 www/config/RemovableDeviceView.js create mode 100644 www/form/PartitionSelector.js create mode 100644 www/window/RemovableDeviceEdit.js -- 2.30.2