From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 75F271FF0AF for ; Fri, 06 Nov 2026 14:38:53 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id B4A2C216E8; Fri, 06 Nov 2026 14:38:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stuttgart.de; s=s1022; t=1793972325; bh=1w96pKA7COTquki0sVs/nyBNJ6C2yp54UWQ5YpMtado=; h=From:To:Cc:Subject:Date:From; b=DXZwNCqYksS2YnSjKkAhmIwxeMRIGO/dcl+g68SI6lyA8GmVIXIMhDrOGW9QKkvvD fM64e5co/CiSIGKmAUm4GghOx0D8KXionGzURLPGHEupOxqslpV3FnB67XIskFNIlB zqRUCMOsUCEMqQltz7Z2lkoy2s3fCVC0MYdAchjb8ca5vUVhsfgvs9M0rJE+XFawdB b7IdOzajOCKhR4k2m4iWVQvf1NWWpkIjRj5N7uxKLUBGHBBOdZ5DjBwoRrBAKNg35d s9TzI5Mv5hcajMbCV+0dDuEGvLdJj7fZy9wbPj1YdFetILB+JtVadlipMWAFB8WCCd IFeKSzwCvyZDw== From: Jakob-Tobias Winter To: pbs-devel@lists.proxmox.com Subject: [RFC PATCH proxmox-backup 0/4] fix 7333 use multiple tape drives for restoring Date: Fri, 6 Nov 2026 14:38:23 +0100 Message-ID: <20261106133826.1692620-2-tobias@stuttgart.de> X-Mailer: git-send-email 2.53.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 4O6UCBFK7M2IVKQGP65YFMRVSPKATKA6 X-Message-ID-Hash: 4O6UCBFK7M2IVKQGP65YFMRVSPKATKA6 X-MailFrom: prvs=957be33b11=tobias@stuttgart.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Jakob-Tobias Winter X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Restoring a media set reads one tape after the other with a single drive, even if the library has more drives that are idle. This series lets a restore use up to three additional drives, so several tapes are read at the same time [0]. Even when the host cannot keep up with the combined data rate, loading, positioning and changer moves on one drive then overlap with reading on the others. I am sending this as RFC to discuss the approach. The change is kept small and reuses the existing selective restore: Patch 1 adds a pool of drives that runs one job per media. With a single drive, this is the existing loop, so a restore with one drive behaves as before. With several drives, each drive runs in its own thread and takes the media already loaded in it, else the next one that is not loaded in another drive of the pool. Changer operations are serialized, so two drives do not pick the same free slot when unloading. The first error, or an abort, stops all drives. As the lines of all drives are interleaved in the task log, every message of a drive starts with "drive 'NAME': ". Patch 2 loads the media of the first two phases of the selective restore through the pool, without functional change. Patch 3 adds the optional 'additional-drives' parameter to the API and to 'proxmox-tape restore'. Every drive is checked for Tape.Read, locked and marked as in use for the whole task. A selective restore then reads the snapshot archives (phase 1) and the chunks (phase 2) on all drives. A full restore writes the snapshot archives straight into the datastore. That is safe with a single drive, because a tape backup writes the chunks of a snapshot before its snapshot archive, to the same or an earlier media. With several drives, a snapshot archive can be read while an earlier media is still being restored, and an aborted restore would leave snapshots with missing chunks behind. So with more than one drive, a full restore runs the phases of the selective restore for all snapshots, which only add the snapshots once all chunks are restored. The restore window already takes that path whenever it sends a namespace mapping. The trade-offs are that the media set catalog is needed, every media is loaded twice (once for the snapshot archives, once for the chunks), and the snapshots only show up at the end. Patch 4 adds a multi-select field for the additional drives to the restore window. On abort or error, the existing cleanup of the selective restore removes the temporary directory, so an abort before phase 3 does not add any snapshot. Chunks that were already written stay until garbage collection removes them, as today. Open questions: - Is 'drive' plus 'additional-drives' fine, or would you prefer something else, e.g. a 'drives' list, or the changer plus "use all free drives"? - Is a maximum of four drives in total OK or can we drop the limit? - Standalone drives (without changer) work, but the pool cannot know which tape is in which drive, so the operator has to follow the per-drive "insert media" notifications. Is that OK? - Should the "drive 'NAME': " prefix also be used with a single drive, for consistency? Testing: I built the series on top of 4.2.8, and tested it on a tape library with four drives (LTO-7 media), with one and with several drives. With four drives, each drive read at about 260-290 MB/s. [0] https://bugzilla.proxmox.com/show_bug.cgi?id=7333 Jakob-Tobias Winter (4): tape: drive: add a pool to use several drives for one task tape: restore: use a drive pool for the selective restore close #7333: tape: restore: allow using several drives in parallel ui: tape restore: add selector for additional drives src/api2/tape/restore.rs | 179 +++++++++++------ src/bin/proxmox-tape.rs | 10 + src/tape/drive/mod.rs | 41 +++- src/tape/drive/pool.rs | 358 +++++++++++++++++++++++++++++++++ www/tape/window/TapeRestore.js | 15 ++ 5 files changed, 540 insertions(+), 63 deletions(-) create mode 100644 src/tape/drive/pool.rs base-commit: 1df6dff9c0acdd11386245502b2c7aeed2f440dd -- 2.53.0