From: Jakob-Tobias Winter <tobias@stuttgart.de>
To: pbs-devel@lists.proxmox.com
Cc: Jakob-Tobias Winter <tobias@stuttgart.de>
Subject: [PATCH proxmox-backup 0/4] fix 7333 use multiple tape drives for restoring
Date: Thu, 12 Nov 2026 12:37:33 +0100 [thread overview]
Message-ID: <20261112113737.748541-1-tobias@stuttgart.de> (raw)
Restoring a media set reads one tape after the other with a single
drive, even if the library has more drives that are idle. This series
lets a restore use several drives of a changer, so several tapes are
read at the same time [0]. Even when the host cannot keep up with the
combined data rate, loading, positioning and changer moves on one
drive then overlap with reading on the others.
This is the follow-up to the RFC [1], thanks to Dominik for the
review.
Patch 1 adds a pool of drives that runs one job per media. With a
single drive, this is the existing loop, so a restore with one drive
behaves as before. With several drives, the media in the drives are
unloaded first, and the task fails if a needed media is loaded in a
drive that is not used. Then each drive runs in its own thread, where
it takes the next media, loads it, runs the job and unloads it again.
Changer operations are serialized, so two drives do not pick the same
free slot when unloading. The first error, or an abort, stops all
drives, which then unload their media as well. As the lines of all
drives are interleaved in the task log, every message of a drive
starts with "drive 'NAME': ".
Patch 2 loads the media of the first two phases of the selective
restore through the pool, without functional change.
Patch 3 adds the 'drives' parameter to the API and to 'proxmox-tape
restore'. It replaces 'drive', which still works, but is deprecated.
Several drives have to be in the same changer. Every drive is checked
for Tape.Read, locked and marked as in use for the whole task. A
selective restore then reads the snapshot archives (phase 1) and the
chunks (phase 2) on all drives.
A full restore writes the snapshot archives straight into the
datastore. That is safe with a single drive, because a tape backup
writes the chunks of a snapshot before its snapshot archive, to the
same or an earlier media. With several drives, a snapshot archive can
be read while an earlier media is still being restored, and an aborted
restore would leave snapshots with missing chunks behind. So with more
than one drive, a full restore runs the phases of the selective restore
for all snapshots, which only add the snapshots once all chunks are
restored. The restore window already takes that path whenever it sends
a namespace mapping. The trade-offs are that the media set catalog is
needed, every media is loaded twice (once for the snapshot archives,
once for the chunks), the snapshots only show up at the end, and
snapshots that already exist in the datastore are skipped with a
warning instead of silently.
Restoring everything in a single pass, with the snapshots in a
temporary directory until all chunks are there, would avoid the second
load, as suggested in the review of the RFC. I left that out to keep
the changes to the restore logic small, and would look into it as a
follow-up.
Patch 4 turns the drive selector of the restore window into a
multi-select field, which only accepts several drives if they are in
the same changer.
On abort or error, the existing cleanup of the selective restore
removes the temporary directory, so an abort before phase 3 does not
add any snapshot. Chunks that were already written stay until garbage
collection removes them, as today.
Changes since the RFC [1]:
- 'drives' replaces 'drive', instead of adding 'additional-drives',
and there is no limit for the number of drives anymore
- several drives have to be in the same changer, the API and the
restore window check that
- the drives are unloaded first, a needed media in a drive that is not
used is an error, and each drive takes the next media in order and
unloads it after use, instead of preferring the media loaded in it
- a failing drive aborts the task, instead of using a stop flag and a
task context of its own, and a poisoned lock is not ignored anymore
- the pool takes the maps of the two restore phases as they are and
looks up the media labels in the inventory itself
- one multi-select field in the restore window instead of a second
field for additional drives
- rebased on a newer master
Testing: I built this version on top of 4.2.8 and tested it with
several drives on a tape library with LTO-7 media, where four of the
drives are available to PBS: restoring a whole media set, restoring a
selected snapshot, aborting a restore, after which the drives unload
their media, and the error for a needed media that is loaded in a
drive that is not used. I also tested the unloading of drives that
hold a media when the restore starts, and the restore window.
In addition, with real daemons and four virtual tape drives, each with
a copy of the tapes: with one drive, the task logs of a full restore,
a restore of selected snapshots and one with a namespace mapping are
the same as on master. With two to four drives, the same three kinds
of restore give datastores with the content of the source, which also
verify and read back correctly. A drive that fails stops the others
and unloads its media, and the task ends with its error, without
adding a snapshot. Drives that are not in the same changer are
refused. The test suite passes and clippy shows no new warnings.
[0] https://bugzilla.proxmox.com/show_bug.cgi?id=7333
[1] https://lore.proxmox.com/pbs-devel/20261106133826.1692620-2-tobias@stuttgart.de/
Jakob-Tobias Winter (4):
tape: drive: add a pool to use several drives for one task
tape: restore: use a drive pool for the selective restore
close #7333: tape: restore: allow using several drives in parallel
ui: tape restore: allow selecting several drives
src/api2/tape/restore.rs | 180 ++++++++++++++++-------
src/bin/proxmox-tape.rs | 15 +-
src/tape/drive/mod.rs | 40 ++++-
src/tape/drive/pool.rs | 259 +++++++++++++++++++++++++++++++++
www/tape/form/DriveSelector.js | 17 +++
www/tape/window/TapeRestore.js | 5 +-
6 files changed, 455 insertions(+), 61 deletions(-)
create mode 100644 src/tape/drive/pool.rs
base-commit: 68577e65400958d9f7e76a7722669fb359b69d4b
--
2.53.0
next reply other threads:[~2026-11-12 11:41 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 11:37 Jakob-Tobias Winter [this message]
2026-11-12 11:37 ` [PATCH proxmox-backup 1/4] tape: drive: add a pool to use several drives for one task Jakob-Tobias Winter
2026-11-12 11:37 ` [PATCH proxmox-backup 2/4] tape: restore: use a drive pool for the selective restore Jakob-Tobias Winter
2026-11-12 11:37 ` [PATCH proxmox-backup 3/4] close #7333: tape: restore: allow using several drives in parallel Jakob-Tobias Winter
2026-11-12 11:37 ` [PATCH proxmox-backup 4/4] ui: tape restore: allow selecting several drives Jakob-Tobias Winter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261112113737.748541-1-tobias@stuttgart.de \
--to=tobias@stuttgart.de \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.