public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Shannon Sterz <s.sterz@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v7 0/4] refactor datastore locking to use tmpfs
Date: Tue, 11 Mar 2025 14:51:56 +0100	[thread overview]
Message-ID: <20250311135200.309896-1-s.sterz@proxmox.com> (raw)

The goal of this series is to make it safer to remove backup groups &
snapshots by separating the corresponding directories from their lock
files. By moving the lock files to the tmpfs-backed '/run' directory,
we also make sure that the lock files get cleaned up when the system
reboots.

This series refactors the locking mechanism inside the `DataStore`,
`BackupDir` and `BackupGroup` traits. In a first step locking methods
are added and the existing code is refactored to use them. Commit two
derives a lock file name under '/run' for each group/snapshot. It also
adds double stat'ing. To avoid issues when upgrading, the file
`/run/proxmox-backup/old-locking` is created which is used to determine
whether the system has been rebooted and we can safely use the new
locking mechanism.

The third commit refactors locking for manifests and brings it in-line
with the group/snapshot locks. Finally, the last commit fixes a race
condition when changing the owner of a datastore.

This version is based on a fairly old series from 2022 [1].

[1]: https://lore.proxmox.com/pbs-devel/20220824124829.392189-1-s.sterz@proxmox.com/

----
changes from v6:
* add old locking safe guards to avoid different versions of the locking
  mechanism being used at the same time (see discussion here [2]).

[2]: https://lore.proxmox.com/pbs-devel/20250306120810.361035-1-m.sandoval@proxmox.com/T/#u

changes from v5:
* re-phrase commit messages to make it clear which commit actually
  fixes the issue and what the commit implies in-terms of semantic
  changes for error messages (thanks @ Thomas Lamprecht)
* make it so the series applies cleanly again and clean up a newly
  added usage of `lock_dir_noblock`

changes from v4 (thanks @ Wolfgang Bumiller):
* stop using `to_string_lossy()`
* switch funtion signature of `create_locked_backup_group()` and
  `create_locked_backup_dir` to use the `Arc` version of a datastore.
* smaller clippy fixes
* rebased on current master

changes from v3:
* moved patch 2 to the front so it can be applied separatelly more
  easily
* rebased on current master

changes from v2:
* different encoding scheme for lock file names
* refactored locking methods to be used by the new BackupDir and
  BackupGroup traits
* adapted lock file names to include namespaces

changes from v1 (thanks @ Wolfgang Bumiller & Thomas Lamprecht):
* split adding locking helpers and move '/run' into two commits
* instead of stat'ing the path of lock file twice, only use the file
  descriptor for one of the stat'ing procedures instead
* several improvements to helper functions and documentation

Shannon Sterz (4):
  datastore/api/backup: prepare for fix of #3935 by adding lock helpers
  fix #3935: datastore/api/backup: move datastore locking to '/run'
  fix #3935: datastore: move manifest locking to new locking method
  fix: api: avoid race condition in set_backup_owner

 Cargo.toml                           |   2 +-
 debian/postinst                      |   5 +
 pbs-config/src/lib.rs                |  32 +++-
 pbs-datastore/Cargo.toml             |   1 +
 pbs-datastore/src/backup_info.rs     | 230 ++++++++++++++++++++++++---
 pbs-datastore/src/datastore.rs       |  70 +++-----
 pbs-datastore/src/snapshot_reader.rs |  20 ++-
 src/api2/admin/datastore.rs          |   9 +-
 src/api2/backup/environment.rs       |  14 +-
 src/api2/backup/mod.rs               |   8 +-
 src/api2/reader/mod.rs               |   7 +-
 src/backup/verify.rs                 |  12 +-
 src/server/sync.rs                   |  10 +-
 13 files changed, 302 insertions(+), 118 deletions(-)

--
2.39.5



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


             reply	other threads:[~2025-03-11 13:52 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-03-11 13:51 Shannon Sterz [this message]
2025-03-11 13:51 ` [pbs-devel] [PATCH proxmox-backup v7 1/4] datastore/api/backup: prepare for fix of #3935 by adding lock helpers Shannon Sterz
2025-03-14  8:30   ` Christian Ebner
2025-03-11 13:51 ` [pbs-devel] [PATCH proxmox-backup v7 2/4] fix #3935: datastore/api/backup: move datastore locking to '/run' Shannon Sterz
2025-03-11 13:51 ` [pbs-devel] [PATCH proxmox-backup v7 3/4] fix #3935: datastore: move manifest locking to new locking method Shannon Sterz
2025-03-11 13:52 ` [pbs-devel] [PATCH proxmox-backup v7 4/4] fix: api: avoid race condition in set_backup_owner Shannon Sterz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250311135200.309896-1-s.sterz@proxmox.com \
    --to=s.sterz@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal