* [pbs-devel] [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper
@ 2024-11-27 14:11 Fabian Grünbichler
2024-11-27 14:11 ` [pbs-devel] [RFC proxmox-backup 2/2] GC: add check for nested datastore Fabian Grünbichler
2024-11-27 14:27 ` [pbs-devel] applied: [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Thomas Lamprecht
0 siblings, 2 replies; 4+ messages in thread
From: Fabian Grünbichler @ 2024-11-27 14:11 UTC (permalink / raw)
To: pbs-devel
and improve the variable namign while we are at it. this allows the check to be
re-used in other code paths, like when starting a garbage collection.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
Notes:
no semantic changes intended, for *after* the release
pbs-api-types/src/datastore.rs | 39 ++++++++++++++++++++++++++++++++++
src/api2/config/datastore.rs | 29 +++----------------------
2 files changed, 42 insertions(+), 26 deletions(-)
diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index d3876838b..ddd8d3c6b 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -441,6 +441,45 @@ impl DataStoreConfig {
Ok(())
}
+
+ pub fn ensure_not_nested(&self, stores: &[DataStoreConfig]) -> Result<(), Error> {
+ let our_absolute_path = PathBuf::from(self.absolute_path());
+ let removable = self.backing_device.is_some();
+ for other_store in stores {
+ if self == other_store {
+ continue;
+ };
+
+ // Relative paths must not be nested on the backing device of removable datastores
+ if removable && other_store.backing_device == self.backing_device {
+ let our_relative_path = Path::new(&self.path);
+ let other_relative_path = Path::new(&other_store.path);
+ if our_relative_path.starts_with(other_relative_path)
+ || other_relative_path.starts_with(our_relative_path)
+ {
+ bail!(
+ "paths on backing device must not be nested - {path:?} already used by '{store}'!",
+ path = other_relative_path,
+ store = other_store.name,
+ );
+ }
+ }
+
+ // No two datastores should have a nested absolute path
+ let other_absolute_path = PathBuf::from(other_store.absolute_path());
+ if other_absolute_path.starts_with(&our_absolute_path)
+ || our_absolute_path.starts_with(&other_absolute_path)
+ {
+ bail!(
+ "nested datastores not allowed: '{}' already in {:?}",
+ other_store.name,
+ other_absolute_path,
+ );
+ }
+ }
+
+ Ok(())
+ }
}
#[api(
diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
index 7c087d9fc..d8bae2078 100644
--- a/src/api2/config/datastore.rs
+++ b/src/api2/config/datastore.rs
@@ -82,32 +82,9 @@ pub(crate) fn do_create_datastore(
bail!("cannot create datastore in root path");
}
- let new_store_path = PathBuf::from(&datastore.absolute_path());
- let removable = datastore.backing_device.is_some();
- for store in config.convert_to_typed_array::<DataStoreConfig>("datastore")? {
- // Relative paths must not be nested on the backing device of removable datastores
- if removable && store.backing_device == datastore.backing_device {
- let new_path = Path::new(&datastore.path);
- let path = Path::new(&store.path);
- if new_path.starts_with(path) || path.starts_with(new_path) {
- param_bail!(
- "path",
- "paths on backing device must not be nested - {path:?} already used by '{store}'!",
- store = store.name
- );
- }
- }
-
- // No two datastores should have a nested absolute path
- let store_path = PathBuf::from(store.absolute_path());
- if store_path.starts_with(&new_store_path) || new_store_path.starts_with(&store_path) {
- param_bail!(
- "path",
- "nested datastores not allowed: '{}' already in {:?}",
- store.name,
- store_path,
- );
- }
+ let existing_stores = config.convert_to_typed_array("datastore")?;
+ if let Err(err) = datastore.ensure_not_nested(&existing_stores) {
+ param_bail!("path", err);
}
let need_unmount = datastore.backing_device.is_some();
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [pbs-devel] [RFC proxmox-backup 2/2] GC: add check for nested datastore
2024-11-27 14:11 [pbs-devel] [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Fabian Grünbichler
@ 2024-11-27 14:11 ` Fabian Grünbichler
2024-11-27 14:31 ` [pbs-devel] applied: " Thomas Lamprecht
2024-11-27 14:27 ` [pbs-devel] applied: [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Thomas Lamprecht
1 sibling, 1 reply; 4+ messages in thread
From: Fabian Grünbichler @ 2024-11-27 14:11 UTC (permalink / raw)
To: pbs-devel
these are particularly problematic since GC will walk the whole datastore tree
on the file system, and will thus pick up indices (but not chunks!) from nested
directories that are ignored in other code paths that use our regular
iterators..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
Notes:
a similar check might also be sensible for mounting and should now be fairly
easy to implement there as well..
pbs-datastore/src/datastore.rs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 33bc1f72e..4c062e244 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -1150,6 +1150,17 @@ impl DataStore {
// writer" information and thus no safe atime cutoff
let _exclusive_lock = self.inner.chunk_store.try_exclusive_lock()?;
+ let (config, _digest) = pbs_config::datastore::config()?;
+ let gc_store_config: DataStoreConfig = config.lookup("datastore", &self.name())?;
+ let all_stores = config.convert_to_typed_array("datastore")?;
+ if let Err(err) = gc_store_config.ensure_not_nested(&all_stores) {
+ info!(
+ "Current datastore path: {path}",
+ path = gc_store_config.absolute_path()
+ );
+ bail!("Aborting GC for safety reasons: {err}");
+ }
+
let phase1_start_time = proxmox_time::epoch_i64();
let oldest_writer = self
.inner
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [pbs-devel] applied: [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper
2024-11-27 14:11 [pbs-devel] [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Fabian Grünbichler
2024-11-27 14:11 ` [pbs-devel] [RFC proxmox-backup 2/2] GC: add check for nested datastore Fabian Grünbichler
@ 2024-11-27 14:27 ` Thomas Lamprecht
1 sibling, 0 replies; 4+ messages in thread
From: Thomas Lamprecht @ 2024-11-27 14:27 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Fabian Grünbichler
Am 27.11.24 um 15:11 schrieb Fabian Grünbichler:
> and improve the variable namign while we are at it. this allows the check to be
> re-used in other code paths, like when starting a garbage collection.
>
> Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
> ---
>
> Notes:
> no semantic changes intended, for *after* the release
>
> pbs-api-types/src/datastore.rs | 39 ++++++++++++++++++++++++++++++++++
> src/api2/config/datastore.rs | 29 +++----------------------
> 2 files changed, 42 insertions(+), 26 deletions(-)
>
>
applied, thanks!
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [pbs-devel] applied: [RFC proxmox-backup 2/2] GC: add check for nested datastore
2024-11-27 14:11 ` [pbs-devel] [RFC proxmox-backup 2/2] GC: add check for nested datastore Fabian Grünbichler
@ 2024-11-27 14:31 ` Thomas Lamprecht
0 siblings, 0 replies; 4+ messages in thread
From: Thomas Lamprecht @ 2024-11-27 14:31 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Fabian Grünbichler
Am 27.11.24 um 15:11 schrieb Fabian Grünbichler:
> these are particularly problematic since GC will walk the whole datastore tree
> on the file system, and will thus pick up indices (but not chunks!) from nested
> directories that are ignored in other code paths that use our regular
> iterators..
>
> Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
> ---
>
> Notes:
> a similar check might also be sensible for mounting and should now be fairly
> easy to implement there as well..
>
> pbs-datastore/src/datastore.rs | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
>
applied, thanks!
It's a sensible improvement and cost is low, especially compared to what GC does
in general. That said, it's naturally not a silver bullet, through a symlink or
bind mount one could construct a datastore that would not be detected, but
probably obvious to you already so just mentioning for the sake of completeness.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-11-27 14:32 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-11-27 14:11 [pbs-devel] [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Fabian Grünbichler
2024-11-27 14:11 ` [pbs-devel] [RFC proxmox-backup 2/2] GC: add check for nested datastore Fabian Grünbichler
2024-11-27 14:31 ` [pbs-devel] applied: " Thomas Lamprecht
2024-11-27 14:27 ` [pbs-devel] applied: [PATCH proxmox-backup 1/2] datastore: extract nesting check into helper Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox