* [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
@ 2024-07-12 7:54 Christoph Heiss
2024-07-12 8:16 ` Turkijan
2024-07-18 10:29 ` Thomas Lamprecht
0 siblings, 2 replies; 8+ messages in thread
From: Christoph Heiss @ 2024-07-12 7:54 UTC (permalink / raw)
To: pbs-devel
Saves us from converting the paths to raw C strings ourselves, thus
simplifying it quite a bit.
Since the `nix` crate does not provide link() directly, only linkat(),
that is used instead - which behaves exactly the same as link() when
using AT_FDCWD and an empty flag set.
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
Changes v1 -> v2:
* use `AtFlags` instead of deprecated `LinkatFlags` (thanks Christian!)
* use an empty flag set, as AT_SYMLINK_NOFOLLOW is the default anyway
for linkat()
* adjust accompanying comment to mention `linkat` instead of `link`
proxmox-sys/src/fs/file.rs | 53 +++++++++++++++++---------------------
1 file changed, 23 insertions(+), 30 deletions(-)
diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
index ac513891..b2d6ddf6 100644
--- a/proxmox-sys/src/fs/file.rs
+++ b/proxmox-sys/src/fs/file.rs
@@ -10,7 +10,6 @@ use nix::errno::Errno;
use nix::fcntl::OFlag;
use nix::sys::stat;
use nix::unistd;
-use nix::NixPath;
use serde_json::Value;
use proxmox_lang::try_block;
@@ -266,30 +265,32 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
// rotate the file into place, but use `RENAME_NOREPLACE`, so in case 2 processes race against
// the initialization, the first one wins!
- let rename_result = temp_file_name.with_nix_path(|c_file_name| {
- path.with_nix_path(|new_path| unsafe {
- // This also works on file systems which don't support hardlinks (eg. vfat)
- match Errno::result(libc::renameat2(
- libc::AT_FDCWD,
- c_file_name.as_ptr(),
- libc::AT_FDCWD,
- new_path.as_ptr(),
- libc::RENAME_NOREPLACE,
- )) {
- Err(Errno::EINVAL) => (), // dumb file system, try `link`+`unlink`
- other => return other,
- };
- // but some file systems don't support `RENAME_NOREPLACE`
- // so we just use `link` + `unlink` instead
- let result = Errno::result(libc::link(c_file_name.as_ptr(), new_path.as_ptr()));
- let _ = libc::unlink(c_file_name.as_ptr());
+ let rename_result = match nix::fcntl::renameat2(
+ None, // AT_FDCWD
+ &temp_file_name,
+ None, // AT_FDCWD
+ path,
+ nix::fcntl::RenameFlags::RENAME_NOREPLACE,
+ ) {
+ Err(Errno::EINVAL) => {
+ // some file systems don't support `RENAME_NOREPLACE`
+ // so we just use `linkat` + `unlink` instead
+ let result = nix::unistd::linkat(
+ None,
+ &temp_file_name,
+ None,
+ &path.to_path_buf(),
+ nix::unistd::AtFlags::empty(),
+ );
+ let _ = nix::unistd::unlink(&temp_file_name);
result
- })
- });
+ }
+ other => other,
+ };
match rename_result {
- Ok(Ok(Ok(_))) => Ok(file),
- Ok(Ok(Err(err))) => {
+ Ok(_) => Ok(file),
+ Err(err) => {
// if another process has already raced ahead and created
// the file, let's just open theirs instead:
let _ = nix::unistd::unlink(&temp_file_name);
@@ -308,14 +309,6 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
);
}
}
- Ok(Err(err)) => {
- let _ = nix::unistd::unlink(&temp_file_name);
- bail!("with_nix_path {:?} failed - {}", path, err);
- }
- Err(err) => {
- let _ = nix::unistd::unlink(&temp_file_name);
- bail!("with_nix_path {:?} failed - {}", temp_file_name, err);
- }
}
}
--
2.45.1
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-12 7:54 [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate Christoph Heiss
@ 2024-07-12 8:16 ` Turkijan
2024-07-12 9:39 ` Christian Ebner
2024-07-18 10:29 ` Thomas Lamprecht
1 sibling, 1 reply; 8+ messages in thread
From: Turkijan @ 2024-07-12 8:16 UTC (permalink / raw)
To: Proxmox Backup Server development discussion
[-- Attachment #1.1: Type: text/plain, Size: 4359 bytes --]
how to update this for my test proxmox backup server?
On Fri, Jul 12, 2024 at 3:55 AM Christoph Heiss <c.heiss@proxmox.com> wrote:
> Saves us from converting the paths to raw C strings ourselves, thus
> simplifying it quite a bit.
>
> Since the `nix` crate does not provide link() directly, only linkat(),
> that is used instead - which behaves exactly the same as link() when
> using AT_FDCWD and an empty flag set.
>
> No functional changes.
>
> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
> ---
> Changes v1 -> v2:
> * use `AtFlags` instead of deprecated `LinkatFlags` (thanks Christian!)
> * use an empty flag set, as AT_SYMLINK_NOFOLLOW is the default anyway
> for linkat()
> * adjust accompanying comment to mention `linkat` instead of `link`
>
> proxmox-sys/src/fs/file.rs | 53 +++++++++++++++++---------------------
> 1 file changed, 23 insertions(+), 30 deletions(-)
>
> diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
> index ac513891..b2d6ddf6 100644
> --- a/proxmox-sys/src/fs/file.rs
> +++ b/proxmox-sys/src/fs/file.rs
> @@ -10,7 +10,6 @@ use nix::errno::Errno;
> use nix::fcntl::OFlag;
> use nix::sys::stat;
> use nix::unistd;
> -use nix::NixPath;
> use serde_json::Value;
>
> use proxmox_lang::try_block;
> @@ -266,30 +265,32 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
>
> // rotate the file into place, but use `RENAME_NOREPLACE`, so in case
> 2 processes race against
> // the initialization, the first one wins!
> - let rename_result = temp_file_name.with_nix_path(|c_file_name| {
> - path.with_nix_path(|new_path| unsafe {
> - // This also works on file systems which don't support
> hardlinks (eg. vfat)
> - match Errno::result(libc::renameat2(
> - libc::AT_FDCWD,
> - c_file_name.as_ptr(),
> - libc::AT_FDCWD,
> - new_path.as_ptr(),
> - libc::RENAME_NOREPLACE,
> - )) {
> - Err(Errno::EINVAL) => (), // dumb file system, try
> `link`+`unlink`
> - other => return other,
> - };
> - // but some file systems don't support `RENAME_NOREPLACE`
> - // so we just use `link` + `unlink` instead
> - let result = Errno::result(libc::link(c_file_name.as_ptr(),
> new_path.as_ptr()));
> - let _ = libc::unlink(c_file_name.as_ptr());
> + let rename_result = match nix::fcntl::renameat2(
> + None, // AT_FDCWD
> + &temp_file_name,
> + None, // AT_FDCWD
> + path,
> + nix::fcntl::RenameFlags::RENAME_NOREPLACE,
> + ) {
> + Err(Errno::EINVAL) => {
> + // some file systems don't support `RENAME_NOREPLACE`
> + // so we just use `linkat` + `unlink` instead
> + let result = nix::unistd::linkat(
> + None,
> + &temp_file_name,
> + None,
> + &path.to_path_buf(),
> + nix::unistd::AtFlags::empty(),
> + );
> + let _ = nix::unistd::unlink(&temp_file_name);
> result
> - })
> - });
> + }
> + other => other,
> + };
>
> match rename_result {
> - Ok(Ok(Ok(_))) => Ok(file),
> - Ok(Ok(Err(err))) => {
> + Ok(_) => Ok(file),
> + Err(err) => {
> // if another process has already raced ahead and created
> // the file, let's just open theirs instead:
> let _ = nix::unistd::unlink(&temp_file_name);
> @@ -308,14 +309,6 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
> );
> }
> }
> - Ok(Err(err)) => {
> - let _ = nix::unistd::unlink(&temp_file_name);
> - bail!("with_nix_path {:?} failed - {}", path, err);
> - }
> - Err(err) => {
> - let _ = nix::unistd::unlink(&temp_file_name);
> - bail!("with_nix_path {:?} failed - {}", temp_file_name, err);
> - }
> }
> }
>
> --
> 2.45.1
>
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>
>
[-- Attachment #1.2: Type: text/html, Size: 5876 bytes --]
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-12 7:54 [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate Christoph Heiss
2024-07-12 8:16 ` Turkijan
@ 2024-07-18 10:29 ` Thomas Lamprecht
2024-07-18 11:19 ` Christian Ebner
1 sibling, 1 reply; 8+ messages in thread
From: Thomas Lamprecht @ 2024-07-18 10:29 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Christoph Heiss
Am 12/07/2024 um 09:54 schrieb Christoph Heiss:
> Saves us from converting the paths to raw C strings ourselves, thus
> simplifying it quite a bit.
>
> Since the `nix` crate does not provide link() directly, only linkat(),
> that is used instead - which behaves exactly the same as link() when
> using AT_FDCWD and an empty flag set.
>
> No functional changes.
>
OK by me in general, but I get the following errors on build (well `cargo check`):
```
Checking proxmox-sys v0.6.0 (/root/sources/pbs/proxmox/proxmox-sys)
error[E0603]: struct `AtFlags` is private
--> proxmox-sys/src/fs/file.rs:283:30
|
283 | nix::unistd::AtFlags::empty(),
| ^^^^^^^ private struct
|
note: the struct `AtFlags` is defined here
--> /usr/share/cargo/registry/nix-0.26.1/src/unistd.rs:6:30
|
6 | use crate::fcntl::{at_rawfd, AtFlags};
| ^^^^^^^
help: consider importing this struct instead
|
283 | nix::fcntl::AtFlags::empty(),
| ~~~~~~~~~~~~~~~~~~~
help: import `AtFlags` directly
|
283 | nix::fcntl::AtFlags(),
| ~~~~~~~~~~~~~~~~~~~
error[E0308]: mismatched types
--> proxmox-sys/src/fs/file.rs:283:17
|
278 | let result = nix::unistd::linkat(
| ------------------- arguments to this function are incorrect
...
283 | nix::unistd::AtFlags::empty(),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `LinkatFlags`, found `AtFlags`
|
note: function defined here
--> /usr/share/cargo/registry/nix-0.26.1/src/unistd.rs:1252:8
|
1252 | pub fn linkat<P: ?Sized + NixPath>(
| ^^^^^^
Some errors have detailed explanations: E0308, E0603.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `proxmox-sys` (lib) due to 2 previous errors
```
Seems like you tested this with upstream nix, as the new type only works
since nix version 0.28, while we still got 0.26.1.
We might be able to upgrade nix, but that is often quite a bit more
involved.
Alternatively we could go with your v1 approach and apply
```
diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
index b2d6ddf6..28c7f88b 100644
--- a/proxmox-sys/src/fs/file.rs
+++ b/proxmox-sys/src/fs/file.rs
@@ -280,7 +280,7 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
&temp_file_name,
None,
&path.to_path_buf(),
- nix::unistd::AtFlags::empty(),
+ nix::unistd::LinkatFlags::NoSymlinkFollow,
);
let _ = nix::unistd::unlink(&temp_file_name);
result
```
On top for now.
Oh, please always test such changes with the crate versions provided by our
Debian package provided crate registry.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-18 10:29 ` Thomas Lamprecht
@ 2024-07-18 11:19 ` Christian Ebner
2024-07-18 12:03 ` Thomas Lamprecht
0 siblings, 1 reply; 8+ messages in thread
From: Christian Ebner @ 2024-07-18 11:19 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Thomas Lamprecht,
Christoph Heiss
> On 18.07.2024 12:29 CEST Thomas Lamprecht <t.lamprecht@proxmox.com> wrote:
>
>
> Am 12/07/2024 um 09:54 schrieb Christoph Heiss:
> > Saves us from converting the paths to raw C strings ourselves, thus
> > simplifying it quite a bit.
> >
> > Since the `nix` crate does not provide link() directly, only linkat(),
> > that is used instead - which behaves exactly the same as link() when
> > using AT_FDCWD and an empty flag set.
> >
> > No functional changes.
> >
>
> OK by me in general, but I get the following errors on build (well `cargo check`):
>
>
> ```
> Checking proxmox-sys v0.6.0 (/root/sources/pbs/proxmox/proxmox-sys)
> error[E0603]: struct `AtFlags` is private
> --> proxmox-sys/src/fs/file.rs:283:30
> |
> 283 | nix::unistd::AtFlags::empty(),
> | ^^^^^^^ private struct
> |
> note: the struct `AtFlags` is defined here
> --> /usr/share/cargo/registry/nix-0.26.1/src/unistd.rs:6:30
> |
> 6 | use crate::fcntl::{at_rawfd, AtFlags};
> | ^^^^^^^
> help: consider importing this struct instead
> |
> 283 | nix::fcntl::AtFlags::empty(),
> | ~~~~~~~~~~~~~~~~~~~
> help: import `AtFlags` directly
> |
> 283 | nix::fcntl::AtFlags(),
> | ~~~~~~~~~~~~~~~~~~~
>
> error[E0308]: mismatched types
> --> proxmox-sys/src/fs/file.rs:283:17
> |
> 278 | let result = nix::unistd::linkat(
> | ------------------- arguments to this function are incorrect
> ...
> 283 | nix::unistd::AtFlags::empty(),
> | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `LinkatFlags`, found `AtFlags`
> |
> note: function defined here
> --> /usr/share/cargo/registry/nix-0.26.1/src/unistd.rs:1252:8
> |
> 1252 | pub fn linkat<P: ?Sized + NixPath>(
> | ^^^^^^
>
> Some errors have detailed explanations: E0308, E0603.
> For more information about an error, try `rustc --explain E0308`.
> error: could not compile `proxmox-sys` (lib) due to 2 previous errors
> ```
>
> Seems like you tested this with upstream nix, as the new type only works
> since nix version 0.28, while we still got 0.26.1.
>
> We might be able to upgrade nix, but that is often quite a bit more
> involved.
>
> Alternatively we could go with your v1 approach and apply
>
> ```
> diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
> index b2d6ddf6..28c7f88b 100644
> --- a/proxmox-sys/src/fs/file.rs
> +++ b/proxmox-sys/src/fs/file.rs
> @@ -280,7 +280,7 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
> &temp_file_name,
> None,
> &path.to_path_buf(),
> - nix::unistd::AtFlags::empty(),
> + nix::unistd::LinkatFlags::NoSymlinkFollow,
> );
> let _ = nix::unistd::unlink(&temp_file_name);
> result
> ```
>
> On top for now.
>
> Oh, please always test such changes with the crate versions provided by our
> Debian package provided crate registry.
>
Partially my fault, as I overlooked in my suggestion that linkat() does not accept `AtFlags` in version 0.26.1 just yet [0].
The suggested replacement type is however already there, but the module namespace is incorrect in the patch. Should have been [1].
[0] https://docs.rs/nix/0.26.1/nix/unistd/fn.linkat.html
[1] https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-18 11:19 ` Christian Ebner
@ 2024-07-18 12:03 ` Thomas Lamprecht
2024-07-18 12:13 ` Christian Ebner
0 siblings, 1 reply; 8+ messages in thread
From: Thomas Lamprecht @ 2024-07-18 12:03 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Christian Ebner,
Christoph Heiss
Am 18/07/2024 um 13:19 schrieb Christian Ebner:
> Partially my fault, as I overlooked in my suggestion that linkat() does not accept `AtFlags` in version 0.26.1 just yet [0].
> The suggested replacement type is however already there, but the module namespace is incorrect in the patch. Should have been [1].
>
> [0] https://docs.rs/nix/0.26.1/nix/unistd/fn.linkat.html
> [1] https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html
>
The type is there, but isn't accepted by `nix::unistd::linkat`, nor is there
a variant of that method that would accept it in this version FWICS.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-18 12:03 ` Thomas Lamprecht
@ 2024-07-18 12:13 ` Christian Ebner
2024-07-18 12:19 ` Thomas Lamprecht
0 siblings, 1 reply; 8+ messages in thread
From: Christian Ebner @ 2024-07-18 12:13 UTC (permalink / raw)
To: Thomas Lamprecht, Proxmox Backup Server development discussion,
Christoph Heiss
On 7/18/24 14:03, Thomas Lamprecht wrote:
> Am 18/07/2024 um 13:19 schrieb Christian Ebner:
>> Partially my fault, as I overlooked in my suggestion that linkat() does not accept `AtFlags` in version 0.26.1 just yet [0].
>> The suggested replacement type is however already there, but the module namespace is incorrect in the patch. Should have been [1].
>>
>> [0] https://docs.rs/nix/0.26.1/nix/unistd/fn.linkat.html
>> [1] https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html
>>
>
> The type is there, but isn't accepted by `nix::unistd::linkat`, nor is there
> a variant of that method that would accept it in this version FWICS.
No, there is not, this will not work with nix v0.26.1 as my suggestion
was wrong. Just wanted to mention that I did overlook this when
suggesting this to Christoph. So I guess it is best to apply v1 and only
later on replace the type, when using nix version >= v0.28
Sorry if I caused noise, should have tested this myself before even
replying to the v1 :(
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate
2024-07-18 12:13 ` Christian Ebner
@ 2024-07-18 12:19 ` Thomas Lamprecht
0 siblings, 0 replies; 8+ messages in thread
From: Thomas Lamprecht @ 2024-07-18 12:19 UTC (permalink / raw)
To: Christian Ebner, Proxmox Backup Server development discussion,
Christoph Heiss
Am 18/07/2024 um 14:13 schrieb Christian Ebner:
> On 7/18/24 14:03, Thomas Lamprecht wrote:
>> Am 18/07/2024 um 13:19 schrieb Christian Ebner:
>>> Partially my fault, as I overlooked in my suggestion that linkat() does not accept `AtFlags` in version 0.26.1 just yet [0].
>>> The suggested replacement type is however already there, but the module namespace is incorrect in the patch. Should have been [1].
>>>
>>> [0] https://docs.rs/nix/0.26.1/nix/unistd/fn.linkat.html
>>> [1] https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html
>>>
>>
>> The type is there, but isn't accepted by `nix::unistd::linkat`, nor is there
>> a variant of that method that would accept it in this version FWICS.
>
> No, there is not, this will not work with nix v0.26.1 as my suggestion
> was wrong. Just wanted to mention that I did overlook this when
> suggesting this to Christoph. So I guess it is best to apply v1 and only
> later on replace the type, when using nix version >= v0.28
>
> Sorry if I caused noise, should have tested this myself before even
> replying to the v1 :(
Nah, suggesting things is definitely okay, even encouraged, no worries
here.
Ultimately, both the patch author (first in line) and the maintainer
applying the patch (last in line) are responsible for verifying its
merit and feasibility in the actual environment to which it would be
applied to and compiled/executed in.
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-07-18 12:18 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-07-12 7:54 [pbs-devel] [PATCH proxmox v2] sys: file: use renameat2() from `nix` crate Christoph Heiss
2024-07-12 8:16 ` Turkijan
2024-07-12 9:39 ` Christian Ebner
2024-07-18 10:29 ` Thomas Lamprecht
2024-07-18 11:19 ` Christian Ebner
2024-07-18 12:03 ` Thomas Lamprecht
2024-07-18 12:13 ` Christian Ebner
2024-07-18 12:19 ` Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox