public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate
@ 2024-07-11 17:07 Christoph Heiss
  2024-07-11 17:11 ` Turkijan
  2024-07-12  7:37 ` Christian Ebner
  0 siblings, 2 replies; 5+ messages in thread
From: Christoph Heiss @ 2024-07-11 17:07 UTC (permalink / raw)
  To: pbs-devel

Saves us from converting the paths to raw C strings ourselves, thus
simplifying it quite a bit.

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
---
 proxmox-sys/src/fs/file.rs | 51 ++++++++++++++++----------------------
 1 file changed, 22 insertions(+), 29 deletions(-)

diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
index ac513891..3b20c4ea 100644
--- a/proxmox-sys/src/fs/file.rs
+++ b/proxmox-sys/src/fs/file.rs
@@ -10,7 +10,6 @@ use nix::errno::Errno;
 use nix::fcntl::OFlag;
 use nix::sys::stat;
 use nix::unistd;
-use nix::NixPath;
 use serde_json::Value;
 
 use proxmox_lang::try_block;
@@ -266,30 +265,32 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
 
     // rotate the file into place, but use `RENAME_NOREPLACE`, so in case 2 processes race against
     // the initialization, the first one wins!
-    let rename_result = temp_file_name.with_nix_path(|c_file_name| {
-        path.with_nix_path(|new_path| unsafe {
-            // This also works on file systems which don't support hardlinks (eg. vfat)
-            match Errno::result(libc::renameat2(
-                libc::AT_FDCWD,
-                c_file_name.as_ptr(),
-                libc::AT_FDCWD,
-                new_path.as_ptr(),
-                libc::RENAME_NOREPLACE,
-            )) {
-                Err(Errno::EINVAL) => (), // dumb file system, try `link`+`unlink`
-                other => return other,
-            };
-            // but some file systems don't support `RENAME_NOREPLACE`
+    let rename_result = match nix::fcntl::renameat2(
+        None, // AT_FDCWD
+        &temp_file_name,
+        None, // AT_FDCWD
+        path,
+        nix::fcntl::RenameFlags::RENAME_NOREPLACE,
+    ) {
+        Err(Errno::EINVAL) => {
+            // some file systems don't support `RENAME_NOREPLACE`
             // so we just use `link` + `unlink` instead
-            let result = Errno::result(libc::link(c_file_name.as_ptr(), new_path.as_ptr()));
-            let _ = libc::unlink(c_file_name.as_ptr());
+            let result = nix::unistd::linkat(
+                None,
+                &temp_file_name,
+                None,
+                &path.to_path_buf(),
+                nix::unistd::LinkatFlags::NoSymlinkFollow,
+            );
+            let _ = nix::unistd::unlink(&temp_file_name);
             result
-        })
-    });
+        }
+        other => other,
+    };
 
     match rename_result {
-        Ok(Ok(Ok(_))) => Ok(file),
-        Ok(Ok(Err(err))) => {
+        Ok(_) => Ok(file),
+        Err(err) => {
             // if another process has already raced ahead and created
             // the file, let's just open theirs instead:
             let _ = nix::unistd::unlink(&temp_file_name);
@@ -308,14 +309,6 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
                 );
             }
         }
-        Ok(Err(err)) => {
-            let _ = nix::unistd::unlink(&temp_file_name);
-            bail!("with_nix_path {:?} failed - {}", path, err);
-        }
-        Err(err) => {
-            let _ = nix::unistd::unlink(&temp_file_name);
-            bail!("with_nix_path {:?} failed - {}", temp_file_name, err);
-        }
     }
 }
 
-- 
2.45.1



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate
  2024-07-11 17:07 [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate Christoph Heiss
@ 2024-07-11 17:11 ` Turkijan
  2024-07-11 19:39   ` Dietmar Maurer
  2024-07-12  7:37 ` Christian Ebner
  1 sibling, 1 reply; 5+ messages in thread
From: Turkijan @ 2024-07-11 17:11 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion


[-- Attachment #1.1: Type: text/plain, Size: 3828 bytes --]

Have you changed dev package?


On Thu, Jul 11, 2024 at 1:08 PM Christoph Heiss <c.heiss@proxmox.com> wrote:

> Saves us from converting the paths to raw C strings ourselves, thus
> simplifying it quite a bit.
>
> No functional changes.
>
> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
> ---
>  proxmox-sys/src/fs/file.rs | 51 ++++++++++++++++----------------------
>  1 file changed, 22 insertions(+), 29 deletions(-)
>
> diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
> index ac513891..3b20c4ea 100644
> --- a/proxmox-sys/src/fs/file.rs
> +++ b/proxmox-sys/src/fs/file.rs
> @@ -10,7 +10,6 @@ use nix::errno::Errno;
>  use nix::fcntl::OFlag;
>  use nix::sys::stat;
>  use nix::unistd;
> -use nix::NixPath;
>  use serde_json::Value;
>
>  use proxmox_lang::try_block;
> @@ -266,30 +265,32 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
>
>      // rotate the file into place, but use `RENAME_NOREPLACE`, so in case
> 2 processes race against
>      // the initialization, the first one wins!
> -    let rename_result = temp_file_name.with_nix_path(|c_file_name| {
> -        path.with_nix_path(|new_path| unsafe {
> -            // This also works on file systems which don't support
> hardlinks (eg. vfat)
> -            match Errno::result(libc::renameat2(
> -                libc::AT_FDCWD,
> -                c_file_name.as_ptr(),
> -                libc::AT_FDCWD,
> -                new_path.as_ptr(),
> -                libc::RENAME_NOREPLACE,
> -            )) {
> -                Err(Errno::EINVAL) => (), // dumb file system, try
> `link`+`unlink`
> -                other => return other,
> -            };
> -            // but some file systems don't support `RENAME_NOREPLACE`
> +    let rename_result = match nix::fcntl::renameat2(
> +        None, // AT_FDCWD
> +        &temp_file_name,
> +        None, // AT_FDCWD
> +        path,
> +        nix::fcntl::RenameFlags::RENAME_NOREPLACE,
> +    ) {
> +        Err(Errno::EINVAL) => {
> +            // some file systems don't support `RENAME_NOREPLACE`
>              // so we just use `link` + `unlink` instead
> -            let result = Errno::result(libc::link(c_file_name.as_ptr(),
> new_path.as_ptr()));
> -            let _ = libc::unlink(c_file_name.as_ptr());
> +            let result = nix::unistd::linkat(
> +                None,
> +                &temp_file_name,
> +                None,
> +                &path.to_path_buf(),
> +                nix::unistd::LinkatFlags::NoSymlinkFollow,
> +            );
> +            let _ = nix::unistd::unlink(&temp_file_name);
>              result
> -        })
> -    });
> +        }
> +        other => other,
> +    };
>
>      match rename_result {
> -        Ok(Ok(Ok(_))) => Ok(file),
> -        Ok(Ok(Err(err))) => {
> +        Ok(_) => Ok(file),
> +        Err(err) => {
>              // if another process has already raced ahead and created
>              // the file, let's just open theirs instead:
>              let _ = nix::unistd::unlink(&temp_file_name);
> @@ -308,14 +309,6 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
>                  );
>              }
>          }
> -        Ok(Err(err)) => {
> -            let _ = nix::unistd::unlink(&temp_file_name);
> -            bail!("with_nix_path {:?} failed - {}", path, err);
> -        }
> -        Err(err) => {
> -            let _ = nix::unistd::unlink(&temp_file_name);
> -            bail!("with_nix_path {:?} failed - {}", temp_file_name, err);
> -        }
>      }
>  }
>
> --
> 2.45.1
>
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 5321 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate
  2024-07-11 17:11 ` Turkijan
@ 2024-07-11 19:39   ` Dietmar Maurer
  0 siblings, 0 replies; 5+ messages in thread
From: Dietmar Maurer @ 2024-07-11 19:39 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Turkijan

> Have you changed dev package?

FYI: We have the following dev workflow

1. Patches are sent to the list
2. The may get applied into the git repository
3. After testing, we build debian packages
4. We upload the debian packages to the public repositories

Does this answer your question?


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate
  2024-07-11 17:07 [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate Christoph Heiss
  2024-07-11 17:11 ` Turkijan
@ 2024-07-12  7:37 ` Christian Ebner
  2024-07-12  7:44   ` Christoph Heiss
  1 sibling, 1 reply; 5+ messages in thread
From: Christian Ebner @ 2024-07-12  7:37 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Christoph Heiss

one small nit inline:
> On 11.07.2024 19:07 CEST Christoph Heiss <c.heiss@proxmox.com> wrote:
> 
>  
> Saves us from converting the paths to raw C strings ourselves, thus
> simplifying it quite a bit.
> 
> No functional changes.
> 
> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
> ---
>  proxmox-sys/src/fs/file.rs | 51 ++++++++++++++++----------------------
>  1 file changed, 22 insertions(+), 29 deletions(-)
> 
> diff --git a/proxmox-sys/src/fs/file.rs b/proxmox-sys/src/fs/file.rs
> index ac513891..3b20c4ea 100644
> --- a/proxmox-sys/src/fs/file.rs
> +++ b/proxmox-sys/src/fs/file.rs
> @@ -10,7 +10,6 @@ use nix::errno::Errno;
>  use nix::fcntl::OFlag;
>  use nix::sys::stat;
>  use nix::unistd;
> -use nix::NixPath;
>  use serde_json::Value;
>  
>  use proxmox_lang::try_block;
> @@ -266,30 +265,32 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
>  
>      // rotate the file into place, but use `RENAME_NOREPLACE`, so in case 2 processes race against
>      // the initialization, the first one wins!
> -    let rename_result = temp_file_name.with_nix_path(|c_file_name| {
> -        path.with_nix_path(|new_path| unsafe {
> -            // This also works on file systems which don't support hardlinks (eg. vfat)
> -            match Errno::result(libc::renameat2(
> -                libc::AT_FDCWD,
> -                c_file_name.as_ptr(),
> -                libc::AT_FDCWD,
> -                new_path.as_ptr(),
> -                libc::RENAME_NOREPLACE,
> -            )) {
> -                Err(Errno::EINVAL) => (), // dumb file system, try `link`+`unlink`
> -                other => return other,
> -            };
> -            // but some file systems don't support `RENAME_NOREPLACE`
> +    let rename_result = match nix::fcntl::renameat2(
> +        None, // AT_FDCWD
> +        &temp_file_name,
> +        None, // AT_FDCWD
> +        path,
> +        nix::fcntl::RenameFlags::RENAME_NOREPLACE,
> +    ) {
> +        Err(Errno::EINVAL) => {
> +            // some file systems don't support `RENAME_NOREPLACE`
>              // so we just use `link` + `unlink` instead
> -            let result = Errno::result(libc::link(c_file_name.as_ptr(), new_path.as_ptr()));
> -            let _ = libc::unlink(c_file_name.as_ptr());
> +            let result = nix::unistd::linkat(
> +                None,
> +                &temp_file_name,
> +                None,
> +                &path.to_path_buf(),
> +                nix::unistd::LinkatFlags::NoSymlinkFollow,

According to https://docs.rs/nix/0.29.0/nix/unistd/type.LinkatFlags.html
`LinkatFlags` will be deprecated with version 0.28. We currently use nix 0.16.1, which already provides the suggested replacment using the `AtFlags` https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html.

So it would make sense to already use these instead.

> +            );
> +            let _ = nix::unistd::unlink(&temp_file_name);
>              result
> -        })
> -    });
> +        }
> +        other => other,
> +    };
>  
>      match rename_result {
> -        Ok(Ok(Ok(_))) => Ok(file),
> -        Ok(Ok(Err(err))) => {
> +        Ok(_) => Ok(file),
> +        Err(err) => {
>              // if another process has already raced ahead and created
>              // the file, let's just open theirs instead:
>              let _ = nix::unistd::unlink(&temp_file_name);
> @@ -308,14 +309,6 @@ pub fn atomic_open_or_create_file<P: AsRef<Path>>(
>                  );
>              }
>          }
> -        Ok(Err(err)) => {
> -            let _ = nix::unistd::unlink(&temp_file_name);
> -            bail!("with_nix_path {:?} failed - {}", path, err);
> -        }
> -        Err(err) => {
> -            let _ = nix::unistd::unlink(&temp_file_name);
> -            bail!("with_nix_path {:?} failed - {}", temp_file_name, err);
> -        }
>      }
>  }
>  
> -- 
> 2.45.1
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate
  2024-07-12  7:37 ` Christian Ebner
@ 2024-07-12  7:44   ` Christoph Heiss
  0 siblings, 0 replies; 5+ messages in thread
From: Christoph Heiss @ 2024-07-12  7:44 UTC (permalink / raw)
  To: Christian Ebner; +Cc: Proxmox Backup Server development discussion

Thanks for the review!

On Fri, Jul 12, 2024 at 09:37:03AM GMT, Christian Ebner wrote:
> one small nit inline:
> > On 11.07.2024 19:07 CEST Christoph Heiss <c.heiss@proxmox.com> wrote:
> > [..]
> > +            // some file systems don't support `RENAME_NOREPLACE`
> >              // so we just use `link` + `unlink` instead
> > -            let result = Errno::result(libc::link(c_file_name.as_ptr(), new_path.as_ptr()));
> > -            let _ = libc::unlink(c_file_name.as_ptr());
> > +            let result = nix::unistd::linkat(
> > +                None,
> > +                &temp_file_name,
> > +                None,
> > +                &path.to_path_buf(),
> > +                nix::unistd::LinkatFlags::NoSymlinkFollow,
>
> According to
> https://docs.rs/nix/0.29.0/nix/unistd/type.LinkatFlags.html
> `LinkatFlags` will be deprecated with version 0.28. We currently use
> nix 0.16.1, which already provides the suggested replacment using the
> `AtFlags` https://docs.rs/nix/0.26.1/nix/fcntl/struct.AtFlags.html.
>
> So it would make sense to already use these instead.

Nice catch! I'll send a v2 :^)



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2024-07-12  7:44 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-07-11 17:07 [pbs-devel] [PATCH proxmox] sys: file: use renameat2() from `nix` crate Christoph Heiss
2024-07-11 17:11 ` Turkijan
2024-07-11 19:39   ` Dietmar Maurer
2024-07-12  7:37 ` Christian Ebner
2024-07-12  7:44   ` Christoph Heiss

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal