* [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore
@ 2024-01-24 10:15 Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 1/3] compression: Refactor ZipEntry creation and add FileType enum Filip Schauer
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Include symlinks when restoring files from a backup as a zip file.
Resulting ZIP files were successfully tested on Linux with: zipinfo,
unzip, unar
On Windows the extracted symlinks show up as regular files containing
the path to the destination. As far as I am aware Windows Explorer does
not support symlinks in ZIP files.
Changes since v3:
* Move content to Regular FileType enum
* Make ZipEntry private
* Add take_reader function to FileType enum
Changes since v2:
* Add a FileType enum and embed the symlink target into the enum.
* Add unit tests for ZipEncoder to the proxmox-compression crate
Changes since v1:
* Use P instead of &Path
* Fix compile error due to misplaced comma
* Check content before symlink_target, since regular files are more
common than symlinks
proxmox:
Filip Schauer (3):
compression: Refactor ZipEntry creation and add FileType enum
compression: Add support for symlinks in zip files
compression: Add unit tests for the ZipEncoder
proxmox-compression/Cargo.toml | 2 +-
proxmox-compression/src/zip.rs | 159 +++++++++++++++++++------------
proxmox-compression/tests/zip.rs | 123 ++++++++++++++++++++++++
3 files changed, 223 insertions(+), 61 deletions(-)
create mode 100644 proxmox-compression/tests/zip.rs
proxmox-backup:
Filip Schauer (2):
pxar: Adopt FileType enum when adding a zip entry
fix #4995: pxar: Include symlinks in zip file creation
pbs-client/src/pxar/extract.rs | 59 +++++++++++++++++++++-------------
1 file changed, 36 insertions(+), 23 deletions(-)
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pbs-devel] [PATCH v4 proxmox 1/3] compression: Refactor ZipEntry creation and add FileType enum
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
@ 2024-01-24 10:15 ` Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 2/3] compression: Add support for symlinks in zip files Filip Schauer
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Refactor the process of adding an entry to the ZipEncoder. Instead of
passing a ZipEntry and the content seperately, pass all parameters
required to create a ZipEntry directly to add_entry, including a new
FileType enum that contains the file content if it is a regular file.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
proxmox-compression/src/zip.rs | 145 +++++++++++++++++++--------------
1 file changed, 82 insertions(+), 63 deletions(-)
diff --git a/proxmox-compression/src/zip.rs b/proxmox-compression/src/zip.rs
index d2d3fd8..6229990 100644
--- a/proxmox-compression/src/zip.rs
+++ b/proxmox-compression/src/zip.rs
@@ -17,6 +17,7 @@ use std::time::SystemTime;
use anyhow::{format_err, Error, Result};
use endian_trait::Endian;
use futures::ready;
+use libc::{S_IFDIR, S_IFREG};
use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadBuf};
use crc32fast::Hasher;
@@ -71,6 +72,20 @@ fn epoch_to_dos(epoch: i64) -> (u16, u16) {
(date, time)
}
+pub enum FileType<R> {
+ Directory,
+ Regular(R),
+}
+
+impl<R: AsyncRead + Unpin + Send> FileType<R> {
+ fn take_reader(&mut self) -> Option<HashWrapper<&mut (dyn AsyncRead + Unpin + Send)>> {
+ match self {
+ FileType::Directory => None,
+ FileType::Regular(content) => Some(HashWrapper::new(content)),
+ }
+ }
+}
+
#[derive(Endian)]
#[repr(C, packed)]
struct Zip64Field {
@@ -192,9 +207,7 @@ where
}
/// Represents an Entry in a ZIP File
-///
-/// used to add to a ZipEncoder
-pub struct ZipEntry {
+struct ZipEntry {
filename: OsString,
mtime: i64,
mode: u16,
@@ -202,16 +215,20 @@ pub struct ZipEntry {
uncompressed_size: u64,
compressed_size: u64,
offset: u64,
- is_file: bool,
is_utf8_filename: bool,
}
impl ZipEntry {
/// Creates a new ZipEntry
///
- /// if is_file is false the path will contain an trailing separator,
+ /// if file_type is Directory the path will contain a trailing separator,
/// so that the zip file understands that it is a directory
- pub fn new<P: AsRef<Path>>(path: P, mtime: i64, mode: u16, is_file: bool) -> Self {
+ fn new<P: AsRef<Path>, R: AsyncRead + Unpin>(
+ path: P,
+ mtime: i64,
+ mode: u16,
+ file_type: &FileType<R>,
+ ) -> Self {
let mut relpath = PathBuf::new();
for comp in path.as_ref().components() {
@@ -220,22 +237,26 @@ impl ZipEntry {
}
}
- if !is_file {
+ if matches!(file_type, FileType::Directory) {
relpath.push(""); // adds trailing slash
}
let filename: OsString = relpath.into();
let is_utf8_filename = filename.to_str().is_some();
+ let file_type_attr = match file_type {
+ FileType::Regular(_) => S_IFREG,
+ FileType::Directory => S_IFDIR,
+ };
+
Self {
filename,
crc32: 0,
mtime,
- mode,
+ mode: mode | file_type_attr as u16,
uncompressed_size: 0,
compressed_size: 0,
offset: 0,
- is_file,
is_utf8_filename,
}
}
@@ -342,6 +363,8 @@ impl ZipEntry {
)
};
+ let is_directory = (self.mode & 0xF000) == S_IFDIR as u16;
+
write_struct(
&mut buf,
CentralDirectoryFileHeader {
@@ -360,7 +383,7 @@ impl ZipEntry {
comment_len: 0,
start_disk: 0,
internal_flags: 0,
- external_flags: (self.mode as u32) << 16 | (!self.is_file as u32) << 4,
+ external_flags: (self.mode as u32) << 16 | (is_directory as u32) << 4,
offset,
},
)
@@ -437,7 +460,7 @@ where
/// use anyhow::{Error, Result};
/// use tokio::fs::File;
///
-/// use proxmox_compression::zip::{ZipEncoder, ZipEntry};
+/// use proxmox_compression::zip::{FileType, ZipEncoder};
///
/// //#[tokio::main]
/// async fn main_() -> Result<(), Error> {
@@ -445,12 +468,12 @@ where
/// let mut source = File::open("foo.txt").await?;
///
/// let mut zip = ZipEncoder::new(target);
-/// zip.add_entry(ZipEntry::new(
+/// zip.add_entry(
/// "foo.txt",
/// 0,
/// 0o100755,
-/// true,
-/// ), Some(source)).await?;
+/// FileType::Regular(source),
+/// ).await?;
///
/// zip.finish().await?;
///
@@ -475,34 +498,38 @@ impl<W: AsyncWrite + Unpin> ZipEncoder<W> {
}
}
- pub async fn add_entry<R: AsyncRead + Unpin>(
+ pub async fn add_entry<P: AsRef<Path>, R: AsyncRead + Unpin + Send>(
&mut self,
- mut entry: ZipEntry,
- content: Option<R>,
+ path: P,
+ mtime: i64,
+ mode: u16,
+ mut file_type: FileType<R>,
) -> Result<(), Error> {
let mut target = self
.target
.take()
.ok_or_else(|| format_err!("had no target during add entry"))?;
+ let mut entry = ZipEntry::new(path, mtime, mode, &file_type);
entry.offset = self.byte_count.try_into()?;
self.byte_count += entry.write_local_header(&mut target).await?;
- if let Some(content) = content {
- let mut reader = HashWrapper::new(content);
+
+ if !matches!(file_type, FileType::Directory) {
let mut enc = DeflateEncoder::with_quality(target, Level::Fastest);
- enc.compress(&mut reader).await?;
+ if let Some(mut reader) = file_type.take_reader() {
+ enc.compress(&mut reader).await?;
+ entry.crc32 = reader.finish().0;
+ }
+
let total_in = enc.total_in();
let total_out = enc.total_out();
target = enc.into_inner();
- let (crc32, _reader) = reader.finish();
-
self.byte_count += total_out as usize;
entry.compressed_size = total_out;
entry.uncompressed_size = total_in;
-
- entry.crc32 = crc32;
}
+
self.byte_count += entry.write_data_descriptor(&mut target).await?;
self.target = Some(target);
@@ -640,46 +667,38 @@ where
}
};
- let entry_path = entry.path().to_owned();
let encoder = &mut encoder;
-
- let entry = async move {
- let entry_path_no_base = entry.path().strip_prefix(base_path)?;
- let metadata = entry.metadata()?;
- let mtime = match metadata
- .modified()
- .unwrap_or_else(|_| SystemTime::now())
- .duration_since(SystemTime::UNIX_EPOCH)
- {
- Ok(dur) => dur.as_secs() as i64,
- Err(time_error) => -(time_error.duration().as_secs() as i64),
- };
- let mode = metadata.mode() as u16;
-
- if entry.file_type().is_file() {
- let file = tokio::fs::File::open(entry.path()).await?;
- let ze = ZipEntry::new(entry_path_no_base, mtime, mode, true);
- Ok(Some((ze, Some(file))))
- } else if entry.file_type().is_dir() {
- let ze = ZipEntry::new(entry_path_no_base, mtime, mode, false);
- let content: Option<tokio::fs::File> = None;
- Ok(Some((ze, content)))
- } else {
- // ignore other file types
- Ok::<_, Error>(None)
- }
- }
- .await;
- match entry {
- Ok(Some((ze, content))) => encoder.add_entry(ze, content).await?,
- Ok(None) => {}
- Err(err) => {
- eprintln!(
- "zip: error encoding file or directory '{}': {}",
- entry_path.display(),
- err
- );
- }
+ let entry_path_no_base = entry.path().strip_prefix(base_path)?;
+ let metadata = entry.metadata()?;
+ let mtime = match metadata
+ .modified()
+ .unwrap_or_else(|_| SystemTime::now())
+ .duration_since(SystemTime::UNIX_EPOCH)
+ {
+ Ok(dur) => dur.as_secs() as i64,
+ Err(time_error) => -(time_error.duration().as_secs() as i64),
+ };
+ let mode = metadata.mode() as u16;
+
+ if entry.file_type().is_file() {
+ let file = tokio::fs::File::open(entry.path()).await?;
+ encoder
+ .add_entry(
+ entry_path_no_base,
+ mtime,
+ mode,
+ FileType::Regular(Box::new(file)),
+ )
+ .await?;
+ } else if entry.file_type().is_dir() {
+ encoder
+ .add_entry(
+ entry_path_no_base,
+ mtime,
+ mode,
+ FileType::<&[u8]>::Directory,
+ )
+ .await?;
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pbs-devel] [PATCH v4 proxmox 2/3] compression: Add support for symlinks in zip files
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 1/3] compression: Refactor ZipEntry creation and add FileType enum Filip Schauer
@ 2024-01-24 10:15 ` Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 3/3] compression: Add unit tests for the ZipEncoder Filip Schauer
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
proxmox-compression/src/zip.rs | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/proxmox-compression/src/zip.rs b/proxmox-compression/src/zip.rs
index 6229990..aec11a4 100644
--- a/proxmox-compression/src/zip.rs
+++ b/proxmox-compression/src/zip.rs
@@ -17,7 +17,7 @@ use std::time::SystemTime;
use anyhow::{format_err, Error, Result};
use endian_trait::Endian;
use futures::ready;
-use libc::{S_IFDIR, S_IFREG};
+use libc::{S_IFDIR, S_IFLNK, S_IFREG};
use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadBuf};
use crc32fast::Hasher;
@@ -75,6 +75,7 @@ fn epoch_to_dos(epoch: i64) -> (u16, u16) {
pub enum FileType<R> {
Directory,
Regular(R),
+ Symlink(OsString),
}
impl<R: AsyncRead + Unpin + Send> FileType<R> {
@@ -82,6 +83,12 @@ impl<R: AsyncRead + Unpin + Send> FileType<R> {
match self {
FileType::Directory => None,
FileType::Regular(content) => Some(HashWrapper::new(content)),
+ FileType::Symlink(symlink_target) => {
+ let cursor = std::io::Cursor::new(symlink_target.as_bytes());
+ let reader_box = Box::new(cursor) as Box<dyn AsyncRead + Unpin + Send>;
+ let leaked_ref = Box::leak(reader_box);
+ Some(HashWrapper::new(leaked_ref))
+ }
}
}
}
@@ -247,6 +254,7 @@ impl ZipEntry {
let file_type_attr = match file_type {
FileType::Regular(_) => S_IFREG,
FileType::Directory => S_IFDIR,
+ FileType::Symlink(_) => S_IFLNK,
};
Self {
@@ -699,6 +707,16 @@ where
FileType::<&[u8]>::Directory,
)
.await?;
+ } else if entry.file_type().is_symlink() {
+ let target = std::fs::read_link(entry.path())?;
+ encoder
+ .add_entry(
+ entry_path_no_base,
+ mtime,
+ mode,
+ FileType::<io::Cursor<&[u8]>>::Symlink(target.into()),
+ )
+ .await?;
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pbs-devel] [PATCH v4 proxmox 3/3] compression: Add unit tests for the ZipEncoder
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 1/3] compression: Refactor ZipEntry creation and add FileType enum Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 2/3] compression: Add support for symlinks in zip files Filip Schauer
@ 2024-01-24 10:15 ` Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 1/2] pxar: Adopt FileType enum when adding a zip entry Filip Schauer
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
proxmox-compression/Cargo.toml | 2 +-
proxmox-compression/tests/zip.rs | 123 +++++++++++++++++++++++++++++++
2 files changed, 124 insertions(+), 1 deletion(-)
create mode 100644 proxmox-compression/tests/zip.rs
diff --git a/proxmox-compression/Cargo.toml b/proxmox-compression/Cargo.toml
index 49735cb..8c9c47e 100644
--- a/proxmox-compression/Cargo.toml
+++ b/proxmox-compression/Cargo.toml
@@ -27,5 +27,5 @@ proxmox-io = { workspace = true, features = [ "tokio" ] }
proxmox-lang.workspace = true
[dev-dependencies]
-tokio = { workspace = true, features = [ "macros" ] }
+tokio = { workspace = true, features = [ "macros", "rt" ] }
diff --git a/proxmox-compression/tests/zip.rs b/proxmox-compression/tests/zip.rs
new file mode 100644
index 0000000..1f76037
--- /dev/null
+++ b/proxmox-compression/tests/zip.rs
@@ -0,0 +1,123 @@
+use proxmox_compression::zip::{FileType, ZipEncoder};
+
+use anyhow::{ensure, Error};
+use flate2::{Decompress, FlushDecompress};
+use std::ffi::OsString;
+use tokio::test;
+
+fn check_zip_with_one_file(
+ bytes: Vec<u8>,
+ expected_file_name: &str,
+ expected_content: &[u8],
+ expected_file_attributes: u16,
+ is_file: bool,
+) -> Result<(), Error> {
+ ensure!(
+ &bytes[..4] == b"PK\x03\x04",
+ "ZIP magic number does not match"
+ );
+
+ let general_purpose_flags = &bytes[6..8];
+ let size_compressed = &bytes[18..22];
+ let size_uncompressed = &bytes[22..26];
+ let file_name_len = (bytes[26] as usize) | ((bytes[27] as usize) << 8);
+ let extra_len = (bytes[28] as usize) | ((bytes[29] as usize) << 8);
+ let file_name = &bytes[30..30 + file_name_len];
+ let mut offset = 30 + file_name_len;
+
+ assert_eq!(expected_file_name.as_bytes(), file_name);
+
+ offset += extra_len;
+
+ if is_file {
+ let mut decompress = Decompress::new(false);
+ let mut decompressed_data = vec![0u8; expected_content.len()];
+ decompress.decompress(
+ &bytes[offset..],
+ &mut decompressed_data,
+ FlushDecompress::Finish,
+ )?;
+
+ assert_eq!(decompressed_data, expected_content);
+
+ offset += decompress.total_in() as usize;
+ }
+
+ // Optional data descriptor
+ if &bytes[offset..offset + 4] == b"PK\x07\x08" {
+ offset += 4;
+
+ if (general_purpose_flags[0] & 0x08) == 0x08 {
+ offset += 12;
+
+ if size_compressed == b"\xff\xff\xff\xff" && size_uncompressed == b"\xff\xff\xff\xff" {
+ offset += 8;
+ }
+ }
+ }
+
+ ensure!(
+ &bytes[offset..offset + 4] == b"PK\x01\x02",
+ "Missing central directory file header"
+ );
+
+ let external_file_attributes = &bytes[offset + 38..offset + 42];
+ let file_attributes = u16::from_le_bytes(external_file_attributes[2..4].try_into()?);
+
+ assert_eq!(expected_file_attributes, file_attributes);
+
+ Ok(())
+}
+
+#[test]
+async fn test_zip_text_file() -> Result<(), Error> {
+ let mut target = Vec::new();
+
+ let source_contents = "bar";
+ let source = source_contents.as_bytes();
+
+ let mut zip_encoder = ZipEncoder::new(&mut target);
+ zip_encoder
+ .add_entry("foo.txt", 0, 0o755, FileType::Regular(source))
+ .await?;
+
+ zip_encoder.finish().await?;
+
+ check_zip_with_one_file(target, "foo.txt", b"bar", 0o100755, true)?;
+
+ Ok(())
+}
+
+#[test]
+async fn test_zip_symlink() -> Result<(), Error> {
+ let mut target = Vec::new();
+
+ let link_path = OsString::from("/dev/null");
+
+ let mut zip_encoder = ZipEncoder::new(&mut target);
+ zip_encoder
+ .add_entry("link", 0, 0o755, FileType::<&[u8]>::Symlink(link_path))
+ .await?;
+
+ zip_encoder.finish().await?;
+
+ check_zip_with_one_file(target, "link", b"/dev/null", 0o120755, true)?;
+
+ Ok(())
+}
+
+#[test]
+async fn test_zip_directory() -> Result<(), Error> {
+ let mut target = Vec::new();
+
+ let mut zip_encoder = ZipEncoder::new(&mut target);
+ zip_encoder
+ .add_entry("directory", 0, 0o755, FileType::<&[u8]>::Directory)
+ .await?;
+
+ zip_encoder.finish().await?;
+
+ check_zip_with_one_file(target, "directory/", b"", 0o40755, false)?;
+
+ Ok(())
+}
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pbs-devel] [PATCH v4 backup 1/2] pxar: Adopt FileType enum when adding a zip entry
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
` (2 preceding siblings ...)
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 3/3] compression: Add unit tests for the ZipEncoder Filip Schauer
@ 2024-01-24 10:15 ` Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 2/2] fix #4995: pxar: Include symlinks in zip file creation Filip Schauer
2024-01-29 12:44 ` [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Folke Gleumes
5 siblings, 0 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Use a FileType enum instead of a boolean to specify whether a ZipEntry
is a directory or a regular file.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
| 48 ++++++++++++++++++----------------
1 file changed, 25 insertions(+), 23 deletions(-)
--git a/pbs-client/src/pxar/extract.rs b/pbs-client/src/pxar/extract.rs
index af18ecfc..a7f94bf6 100644
--- a/pbs-client/src/pxar/extract.rs
+++ b/pbs-client/src/pxar/extract.rs
@@ -24,7 +24,7 @@ use proxmox_io::{sparse_copy, sparse_copy_async};
use proxmox_sys::c_result;
use proxmox_sys::fs::{create_path, CreateOptions};
-use proxmox_compression::zip::{ZipEncoder, ZipEntry};
+use proxmox_compression::zip::{FileType, ZipEncoder};
use crate::pxar::dir_stack::PxarDirStack;
use crate::pxar::metadata;
@@ -993,13 +993,13 @@ where
let path = entry.path().strip_prefix(&prefix)?;
if path != Path::new("/") {
let metadata = entry.metadata();
- let entry = ZipEntry::new(
+ zip.add_entry(
path,
metadata.stat.mtime.secs,
metadata.stat.mode as u16,
- false,
- );
- zip.add_entry::<FileContents<T>>(entry, None).await?;
+ FileType::<FileContents<T>>::Directory,
+ )
+ .await?;
}
let mut decoder = dir.decode_full().await?;
@@ -1012,15 +1012,16 @@ where
match entry.kind() {
EntryKind::File { .. } => {
log::debug!("adding '{}' to zip", path.display());
- let entry = ZipEntry::new(
- path,
- metadata.stat.mtime.secs,
- metadata.stat.mode as u16,
- true,
- );
- zip.add_entry(entry, decoder.contents())
+ if let Some(contents) = decoder.contents() {
+ zip.add_entry(
+ path,
+ metadata.stat.mtime.secs,
+ metadata.stat.mode as u16,
+ FileType::Regular(contents),
+ )
.await
.context("could not send file entry")?;
+ }
}
EntryKind::Hardlink(_) => {
let entry = root
@@ -1030,25 +1031,26 @@ where
let realfile = accessor.follow_hardlink(&entry).await?;
let metadata = realfile.entry().metadata();
log::debug!("adding '{}' to zip", path.display());
- let entry = ZipEntry::new(
- path,
- metadata.stat.mtime.secs,
- metadata.stat.mode as u16,
- true,
- );
- zip.add_entry(entry, decoder.contents())
+ if let Some(contents) = decoder.contents() {
+ zip.add_entry(
+ path,
+ metadata.stat.mtime.secs,
+ metadata.stat.mode as u16,
+ FileType::Regular(contents),
+ )
.await
.context("could not send file entry")?;
+ }
}
EntryKind::Directory => {
log::debug!("adding '{}' to zip", path.display());
- let entry = ZipEntry::new(
+ zip.add_entry(
path,
metadata.stat.mtime.secs,
metadata.stat.mode as u16,
- false,
- );
- zip.add_entry::<FileContents<T>>(entry, None).await?;
+ FileType::<FileContents<T>>::Directory,
+ )
+ .await?;
}
_ => {} // ignore all else
};
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pbs-devel] [PATCH v4 backup 2/2] fix #4995: pxar: Include symlinks in zip file creation
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
` (3 preceding siblings ...)
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 1/2] pxar: Adopt FileType enum when adding a zip entry Filip Schauer
@ 2024-01-24 10:15 ` Filip Schauer
2024-01-29 12:44 ` [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Folke Gleumes
5 siblings, 0 replies; 7+ messages in thread
From: Filip Schauer @ 2024-01-24 10:15 UTC (permalink / raw)
To: pbs-devel
Include symlinks when restoring a backup as a zip file.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
| 11 +++++++++++
1 file changed, 11 insertions(+)
--git a/pbs-client/src/pxar/extract.rs b/pbs-client/src/pxar/extract.rs
index a7f94bf6..96053e42 100644
--- a/pbs-client/src/pxar/extract.rs
+++ b/pbs-client/src/pxar/extract.rs
@@ -1052,6 +1052,17 @@ where
)
.await?;
}
+ EntryKind::Symlink(link) => {
+ log::debug!("adding '{}' to zip", path.display());
+ let realpath = Path::new(link);
+ zip.add_entry(
+ path,
+ metadata.stat.mtime.secs,
+ metadata.stat.mode as u16,
+ FileType::<FileContents<T>>::Symlink(realpath.into()),
+ )
+ .await?;
+ }
_ => {} // ignore all else
};
}
--
2.39.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
` (4 preceding siblings ...)
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 2/2] fix #4995: pxar: Include symlinks in zip file creation Filip Schauer
@ 2024-01-29 12:44 ` Folke Gleumes
5 siblings, 0 replies; 7+ messages in thread
From: Folke Gleumes @ 2024-01-29 12:44 UTC (permalink / raw)
To: Proxmox Backup Server development discussion
On Wed, 2024-01-24 at 11:15 +0100, Filip Schauer wrote:
> Include symlinks when restoring files from a backup as a zip file.
>
> Resulting ZIP files were successfully tested on Linux with: zipinfo,
> unzip, unar
>
> On Windows the extracted symlinks show up as regular files containing
> the path to the destination. As far as I am aware Windows Explorer
> does
> not support symlinks in ZIP files.
>
> Changes since v3:
> * Move content to Regular FileType enum
> * Make ZipEntry private
> * Add take_reader function to FileType enum
>
> Changes since v2:
> * Add a FileType enum and embed the symlink target into the enum.
> * Add unit tests for ZipEncoder to the proxmox-compression crate
>
> Changes since v1:
> * Use P instead of &Path
> * Fix compile error due to misplaced comma
> * Check content before symlink_target, since regular files are more
> common than symlinks
>
> proxmox:
>
> Filip Schauer (3):
> compression: Refactor ZipEntry creation and add FileType enum
> compression: Add support for symlinks in zip files
> compression: Add unit tests for the ZipEncoder
>
> proxmox-compression/Cargo.toml | 2 +-
> proxmox-compression/src/zip.rs | 159 +++++++++++++++++++----------
> --
> proxmox-compression/tests/zip.rs | 123 ++++++++++++++++++++++++
> 3 files changed, 223 insertions(+), 61 deletions(-)
> create mode 100644 proxmox-compression/tests/zip.rs
>
> proxmox-backup:
>
> Filip Schauer (2):
> pxar: Adopt FileType enum when adding a zip entry
> fix #4995: pxar: Include symlinks in zip file creation
>
> pbs-client/src/pxar/extract.rs | 59 +++++++++++++++++++++-----------
> --
> 1 file changed, 36 insertions(+), 23 deletions(-)
>
Tested this with some handcrafted directory/symlink structures and my
home directory for some organic data. Restore was tested on Windows 11
and Debian 12.
Everything worked as intended.
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-01-29 12:44 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-24 10:15 [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 1/3] compression: Refactor ZipEntry creation and add FileType enum Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 2/3] compression: Add support for symlinks in zip files Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 proxmox 3/3] compression: Add unit tests for the ZipEncoder Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 1/2] pxar: Adopt FileType enum when adding a zip entry Filip Schauer
2024-01-24 10:15 ` [pbs-devel] [PATCH v4 backup 2/2] fix #4995: pxar: Include symlinks in zip file creation Filip Schauer
2024-01-29 12:44 ` [pbs-devel] [PATCH v4 many] fix #4995: Include symlinks in zip file restore Folke Gleumes
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal