Re: [pbs-devel] [PATCH v3 proxmox 3/3] compression: Add unit tests for the ZipEncoder

Re: [pbs-devel] [PATCH v3 proxmox 3/3] compression: Add unit tests for the ZipEncoder

[Kevin Newman]

[-- Attachment #1.1: Type: text/plain, Size: 251 bytes --]

Hi,

I was curious if this patch ever made it into a PBS release? I'm using
3.2.2 and still see missing symlinks in file restores. I'm thinking about
upgrading but I don't think I see this fix in the proxmox-compression got
repo yet either.


Thanks!

[-- Attachment #1.2: Type: text/html, Size: 403 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH v3 proxmox 3/3] compression: Add unit tests for the ZipEncoder

[Fabian Grünbichler]

> Kevin Newman <kevinjnewman@gmail.com> hat am 02.07.2025 19:50 CEST geschrieben:
> Hi,
> 
> I was curious if this patch ever made it into a PBS release? I'm using 3.2.2 and still see missing symlinks in file restores. I'm thinking about upgrading but I don't think I see this fix in the proxmox-compression got repo yet either.

https://lore.proxmox.com/pbs-devel/014673339f88b31a362d2f0ee666453ac978865c.camel@proxmox.com/

doesn't seem like it ever got applied

@Filip maybe something to rebase and resend?


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH v3 proxmox 3/3] compression: Add unit tests for the ZipEncoder

[Filip Schauer]

On 09/07/2025 11:09, Fabian Grünbichler wrote:
>> Kevin Newman <kevinjnewman@gmail.com> hat am 02.07.2025 19:50 CEST geschrieben:
>> Hi,
>>
>> I was curious if this patch ever made it into a PBS release? I'm using 3.2.2 and still see missing symlinks in file restores. I'm thinking about upgrading but I don't think I see this fix in the proxmox-compression got repo yet either.
> https://lore.proxmox.com/pbs-devel/014673339f88b31a362d2f0ee666453ac978865c.camel@proxmox.com/
>
> doesn't seem like it ever got applied
>
> @Filip maybe something to rebase and resend?

I replied to the bug report:
https://bugzilla.proxmox.com/show_bug.cgi?id=4995#c4



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH v3 many] fix #4995: Include symlinks in zip file restore

[Filip Schauer]

Include symlinks when restoring files from a backup as a zip file.

Resulting ZIP files were successfully tested on Linux with: zipinfo,
unzip, unar

On Windows the extracted symlinks show up as regular files containing
the path to the destination. As far as I am aware Windows Explorer does
not support symlinks in ZIP files.

Changes since v2:
* Add a FileType enum and embed the symlink target into the enum.
* Add unit tests for ZipEncoder to the proxmox-compression crate

Changes since v1:
* Use P instead of &Path
* Fix compile error due to misplaced comma
* Check content before symlink_target, since regular files are more
  common than symlinks

proxmox:

Filip Schauer (3):
  compression: Add a FileType enum to ZipEntry
  compression: Add support for symlinks in zip files
  compression: Add unit tests for the ZipEncoder

 proxmox-compression/Cargo.toml   |   2 +-
 proxmox-compression/src/zip.rs   |  62 ++++++++++----
 proxmox-compression/tests/zip.rs | 134 +++++++++++++++++++++++++++++++
 3 files changed, 181 insertions(+), 17 deletions(-)
 create mode 100644 proxmox-compression/tests/zip.rs

proxmox-backup:

Filip Schauer (2):
  pxar: Adopt FileType enum when creating a ZipEntry
  fix #4995: pxar: Include symlinks in zip file creation

 pbs-client/src/pxar/extract.rs | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

-- 
2.39.2

[pbs-devel] [PATCH v3 proxmox 3/3] compression: Add unit tests for the ZipEncoder

[Filip Schauer]

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 proxmox-compression/Cargo.toml   |   2 +-
 proxmox-compression/tests/zip.rs | 134 +++++++++++++++++++++++++++++++
 2 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 proxmox-compression/tests/zip.rs

diff --git a/proxmox-compression/Cargo.toml b/proxmox-compression/Cargo.toml
index 49735cb..8c9c47e 100644
--- a/proxmox-compression/Cargo.toml
+++ b/proxmox-compression/Cargo.toml
@@ -27,5 +27,5 @@ proxmox-io = { workspace = true, features = [ "tokio" ] }
 proxmox-lang.workspace = true
 
 [dev-dependencies]
-tokio = { workspace = true, features = [ "macros" ] }
+tokio = { workspace = true, features = [ "macros", "rt" ] }
 
diff --git a/proxmox-compression/tests/zip.rs b/proxmox-compression/tests/zip.rs
new file mode 100644
index 0000000..f684ccc
--- /dev/null
+++ b/proxmox-compression/tests/zip.rs
@@ -0,0 +1,134 @@
+use proxmox_compression::zip::{FileType, ZipEncoder, ZipEntry};
+
+use anyhow::{ensure, Error};
+use flate2::{Decompress, FlushDecompress};
+use std::ffi::OsString;
+use tokio::test;
+
+fn check_zip_with_one_file(
+    bytes: Vec<u8>,
+    expected_file_name: &str,
+    expected_content: &[u8],
+    expected_file_attributes: u16,
+    is_file: bool,
+) -> Result<(), Error> {
+    ensure!(
+        &bytes[..4] == b"PK\x03\x04",
+        "ZIP magic number does not match"
+    );
+
+    let general_purpose_flags = &bytes[6..8];
+    let size_compressed = &bytes[18..22];
+    let size_uncompressed = &bytes[22..26];
+    let file_name_len = (bytes[26] as usize) | ((bytes[27] as usize) << 8);
+    let extra_len = (bytes[28] as usize) | ((bytes[29] as usize) << 8);
+    let file_name = &bytes[30..30 + file_name_len];
+    let mut offset = 30 + file_name_len;
+
+    assert_eq!(expected_file_name.as_bytes(), file_name);
+
+    offset += extra_len;
+
+    if is_file {
+        let mut decompress = Decompress::new(false);
+        let mut decompressed_data = vec![0u8; expected_content.len()];
+        decompress.decompress(
+            &bytes[offset..],
+            &mut decompressed_data,
+            FlushDecompress::Finish,
+        )?;
+
+        assert_eq!(decompressed_data, expected_content);
+
+        offset += decompress.total_in() as usize;
+    }
+
+    // Optional data descriptor
+    if &bytes[offset..offset + 4] == b"PK\x07\x08" {
+        offset += 4;
+
+        if (general_purpose_flags[0] & 0x08) == 0x08 {
+            offset += 12;
+
+            if size_compressed == b"\xff\xff\xff\xff" && size_uncompressed == b"\xff\xff\xff\xff" {
+                offset += 8;
+            }
+        }
+    }
+
+    ensure!(
+        &bytes[offset..offset + 4] == b"PK\x01\x02",
+        "Missing central directory file header"
+    );
+
+    let external_file_attributes = &bytes[offset + 38..offset + 42];
+    let file_attributes = u16::from_le_bytes(external_file_attributes[2..4].try_into()?);
+
+    assert_eq!(expected_file_attributes, file_attributes);
+
+    Ok(())
+}
+
+#[test]
+async fn test_zip_text_file() -> Result<(), Error> {
+    let mut target = Vec::new();
+
+    let source_contents = "bar";
+    let source = source_contents.as_bytes();
+
+    let mut zip_encoder = ZipEncoder::new(&mut target);
+    zip_encoder
+        .add_entry(
+            ZipEntry::new("foo.txt", 0, 0o755, FileType::Regular),
+            Some(source),
+        )
+        .await?;
+
+    zip_encoder.finish().await?;
+
+    check_zip_with_one_file(target, "foo.txt", b"bar", 0o100755, true)?;
+
+    Ok(())
+}
+
+#[test]
+async fn test_zip_symlink() -> Result<(), Error> {
+    let mut target = Vec::new();
+
+    let link_path = OsString::from("/dev/null");
+
+    let mut zip_encoder = ZipEncoder::new(&mut target);
+    let content: Option<tokio::fs::File> = None;
+    zip_encoder
+        .add_entry(
+            ZipEntry::new("link", 0, 0o755, FileType::Symlink(link_path)),
+            content,
+        )
+        .await?;
+
+    zip_encoder.finish().await?;
+
+    check_zip_with_one_file(target, "link", b"/dev/null", 0o120755, true)?;
+
+    Ok(())
+}
+
+#[test]
+async fn test_zip_directory() -> Result<(), Error> {
+    let mut target = Vec::new();
+
+    let mut zip_encoder = ZipEncoder::new(&mut target);
+    let content: Option<tokio::fs::File> = None;
+    zip_encoder
+        .add_entry(
+            ZipEntry::new("directory", 0, 0o755, FileType::Directory),
+            content,
+        )
+        .await?;
+
+    zip_encoder.finish().await?;
+
+    check_zip_with_one_file(target, "directory/", b"", 0o40755, false)?;
+
+    Ok(())
+}
-- 
2.39.2