* [PATCH pve-storage 0/1] storage: add xz support @ 2026-03-05 12:08 mail 2026-03-05 12:08 ` [PATCH pve-storage 1/1] " mail 2026-03-05 16:25 ` [PATCH pve-storage 0/1] " Max R. Carrara 0 siblings, 2 replies; 4+ messages in thread From: mail @ 2026-03-05 12:08 UTC (permalink / raw) To: pve-devel; +Cc: Roman Ondráček From: Roman Ondráček <mail@romanondracek.cz> A popular ISO compressed exclusively with xz is FreeBSD [1]. Since this requires adding `xz` to the list of known compression formats we add decompression methods for vmz and tar. [1] https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/15.0/ Signed-off-by: Roman Ondráček <mail@romanondracek.cz> Roman Ondráček (1): storage: add xz support debian/control | 1 + src/PVE/Storage.pm | 5 ++++- src/PVE/Storage/Plugin.pm | 2 +- src/test/archive_info_test.pm | 6 ++++-- src/test/list_volumes_test.pm | 11 ++++++++++- src/test/parse_volname_test.pm | 8 ++++---- src/test/path_to_volume_id_test.pm | 12 +++++++----- 7 files changed, 31 insertions(+), 14 deletions(-) -- 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH pve-storage 1/1] storage: add xz support 2026-03-05 12:08 [PATCH pve-storage 0/1] storage: add xz support mail @ 2026-03-05 12:08 ` mail 2026-03-05 14:52 ` David Riley 2026-03-05 16:25 ` [PATCH pve-storage 0/1] " Max R. Carrara 1 sibling, 1 reply; 4+ messages in thread From: mail @ 2026-03-05 12:08 UTC (permalink / raw) To: pve-devel; +Cc: Roman Ondráček From: Roman Ondráček <mail@romanondracek.cz> Signed-off-by: Roman Ondráček <mail@romanondracek.cz> --- debian/control | 1 + src/PVE/Storage.pm | 5 ++++- src/PVE/Storage/Plugin.pm | 2 +- src/test/archive_info_test.pm | 6 ++++-- src/test/list_volumes_test.pm | 11 ++++++++++- src/test/parse_volname_test.pm | 8 ++++---- src/test/path_to_volume_id_test.pm | 12 +++++++----- 7 files changed, 31 insertions(+), 14 deletions(-) diff --git a/debian/control b/debian/control index 6bd55a2..9ef10e6 100644 --- a/debian/control +++ b/debian/control @@ -50,6 +50,7 @@ Depends: bzip2, smbclient, thin-provisioning-tools, udev, + xz-utils, zstd, ${misc:Depends}, ${perl:Depends}, diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index 6e87bac..b9ae6d7 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE/Storage.pm @@ -115,7 +115,7 @@ PVE::Storage::Plugin->init(); our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i; -our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i; +our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2|xz))/i; our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/; @@ -1761,18 +1761,21 @@ sub decompressor_info { lzo => ['tar', '--lzop'], zst => ['tar', '--zstd'], bz2 => ['tar', '--bzip2'], + xz => ['tar', '--xz'], }, vma => { gz => ['zcat'], lzo => ['lzop', '-d', '-c'], zst => ['zstd', '-q', '-d', '-c'], bz2 => ['bzcat', '-q'], + xz => ['xz', '-q', '-d', '-c'], }, iso => { gz => ['zcat'], lzo => ['lzop', '-d', '-c'], zst => ['zstd', '-q', '-d', '-c'], bz2 => ['bzcat', '-q'], + xz => ['xz', '-q', '-d', '-c'], }, }; diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm index 6f3d691..5b0216e 100644 --- a/src/PVE/Storage/Plugin.pm +++ b/src/PVE/Storage/Plugin.pm @@ -21,7 +21,7 @@ use JSON; use base qw(PVE::SectionConfig); -use constant KNOWN_COMPRESSION_FORMATS => ('gz', 'lzo', 'zst', 'bz2'); +use constant KNOWN_COMPRESSION_FORMATS => ('gz', 'lzo', 'zst', 'bz2', 'xz'); use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS); use constant LOG_EXT => ".log"; diff --git a/src/test/archive_info_test.pm b/src/test/archive_info_test.pm index c5c3992..b8dfbff 100644 --- a/src/test/archive_info_test.pm +++ b/src/test/archive_info_test.pm @@ -115,19 +115,21 @@ my $tests = [ }, ]; -# add new compression fromats to test +# add new compression formats to test my $decompressor = { tar => { gz => ['tar', '-z'], lzo => ['tar', '--lzop'], zst => ['tar', '--zstd'], bz2 => ['tar', '--bzip2'], + xz => ['tar', '--xz'], }, vma => { gz => ['zcat'], lzo => ['lzop', '-d', '-c'], zst => ['zstd', '-q', '-d', '-c'], bz2 => ['bzcat', '-q'], + xz => ['xz', '-q', '-d', '-c'], }, }; @@ -167,7 +169,7 @@ for my $virt (sort keys %$bkp_suffix) { my $non_bkp_suffix = { 'openvz' => ['zip', 'tgz.lzo', 'zip.gz', ''], 'lxc' => ['zip', 'tgz.lzo', 'zip.gz', ''], - 'qemu' => ['vma.xz', 'vms.gz', 'vmx.zst', ''], + 'qemu' => ['vms.gz', 'vmx.zst', ''], 'none' => ['tar.gz'], }; diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm index 0876902..d79e991 100644 --- a/src/test/list_volumes_test.pm +++ b/src/test/list_volumes_test.pm @@ -90,6 +90,7 @@ my @tests = ( "$storage_dir/images/16110/vm-16110-disk-1.raw", "$storage_dir/images/16110/vm-16110-disk-2.vmdk", "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz", + "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo", "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma", "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst", @@ -136,6 +137,15 @@ my @tests = ( 'vmid' => '16110', 'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz', }, + { + 'content' => 'backup', + 'ctime' => 1585602760, + 'format' => 'vma.xz', + 'size' => DEFAULT_SIZE, + 'subtype' => 'qemu', + 'vmid' => '16110', + 'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz', + }, { 'content' => 'backup', 'ctime' => 1585602765, @@ -457,7 +467,6 @@ my @tests = ( "$storage_dir/private/subvol-19254-disk-0/19254", "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz", "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo", - "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz", ], expected => [], # returns empty list diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm index 5c9478a..035e3ee 100644 --- a/src/test/parse_volname_test.pm +++ b/src/test/parse_volname_test.pm @@ -247,9 +247,9 @@ foreach my $s (@$disk_suffix) { # create more test cases for backup files matches my $bkp_suffix = { - qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'], - lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'], - openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst'], + qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.xz', 'vma.zst'], + lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.xz', 'tar.zst', 'tar.bz2'], + openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.xz', 'tar.zst'], }; foreach my $virt (keys %$bkp_suffix) { @@ -277,7 +277,7 @@ foreach my $virt (keys %$bkp_suffix) { # create more test cases for failed backup files matches my $non_bkp_suffix = { - qemu => ['vms.gz', 'vma.xz'], + qemu => ['vms.gz'], lxc => ['zip.gz', 'tgz.lzo'], }; foreach my $virt (keys %$non_bkp_suffix) { diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm index dfa51e6..ad3f39a 100644 --- a/src/test/path_to_volume_id_test.pm +++ b/src/test/path_to_volume_id_test.pm @@ -92,6 +92,13 @@ my @tests = ( 'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo', ], }, + { + description => 'Backup, vma.xz', + volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", + expected => [ + 'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz', + ], + }, { description => 'Backup, vma.zst', volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst", @@ -212,11 +219,6 @@ my @tests = ( volname => "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo", expected => [''], }, - { - description => 'Backup, wrong ending, qemu, vma.xz', - volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", - expected => [''], - }, { description => 'Backup, wrong format, qemu, vms.gz', volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz", -- 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH pve-storage 1/1] storage: add xz support 2026-03-05 12:08 ` [PATCH pve-storage 1/1] " mail @ 2026-03-05 14:52 ` David Riley 0 siblings, 0 replies; 4+ messages in thread From: David Riley @ 2026-03-05 14:52 UTC (permalink / raw) To: mail, pve-devel no deep review I just noted a small thing, comment inline On 3/5/26 1:08 PM, mail@romanondracek.cz wrote: > From: Roman Ondráček <mail@romanondracek.cz> > > Signed-off-by: Roman Ondráček <mail@romanondracek.cz> > --- > debian/control | 1 + > src/PVE/Storage.pm | 5 ++++- > src/PVE/Storage/Plugin.pm | 2 +- > src/test/archive_info_test.pm | 6 ++++-- > src/test/list_volumes_test.pm | 11 ++++++++++- > src/test/parse_volname_test.pm | 8 ++++---- > src/test/path_to_volume_id_test.pm | 12 +++++++----- > 7 files changed, 31 insertions(+), 14 deletions(-) > > diff --git a/debian/control b/debian/control > index 6bd55a2..9ef10e6 100644 > --- a/debian/control > +++ b/debian/control > @@ -50,6 +50,7 @@ Depends: bzip2, > smbclient, > thin-provisioning-tools, > udev, > + xz-utils, > zstd, > ${misc:Depends}, > ${perl:Depends}, > diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm > index 6e87bac..b9ae6d7 100755 > --- a/src/PVE/Storage.pm > +++ b/src/PVE/Storage.pm > @@ -115,7 +115,7 @@ PVE::Storage::Plugin->init(); > > our $ISO_EXT_RE_0 = qr/\.(?:iso|img)/i; > > -our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2))/i; > +our $VZTMPL_EXT_RE_1 = qr/\.(?|(tar)(?!\.)|tar\.(gz|xz|zst|bz2|xz))/i; seems like xz was already part of the regex. > > our $BACKUP_EXT_RE_2 = qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/; > > @@ -1761,18 +1761,21 @@ sub decompressor_info { > lzo => ['tar', '--lzop'], > zst => ['tar', '--zstd'], > bz2 => ['tar', '--bzip2'], > + xz => ['tar', '--xz'], > }, > vma => { > gz => ['zcat'], > lzo => ['lzop', '-d', '-c'], > zst => ['zstd', '-q', '-d', '-c'], > bz2 => ['bzcat', '-q'], > + xz => ['xz', '-q', '-d', '-c'], > }, > iso => { > gz => ['zcat'], > lzo => ['lzop', '-d', '-c'], > zst => ['zstd', '-q', '-d', '-c'], > bz2 => ['bzcat', '-q'], > + xz => ['xz', '-q', '-d', '-c'], > }, > }; > > diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm > index 6f3d691..5b0216e 100644 > --- a/src/PVE/Storage/Plugin.pm > +++ b/src/PVE/Storage/Plugin.pm > @@ -21,7 +21,7 @@ use JSON; > > use base qw(PVE::SectionConfig); > > -use constant KNOWN_COMPRESSION_FORMATS => ('gz', 'lzo', 'zst', 'bz2'); > +use constant KNOWN_COMPRESSION_FORMATS => ('gz', 'lzo', 'zst', 'bz2', 'xz'); > use constant COMPRESSOR_RE => join('|', KNOWN_COMPRESSION_FORMATS); > > use constant LOG_EXT => ".log"; > diff --git a/src/test/archive_info_test.pm b/src/test/archive_info_test.pm > index c5c3992..b8dfbff 100644 > --- a/src/test/archive_info_test.pm > +++ b/src/test/archive_info_test.pm > @@ -115,19 +115,21 @@ my $tests = [ > }, > ]; > > -# add new compression fromats to test > +# add new compression formats to test > my $decompressor = { > tar => { > gz => ['tar', '-z'], > lzo => ['tar', '--lzop'], > zst => ['tar', '--zstd'], > bz2 => ['tar', '--bzip2'], > + xz => ['tar', '--xz'], > }, > vma => { > gz => ['zcat'], > lzo => ['lzop', '-d', '-c'], > zst => ['zstd', '-q', '-d', '-c'], > bz2 => ['bzcat', '-q'], > + xz => ['xz', '-q', '-d', '-c'], > }, > }; > > @@ -167,7 +169,7 @@ for my $virt (sort keys %$bkp_suffix) { > my $non_bkp_suffix = { > 'openvz' => ['zip', 'tgz.lzo', 'zip.gz', ''], > 'lxc' => ['zip', 'tgz.lzo', 'zip.gz', ''], > - 'qemu' => ['vma.xz', 'vms.gz', 'vmx.zst', ''], > + 'qemu' => ['vms.gz', 'vmx.zst', ''], > 'none' => ['tar.gz'], > }; > > diff --git a/src/test/list_volumes_test.pm b/src/test/list_volumes_test.pm > index 0876902..d79e991 100644 > --- a/src/test/list_volumes_test.pm > +++ b/src/test/list_volumes_test.pm > @@ -90,6 +90,7 @@ my @tests = ( > "$storage_dir/images/16110/vm-16110-disk-1.raw", > "$storage_dir/images/16110/vm-16110-disk-2.vmdk", > "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz", > + "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", > "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_45.vma.lzo", > "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma", > "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst", > @@ -136,6 +137,15 @@ my @tests = ( > 'vmid' => '16110', > 'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_11_40.vma.gz', > }, > + { > + 'content' => 'backup', > + 'ctime' => 1585602760, > + 'format' => 'vma.xz', > + 'size' => DEFAULT_SIZE, > + 'subtype' => 'qemu', > + 'vmid' => '16110', > + 'volid' => 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz', > + }, > { > 'content' => 'backup', > 'ctime' => 1585602765, > @@ -457,7 +467,6 @@ my @tests = ( > "$storage_dir/private/subvol-19254-disk-0/19254", > "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.zip.gz", > "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo", > - "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", > "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz", > ], > expected => [], # returns empty list > diff --git a/src/test/parse_volname_test.pm b/src/test/parse_volname_test.pm > index 5c9478a..035e3ee 100644 > --- a/src/test/parse_volname_test.pm > +++ b/src/test/parse_volname_test.pm > @@ -247,9 +247,9 @@ foreach my $s (@$disk_suffix) { > > # create more test cases for backup files matches > my $bkp_suffix = { > - qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.zst'], > - lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst', 'tar.bz2'], > - openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.zst'], > + qemu => ['vma', 'vma.gz', 'vma.lzo', 'vma.xz', 'vma.zst'], > + lxc => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.xz', 'tar.zst', 'tar.bz2'], > + openvz => ['tar', 'tgz', 'tar.gz', 'tar.lzo', 'tar.xz', 'tar.zst'], > }; > > foreach my $virt (keys %$bkp_suffix) { > @@ -277,7 +277,7 @@ foreach my $virt (keys %$bkp_suffix) { > > # create more test cases for failed backup files matches > my $non_bkp_suffix = { > - qemu => ['vms.gz', 'vma.xz'], > + qemu => ['vms.gz'], > lxc => ['zip.gz', 'tgz.lzo'], > }; > foreach my $virt (keys %$non_bkp_suffix) { > diff --git a/src/test/path_to_volume_id_test.pm b/src/test/path_to_volume_id_test.pm > index dfa51e6..ad3f39a 100644 > --- a/src/test/path_to_volume_id_test.pm > +++ b/src/test/path_to_volume_id_test.pm > @@ -92,6 +92,13 @@ my @tests = ( > 'backup', 'local:backup/vzdump-lxc-16112-2020_03_30-21_39_30.tar.lzo', > ], > }, > + { > + description => 'Backup, vma.xz', > + volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", > + expected => [ > + 'backup', 'local:backup/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz', > + ], > + }, > { > description => 'Backup, vma.zst', > volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_13_55.vma.zst", > @@ -212,11 +219,6 @@ my @tests = ( > volname => "$storage_dir/dump/vzdump-openvz-16112-2020_03_30-21_39_30.tgz.lzo", > expected => [''], > }, > - { > - description => 'Backup, wrong ending, qemu, vma.xz', > - volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vma.xz", > - expected => [''], > - }, > { > description => 'Backup, wrong format, qemu, vms.gz', > volname => "$storage_dir/dump/vzdump-qemu-16110-2020_03_30-21_12_40.vms.gz", ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH pve-storage 0/1] storage: add xz support 2026-03-05 12:08 [PATCH pve-storage 0/1] storage: add xz support mail 2026-03-05 12:08 ` [PATCH pve-storage 1/1] " mail @ 2026-03-05 16:25 ` Max R. Carrara 1 sibling, 0 replies; 4+ messages in thread From: Max R. Carrara @ 2026-03-05 16:25 UTC (permalink / raw) To: mail, pve-devel On Thu Mar 5, 2026 at 1:08 PM CET, mail wrote: > From: Roman Ondráček <mail@romanondracek.cz> > > A popular ISO compressed exclusively with xz is FreeBSD [1]. > > Since this requires adding `xz` to the list of known compression > formats we add decompression methods for vmz and tar. > > [1] https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/15.0/ > > Signed-off-by: Roman Ondráček <mail@romanondracek.cz> Hi! Thanks a lot for your patch! Have you by change already signed a contributor license agreement? If not, see here: https://www.proxmox.com/en/about/open-source/developers - Max > > Roman Ondráček (1): > storage: add xz support > > debian/control | 1 + > src/PVE/Storage.pm | 5 ++++- > src/PVE/Storage/Plugin.pm | 2 +- > src/test/archive_info_test.pm | 6 ++++-- > src/test/list_volumes_test.pm | 11 ++++++++++- > src/test/parse_volname_test.pm | 8 ++++---- > src/test/path_to_volume_id_test.pm | 12 +++++++----- > 7 files changed, 31 insertions(+), 14 deletions(-) > > -- > 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-03-05 16:25 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-03-05 12:08 [PATCH pve-storage 0/1] storage: add xz support mail 2026-03-05 12:08 ` [PATCH pve-storage 1/1] " mail 2026-03-05 14:52 ` David Riley 2026-03-05 16:25 ` [PATCH pve-storage 0/1] " Max R. Carrara
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.