From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 23F7B1FF0E0 for ; Thu, 09 Jul 2026 10:20:56 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 1E95121487; Thu, 09 Jul 2026 10:20:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783366854; x=1783971654; darn=lists.proxmox.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ccUj3pwZdu5n37aYFmKHnL3HBV/7B2P+eRlPSJHFA8Y=; b=ModUB3GJx1l8FJ2XVsFmOgbZBKRlMi6jOouD6TAxFCtVFdsOe7wSaG5W6t5tU/aBn5 asnY0KnGMNHTU07utTjfAXr9OTLQi73f3eNM4W1HZRDhbmyq9gmeVbBTdmRyPtsJqOv4 6dy95r5MAMvzkelQafJTrVNN+TNKNBw86xa5rEv0FubszWFAwhJmV8VSbIhUPIDhreue EsMep5aTyc5PA65yCu/nOnh27kpxIwRrl0QhQbNHQ/tuzwlt/741XGyf12O4ai881jkH jzF51FxEFBytJxLi4sxWRj1WWKngY+cXkyRoITyJAWp3nPiiN9HiCDbYMGANZP/yH1Sd sxTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783366854; x=1783971654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ccUj3pwZdu5n37aYFmKHnL3HBV/7B2P+eRlPSJHFA8Y=; b=nfilXiiuDxacnYNKSr+C23b3uR9Y7pK0VGKmDge1gOyunVbLssnuVZdv1P7tIH4iOl rgdJzKy2BfPnaBoG4pf9F2L4hQ2W8GnuGQi7v1bZCzdH31lAu1FaxUPNTURf88UG0tNm HIKeXiHf9zLj0HuWDfn8rCOxhsfsbrRteKPkyRopM5Yfivj07ZUxwd3NUqhASC4LEWxt xIJXh768DVj+fDvnV+qG5XL2DNCHHes+eEtE/Iv8TtZCyjaCXSMaEMXk/fEY6rlvCIQC zjbtPup7YZ0c9ILN5IqhKzZ0gt2/UpooNXCVdfT+YkCYo0DZ4F2XDyjFUIhzXVO0KPnL ELzQ== X-Gm-Message-State: AOJu0YyiHSARjHs/vx6sMXsSP/+J8Uv4qHYwX9MIzu0nn5truiRcI3g7 RAioA/ZbIlmM3hwLGO82x8alFZ4SqMQ1Kopo7lawts9gCROiRuhIPYrvJzlNXIb/ X-Gm-Gg: AfdE7ckciK01RTAyN0pT8OfHQT/rumP5VOc66wxDStHFsTPUWIP4cbsatpeFZ5GP94Z 5QA2ob3GR+jhvLGRrRtfXF+zfKygyIDjD7Mtg5hdt7vFq7mAtFpqOWHW4091R5DOMbf5bV+u7Sz jdYvd4vgzfNyR++awqrMS07LVIeN/8XHKazAg5dNbHg5HnF9cX8s4cKlszGyc3hTmbAgNJFC3Eg 9p2TyDxCFIBY1aha870D0TDdjJyEN8fytHO7f1z0dPE6IPqcaqtrwzUxKwnmWXoeiyGRJVHFE34 dnJRrGW63Jk4UJegMf55asy9ZtCtuIDvR4maP+DkR2Z1TdJsSGO6Xe9q4yaq9i3bFqxCBnQE7xf 96LETdzARnRprBbp2BydOBKjLI8Bz85twJJX/diTMXElCy2+wA3zKtexVnu6xi7cwZ6x5DuRLpy HaJKEsYbxo9zleVw== X-Received: by 2002:adf:ed06:0:b0:474:9002:c72c with SMTP id ffacd0b85a97d-47de66a6c12mr1791857f8f.34.1783366854361; Mon, 06 Jul 2026 12:40:54 -0700 (PDT) From: copystring To: pve-devel@lists.proxmox.com subject: SPAM: [RFC PATCH 1/4] lxc: create: add isolated OCI rootfs preparation Date: Mon, 6 Jul 2026 21:39:29 +0200 Message-ID: <20260706193934.279992-2-copystring@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260706193934.279992-1-copystring@gmail.com> References: <20260706193934.279992-1-copystring@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 3 AWL -0.425 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FORGED_GMAIL_RCVD 1 'From' gmail.com does not match 'Received' headers FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust RCVD_IN_SBL_CSS 3.558 Received via a relay in Spamhaus SBL-CSS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record X-MailFrom: copystring@gmail.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation Message-ID-Hash: 62HMLJCORPC42XMJR2PIXUUCWGSUVLCD X-Message-ID-Hash: 62HMLJCORPC42XMJR2PIXUUCWGSUVLCD X-Mailman-Approved-At: Thu, 09 Jul 2026 10:20:40 +0200 CC: copystring X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Prepare a new rootfs volume from an OCI archive without mounting any existing mpX mount points. This keeps the active container config untouched until a later API layer can switch rootfs atomically. The helper preserves rootfs options, carries over only idmap-related LXC entries into the temporary extraction config, returns OCI-derived runtime metadata from the unshared worker, and tears down the new volume on prepare or cleanup failure. Signed-off-by: copystring --- src/PVE/LXC/Create.pm | 95 +++++++++ src/test/Makefile | 4 +- src/test/oci-rootfs-replace-test.pm | 250 +++++++++++++++++++++++ src/test/run_oci_rootfs_replace_tests.pl | 10 + 4 files changed, 358 insertions(+), 1 deletion(-) create mode 100644 src/test/oci-rootfs-replace-test.pm create mode 100644 src/test/run_oci_rootfs_replace_tests.pl diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm index 69065b9..f809afb 100644 --- a/src/PVE/LXC/Create.pm +++ b/src/PVE/LXC/Create.pm @@ -783,6 +783,101 @@ sub restore_oci_archive { return $conf; # it's a reference anyway, so return mostly for convenience. } +my sub oci_rootfs_size_to_gb { + my ($size) = @_; + + my $size_mib = int(($size + 1024 * 1024 - 1) / (1024 * 1024)); + my $size_gb = sprintf('%.6f', $size_mib / 1024); + $size_gb =~ s/0+$//; + $size_gb =~ s/\.$//; + + return $size_gb; +} + +my sub oci_config_lxc_copy { + my ($conf) = @_; + + return [map { [@$_] } grep { $_->[0] eq 'lxc.idmap' } @{ $conf->{lxc} // [] }]; +} + +sub create_oci_rootfs { + my ($storage_cfg, $vmid, $archive, $storage, $size, $conf, $rootdir_base) = @_; + + die "archive '$archive' is not an OCI image archive\n" + if !archive_is_oci_format($storage_cfg, $archive); + + my $size_gb = oci_rootfs_size_to_gb($size); + my $rootfs = PVE::LXC::Config->parse_volume('rootfs', $conf->{rootfs}); + my $new_rootfs_config = { + %$rootfs, + volume => "$storage:$size_gb", + }; + delete $new_rootfs_config->{type}; + delete $new_rootfs_config->{size}; + + my $settings = { + rootfs => PVE::LXC::Config->print_ct_mountpoint($new_rootfs_config, 1), + }; + my $new_conf = { + lxc => oci_config_lxc_copy($conf), + }; + $new_conf->{unprivileged} = $conf->{unprivileged} if exists($conf->{unprivileged}); + + $rootdir_base //= "/var/lib/lxc/$vmid"; + File::Path::make_path($rootdir_base); + my $rootdir = "$rootdir_base/.oci-replace-rootfs"; + + my $vollist = PVE::LXC::create_disks($storage_cfg, $vmid, $settings, $new_conf); + my $new_rootfs = PVE::LXC::Config->parse_volume('rootfs', $new_conf->{rootfs}); + my $new_volid = $new_rootfs->{volume}; + + eval { + my (undef, $root_uid, $root_gid) = PVE::LXC::parse_id_maps($conf); + + $new_conf = PVE::LXC::run_unshared(sub { + die "temporary OCI rootfs path '$rootdir' already exists\n" if -e $rootdir; + + my @mounted; + eval { + mkdir $rootdir + or die "failed to create temporary OCI rootfs path '$rootdir': $!\n"; + + my $mountpoint = { %$new_rootfs, mp => '/', ro => 0 }; + PVE::LXC::mountpoint_mount( + $mountpoint, $rootdir, $storage_cfg, undef, $root_uid, $root_gid, + ); + push @mounted, $rootdir; + + restore_oci_archive($storage_cfg, $archive, $rootdir, $new_conf); + PVE::LXC::Setup->new($new_conf, $rootdir); # detect OS and architecture + }; + my $err = $@; + my $cleanup_err = ''; + + foreach my $mount (reverse @mounted) { + eval { + PVE::Tools::run_command(['/bin/umount', $mount], errfunc => sub { }); + }; + $cleanup_err .= "can't unmount temporary OCI rootfs path '$mount': $@\n" if $@; + } + + $cleanup_err .= "failed to remove temporary OCI rootfs path '$rootdir': $!\n" + if -d $rootdir && !rmdir $rootdir; + + die $err if $err; + die $cleanup_err if $cleanup_err; + + return $new_conf; + }); + }; + if (my $err = $@) { + PVE::LXC::destroy_disks($storage_cfg, $vollist); + die $err; + } + + return ($new_conf->{rootfs}, $new_volid, $new_conf); +} + sub resolve_oci_user { my ($userstr, $rootdir) = @_; diff --git a/src/test/Makefile b/src/test/Makefile index 91ae6ff..8da35ee 100644 --- a/src/test/Makefile +++ b/src/test/Makefile @@ -2,7 +2,7 @@ RUN_USERNS := lxc-usernsexec -m "u:0:`id -u`:1" -m "g:0:`id -g`:1" -- all: test -test: test_setup test_snapshot test_bindmount test_idmap +test: test_setup test_snapshot test_bindmount test_idmap test_oci_rootfs_replace test_setup: run_setup_tests.pl if test -e /run/lock/sbuild; then \ @@ -24,5 +24,7 @@ test_bindmount: bindmount_test.pl test_idmap: run_idmap_tests.pl ./run_idmap_tests.pl +test_oci_rootfs_replace: run_oci_rootfs_replace_tests.pl + ./run_oci_rootfs_replace_tests.pl clean: rm -rf tmprootfs diff --git a/src/test/oci-rootfs-replace-test.pm b/src/test/oci-rootfs-replace-test.pm new file mode 100644 index 0000000..88f8485 --- /dev/null +++ b/src/test/oci-rootfs-replace-test.pm @@ -0,0 +1,250 @@ +package PVE::LXC::Test; + +use strict; +use warnings; + +use lib qw(..); + +use File::Temp qw(tempdir); +use Storable qw(dclone); +use Test::MockModule; +use Test::More; + +use PVE::LXC; +use PVE::LXC::Config; +use PVE::LXC::Create; +use PVE::LXC::Setup; +use PVE::Tools; + +my $storage_cfg = {}; +my $vmid = 900; +my $archive = 'local:vztmpl/test-oci.tar'; +my $storage = 'local-lvm'; +my $size = 2 * 1024 * 1024 * 1024; +my $new_volid = 'local-lvm:vm-900-disk-9'; + +my $base_conf = { + rootfs => 'local-lvm:vm-900-disk-0,size=2G,acl=1,mountoptions=noatime,quota=1', + mp0 => 'local-lvm:vm-900-disk-1,mp=/data,size=1G', + mp1 => '/host/path,mp=/bind', + unprivileged => 1, + lxc => [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ['lxc.init.cwd', '/old-cwd'], + ], +}; + +my ( + $fail_restore, + $fail_umount, + $create_disks_settings, + $create_disks_conf, + @destroy_disks_calls, + @mountpoint_mount_calls, + @restore_oci_calls, + @setup_calls, + @run_command_calls, +); + +my $create_module = Test::MockModule->new('PVE::LXC::Create'); +$create_module->mock( + archive_is_oci_format => sub { + my ($cfg, $volid) = @_; + is($cfg, $storage_cfg, 'archive check uses storage config'); + is($volid, $archive, 'archive check uses requested OCI archive'); + return 1; + }, + restore_oci_archive => sub { + my ($cfg, $volid, $rootdir, $conf) = @_; + push @restore_oci_calls, [$cfg, $volid, $rootdir, dclone($conf)]; + die "mocked OCI extraction failure\n" if $fail_restore; + + $conf->{entrypoint} = '/usr/bin/new-entrypoint'; + $conf->{env} = "FOO=bar\0PATH=/usr/bin"; + push $conf->{lxc}->@*, ['lxc.signal.halt', 'SIGUSR1']; + }, +); + +my $lxc_module = Test::MockModule->new('PVE::LXC'); +$lxc_module->mock( + create_disks => sub { + my ($cfg, $got_vmid, $settings, $conf) = @_; + is($cfg, $storage_cfg, 'create_disks uses storage config'); + is($got_vmid, $vmid, 'create_disks uses vmid'); + $create_disks_settings = dclone($settings); + $create_disks_conf = dclone($conf); + $conf->{rootfs} = 'local-lvm:vm-900-disk-9,size=2G,acl=1,mountoptions=noatime,quota=1'; + return [$new_volid]; + }, + destroy_disks => sub { + push @destroy_disks_calls, dclone(\@_); + }, + mount_all => sub { + die "mount_all must not be used for OCI rootfs replacement\n"; + }, + mountpoint_mount => sub { + my ($mountpoint, $rootdir, $cfg, $snapname, $root_uid, $root_gid) = @_; + push @mountpoint_mount_calls, + [dclone($mountpoint), $rootdir, $cfg, $snapname, $root_uid, $root_gid]; + }, + parse_id_maps => sub { + return ([], 100000, 100000); + }, + run_unshared => sub { + my ($code) = @_; + return $code->(); + }, +); + +my $setup_module = Test::MockModule->new('PVE::LXC::Setup'); +$setup_module->mock( + new => sub { + my ($class, $conf, $rootdir) = @_; + push @setup_calls, [$class, dclone($conf), $rootdir]; + $conf->{ostype} = 'debian'; + $conf->{arch} = 'amd64'; + return bless {}, $class; + }, +); + +my $tools_module = Test::MockModule->new('PVE::Tools'); +$tools_module->mock( + run_command => sub { + my ($cmd, %param) = @_; + push @run_command_calls, dclone($cmd); + die "mocked umount failure\n" if $fail_umount; + }, +); + +sub reset_state { + $fail_restore = 0; + $fail_umount = 0; + $create_disks_settings = undef; + $create_disks_conf = undef; + @destroy_disks_calls = (); + @mountpoint_mount_calls = (); + @restore_oci_calls = (); + @setup_calls = (); + @run_command_calls = (); +} + +sub run_create_oci_rootfs { + my ($conf, $rootdir_base) = @_; + + return PVE::LXC::Create::create_oci_rootfs( + $storage_cfg, + $vmid, + $archive, + $storage, + $size, + $conf, + $rootdir_base, + ); +} + +subtest 'prepares only a new rootfs and preserves OCI-derived config' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + + my ($new_rootfs_conf, $got_new_volid, $oci_conf) = run_create_oci_rootfs($conf, $tmpdir); + + is($got_new_volid, $new_volid, 'returns new rootfs volid'); + is($new_rootfs_conf, 'local-lvm:vm-900-disk-9,size=2G,acl=1,mountoptions=noatime,quota=1', + 'returns new rootfs config'); + is_deeply($conf, $orig_conf, 'source CT config is not mutated while preparing rootfs'); + + my $requested_rootfs = PVE::LXC::Config->parse_volume('rootfs', $create_disks_settings->{rootfs}); + is($requested_rootfs->{volume}, 'local-lvm:2', 'allocates new rootfs on target storage'); + is($requested_rootfs->{acl}, 1, 'preserves rootfs acl option'); + is($requested_rootfs->{quota}, 1, 'preserves rootfs quota option'); + is($requested_rootfs->{mountoptions}, 'noatime', 'preserves rootfs mount options'); + + is_deeply( + $create_disks_conf->{lxc}, + [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ], + 'copies only idmap lxc entries into the temporary OCI config', + ); + ok(!exists($create_disks_conf->{mp0}), 'does not copy mp0 into temporary OCI config'); + ok(!exists($create_disks_conf->{mp1}), 'does not copy mp1 into temporary OCI config'); + + is(scalar(@mountpoint_mount_calls), 1, 'mounts exactly one filesystem'); + my ($mountpoint, $rootdir, $got_storage_cfg, $snapname, $root_uid, $root_gid) = + $mountpoint_mount_calls[0]->@*; + is($mountpoint->{volume}, $new_volid, 'mounts the newly allocated rootfs'); + is($mountpoint->{mp}, '/', 'mounts it as root only'); + is($rootdir, "$tmpdir/.oci-replace-rootfs", 'uses isolated temporary rootfs path'); + is($got_storage_cfg, $storage_cfg, 'mount uses storage config'); + is($snapname, undef, 'mount does not use snapshot'); + is($root_uid, 100000, 'mount uses mapped root uid'); + is($root_gid, 100000, 'mount uses mapped root gid'); + + is(scalar(@restore_oci_calls), 1, 'extracts OCI image once'); + is($restore_oci_calls[0]->[2], "$tmpdir/.oci-replace-rootfs", 'extracts into new rootfs'); + is(scalar(@setup_calls), 1, 'detects OS and architecture on the new rootfs'); + is($setup_calls[0]->[2], "$tmpdir/.oci-replace-rootfs", 'runs setup detection on new rootfs'); + + is($oci_conf->{entrypoint}, '/usr/bin/new-entrypoint', 'returns OCI entrypoint'); + is($oci_conf->{env}, "FOO=bar\0PATH=/usr/bin", 'returns OCI environment'); + is($oci_conf->{ostype}, 'debian', 'returns detected ostype'); + is($oci_conf->{arch}, 'amd64', 'returns detected architecture'); + is_deeply( + $oci_conf->{lxc}, + [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ['lxc.signal.halt', 'SIGUSR1'], + ], + 'returns idmap plus OCI runtime lxc entries', + ); + + is_deeply(\@destroy_disks_calls, [], 'does not destroy disks after successful prepare'); + is_deeply(\@run_command_calls, [['/bin/umount', "$tmpdir/.oci-replace-rootfs"]], + 'unmounts the temporary rootfs'); + ok(!-e "$tmpdir/.oci-replace-rootfs", 'removes temporary rootfs directory'); +}; + +subtest 'destroys newly allocated rootfs if OCI extraction fails' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + $fail_restore = 1; + + eval { run_create_oci_rootfs($conf, $tmpdir) }; + like($@, qr/mocked OCI extraction failure/, 'propagates OCI extraction failure'); + is_deeply($conf, $orig_conf, 'source CT config stays unchanged on extraction failure'); + is(scalar(@mountpoint_mount_calls), 1, 'mounted the new rootfs before extraction failure'); + is_deeply(\@run_command_calls, [['/bin/umount', "$tmpdir/.oci-replace-rootfs"]], + 'unmounts temporary rootfs after extraction failure'); + is_deeply( + \@destroy_disks_calls, + [[$storage_cfg, [$new_volid]]], + 'destroys the newly allocated rootfs on extraction failure', + ); + ok(!-e "$tmpdir/.oci-replace-rootfs", 'removes temporary rootfs directory after failure'); +}; + +subtest 'destroys newly allocated rootfs if cleanup after successful extraction fails' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + $fail_umount = 1; + + eval { run_create_oci_rootfs($conf, $tmpdir) }; + like($@, qr/can't unmount temporary OCI rootfs path/, 'propagates cleanup failure'); + is_deeply($conf, $orig_conf, 'source CT config stays unchanged on cleanup failure'); + is_deeply( + \@destroy_disks_calls, + [[$storage_cfg, [$new_volid]]], + 'destroys the newly allocated rootfs on cleanup failure', + ); +}; + +done_testing(); diff --git a/src/test/run_oci_rootfs_replace_tests.pl b/src/test/run_oci_rootfs_replace_tests.pl new file mode 100644 index 0000000..a13ae8e --- /dev/null +++ b/src/test/run_oci_rootfs_replace_tests.pl @@ -0,0 +1,10 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +use TAP::Harness; + +my $harness = TAP::Harness->new({ "verbosity" => -2 }); +my $res = $harness->runtests("oci-rootfs-replace-test.pm"); +exit -1 if $res->{failed}; -- 2.55.0.windows.2