From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 221771FF0E0 for ; Thu, 09 Jul 2026 10:21:29 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 8452421522; Thu, 09 Jul 2026 10:20:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783366862; x=1783971662; darn=lists.proxmox.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ccUj3pwZdu5n37aYFmKHnL3HBV/7B2P+eRlPSJHFA8Y=; b=a/+Shn0KT+W4QMLyW+uWGwfcVgDw6AezSkHnCOCGaWGnzLZtxbKTfhrEuw97PqnzmT jqQ6YjP188SPoPQWFR+d+A80J7Gixm3rx8B2BtUosp1c3GipYUzpXo4i7gqEQvG5js/M /1G+wNJxgvvJtHOu97g++rVtXqO9JsxsVU8/BQ5nk/GAOlvSto+zJJHhR7oISrLlpB3+ YHSQWT0BG0LyJ103YNWmO04+F+ZR1AA4NSVWYdTcAhn9u+mDFz+d3HJaTuRUHeWqmX15 bAtV7fI8jLHWfbAkaAgyEK6sjXtMfI7tD0uHXeKnUaSzVVBH/cGQq3JNbMi4w0fagpKn /DUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783366862; x=1783971662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ccUj3pwZdu5n37aYFmKHnL3HBV/7B2P+eRlPSJHFA8Y=; b=GSeL/EwyCaFbTJ+LJtADqZpS9lGUvY6+T7v0O5vQ8pRMyFPDs8BgamnT/ZgQJMbsrP St9dqAkcsow/7Khoi3DwEmBTPNEkF6AonL8MzcTuXDFwkVMIFOznSGutqTIgLejf/hqW upHqch0gb2OJ60SGGEHeTnJGxGYvzgreUyJGjaRpQ+U9YRjKsi7rG+TZYKSvQGXSq6SM 5rDtm3QgRRVIdY9eXilBiKNE0P3AC2LAKMNeTdI0NPcAgGzOy0OO6DbAha5nvUHevpKm lu5bkmsP7nkVPikwAr3rte5BVUIbO1z4ksnCJxf2vbthDNtSbPR8E3qmRhXja/ltkCi6 rqig== X-Gm-Message-State: AOJu0YxNPS2Hjd+81byeqXvQ/MRJJHa40Xu5pxfvDhl7LKncFvxC8z4t zvrhVvpdPAPHv8QP4mrRGbkkrl96ceuOkPrJr+RaxQH8GKmNQ2JKfFM/s3Mfr6xa X-Gm-Gg: AfdE7cn83XuxRcr0Hyj+9WfYiCQF//ksIdAKqhEy2foQDTfnyx+GQAaJKAnjZOETbqB 7Yd8u+bz0uPY5aIk20UOmnl33xJ9rI30dmUGUBDhpivvhq+9KpqzbO37GnDnMHBi5pblJjXZLoN 3g82XBY8Wv1uCn3hi39IPw3IskDyjk5PP3rrQAsxOTUZZSBlxzYpplqvmby4SxgYdccOwmriFKJ rYAKpaBOM/Ggo/w/x2r4EJ3VHVxrpOKaeYOgtnzelOCBZLI7w+HM8kTEsEfNhtkADz1k6XgVhqC Tiwnen/XEVrv1MnXh264b6dw/6ddDZxBEeVMwiN7oCK/mJ15hVFl1k/qrq9aUgxUFKsmLhYRce0 kF6comJj8FFjQ2YZqIAg4Bdu/v5wk7D7yO5NUfmN/I15Qu01rbCrvRdcKohsa6tzK05+bhDW/r7 F/cBuYUwn5XCpGu1G7dpjaCc7r X-Received: by 2002:a05:600c:3145:b0:490:e974:e006 with SMTP id 5b1f17b1804b1-493df099f12mr23103145e9.29.1783366861876; Mon, 06 Jul 2026 12:41:01 -0700 (PDT) From: copystring To: pve-devel@lists.proxmox.com subject: SPAM: [RFC PATCH 1/4] lxc: create: add isolated OCI rootfs preparation Date: Mon, 6 Jul 2026 21:40:54 +0200 Message-ID: <20260706194059.280257-2-copystring@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260706194059.280257-1-copystring@gmail.com> References: <20260706194059.280257-1-copystring@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 3 AWL 0.311 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust RCVD_IN_SBL_CSS 3.558 Received via a relay in Spamhaus SBL-CSS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record X-MailFrom: copystring@gmail.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation Message-ID-Hash: 6TRX5LOXW7CYFK45WSOEPP7U6GKPYCV3 X-Message-ID-Hash: 6TRX5LOXW7CYFK45WSOEPP7U6GKPYCV3 X-Mailman-Approved-At: Thu, 09 Jul 2026 10:20:40 +0200 CC: copystring X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Prepare a new rootfs volume from an OCI archive without mounting any existing mpX mount points. This keeps the active container config untouched until a later API layer can switch rootfs atomically. The helper preserves rootfs options, carries over only idmap-related LXC entries into the temporary extraction config, returns OCI-derived runtime metadata from the unshared worker, and tears down the new volume on prepare or cleanup failure. Signed-off-by: copystring --- src/PVE/LXC/Create.pm | 95 +++++++++ src/test/Makefile | 4 +- src/test/oci-rootfs-replace-test.pm | 250 +++++++++++++++++++++++ src/test/run_oci_rootfs_replace_tests.pl | 10 + 4 files changed, 358 insertions(+), 1 deletion(-) create mode 100644 src/test/oci-rootfs-replace-test.pm create mode 100644 src/test/run_oci_rootfs_replace_tests.pl diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm index 69065b9..f809afb 100644 --- a/src/PVE/LXC/Create.pm +++ b/src/PVE/LXC/Create.pm @@ -783,6 +783,101 @@ sub restore_oci_archive { return $conf; # it's a reference anyway, so return mostly for convenience. } +my sub oci_rootfs_size_to_gb { + my ($size) = @_; + + my $size_mib = int(($size + 1024 * 1024 - 1) / (1024 * 1024)); + my $size_gb = sprintf('%.6f', $size_mib / 1024); + $size_gb =~ s/0+$//; + $size_gb =~ s/\.$//; + + return $size_gb; +} + +my sub oci_config_lxc_copy { + my ($conf) = @_; + + return [map { [@$_] } grep { $_->[0] eq 'lxc.idmap' } @{ $conf->{lxc} // [] }]; +} + +sub create_oci_rootfs { + my ($storage_cfg, $vmid, $archive, $storage, $size, $conf, $rootdir_base) = @_; + + die "archive '$archive' is not an OCI image archive\n" + if !archive_is_oci_format($storage_cfg, $archive); + + my $size_gb = oci_rootfs_size_to_gb($size); + my $rootfs = PVE::LXC::Config->parse_volume('rootfs', $conf->{rootfs}); + my $new_rootfs_config = { + %$rootfs, + volume => "$storage:$size_gb", + }; + delete $new_rootfs_config->{type}; + delete $new_rootfs_config->{size}; + + my $settings = { + rootfs => PVE::LXC::Config->print_ct_mountpoint($new_rootfs_config, 1), + }; + my $new_conf = { + lxc => oci_config_lxc_copy($conf), + }; + $new_conf->{unprivileged} = $conf->{unprivileged} if exists($conf->{unprivileged}); + + $rootdir_base //= "/var/lib/lxc/$vmid"; + File::Path::make_path($rootdir_base); + my $rootdir = "$rootdir_base/.oci-replace-rootfs"; + + my $vollist = PVE::LXC::create_disks($storage_cfg, $vmid, $settings, $new_conf); + my $new_rootfs = PVE::LXC::Config->parse_volume('rootfs', $new_conf->{rootfs}); + my $new_volid = $new_rootfs->{volume}; + + eval { + my (undef, $root_uid, $root_gid) = PVE::LXC::parse_id_maps($conf); + + $new_conf = PVE::LXC::run_unshared(sub { + die "temporary OCI rootfs path '$rootdir' already exists\n" if -e $rootdir; + + my @mounted; + eval { + mkdir $rootdir + or die "failed to create temporary OCI rootfs path '$rootdir': $!\n"; + + my $mountpoint = { %$new_rootfs, mp => '/', ro => 0 }; + PVE::LXC::mountpoint_mount( + $mountpoint, $rootdir, $storage_cfg, undef, $root_uid, $root_gid, + ); + push @mounted, $rootdir; + + restore_oci_archive($storage_cfg, $archive, $rootdir, $new_conf); + PVE::LXC::Setup->new($new_conf, $rootdir); # detect OS and architecture + }; + my $err = $@; + my $cleanup_err = ''; + + foreach my $mount (reverse @mounted) { + eval { + PVE::Tools::run_command(['/bin/umount', $mount], errfunc => sub { }); + }; + $cleanup_err .= "can't unmount temporary OCI rootfs path '$mount': $@\n" if $@; + } + + $cleanup_err .= "failed to remove temporary OCI rootfs path '$rootdir': $!\n" + if -d $rootdir && !rmdir $rootdir; + + die $err if $err; + die $cleanup_err if $cleanup_err; + + return $new_conf; + }); + }; + if (my $err = $@) { + PVE::LXC::destroy_disks($storage_cfg, $vollist); + die $err; + } + + return ($new_conf->{rootfs}, $new_volid, $new_conf); +} + sub resolve_oci_user { my ($userstr, $rootdir) = @_; diff --git a/src/test/Makefile b/src/test/Makefile index 91ae6ff..8da35ee 100644 --- a/src/test/Makefile +++ b/src/test/Makefile @@ -2,7 +2,7 @@ RUN_USERNS := lxc-usernsexec -m "u:0:`id -u`:1" -m "g:0:`id -g`:1" -- all: test -test: test_setup test_snapshot test_bindmount test_idmap +test: test_setup test_snapshot test_bindmount test_idmap test_oci_rootfs_replace test_setup: run_setup_tests.pl if test -e /run/lock/sbuild; then \ @@ -24,5 +24,7 @@ test_bindmount: bindmount_test.pl test_idmap: run_idmap_tests.pl ./run_idmap_tests.pl +test_oci_rootfs_replace: run_oci_rootfs_replace_tests.pl + ./run_oci_rootfs_replace_tests.pl clean: rm -rf tmprootfs diff --git a/src/test/oci-rootfs-replace-test.pm b/src/test/oci-rootfs-replace-test.pm new file mode 100644 index 0000000..88f8485 --- /dev/null +++ b/src/test/oci-rootfs-replace-test.pm @@ -0,0 +1,250 @@ +package PVE::LXC::Test; + +use strict; +use warnings; + +use lib qw(..); + +use File::Temp qw(tempdir); +use Storable qw(dclone); +use Test::MockModule; +use Test::More; + +use PVE::LXC; +use PVE::LXC::Config; +use PVE::LXC::Create; +use PVE::LXC::Setup; +use PVE::Tools; + +my $storage_cfg = {}; +my $vmid = 900; +my $archive = 'local:vztmpl/test-oci.tar'; +my $storage = 'local-lvm'; +my $size = 2 * 1024 * 1024 * 1024; +my $new_volid = 'local-lvm:vm-900-disk-9'; + +my $base_conf = { + rootfs => 'local-lvm:vm-900-disk-0,size=2G,acl=1,mountoptions=noatime,quota=1', + mp0 => 'local-lvm:vm-900-disk-1,mp=/data,size=1G', + mp1 => '/host/path,mp=/bind', + unprivileged => 1, + lxc => [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ['lxc.init.cwd', '/old-cwd'], + ], +}; + +my ( + $fail_restore, + $fail_umount, + $create_disks_settings, + $create_disks_conf, + @destroy_disks_calls, + @mountpoint_mount_calls, + @restore_oci_calls, + @setup_calls, + @run_command_calls, +); + +my $create_module = Test::MockModule->new('PVE::LXC::Create'); +$create_module->mock( + archive_is_oci_format => sub { + my ($cfg, $volid) = @_; + is($cfg, $storage_cfg, 'archive check uses storage config'); + is($volid, $archive, 'archive check uses requested OCI archive'); + return 1; + }, + restore_oci_archive => sub { + my ($cfg, $volid, $rootdir, $conf) = @_; + push @restore_oci_calls, [$cfg, $volid, $rootdir, dclone($conf)]; + die "mocked OCI extraction failure\n" if $fail_restore; + + $conf->{entrypoint} = '/usr/bin/new-entrypoint'; + $conf->{env} = "FOO=bar\0PATH=/usr/bin"; + push $conf->{lxc}->@*, ['lxc.signal.halt', 'SIGUSR1']; + }, +); + +my $lxc_module = Test::MockModule->new('PVE::LXC'); +$lxc_module->mock( + create_disks => sub { + my ($cfg, $got_vmid, $settings, $conf) = @_; + is($cfg, $storage_cfg, 'create_disks uses storage config'); + is($got_vmid, $vmid, 'create_disks uses vmid'); + $create_disks_settings = dclone($settings); + $create_disks_conf = dclone($conf); + $conf->{rootfs} = 'local-lvm:vm-900-disk-9,size=2G,acl=1,mountoptions=noatime,quota=1'; + return [$new_volid]; + }, + destroy_disks => sub { + push @destroy_disks_calls, dclone(\@_); + }, + mount_all => sub { + die "mount_all must not be used for OCI rootfs replacement\n"; + }, + mountpoint_mount => sub { + my ($mountpoint, $rootdir, $cfg, $snapname, $root_uid, $root_gid) = @_; + push @mountpoint_mount_calls, + [dclone($mountpoint), $rootdir, $cfg, $snapname, $root_uid, $root_gid]; + }, + parse_id_maps => sub { + return ([], 100000, 100000); + }, + run_unshared => sub { + my ($code) = @_; + return $code->(); + }, +); + +my $setup_module = Test::MockModule->new('PVE::LXC::Setup'); +$setup_module->mock( + new => sub { + my ($class, $conf, $rootdir) = @_; + push @setup_calls, [$class, dclone($conf), $rootdir]; + $conf->{ostype} = 'debian'; + $conf->{arch} = 'amd64'; + return bless {}, $class; + }, +); + +my $tools_module = Test::MockModule->new('PVE::Tools'); +$tools_module->mock( + run_command => sub { + my ($cmd, %param) = @_; + push @run_command_calls, dclone($cmd); + die "mocked umount failure\n" if $fail_umount; + }, +); + +sub reset_state { + $fail_restore = 0; + $fail_umount = 0; + $create_disks_settings = undef; + $create_disks_conf = undef; + @destroy_disks_calls = (); + @mountpoint_mount_calls = (); + @restore_oci_calls = (); + @setup_calls = (); + @run_command_calls = (); +} + +sub run_create_oci_rootfs { + my ($conf, $rootdir_base) = @_; + + return PVE::LXC::Create::create_oci_rootfs( + $storage_cfg, + $vmid, + $archive, + $storage, + $size, + $conf, + $rootdir_base, + ); +} + +subtest 'prepares only a new rootfs and preserves OCI-derived config' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + + my ($new_rootfs_conf, $got_new_volid, $oci_conf) = run_create_oci_rootfs($conf, $tmpdir); + + is($got_new_volid, $new_volid, 'returns new rootfs volid'); + is($new_rootfs_conf, 'local-lvm:vm-900-disk-9,size=2G,acl=1,mountoptions=noatime,quota=1', + 'returns new rootfs config'); + is_deeply($conf, $orig_conf, 'source CT config is not mutated while preparing rootfs'); + + my $requested_rootfs = PVE::LXC::Config->parse_volume('rootfs', $create_disks_settings->{rootfs}); + is($requested_rootfs->{volume}, 'local-lvm:2', 'allocates new rootfs on target storage'); + is($requested_rootfs->{acl}, 1, 'preserves rootfs acl option'); + is($requested_rootfs->{quota}, 1, 'preserves rootfs quota option'); + is($requested_rootfs->{mountoptions}, 'noatime', 'preserves rootfs mount options'); + + is_deeply( + $create_disks_conf->{lxc}, + [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ], + 'copies only idmap lxc entries into the temporary OCI config', + ); + ok(!exists($create_disks_conf->{mp0}), 'does not copy mp0 into temporary OCI config'); + ok(!exists($create_disks_conf->{mp1}), 'does not copy mp1 into temporary OCI config'); + + is(scalar(@mountpoint_mount_calls), 1, 'mounts exactly one filesystem'); + my ($mountpoint, $rootdir, $got_storage_cfg, $snapname, $root_uid, $root_gid) = + $mountpoint_mount_calls[0]->@*; + is($mountpoint->{volume}, $new_volid, 'mounts the newly allocated rootfs'); + is($mountpoint->{mp}, '/', 'mounts it as root only'); + is($rootdir, "$tmpdir/.oci-replace-rootfs", 'uses isolated temporary rootfs path'); + is($got_storage_cfg, $storage_cfg, 'mount uses storage config'); + is($snapname, undef, 'mount does not use snapshot'); + is($root_uid, 100000, 'mount uses mapped root uid'); + is($root_gid, 100000, 'mount uses mapped root gid'); + + is(scalar(@restore_oci_calls), 1, 'extracts OCI image once'); + is($restore_oci_calls[0]->[2], "$tmpdir/.oci-replace-rootfs", 'extracts into new rootfs'); + is(scalar(@setup_calls), 1, 'detects OS and architecture on the new rootfs'); + is($setup_calls[0]->[2], "$tmpdir/.oci-replace-rootfs", 'runs setup detection on new rootfs'); + + is($oci_conf->{entrypoint}, '/usr/bin/new-entrypoint', 'returns OCI entrypoint'); + is($oci_conf->{env}, "FOO=bar\0PATH=/usr/bin", 'returns OCI environment'); + is($oci_conf->{ostype}, 'debian', 'returns detected ostype'); + is($oci_conf->{arch}, 'amd64', 'returns detected architecture'); + is_deeply( + $oci_conf->{lxc}, + [ + ['lxc.idmap', 'u 0 100000 65536'], + ['lxc.idmap', 'g 0 100000 65536'], + ['lxc.signal.halt', 'SIGUSR1'], + ], + 'returns idmap plus OCI runtime lxc entries', + ); + + is_deeply(\@destroy_disks_calls, [], 'does not destroy disks after successful prepare'); + is_deeply(\@run_command_calls, [['/bin/umount', "$tmpdir/.oci-replace-rootfs"]], + 'unmounts the temporary rootfs'); + ok(!-e "$tmpdir/.oci-replace-rootfs", 'removes temporary rootfs directory'); +}; + +subtest 'destroys newly allocated rootfs if OCI extraction fails' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + $fail_restore = 1; + + eval { run_create_oci_rootfs($conf, $tmpdir) }; + like($@, qr/mocked OCI extraction failure/, 'propagates OCI extraction failure'); + is_deeply($conf, $orig_conf, 'source CT config stays unchanged on extraction failure'); + is(scalar(@mountpoint_mount_calls), 1, 'mounted the new rootfs before extraction failure'); + is_deeply(\@run_command_calls, [['/bin/umount', "$tmpdir/.oci-replace-rootfs"]], + 'unmounts temporary rootfs after extraction failure'); + is_deeply( + \@destroy_disks_calls, + [[$storage_cfg, [$new_volid]]], + 'destroys the newly allocated rootfs on extraction failure', + ); + ok(!-e "$tmpdir/.oci-replace-rootfs", 'removes temporary rootfs directory after failure'); +}; + +subtest 'destroys newly allocated rootfs if cleanup after successful extraction fails' => sub { + reset_state(); + my $tmpdir = tempdir(CLEANUP => 1); + my $conf = dclone($base_conf); + my $orig_conf = dclone($conf); + $fail_umount = 1; + + eval { run_create_oci_rootfs($conf, $tmpdir) }; + like($@, qr/can't unmount temporary OCI rootfs path/, 'propagates cleanup failure'); + is_deeply($conf, $orig_conf, 'source CT config stays unchanged on cleanup failure'); + is_deeply( + \@destroy_disks_calls, + [[$storage_cfg, [$new_volid]]], + 'destroys the newly allocated rootfs on cleanup failure', + ); +}; + +done_testing(); diff --git a/src/test/run_oci_rootfs_replace_tests.pl b/src/test/run_oci_rootfs_replace_tests.pl new file mode 100644 index 0000000..a13ae8e --- /dev/null +++ b/src/test/run_oci_rootfs_replace_tests.pl @@ -0,0 +1,10 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +use TAP::Harness; + +my $harness = TAP::Harness->new({ "verbosity" => -2 }); +my $res = $harness->runtests("oci-rootfs-replace-test.pm"); +exit -1 if $res->{failed}; -- 2.55.0.windows.2