From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 4B6CCE30B for ; Mon, 25 Sep 2023 13:38:58 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 33C991EDB9 for ; Mon, 25 Sep 2023 13:38:58 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 25 Sep 2023 13:38:57 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 0399A48D45 for ; Mon, 25 Sep 2023 13:38:57 +0200 (CEST) From: Christoph Heiss To: pve-devel@lists.proxmox.com Date: Mon, 25 Sep 2023 13:38:49 +0200 Message-ID: <20230925113853.919523-1-c.heiss@proxmox.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.037 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH container v2] setup: fix architecture detection for NixOS containers X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2023 11:38:58 -0000 NixOS is special and deviates in many places from a "standard" Linux system. In this case, /bin/sh does not exist in the filesystem, before the initial activation (aka. first boot) - which creates a symlink at /bin/sh. Due to the currently existing fallback code, only an error message is logged and the architecture is defaulted to x86_64. Still, this is not something users might expect. Thus try a bit harder to detect the architecture for NixOS containers by inspecting the init script, which contains a shebang-line with the full path to the system shell. This moves the architecture detection code to the end of the container creation lifecycle, so that it can be implemented as a plugin subroutine. Therefore this mechanism is now generic enough that it can be adapted to other container OS's in the future if needed. AFAICS `arch` is only used when writing the actual LXC config, so determining it later during creation does not change anything. detect_architecture() has been made a bit more generic; the LXC-specific error was moved out of this function, as well as the chroot(). Ensuring that it is executed from the correct rootdir/chroot should be handled by the caller. Tested by creating a NixOS and a Debian container (to verify that nothing regressed) and checking if the warning "Architecure detection failed: [..]" no longer appears for the NixOS CT and if `arch` in the CT config is correct. Also tested restoring both containers from a local and a PBS backup, as well as migrating both container. Signed-off-by: Christoph Heiss --- v1: https://lists.proxmox.com/pipermail/pve-devel/2023-February/055949.html Changes since v1: * Moved detect_architecture() to PVE::LXC::Tools to avoid a cyclic include * Properly log/report errors from detect_architecture() src/PVE/LXC/Create.pm | 76 ------------------------------------- src/PVE/LXC/Setup.pm | 18 +++++++++ src/PVE/LXC/Setup/Base.pm | 9 +++++ src/PVE/LXC/Setup/NixOS.pm | 17 +++++++++ src/PVE/LXC/Setup/Plugin.pm | 5 +++ src/PVE/LXC/Tools.pm | 50 ++++++++++++++++++++++++ 6 files changed, 99 insertions(+), 76 deletions(-) diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm index f4c3220..277c6a9 100644 --- a/src/PVE/LXC/Create.pm +++ b/src/PVE/LXC/Create.pm @@ -16,72 +16,6 @@ use PVE::VZDump::ConvertOVZ; use PVE::Tools; use POSIX; -sub detect_architecture { - my ($rootdir) = @_; - - # see https://en.wikipedia.org/wiki/Executable_and_Linkable_Format - - my $supported_elf_machine = { - 0x03 => 'i386', - 0x3e => 'amd64', - 0x28 => 'armhf', - 0xb7 => 'arm64', - 0xf3 => 'riscv', - }; - - my $elf_fn = '/bin/sh'; # '/bin/sh' is POSIX mandatory - my $detect_arch = sub { - # chroot avoids a problem where we check the binary of the host system - # if $elf_fn is an absolut symlink (e.g. $rootdir/bin/sh -> /bin/bash) - chroot($rootdir) or die "chroot '$rootdir' failed: $!\n"; - chdir('/') or die "failed to change to root directory\n"; - - open(my $fh, "<", $elf_fn) or die "open '$elf_fn' failed: $!\n"; - binmode($fh); - - my $length = read($fh, my $data, 20) or die "read failed: $!\n"; - - # 4 bytes ELF magic number and 1 byte ELF class, padding, machine - my ($magic, $class, undef, $machine) = unpack("A4CA12n", $data); - - die "'$elf_fn' does not resolve to an ELF!\n" - if (!defined($class) || !defined($magic) || $magic ne "\177ELF"); - - my $arch = $supported_elf_machine->{$machine}; - die "'$elf_fn' has unknown ELF machine '$machine'!\n" - if !defined($arch); - - if ($arch eq 'riscv') { - if ($class eq 1) { - $arch = 'riscv32'; - } elsif ($class eq 2) { - $arch = 'riscv64'; - } else { - die "'$elf_fn' has invalid class '$class'!\n"; - } - } - - return $arch; - }; - - my $arch = eval { PVE::Tools::run_fork_with_timeout(10, $detect_arch); }; - my $err = $@; - - if (!defined($arch) && !defined($err)) { - # on timeout - die "Architecture detection failed: timeout\n"; - } elsif ($err) { - # any other error - $arch = 'amd64'; - print "Architecture detection failed: $err\nFalling back to $arch.\n" . - "Use `pct set VMID --arch ARCH` to change.\n"; - } else { - print "Detected container architecture: $arch\n"; - } - - return $arch; -} - sub restore_archive { my ($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit) = @_; @@ -122,11 +56,6 @@ sub restore_proxmox_backup_archive { PVE::Storage::PBSPlugin::run_raw_client_cmd( $scfg, $storeid, $cmd, $param, userns_cmd => $userns_cmd); - - # if arch is set, we do not try to autodetect it - return if defined($conf->{arch}); - - $conf->{arch} = detect_architecture($rootdir); } sub restore_tar_archive { @@ -187,11 +116,6 @@ sub restore_tar_archive { my $err = $@; close($archive_fh) if defined $archive_fh; die $err if $err && !$no_unpack_error; - - # if arch is set, we do not try to autodetect it - return if defined($conf->{arch}); - - $conf->{arch} = detect_architecture($rootdir); } sub recover_config { diff --git a/src/PVE/LXC/Setup.pm b/src/PVE/LXC/Setup.pm index 891231f..c6a5fe9 100644 --- a/src/PVE/LXC/Setup.pm +++ b/src/PVE/LXC/Setup.pm @@ -131,6 +131,24 @@ sub new { $plugin->{rootgid} = $rootgid; } + # if arch is unset, we try to autodetect it + if (!defined($conf->{arch})) { + my $arch = eval { $self->protected_call(sub { $plugin->detect_architecture() }) }; + + if (my $err = $@) { + warn "Architecture detection failed: $err" if $err; + } + + if (!defined($arch)) { + $arch = 'amd64'; + print "Falling back to $arch.\nUse `pct set VMID --arch ARCH` to change.\n"; + } else { + print "Detected container architecture: $arch\n"; + } + + $conf->{arch} = $arch; + } + return $self; } diff --git a/src/PVE/LXC/Setup/Base.pm b/src/PVE/LXC/Setup/Base.pm index b8f07ea..38f0d68 100644 --- a/src/PVE/LXC/Setup/Base.pm +++ b/src/PVE/LXC/Setup/Base.pm @@ -19,6 +19,8 @@ use PVE::Tools; use PVE::Network; use PVE::LXC::Setup::Plugin; +use PVE::LXC::Tools; + use base qw(PVE::LXC::Setup::Plugin); sub new { @@ -608,6 +610,13 @@ sub ssh_host_key_types_to_generate { }; } +sub detect_architecture { + my ($self) = @_; + + # '/bin/sh' is POSIX mandatory + return PVE::LXC::Tools::detect_elf_architecture('/bin/sh'); +} + sub pre_start_hook { my ($self, $conf) = @_; diff --git a/src/PVE/LXC/Setup/NixOS.pm b/src/PVE/LXC/Setup/NixOS.pm index 845d2d5..c702f3d 100644 --- a/src/PVE/LXC/Setup/NixOS.pm +++ b/src/PVE/LXC/Setup/NixOS.pm @@ -6,6 +6,7 @@ use warnings; use File::Path 'make_path'; use PVE::LXC::Setup::Base; +use PVE::LXC::Tools; use base qw(PVE::LXC::Setup::Base); @@ -37,4 +38,20 @@ sub setup_init { my ($self, $conf) = @_; } +sub detect_architecture { + my ($self) = @_; + + # /bin/sh only exists as a symlink after the initial system activaction on first boot. + # To detect the actual architecture of the system, examine the shebang line of the /sbin/init + # script, which has the full path to the system shell. + my $init_path = '/sbin/init'; + open(my $fh, '<', $init_path) or die "open '$init_path' failed: $!\n"; + + if (<$fh> =~ /^#! ?(\S*)/) { + return PVE::LXC::Tools::detect_elf_architecture($1); + } + + die "could not find a shell\n"; +} + 1; diff --git a/src/PVE/LXC/Setup/Plugin.pm b/src/PVE/LXC/Setup/Plugin.pm index 3d968e7..b9d9c2d 100644 --- a/src/PVE/LXC/Setup/Plugin.pm +++ b/src/PVE/LXC/Setup/Plugin.pm @@ -62,6 +62,11 @@ sub ssh_host_key_types_to_generate { croak "implement me in sub-class\n"; } +sub detect_architecture { + my ($self) = @_; + croak "implement me in sub-class\n"; +} + # hooks sub pre_start_hook { diff --git a/src/PVE/LXC/Tools.pm b/src/PVE/LXC/Tools.pm index 1d83768..fdda4e3 100644 --- a/src/PVE/LXC/Tools.pm +++ b/src/PVE/LXC/Tools.pm @@ -150,4 +150,54 @@ sub can_use_new_mount_api() { return $cached_can_use_new_mount_api; } +# Tries to the architecture of an executable file based on its ELF header. +sub detect_elf_architecture { + my ($elf_fn) = @_; + + # see https://en.wikipedia.org/wiki/Executable_and_Linkable_Format + + my $supported_elf_machine = { + 0x03 => 'i386', + 0x3e => 'amd64', + 0x28 => 'armhf', + 0xb7 => 'arm64', + 0xf3 => 'riscv', + }; + + my $detect_arch = sub { + open(my $fh, "<", $elf_fn) or die "open '$elf_fn' failed: $!\n"; + binmode($fh); + + my $length = read($fh, my $data, 20) or die "read failed: $!\n"; + + # 4 bytes ELF magic number and 1 byte ELF class, padding, machine + my ($magic, $class, undef, $machine) = unpack("A4CA12n", $data); + + die "'$elf_fn' does not resolve to an ELF!\n" + if (!defined($class) || !defined($magic) || $magic ne "\177ELF"); + + my $arch = $supported_elf_machine->{$machine}; + die "'$elf_fn' has unknown ELF machine '$machine'!\n" + if !defined($arch); + + if ($arch eq 'riscv') { + if ($class eq 1) { + $arch = 'riscv32'; + } elsif ($class eq 2) { + $arch = 'riscv64'; + } else { + die "'$elf_fn' has invalid class '$class'!\n"; + } + } + + return $arch; + }; + + my $arch = eval { PVE::Tools::run_fork_with_timeout(10, $detect_arch); }; + my $err = $@ // "timeout\n"; + die $err if !defined($arch); + + return $arch; +} + 1; -- 2.41.0