From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 86B251FF176 for ; Sat, 4 Jan 2025 04:08:24 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 938FD2F3CA; Sat, 4 Jan 2025 04:08:14 +0100 (CET) Date: Sat, 4 Jan 2025 04:07:23 +0100 To: pve-devel@lists.proxmox.com MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Orwa D via pve-devel Precedence: list Cc: Orwa D X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] Bugfix: LXC root filesystem tar extraction (bug in the exclude pattern) Content-Type: multipart/mixed; boundary="===============7068927186848279252==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============7068927186848279252== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 896D8CA4FB for ; Sat, 4 Jan 2025 04:08:13 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5F1C62F43D for ; Sat, 4 Jan 2025 04:07:43 +0100 (CET) Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Sat, 4 Jan 2025 04:07:41 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-436637e8c8dso129823665e9.1 for ; Fri, 03 Jan 2025 19:07:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735960055; x=1736564855; darn=lists.proxmox.com; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=/b6UERMrIj8Nf4J1XHUlaWRBkXxQqfi/WLBduwgcE5A=; b=O858p9EWuFuMj4o18CqLSMtQIEIfoYrNfDqEu2QooogCav/UvPKj4Kc0DPGv4WlNiP Ai3p7NXoNML058BUC9vIp2PaoGanytNFcPDj3YaWU6Y2R/wIyiwhmToM2OK99pQ+5ScK Xo3EnOtOPIoxOu4TmLh2kM5xxpxYg1I6fE8QISeW+O6sUPGSWO2NGQgi2hTIrGxEEaDv 2VXC+SXIAAV3sLFUIlbIvjaTYKdsOXpjDq4mAU0KHfIj3E3KB/YpHnKeXitc2yQnqbbT MYtqQoUHaCxGj8yhabRgKhS064pHpQd9DDUucsj7Lsduc3gb8I/5ZjHi5Vo8/mBuxKcj KISQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735960055; x=1736564855; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/b6UERMrIj8Nf4J1XHUlaWRBkXxQqfi/WLBduwgcE5A=; b=D3JcIzHMAvpapD1esEfDG/et3t2lrzxnzDyGJ0xWyou3CJKOq5OJ849naadgIsnTP4 nQrSqy30gL0F5MhsljufHLHb3GihT+QkIyz+eT324RPYUFTQAOSbrKlgO+Ge2kKo0e0V Op7p7RCSObl6YZ4/1wWIEJZQ7LX5DMoqJzlOENd7CWN6FOZ0Dp6igMrigxKr25Rtasen 0xtPcUhaY6dJ5uhYqHNhF65XEmUBKtF1oxuATO+EJGniB05Mb+xjp9x9XLsvW1ZqYlzz ed6Czv+Hfgp5E2zrDgSynkUjZ5L3RGeKTv+f116VwMTZawQss6c+Pjc3kIXadj5pXMIX BhUQ== X-Gm-Message-State: AOJu0YzngMOM32Jy1jM2jEpQ6La558gT8Rx77fcECfz8T2X+oU5txnBL 7Triglg4qraJ784LD9RDOY79VYYHJnTW+w6BmQUgGhfIGGhFX/FVyqPZcSfeDD4Bo7EKmQHa2fs AbTZ8XoFGld78vV0mbDz1CeK9QTtfSHNn7yE= X-Gm-Gg: ASbGncs6wWgep1bT0JnBn3VZKr/zMlVnaklVw7/p4CvJE8wMdUVrqSv4kn1xOYyKQQG ayc4ap3mntBNK4Hr8rcp3+O7OPzFFc+OTifz/YtG5bNOk+58iCvz1Jeyqz33Euk6xNiVzw9I= X-Google-Smtp-Source: AGHT+IEL6at3eWoyloQQrbeS6t7xXHgUshDcfg/66ZHZqu2BGZCcTPAdM+ZUYIauZ/rmavfcUb1X70M4HVZ1E4Wurww= X-Received: by 2002:a05:600c:4748:b0:434:f804:a9b0 with SMTP id 5b1f17b1804b1-43668b78818mr415452875e9.29.1735960054543; Fri, 03 Jan 2025 19:07:34 -0800 (PST) MIME-Version: 1.0 From: Orwa D Date: Sat, 4 Jan 2025 04:07:23 +0100 Message-ID: Subject: Bugfix: LXC root filesystem tar extraction (bug in the exclude pattern) To: pve-devel@lists.proxmox.com X-SPAM-LEVEL: Spam detection results: 0 AWL 1.421 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider HTML_MESSAGE 0.001 HTML included in message RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 *TL;DR * *The tar extraction exclude pattern for LXC containers in the source file * */usr/share/perl5/PVE/LXC/**Create.pm* *must be changed from './dev/*' to 'dev/*'* *Steps to reproduce error due to current bug:* 1. Grab any of the root filesystem cloud images from https://cloud-images.ubuntu.com/ 2. Using Proxmox VE web interface, download one of the tar balls using the *Download from URL* UI function 3. Try to create a CT (i.e. LXC container) from the tar-ball template (e.g. https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-root.tar.xz ) *Typical failure message during container creation:* Task viewer: CT 115 - Create OutputStatus Stop Download Logical volume "vm-115-disk-0" created. Creating filesystem with 2097152 4k blocks and 524288 inodes Filesystem UUID: 5d302f80-bded-46b6-a991-00cafe8a6257 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 extracting archive '/mnt/pve/folder2TB/template/cache/jammy-server-cloudimg-amd64-root.tar.xz' tar: dev/console: Cannot mknod: Operation not permitted tar: dev/full: Cannot mknod: Operation not permitted tar: dev/null: Cannot mknod: Operation not permitted tar: dev/ptmx: Cannot mknod: Operation not permitted tar: dev/random: Cannot mknod: Operation not permitted tar: dev/tty: Cannot mknod: Operation not permitted tar: dev/urandom: Cannot mknod: Operation not permitted tar: dev/zero: Cannot mknod: Operation not permitted Total bytes read: 1140961280 (1.1GiB, 45MiB/s) tar: Exiting with failure status due to previous errors Logical volume "vm-115-disk-0" successfully removed. TASK ERROR: unable to create CT 115 - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar xpf - -J --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' -C /var/lib/lxc/115/rootfs --skip-old-files --anchored --exclude './dev/*'' failed: exit code 2 *Proposed fix/solution:* Change the exclude pattern in the tar extraction line above from './dev/*' to 'dev/*' This exclude pattern can be currently found in two locations: - /usr/share/perl5/PVE/LXC/Create.pm - /usr/share/lxc/templates/lxc-local (shell script) *Success container creation message (after the fix):* Task viewer: CT 116 - Create OutputStatus Stop Download Logical volume "vm-116-disk-0" created. Creating filesystem with 2097152 4k blocks and 524288 inodes Filesystem UUID: 35681b1e-4220-4d2a-9e1a-f2e17fc16806 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 extracting archive '/var/lib/vz/template/cache/jammy-server-cloudimg-amd64-root.tar.xz' (extraction command is 'ARRAY(0x64ac4ede7218)') Total bytes read: 1140961280 (1.1GiB, 51MiB/s) Detected container architecture: amd64 Creating SSH host key 'ssh_host_dsa_key' - this may take some time ... done: SHA256:ft9IuYd6J/OiZVN2acYNe20dBADc7nS9kZS4ihD1iDA root@test Creating SSH host key 'ssh_host_rsa_key' - this may take some time ... done: SHA256:Nz+J528gr0ZPQARwvHx6lZgo1hElMxOfuzmFbub3inM root@test Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ... done: SHA256:YRUD2eS1M65WHe4+808sopNtOwGthDM9Qg96O5ljxqw root@test Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ... done: SHA256:nLDsxaLs4/VoUjpo7MLlPhKLsssLAlNkV8nOhOUP1nk root@test TASK OK Prior to the fix, people were repackaging the root filesystem tarballs found on the internet to exclude the `dev` folder, as evident here: https://discuss.linuxcontainers.org/t/simple-script-to-convert-any-gnu-linux-machine-into-a-proxmox-lxc-container/10339 and here https://github.com/my5t3ry/machine-to-proxmox-lxc-ct-converter which would be unnecessary after this fix. Regards, Orwa. ReplyForward Add reaction --===============7068927186848279252== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============7068927186848279252==--