From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	by lore.proxmox.com (Postfix) with ESMTPS id 86B251FF176
	for <inbox@lore.proxmox.com>; Sat,  4 Jan 2025 04:08:24 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 938FD2F3CA;
	Sat,  4 Jan 2025 04:08:14 +0100 (CET)
Date: Sat, 4 Jan 2025 04:07:23 +0100
To: pve-devel@lists.proxmox.com
MIME-Version: 1.0
Message-ID: <mailman.74.1735960093.441.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Orwa D via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Orwa D <orwa.diraneyya@gmail.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] Bugfix: LXC root filesystem tar extraction (bug in the
 exclude pattern)
Content-Type: multipart/mixed; boundary="===============7068927186848279252=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>

--===============7068927186848279252==
Content-Type: message/rfc822
Content-Disposition: inline

Return-Path: <orwa.diraneyya@gmail.com>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by lists.proxmox.com (Postfix) with ESMTPS id 896D8CA4FB
	for <pve-devel@lists.proxmox.com>; Sat,  4 Jan 2025 04:08:13 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 5F1C62F43D
	for <pve-devel@lists.proxmox.com>; Sat,  4 Jan 2025 04:07:43 +0100 (CET)
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by firstgate.proxmox.com (Proxmox) with ESMTPS
	for <pve-devel@lists.proxmox.com>; Sat,  4 Jan 2025 04:07:41 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-436637e8c8dso129823665e9.1
        for <pve-devel@lists.proxmox.com>; Fri, 03 Jan 2025 19:07:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1735960055; x=1736564855; darn=lists.proxmox.com;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=/b6UERMrIj8Nf4J1XHUlaWRBkXxQqfi/WLBduwgcE5A=;
        b=O858p9EWuFuMj4o18CqLSMtQIEIfoYrNfDqEu2QooogCav/UvPKj4Kc0DPGv4WlNiP
         Ai3p7NXoNML058BUC9vIp2PaoGanytNFcPDj3YaWU6Y2R/wIyiwhmToM2OK99pQ+5ScK
         Xo3EnOtOPIoxOu4TmLh2kM5xxpxYg1I6fE8QISeW+O6sUPGSWO2NGQgi2hTIrGxEEaDv
         2VXC+SXIAAV3sLFUIlbIvjaTYKdsOXpjDq4mAU0KHfIj3E3KB/YpHnKeXitc2yQnqbbT
         MYtqQoUHaCxGj8yhabRgKhS064pHpQd9DDUucsj7Lsduc3gb8I/5ZjHi5Vo8/mBuxKcj
         KISQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1735960055; x=1736564855;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/b6UERMrIj8Nf4J1XHUlaWRBkXxQqfi/WLBduwgcE5A=;
        b=D3JcIzHMAvpapD1esEfDG/et3t2lrzxnzDyGJ0xWyou3CJKOq5OJ849naadgIsnTP4
         nQrSqy30gL0F5MhsljufHLHb3GihT+QkIyz+eT324RPYUFTQAOSbrKlgO+Ge2kKo0e0V
         Op7p7RCSObl6YZ4/1wWIEJZQ7LX5DMoqJzlOENd7CWN6FOZ0Dp6igMrigxKr25Rtasen
         0xtPcUhaY6dJ5uhYqHNhF65XEmUBKtF1oxuATO+EJGniB05Mb+xjp9x9XLsvW1ZqYlzz
         ed6Czv+Hfgp5E2zrDgSynkUjZ5L3RGeKTv+f116VwMTZawQss6c+Pjc3kIXadj5pXMIX
         BhUQ==
X-Gm-Message-State: AOJu0YzngMOM32Jy1jM2jEpQ6La558gT8Rx77fcECfz8T2X+oU5txnBL
	7Triglg4qraJ784LD9RDOY79VYYHJnTW+w6BmQUgGhfIGGhFX/FVyqPZcSfeDD4Bo7EKmQHa2fs
	AbTZ8XoFGld78vV0mbDz1CeK9QTtfSHNn7yE=
X-Gm-Gg: ASbGncs6wWgep1bT0JnBn3VZKr/zMlVnaklVw7/p4CvJE8wMdUVrqSv4kn1xOYyKQQG
	ayc4ap3mntBNK4Hr8rcp3+O7OPzFFc+OTifz/YtG5bNOk+58iCvz1Jeyqz33Euk6xNiVzw9I=
X-Google-Smtp-Source: AGHT+IEL6at3eWoyloQQrbeS6t7xXHgUshDcfg/66ZHZqu2BGZCcTPAdM+ZUYIauZ/rmavfcUb1X70M4HVZ1E4Wurww=
X-Received: by 2002:a05:600c:4748:b0:434:f804:a9b0 with SMTP id
 5b1f17b1804b1-43668b78818mr415452875e9.29.1735960054543; Fri, 03 Jan 2025
 19:07:34 -0800 (PST)
MIME-Version: 1.0
From: Orwa D <orwa.diraneyya@gmail.com>
Date: Sat, 4 Jan 2025 04:07:23 +0100
Message-ID: <CAP8PTXROjrL-J3eRV5_HXPV1oku9xD_YVwMRcJ0OUAw=M0xw8w@mail.gmail.com>
Subject: Bugfix: LXC root filesystem tar extraction (bug in the exclude pattern)
To: pve-devel@lists.proxmox.com
X-SPAM-LEVEL: Spam detection results:  0
	AWL                     1.421 Adjusted score from AWL reputation of From: address
	BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
	DKIM_SIGNED               0.1 Message has a DKIM or DK signature, not necessarily valid
	DKIM_VALID               -0.1 Message has at least one valid DKIM or DK signature
	DKIM_VALID_AU            -0.1 Message has a valid DKIM or DK signature from author's domain
	DKIM_VALID_EF            -0.1 Message has a valid DKIM or DK signature from envelope-from domain
	DMARC_PASS               -0.1 DMARC pass policy
	FREEMAIL_FROM           0.001 Sender email is commonly abused enduser mail provider
	HTML_MESSAGE            0.001 HTML included in message
	RCVD_IN_DNSWL_NONE     -0.0001 Sender listed at https://www.dnswl.org/, no trust
	SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
	SPF_PASS               -0.001 SPF: sender matches SPF record
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29

*TL;DR *
*The tar extraction exclude pattern for LXC containers in the source file *
*/usr/share/perl5/PVE/LXC/**Create.pm* *must be changed from './dev/*' to
'dev/*'*

*Steps to reproduce error due to current bug:*
1. Grab any of the root filesystem cloud images from
https://cloud-images.ubuntu.com/
2. Using Proxmox VE web interface, download one of the tar balls using
the *Download
from URL* UI function
3. Try to create a CT (i.e. LXC container) from the tar-ball template (e.g.
https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-root.tar.xz
)

*Typical failure message during container creation:*
Task viewer: CT 115 - Create
OutputStatus
Stop
Download
Logical volume "vm-115-disk-0" created.
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 5d302f80-bded-46b6-a991-00cafe8a6257
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
extracting archive
'/mnt/pve/folder2TB/template/cache/jammy-server-cloudimg-amd64-root.tar.xz'
tar: dev/console: Cannot mknod: Operation not permitted
tar: dev/full: Cannot mknod: Operation not permitted
tar: dev/null: Cannot mknod: Operation not permitted
tar: dev/ptmx: Cannot mknod: Operation not permitted
tar: dev/random: Cannot mknod: Operation not permitted
tar: dev/tty: Cannot mknod: Operation not permitted
tar: dev/urandom: Cannot mknod: Operation not permitted
tar: dev/zero: Cannot mknod: Operation not permitted
Total bytes read: 1140961280 (1.1GiB, 45MiB/s)
tar: Exiting with failure status due to previous errors
Logical volume "vm-115-disk-0" successfully removed.
TASK ERROR: unable to create CT 115 - command 'lxc-usernsexec -m
u:0:100000:65536 -m g:0:100000:65536 -- tar xpf - -J --totals
--one-file-system -p --sparse --numeric-owner --acls --xattrs
'--xattrs-include=user.*' '--xattrs-include=security.capability'
'--warning=no-file-ignored' '--warning=no-xattr-write' -C
/var/lib/lxc/115/rootfs --skip-old-files --anchored --exclude './dev/*''
failed: exit code 2

*Proposed fix/solution:*
Change the exclude pattern in the tar extraction line above from './dev/*'
to 'dev/*'

This exclude pattern can be currently found in two locations:
  - /usr/share/perl5/PVE/LXC/Create.pm
  - /usr/share/lxc/templates/lxc-local (shell script)

*Success container creation message (after the fix):*
Task viewer: CT 116 - Create
OutputStatus
Stop
Download
Logical volume "vm-116-disk-0" created.
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 35681b1e-4220-4d2a-9e1a-f2e17fc16806
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
extracting archive
'/var/lib/vz/template/cache/jammy-server-cloudimg-amd64-root.tar.xz'
(extraction command is 'ARRAY(0x64ac4ede7218)')
Total bytes read: 1140961280 (1.1GiB, 51MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_dsa_key' - this may take some time ...
done: SHA256:ft9IuYd6J/OiZVN2acYNe20dBADc7nS9kZS4ihD1iDA root@test
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:Nz+J528gr0ZPQARwvHx6lZgo1hElMxOfuzmFbub3inM root@test
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:YRUD2eS1M65WHe4+808sopNtOwGthDM9Qg96O5ljxqw root@test
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:nLDsxaLs4/VoUjpo7MLlPhKLsssLAlNkV8nOhOUP1nk root@test
TASK OK

Prior to the fix, people were repackaging the root filesystem tarballs
found on the internet to exclude the `dev` folder, as evident here:
https://discuss.linuxcontainers.org/t/simple-script-to-convert-any-gnu-linux-machine-into-a-proxmox-lxc-container/10339
and here https://github.com/my5t3ry/machine-to-proxmox-lxc-ct-converter
which would be unnecessary after this fix.

Regards,
Orwa.



ReplyForward
Add reaction

--===============7068927186848279252==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

--===============7068927186848279252==--