public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Stefan Reiter <s.reiter@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-restore-vm-data 03/22] initial commit
Date: Tue, 16 Feb 2021 18:06:51 +0100	[thread overview]
Message-ID: <20210216170710.31767-4-s.reiter@proxmox.com> (raw)
In-Reply-To: <20210216170710.31767-1-s.reiter@proxmox.com>

proxmox-restore-vm-data provides means to build a debian package
containing a minimalistic Linux kernel and a corresponding initramfs
image for use in a file-restore VM.

Launched with QEMU/KVM, it boots in 1.6 seconds to userspace (on AMD
2700X) and has a minimal attack surface (no network stack other than
virtio-vsock, no auxiliary device support (USB, etc...), userspace
written in Rust) as opposed to mounting backup archives directly on the
host.

Since our Rust binaries are currently not fully statically linked, we
need to include some libraries into the initramfs as well. This is done
in 'build_initramfs.sh'.

A minimal /init is included as a Rust binary (init-shim-rs), doing only
the bare-minimum userspace setup before handing over control to the
file-restore daemon (see 'proxmox-backup' repository).

The debian package comes with a 'activate-noawait
pbs-file-restore-initramfs' trigger activation to rebuild the cached
initramfs when the base image shipped here updates. This is taken care
of by proxmox-file-restore.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---

Brand new git repo! I called it proxmox-restore-vm-data for lack of any smarter
ideas, open for better names :)

I also decided to include the 5.10 kernel and ZFS 2.0.3 from current pve-kernel
repository pretty last-minute, it seems to work fine though (ZFS isn't used atm
anyway).


 .gitignore                                    |   9 ++
 .gitmodules                                   |   6 +
 Makefile                                      | 103 +++++++++++++
 build_initramfs.sh                            |  42 +++++
 config-base                                   | 144 ++++++++++++++++++
 debian/changelog                              |   6 +
 debian/compat                                 |   1 +
 debian/control                                |  34 +++++
 debian/copyright                              |  22 +++
 debian/install                                |   2 +
 debian/rules                                  |  13 ++
 debian/triggers                               |   1 +
 init-shim-rs/Cargo.lock                       |  51 +++++++
 init-shim-rs/Cargo.toml                       |   9 ++
 init-shim-rs/src/main.rs                      | 122 +++++++++++++++
 ...-OVERRIDE-do-not-build-xr-usb-serial.patch |  30 ++++
 ...2-FIXUP-syntax-error-in-Ubuntu-Sauce.patch |  26 ++++
 submodules/ubuntu-hirsute                     |   1 +
 submodules/zfsonlinux                         |   1 +
 19 files changed, 623 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 .gitmodules
 create mode 100644 Makefile
 create mode 100755 build_initramfs.sh
 create mode 100644 config-base
 create mode 100644 debian/changelog
 create mode 100644 debian/compat
 create mode 100644 debian/control
 create mode 100644 debian/copyright
 create mode 100644 debian/install
 create mode 100755 debian/rules
 create mode 100644 debian/triggers
 create mode 100644 init-shim-rs/Cargo.lock
 create mode 100644 init-shim-rs/Cargo.toml
 create mode 100644 init-shim-rs/src/main.rs
 create mode 100644 patches/kernel/0001-OVERRIDE-do-not-build-xr-usb-serial.patch
 create mode 100644 patches/kernel/0002-FIXUP-syntax-error-in-Ubuntu-Sauce.patch
 create mode 160000 submodules/ubuntu-hirsute
 create mode 160000 submodules/zfsonlinux

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..d331656
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,9 @@
+build/
+init-shim-rs/target/
+
+*.deb
+*.dsc
+*.buildinfo
+*.changes
+*.prepared
+*.tar.gz
diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 0000000..fdd2bb0
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,6 @@
+[submodule "submodules/zfsonlinux"]
+	path = submodules/zfsonlinux
+	url = git://git.proxmox.com/git/mirror_zfs.git
+[submodule "submodules/ubuntu-hirsute"]
+	path = submodules/ubuntu-hirsute
+	url = git://git.proxmox.com/git/mirror_ubuntu-hirsute-kernel.git
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..2276f4b
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,103 @@
+include /usr/share/dpkg/pkg-info.mk
+include /usr/share/dpkg/architecture.mk
+
+PACKAGE=proxmox-restore-vm-data
+
+BUILDDIR=build
+INITRAMFS_BUILDDIR=build/initramfs
+
+ZFSONLINUX_SUBMODULE=submodules/zfsonlinux
+KERNEL_SUBMODULE=submodules/ubuntu-hirsute
+SHIM_DIR=init-shim-rs
+
+KERNEL_IMG=${BUILDDIR}/${KERNEL_SUBMODULE}/arch/x86/boot/bzImage
+INITRAMFS_IMG=${INITRAMFS_BUILDDIR}/initramfs.img
+
+CONFIG=config-base
+
+RUST_SRC=$(wildcard ${SHIM_DIR}/**/*.rs) ${SHIM_DIR}/Cargo.toml
+
+DEB=${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}_${DEB_BUILD_ARCH}.deb
+DSC=${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}.dsc
+
+all: deb
+
+submodules.prepared:
+	git submodule update --init ${KERNEL_SUBMODULE}
+	git submodule update --init --recursive ${ZFSONLINUX_SUBMODULE}
+	touch $@
+
+${BUILDDIR}.prepared: submodules.prepared ${CONFIG}
+	rm -rf ${BUILDDIR}
+	mkdir -p ${BUILDDIR}
+	cp -a submodules debian patches ${BUILDDIR}/
+	cp ${CONFIG} ${BUILDDIR}/${KERNEL_SUBMODULE}
+	cd ${BUILDDIR}/${KERNEL_SUBMODULE}; \
+		for p in ../../patches/kernel/*.patch; do \
+			patch -Np1 < $$p; \
+		done
+	touch $@
+
+kernel.prepared: ${BUILDDIR}.prepared
+	cd ${BUILDDIR}/${KERNEL_SUBMODULE}; \
+		KCONFIG_ALLCONFIG=${CONFIG} make allnoconfig && \
+		make -j$(nproc) prepare scripts
+	touch $@
+
+zfs.prepared: kernel.prepared
+	cd ${BUILDDIR}/${ZFSONLINUX_SUBMODULE}; \
+		sh autogen.sh && \
+		./configure \
+			--enable-linux-builtin \
+			--with-linux=../../${KERNEL_SUBMODULE} \
+			--with-linux-obj=../../${KERNEL_SUBMODULE} && \
+		./copy-builtin ../../${KERNEL_SUBMODULE}
+	# only now can we enable CONFIG_ZFS
+	cd ${BUILDDIR}/${KERNEL_SUBMODULE}; \
+		./scripts/config -e CONFIG_ZFS
+	touch $@
+
+${KERNEL_IMG}: zfs.prepared
+	cd ${BUILDDIR}/${KERNEL_SUBMODULE}; \
+	    make -j$(nproc)
+	mv ${BUILDDIR}/${KERNEL_SUBMODULE}/arch/x86/boot/bzImage ${BUILDDIR}/
+
+${INITRAMFS_IMG}: ${BUILDDIR}.prepared ${RUST_SRC} build_initramfs.sh
+	cd ${SHIM_DIR}; cargo build --release
+	sh build_initramfs.sh
+
+.PHONY: dinstall
+dinstall: deb
+	dpkg -i ${DEB}
+
+.PHONY: deb
+deb: ${DEB}
+${DEB}: ${KERNEL_IMG} ${INITRAMFS_IMG}
+	cd ${BUILDDIR}; dpkg-buildpackage -b -us -uc
+	lintian ${DEB}
+
+.PHONY: dsc
+dsc: ${DSC}
+${DSC}: ${KERNEL_IMG} ${INITRAMFS_IMG}
+	cd ${BUILDDIR}; dpkg-buildpackage -S -us -uc -d
+	lintian ${DSC}
+
+.PHONY: upload
+upload: ${DEB}
+	tar cf - ${DEB} | ssh -X repoman@repo.proxmox.com upload --product pbs --dist buster
+	tar cf - ${DEB} | ssh -X repoman@repo.proxmox.com upload --product pve --dist buster
+
+.PHONY: test-run
+test-run: ${KERNEL_IMG} ${INITRAMFS_IMG}
+	# note: this will always fail since /proxmox-restore-daemon is not
+	# included in the initramfs, but it can be used to test the
+	# kernel/init-shim-rs builds
+	qemu-system-x86_64 -serial stdio -vnc none -enable-kvm \
+		-kernel build/${KERNEL_SUBMODULE}/arch/x86/boot/bzImage \
+		-initrd build/initramfs/initramfs.img
+
+.PHONY: clean
+clean:
+	rm -rf *~ ${BUILDDIR} ${INITRAMFS_BUILDDIR} *.prepared
+	rm -f ${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}.tar.gz
+	rm -f *.deb *.changes *.buildinfo *.dsc
diff --git a/build_initramfs.sh b/build_initramfs.sh
new file mode 100755
index 0000000..72bb483
--- /dev/null
+++ b/build_initramfs.sh
@@ -0,0 +1,42 @@
+#!/bin/sh
+
+set -e
+
+ROOT="root"
+BUILDDIR="build/initramfs"
+INIT="../../init-shim-rs/target/release/init-shim-rs"
+
+PKGS=" \
+    libc6:amd64=2.28-10 \
+    libgcc1:amd64=1:8.3.0-6 \
+    libstdc++6:amd64=8.3.0-6 \
+    libssl1.1:amd64=1.1.1d-0+deb10u4 \
+    libattr1:amd64=1:2.4.48-4 \
+    libacl1:amd64=2.2.53-4
+"
+
+echo "Using build dir: $BUILDDIR"
+rm -rf "$BUILDDIR"
+mkdir -p "$BUILDDIR"
+cd "$BUILDDIR"
+mkdir "$ROOT"
+
+# add necessary packages to initramfs
+for pkg in $PKGS; do
+    apt-get download "$pkg"
+    dpkg-deb -x ./*.deb "$ROOT"
+    rm ./*.deb
+done
+
+rm -rf ${ROOT:?}/usr/share # contains only docs and debian stuff
+
+cp $INIT "$ROOT/init"
+chmod a+x "$ROOT/init" # just to be sure
+
+# tell daemon it's running in the correct environment
+touch "$ROOT/restore-vm-marker"
+
+fakeroot -- sh -c "
+    cd '$ROOT';
+    find . -print0 | cpio --null -oV --format=newc -F ../initramfs.img
+"
diff --git a/config-base b/config-base
new file mode 100644
index 0000000..db52460
--- /dev/null
+++ b/config-base
@@ -0,0 +1,144 @@
+CONFIG_LOCALVERSION="-pbs-restore"
+
+# kernel commandline override
+CONFIG_CMDLINE_BOOL=y
+CONFIG_CMDLINE="console=ttyS0"
+
+# NOTE: ZFS will be enabled from Makefile, since we can only activate it after
+# 'copy-builtin' creates the necessary Kconfig in the kernel tree.
+# CONFIG_ZFS=y
+
+# in case we crash the kernel, so we can at least read the stacktraces
+CONFIG_KALLSYMS=y
+
+# CPU settings
+CONFIG_64BIT=y
+CONFIG_MMU=y
+CONFIG_SMP=y
+CONFIG_X86_X2APIC=y
+CONFIG_ACPI=y
+# not super necessary, but avoids a warning
+CONFIG_RETPOLINE=y
+
+# basic kernel features
+CONFIG_MULTIUSER=y
+CONFIG_POSIX_TIMERS=y
+CONFIG_BUG=y
+CONFIG_FUTEX=y
+CONFIG_EPOLL=y
+CONFIG_AIO=y
+CONFIG_BINFMT_ELF=y
+CONFIG_ELFCORE=y
+CONFIG_PRINTK=y
+CONFIG_EVENTFD=y
+CONFIG_MODULES=y
+
+# initramfs support
+CONFIG_TMPFS=y
+CONFIG_BLK_DEV_INITRD=y
+
+# paravirt acceleration
+CONFIG_KVM_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_PARAVIRT_CLOCK=y
+CONFIG_PARAVIRT_SPINLOCKS=y
+CONFIG_VIRTIO_MENU=y
+CONFIG_VIRTIO=y
+
+# enable terminal on serial for debugging/logging
+CONFIG_TTY=y
+CONFIG_VT=y
+CONFIG_VT_CONSOLE=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_INPUT_KEYBOARD=y
+
+# vsock support
+CONFIG_PCI=y
+CONFIG_NET=y
+CONFIG_UNIX=y
+CONFIG_VSOCKETS=y
+CONFIG_VIRTIO_VSOCKETS=y
+CONFIG_VIRTIO_VSOCKETS_COMMON=y
+
+# block device support, especially virtio-scsi/blk
+CONFIG_BLOCK=y
+CONFIG_BLK_DEV=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_BLK_DEV_SR=y
+CONFIG_VIRT_DRIVERS=y
+CONFIG_SCSI=y
+CONFIG_SCSI_VIRTIO=y
+CONFIG_BLK_MQ_VIRTIO=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_VIRTIO_MMIO=y
+CONFIG_VIRTIO_PCI=y
+
+# md/LVM/device-mapper support
+CONFIG_DM_CRYPT=y
+CONFIG_DM_MIRROR=y
+CONFIG_DM_RAID=y
+CONFIG_DM_SNAPSHOT=y
+CONFIG_DM_THIN_PROVISIONING=y
+CONFIG_DM_UNSTRIPED=y
+CONFIG_MD=y
+CONFIG_MD_AUTODETECT=y
+CONFIG_MD_LINEAR=y
+CONFIG_MD_RAID0=y
+CONFIG_MD_RAID1=y
+CONFIG_MD_RAID10=y
+CONFIG_MD_RAID456=y
+CONFIG_BLK_DEV_MD=y
+CONFIG_BLK_DEV_DM=y
+
+# basic fs features
+CONFIG_FS_POSIX_ACL=y
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_PROC_FS=y
+# proc_sysctl is necessary for ZFS, it panics otherwise
+CONFIG_PROC_SYSCTL=y
+CONFIG_SYSFS=y
+CONFIG_NLS=y
+CONFIG_NLS_UTF8=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_MSDOS_PARTITION=y
+CONFIG_EFI_PARTITION=y
+
+# filesystem support
+CONFIG_MISC_FILESYSTEMS=y
+CONFIG_EXT2_FS=y
+CONFIG_EXT2_FS_XATTR=y
+CONFIG_EXT2_FS_POSIX_ACL=y
+CONFIG_EXT3_FS=y
+CONFIG_EXT3_FS_XATTR=y
+CONFIG_EXT3_FS_POSIX_ACL=y
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_FS_XATTR=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_REISERFS_FS=y
+CONFIG_REISERFS_FS_XATTR=y
+CONFIG_REISERFS_FS_POSIX_ACL=y
+CONFIG_JFS_FS=y
+CONFIG_JFS_POSIX_ACL=y
+CONFIG_XFS_FS=y
+CONFIG_XFS_POSIX_ACL=y
+CONFIG_BTRFS_FS=y
+CONFIG_BTRFS_FS_POSIX_ACL=y
+CONFIG_F2FS_FS=y
+CONFIG_F2FS_FS_XATTR=y
+CONFIG_F2FS_FS_POSIX_ACL=y
+CONFIG_F2FS_FS_COMPRESSION=y
+CONFIG_F2FS_FS_LZO=y
+CONFIG_F2FS_FS_LZ4=y
+CONFIG_F2FS_FS_ZSTD=y
+CONFIG_F2FS_FS_LZORLE=y
+CONFIG_HFS_FS=y
+CONFIG_HFSPLUS_FS=y
+CONFIG_BEFS_FS=y
+CONFIG_SYSV_FS=y
+CONFIG_UFS_FS=y
+CONFIG_ISO9660_FS=y
+CONFIG_NTFS_FS=y
+CONFIG_MSDOS_FS=y
+CONFIG_VFAT_FS=y
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..a389c1b
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,6 @@
+proxmox-restore-vm-data (1.0.0-1) pbs; urgency=medium
+
+  * initial release
+
+ -- Proxmox Support Team <support@proxmox.com>  Tue, 16 Feb 2021 16:49:20 +0100
+
diff --git a/debian/compat b/debian/compat
new file mode 100644
index 0000000..f599e28
--- /dev/null
+++ b/debian/compat
@@ -0,0 +1 @@
+10
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..ff128de
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,34 @@
+Source: proxmox-restore-vm-data
+Section: admin
+Priority: optional
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Build-Depends: asciidoc-base,
+               automake,
+               bc,
+               bison,
+               cpio,
+               debhelper (>= 10~),
+               dh-python,
+               flex,
+               gcc (>= 8.3.0-6),
+               git,
+               libdw-dev,
+               libelf-dev,
+               libtool,
+               lintian,
+               perl-modules,
+               python-minimal,
+               sed,
+               sphinx-common,
+               tar,
+               xmlto,
+               zlib1g-dev,
+Standards-Version: 4.5.1
+Homepage: https://www.proxmox.com
+
+Package: proxmox-restore-vm-data
+Architecture: amd64
+Recommends: proxmox-file-restore
+Description: VM kernel/initramfs images for PBS single file restore
+ Preconfigured images used as base for single file restore of PBS backup
+ snapshots. Useless on their own, use together with proxmox-file-restore.
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 0000000..ce2c0a7
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,22 @@
+Copyright (C) 2020 Proxmox Server Solutions GmbH
+
+This package contains a version of a linux kernel binary image (including
+patches by Ubuntu/Canonical and Proxmox) with built-in ZFS support.
+
+Linux is copyrighted by Linus Torvalds and others and distributed under the
+terms of the GPL-2.0 license.
+The complete text of the GNU General Public License can be found in
+`/usr/share/common-licenses/GPL-2'.
+
+ZFS is licensed under the Common Development and Distribution License (CDDL).
+
+The shipped initramfs image contains several files from other debian packages.
+For their copyright notices see the respective packages in the versions
+mentioned in build_initramfs.sh.
+
+The initramfs also contains a rust binary as /init, built from 'init-shim-rs'
+available in this package's sources. This binary is released under the terms of
+the AGPLv3 by Proxmox Server Solutions GmbH.
+
+This package was put together by Proxmox Server Solutions GmbH
+<support@proxmox.com>.
diff --git a/debian/install b/debian/install
new file mode 100644
index 0000000..5e83453
--- /dev/null
+++ b/debian/install
@@ -0,0 +1,2 @@
+bzImage /usr/lib/x86_64-linux-gnu/proxmox-backup/file-restore/
+initramfs/initramfs.img /usr/lib/x86_64-linux-gnu/proxmox-backup/file-restore/
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..955dd78
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,13 @@
+#!/usr/bin/make -f
+# -*- makefile -*-
+# Sample debian/rules that uses debhelper.
+# This file was originally written by Joey Hess and Craig Small.
+# As a special exception, when this file is copied by dh-make into a
+# dh-make output file, you may use that output file without restriction.
+# This special exception was added by Craig Small in version 0.37 of dh-make.
+
+# Uncomment this to turn on verbose mode.
+#export DH_VERBOSE=1
+
+%:
+	dh $@
diff --git a/debian/triggers b/debian/triggers
new file mode 100644
index 0000000..a7abac5
--- /dev/null
+++ b/debian/triggers
@@ -0,0 +1 @@
+activate-noawait pbs-file-restore-initramfs
diff --git a/init-shim-rs/Cargo.lock b/init-shim-rs/Cargo.lock
new file mode 100644
index 0000000..a293b3c
--- /dev/null
+++ b/init-shim-rs/Cargo.lock
@@ -0,0 +1,51 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+[[package]]
+name = "anyhow"
+version = "1.0.34"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "bf8dcb5b4bbaa28653b647d8c77bd4ed40183b48882e130c1f1ffb73de069fd7"
+
+[[package]]
+name = "bitflags"
+version = "1.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
+
+[[package]]
+name = "cc"
+version = "1.0.62"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f1770ced377336a88a67c473594ccc14eca6f4559217c34f64aac8f83d641b40"
+
+[[package]]
+name = "cfg-if"
+version = "0.1.10"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
+
+[[package]]
+name = "init-shim-rs"
+version = "1.0.0"
+dependencies = [
+ "anyhow",
+ "nix",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.80"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4d58d1b70b004888f764dfbf6a26a3b0342a1632d33968e4a179d8011c760614"
+
+[[package]]
+name = "nix"
+version = "0.19.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "85db2feff6bf70ebc3a4793191517d5f0331100a2f10f9bf93b5e5214f32b7b7"
+dependencies = [
+ "bitflags",
+ "cc",
+ "cfg-if",
+ "libc",
+]
diff --git a/init-shim-rs/Cargo.toml b/init-shim-rs/Cargo.toml
new file mode 100644
index 0000000..013395c
--- /dev/null
+++ b/init-shim-rs/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "init-shim-rs"
+version = "1.0.0"
+authors = ["Stefan Reiter <s.reiter@proxmox.com>"]
+edition = "2018"
+
+[dependencies]
+anyhow = "1.0"
+nix = "0.19"
diff --git a/init-shim-rs/src/main.rs b/init-shim-rs/src/main.rs
new file mode 100644
index 0000000..89aff7b
--- /dev/null
+++ b/init-shim-rs/src/main.rs
@@ -0,0 +1,122 @@
+use anyhow::Error;
+use std::ffi::CStr;
+use std::fs;
+
+const URANDOM_MAJ: u64 = 1;
+const URANDOM_MIN: u64 = 9;
+
+/// Set up a somewhat normal linux userspace environment before starting the restore daemon, and
+/// provide error messages to the user if doing so fails.
+///
+/// This is supposed to run as /init in an initramfs image.
+fn main() {
+    println!("[init-shim] beginning user space setup");
+
+    // /dev is mounted automatically
+    wrap_err("mount /sys", || do_mount("/sys", "sysfs"));
+    wrap_err("mount /proc", || do_mount("/proc", "proc"));
+
+    // make device nodes required by daemon
+    wrap_err("mknod /dev/urandom", || {
+        do_mknod("/dev/urandom", URANDOM_MAJ, URANDOM_MIN)
+    });
+
+    let uptime = read_uptime();
+    println!("[init-shim] reached daemon start after {:.2}s", uptime);
+
+    do_run("/proxmox-restore-daemon");
+}
+
+fn do_mount(target: &str, fstype: &str) -> Result<(), Error> {
+    use nix::mount::{mount, MsFlags};
+    fs::create_dir(target)?;
+    let none_type: Option<&CStr> = None;
+    mount(
+        none_type,
+        target,
+        Some(fstype),
+        MsFlags::MS_NOSUID | MsFlags::MS_NOEXEC,
+        none_type,
+    )?;
+    Ok(())
+}
+
+fn do_mknod(path: &str, maj: u64, min: u64) -> Result<(), Error> {
+    use nix::sys::stat;
+    let dev = stat::makedev(maj, min);
+    stat::mknod(path, stat::SFlag::S_IFCHR, stat::Mode::S_IRWXU, dev)?;
+    Ok(())
+}
+
+fn read_uptime() -> f32 {
+    let uptime = wrap_err("read /proc/uptime", || {
+        fs::read_to_string("/proc/uptime").map_err(|e| e.into())
+    });
+    // this can never fail on a sane kernel, so just unwrap
+    uptime
+        .split_ascii_whitespace()
+        .next()
+        .unwrap()
+        .parse()
+        .unwrap()
+}
+
+fn do_run(cmd: &str) -> ! {
+    use std::io::ErrorKind;
+    use std::process::Command;
+
+    let spawn_res = Command::new(cmd).env("RUST_BACKTRACE", "1").spawn();
+
+    match spawn_res {
+        Ok(mut child) => {
+            let res = wrap_err("wait failed", || child.wait().map_err(|e| e.into()));
+            error(&format!(
+                "child process {} (pid={} exitcode={}) exited unexpectedly, check log for more info",
+                cmd,
+                child.id(),
+                res.code().unwrap_or(-1),
+            ));
+        }
+        Err(err) if err.kind() == ErrorKind::NotFound => {
+            error(&format!(
+                concat!(
+                    "{} missing from image.\n",
+                    "This initramfs should only be run with proxmox-file-restore!"
+                ),
+                cmd
+            ));
+        }
+        Err(err) => {
+            error(&format!(
+                "unexpected error during start of {}: {}",
+                cmd, err
+            ));
+        }
+    }
+}
+
+fn wrap_err<R, F: FnOnce() -> Result<R, Error>>(op: &str, f: F) -> R {
+    match f() {
+        Ok(r) => r,
+        Err(e) => error(&format!("operation '{}' failed: {}", op, e)),
+    }
+}
+
+fn error(msg: &str) -> ! {
+    use nix::sys::reboot;
+
+    println!("\n--------");
+    println!("ERROR: Init shim failed\n");
+    println!("{}", msg);
+    println!("--------\n");
+
+    // in case a fatal error occurs we shut down the VM, there's no sense in continuing and this
+    // will certainly alert whoever started us up in the first place
+    let err = reboot::reboot(reboot::RebootMode::RB_POWER_OFF).unwrap_err();
+    println!("'reboot' syscall failed: {} - cannot continue", err);
+
+    // in case 'reboot' fails just loop forever
+    loop {
+        std::thread::sleep(std::time::Duration::from_secs(600));
+    }
+}
diff --git a/patches/kernel/0001-OVERRIDE-do-not-build-xr-usb-serial.patch b/patches/kernel/0001-OVERRIDE-do-not-build-xr-usb-serial.patch
new file mode 100644
index 0000000..7873602
--- /dev/null
+++ b/patches/kernel/0001-OVERRIDE-do-not-build-xr-usb-serial.patch
@@ -0,0 +1,30 @@
+From 4cf77185b43a29ad2d70749648cac83330030cf9 Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Tue, 17 Nov 2020 14:42:52 +0100
+Subject: [PATCH] OVERRIDE: do not build xr-usb-serial
+
+We don't have USB support in the kernel, so this will fail - and for
+some reason there's no Kconfig setting for this...
+
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+---
+ ubuntu/Makefile | 3 ---
+ 1 file changed, 3 deletions(-)
+
+diff --git a/ubuntu/Makefile b/ubuntu/Makefile
+index 67c6d5b98b53..6e7264845b66 100644
+--- a/ubuntu/Makefile
++++ b/ubuntu/Makefile
+@@ -19,9 +19,6 @@ obj-$(CONFIG_HIO)             += hio/
+ ##
+ ##
+ ##
+-ifeq ($(ARCH),x86)
+-obj-y				+= xr-usb-serial/
+-endif
+ ##
+ ##
+ ##
+-- 
+2.20.1
+
diff --git a/patches/kernel/0002-FIXUP-syntax-error-in-Ubuntu-Sauce.patch b/patches/kernel/0002-FIXUP-syntax-error-in-Ubuntu-Sauce.patch
new file mode 100644
index 0000000..6273847
--- /dev/null
+++ b/patches/kernel/0002-FIXUP-syntax-error-in-Ubuntu-Sauce.patch
@@ -0,0 +1,26 @@
+From 2c972569ef5b641846773bee3b3a0191ba66165e Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Tue, 16 Feb 2021 17:14:41 +0100
+Subject: [PATCH] FIXUP: syntax error in Ubuntu Sauce
+
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+---
+ include/linux/audit.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/include/linux/audit.h b/include/linux/audit.h
+index 55cc03c1bed8..8f84c9503827 100644
+--- a/include/linux/audit.h
++++ b/include/linux/audit.h
+@@ -253,7 +253,7 @@ static inline void audit_log_path_denied(int type, const char *operation)
+ static inline void audit_log_lsm(struct lsmblob *blob, bool exiting)
+ { }
+ static inline int audit_log_task_context(struct audit_buffer *ab,
+-					 struct lsmblob *blob);
++					 struct lsmblob *blob)
+ {
+ 	return 0;
+ }
+-- 
+2.20.1
+
diff --git a/submodules/ubuntu-hirsute b/submodules/ubuntu-hirsute
new file mode 160000
index 0000000..01f2ad6
--- /dev/null
+++ b/submodules/ubuntu-hirsute
@@ -0,0 +1 @@
+Subproject commit 01f2ad60c19fc07666c3cad5e6f527bc46af6303
diff --git a/submodules/zfsonlinux b/submodules/zfsonlinux
new file mode 160000
index 0000000..9f5f866
--- /dev/null
+++ b/submodules/zfsonlinux
@@ -0,0 +1 @@
+Subproject commit 9f5f86626620c52ad1bebf27d17cece6a28d39a0
-- 
2.20.1





  parent reply	other threads:[~2021-02-16 17:07 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-16 17:06 [pbs-devel] [PATCH 00/22] Single file restore for VM images Stefan Reiter
2021-02-16 17:06 ` [pbs-devel] [PATCH pxar 01/22] decoder/aio: add contents() and content_size() calls Stefan Reiter
2021-02-17  7:56   ` Wolfgang Bumiller
2021-02-16 17:06 ` [pbs-devel] [PATCH pxar 02/22] decoder: add peek() Stefan Reiter
2021-02-17  8:20   ` Wolfgang Bumiller
2021-02-17  8:38     ` Stefan Reiter
2021-02-16 17:06 ` Stefan Reiter [this message]
2021-03-15 18:35   ` [pbs-devel] applied: [PATCH proxmox-restore-vm-data 03/22] initial commit Thomas Lamprecht
2021-03-16 15:33     ` Stefan Reiter
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 04/22] api2/admin/datastore: refactor list_dir_content in catalog_reader Stefan Reiter
2021-02-17  7:50   ` [pbs-devel] applied: " Thomas Lamprecht
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 05/22] api2/admin/datastore: accept "/" as path for root Stefan Reiter
2021-02-17  7:50   ` [pbs-devel] applied: " Thomas Lamprecht
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 06/22] api2/admin/datastore: refactor create_zip into pxar/extract Stefan Reiter
2021-02-17  7:50   ` [pbs-devel] applied: " Thomas Lamprecht
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 07/22] pxar/extract: add extract_sub_dir Stefan Reiter
2021-02-17  7:51   ` [pbs-devel] applied: " Thomas Lamprecht
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 08/22] pxar/extract: add sequential variants to create_zip, extract_sub_dir Stefan Reiter
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 09/22] client: extract common functions to proxmox_client_tools module Stefan Reiter
2021-02-17  6:49   ` Dietmar Maurer
2021-02-17  7:58     ` Stefan Reiter
2021-02-17  8:50       ` Dietmar Maurer
2021-02-17  9:47         ` Stefan Reiter
2021-02-17 10:12           ` Dietmar Maurer
2021-02-17  9:13   ` [pbs-devel] applied: " Dietmar Maurer
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 10/22] proxmox_client_tools: extract 'key' from client module Stefan Reiter
2021-02-17  9:11   ` Dietmar Maurer
2021-02-16 17:06 ` [pbs-devel] [PATCH proxmox-backup 11/22] file-restore: add binary and basic commands Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 12/22] file-restore: allow specifying output-format Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 13/22] rest: implement tower service for UnixStream Stefan Reiter
2021-02-17  6:52   ` [pbs-devel] applied: " Dietmar Maurer
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 14/22] client: add VsockClient to connect to virtio-vsock VMs Stefan Reiter
2021-02-17  7:24   ` [pbs-devel] applied: " Dietmar Maurer
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 15/22] file-restore-daemon: add binary with virtio-vsock API server Stefan Reiter
2021-02-17 10:17   ` Dietmar Maurer
2021-02-17 10:25   ` Dietmar Maurer
2021-02-17 10:30     ` Stefan Reiter
2021-02-17 11:13       ` Dietmar Maurer
2021-02-17 11:26   ` Dietmar Maurer
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 16/22] file-restore-daemon: add watchdog module Stefan Reiter
2021-02-17 10:52   ` Wolfgang Bumiller
2021-02-17 11:14     ` Stefan Reiter
2021-02-17 11:29       ` Wolfgang Bumiller
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 17/22] file-restore-daemon: add disk module Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 18/22] file-restore: add basic VM/block device support Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 19/22] file-restore: improve logging of VM with logrotate Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 20/22] debian/client: add postinst hook to rebuild file-restore initramfs Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 21/22] file-restore(-daemon): implement list API Stefan Reiter
2021-02-16 17:07 ` [pbs-devel] [PATCH proxmox-backup 22/22] file-restore: add 'extract' command for VM file restore Stefan Reiter
2021-02-16 17:11 ` [pbs-devel] [PATCH 00/22] Single file restore for VM images Stefan Reiter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210216170710.31767-4-s.reiter@proxmox.com \
    --to=s.reiter@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal