public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* Re: [pbs-devel] building proxmox-backup-qemu
@ 2020-11-30 19:11 Niko Fellner
  2020-12-01 10:04 ` Fabian Grünbichler
  0 siblings, 1 reply; 5+ messages in thread
From: Niko Fellner @ 2020-11-30 19:11 UTC (permalink / raw)
  To: pbs-devel

> I just added it to the build-depends as well.
> [...]
> diff --git a/Cargo.toml b/Cargo.toml
> [...]
> with proxmox-backup checked out next to proxmox-backup-qemu

Thanks, I've applied your patch, and obviously the cbindgen package was installed, but I still get the same error. Here is my log:


After successfully building the latest proxmox-backup in ~/proxmox-backup:
> root@debian:~/proxmox-backup# cd ~ && \
> git clone git://git.proxmox.com/git/proxmox-backup-qemu.git && \
> cd proxmox-backup-qemu && nano Cargo.toml

I applied your patch (change proxmox version to "0.8.0" and use proxmox-backup from path = "../proxmox-backup" instead of git://git.proxmox.com...).

> root@debian:~/proxmox-backup-qemu# mk-build-deps -ir
> dh_testdir
> dh_testroot
> dh_prep
> dh_testdir
> dh_testroot
> dh_install
> dh_installdocs
> dh_installchangelogs
> dh_compress
> dh_fixperms
> dh_installdeb
> dh_gencontrol
> dh_md5sums
> dh_builddeb
> dpkg-deb: Paket »proxmox-backup-qemu-build-deps« wird in »../proxmox-backup-qemu-build-deps_1.0.2-1_all.deb« gebaut.
> The package has been created.
> Attention, the package has been created in the current directory,
> not in ".." as indicated by the message above!
> Vormals nicht ausgewähltes Paket proxmox-backup-qemu-build-deps wird gewählt.
> (Lese Datenbank ... 247817 Dateien und Verzeichnisse sind derzeit installiert.)
> Vorbereitung zum Entpacken von proxmox-backup-qemu-build-deps_1.0.2-1_all.deb ...
> Entpacken von proxmox-backup-qemu-build-deps (1.0.2-1) ...
> Paketlisten werden gelesen... Fertig
> Abhängigkeitsbaum wird aufgebaut.       
> Statusinformationen werden eingelesen.... Fertig
> Abhängigkeiten werden korrigiert ...Starting pkgProblemResolver with broken count: 0
> Starting 2 pkgProblemResolver with broken count: 0
> Done
>  Fertig
> Starting pkgProblemResolver with broken count: 0
> Starting 2 pkgProblemResolver with broken count: 0
> Done
> Die folgenden zusätzlichen Pakete werden installiert:
>   cbindgen librust-bincode-dev
> Vorgeschlagene Pakete:
>   librust-bincode+i128-dev
> Die folgenden NEUEN Pakete werden installiert:
>   cbindgen librust-bincode-dev
> 0 aktualisiert, 2 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
> 1 nicht vollständig installiert oder entfernt.
> Es müssen 1.331 kB an Archiven heruntergeladen werden.
> Nach dieser Operation werden 5.264 kB Plattenplatz zusätzlich benutzt.
> Möchten Sie fortfahren? [J/n] Y
> Holen:1 http://deb.debian.org/debian buster/main amd64 cbindgen amd64 0.14.4-1~deb10u1 [1.316 kB]
> Holen:2 http://deb.debian.org/debian buster/main amd64 librust-bincode-dev amd64 1.0.1-1 [14,6 kB]
> Es wurden 1.331 kB in 0 s geholt (6.094 kB/s).  
> Vormals nicht ausgewähltes Paket cbindgen wird gewählt.
> (Lese Datenbank ... 247821 Dateien und Verzeichnisse sind derzeit installiert.)
> Vorbereitung zum Entpacken von .../cbindgen_0.14.4-1~deb10u1_amd64.deb ...
> Entpacken von cbindgen (0.14.4-1~deb10u1) ...
> Vormals nicht ausgewähltes Paket librust-bincode-dev:amd64 wird gewählt.
> Vorbereitung zum Entpacken von .../librust-bincode-dev_1.0.1-1_amd64.deb ...
> Entpacken von librust-bincode-dev:amd64 (1.0.1-1) ...
> cbindgen (0.14.4-1~deb10u1) wird eingerichtet ...
> librust-bincode-dev:amd64 (1.0.1-1) wird eingerichtet ...
> proxmox-backup-qemu-build-deps (1.0.2-1) wird eingerichtet ...
> Trigger für man-db (2.8.5-2) werden verarbeitet ...

> root@debian:~/proxmox-backup-qemu# make
> cargo build 
> error: no matching package named `cbindgen` found
> location searched: registry `https://github.com/rust-lang/crates.io-index`
> perhaps you meant: bindgen
> required by package `proxmox-backup-qemu v1.0.2 (/root/proxmox-backup-qemu)`
> make: *** [Makefile:22: all] Fehler 101

> root@debian:~/proxmox-backup-qemu# apt-get install cbindgen
> Paketlisten werden gelesen... Fertig
> Abhängigkeitsbaum wird aufgebaut.       
> Statusinformationen werden eingelesen.... Fertig
> cbindgen ist schon die neueste Version (0.14.4-1~deb10u1).
> cbindgen wurde als manuell installiert festgelegt.
> 0 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.

> root@debian:~/proxmox-backup-qemu# cargo build
> error: no matching package named `cbindgen` found
> location searched: registry `https://github.com/rust-lang/crates.io-index`
> perhaps you meant: bindgen
> required by package `proxmox-backup-qemu v1.0.2 (/root/proxmox-backup-qemu)`

> root@debian:~/proxmox-backup-qemu# cat /etc/apt/sources.list.d/proxmox-devel.list 
> deb http://download.proxmox.com/debian/devel/ buster main

> root@debian:~/proxmox-backup-qemu# ls -a /usr/share/cargo/registry/c
> cc-1.0.50/               clang-sys-0.29.3/        crossbeam-channel-0.4.2/
> cexpr-0.4.0/             clap-2.33.0/             crossbeam-utils-0.7.2/
> cfg-if-0.1.10/           crc32fast-1.2.0/         
> root@debian:~/proxmox-backup-qemu# ls -a /usr/share/cargo/registry/c^C

Is the "location searched: registry `https://github.com/rust-lang/crates.io-index`" correct?



^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: [pbs-devel] building proxmox-backup-qemu
@ 2020-12-02 19:55 Niko Fellner
  0 siblings, 0 replies; 5+ messages in thread
From: Niko Fellner @ 2020-12-02 19:55 UTC (permalink / raw)
  To: pbs-devel

> 3.) install librust-cbindgen-dev in addition to cbindgen (the latter is
> the binary, the former contains the source code to satisfy cargo that 
> the crate is available)

That was the missing part for me! Maybe you can add it to the dependencies aswell?

Thanks a lot! :)

Here is everything together:

# Basic setup
echo 'deb http://download.proxmox.com/debian/devel/ buster main' >> /etc/apt/sources.list.d/proxmox-devel.list
wget http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg && \
apt-get update && \ 
apt-get install -y devscripts debcargo clang

# Build PBS
apt-get update && apt-get dist-upgrade -y && \
git clone git://git.proxmox.com/git/proxmox-backup.git && cd proxmox-backup/ && \
#git checkout v1.0.3 && \
mk-build-deps -ir; mk-build-deps -ir && \
make && make install

# Build proxmox-backup-qemu
cd ~ && git clone git://git.proxmox.com/git/proxmox-backup-qemu.git && \
cd proxmox-backup-qemu && \
# Optionally: add some changes to the rust sourcecode :) \
sed -i 's/speed={:.2}MB/speed={:.2}MiB/' src/restore.rs && \
# Changes to Cargo.toml according to https://lists.proxmox.com/pipermail/pbs-devel/2020-November/001635.html \
sed -i 's/proxmox = { version = "0.7.0"/proxmox = { version = "0.8.0"/' Cargo.toml && \
sed -i 's/proxmox-backup = { git = "git:\/\/git.proxmox.com\/git\/proxmox-backup.git", tag = "v1.0.4" }/#proxmox-backup = { git = "git:\/\/git.proxmox.com\/git\/proxmox-backup.git", tag = "v1.0.4" }/' Cargo.toml && \
sed -i 's/#proxmox-backup = { path = "..\/proxmox-backup" }/proxmox-backup = { path = "..\/proxmox-backup" }/' Cargo.toml && \
apt-get install -y librust-cbindgen-dev && \
export BUILD_MODE=release && \
mk-build-deps -ir && make && make deb && make install

# You can use the .deb files in the proxmox-backup-qemu directory directly now or ship them to the target Proxmox installation
dpkg -i libproxmox-backup-qemu*



# Optionally - if you want to build a new pbs-restore binary (for me that's not neccessary; I'm happy with the libs from proxmox-backup-qemu):
# Build pve-qemu. Requires either proxmox-backup-qemu libs (see above) or a PVE installation
cd ~ && git clone git://git.proxmox.com/git/pve-qemu.git && \
cd pve-qemu && apt-get install -y libaio-dev libcap-ng-dev libfdt-dev libglusterfs-dev libiscsi-dev libjemalloc-dev \
libjpeg-dev libnuma-dev libpci-dev libpixman-1-dev librbd-dev libsdl1.2-dev \
libseccomp-dev libspice-protocol-dev libspice-server-dev libusb-1.0-0-dev libusbredirparser-dev texi2html texinfo xfslibs-dev && \
mk-build-deps -ir && make && \
apt-get install -y ceph-common numactl python-cephfs python-prettytable python-rados python-rbd python-requests python3-prettytable libboost-program-options1.67.0 libgoogle-perftools4 libradosstriper1 python-certifi python-chardet python-idna python-urllib3 python-chardet libcephfs2 python-pkg-resources python-six libtcmalloc-minimal4 && \
dpkg -i pve-qemu*.deb



^ permalink raw reply	[flat|nested] 5+ messages in thread
* [pbs-devel] building proxmox-backup-qemu
@ 2020-11-29 10:20 Niko Fellner
  2020-11-30 13:00 ` Fabian Grünbichler
  0 siblings, 1 reply; 5+ messages in thread
From: Niko Fellner @ 2020-11-29 10:20 UTC (permalink / raw)
  To: pbs-devel

[-- Attachment #1: Type: text/plain, Size: 2458 bytes --]

Hi,

I originally posted this question in #3163#c9, but here my question is better placed:

> This is because for restore, the binary is actually created from our patched
> QEMU source code which just links against the Rust library, see
> 'pbs-restore.c' in our QEMU sources (i.e. clone pve-qemu repo, then apply
> the patches in 'debian/patches/series' to the 'qemu' submodule).
>


So if I want to build the "pbs-restore" binary, I need to make "pve-qemu":

> cd ~ && git clone git://git.proxmox.com/git/pve-qemu.git && \
> cd pve-qemu && apt-get install libaio-dev libcap-ng-dev libfdt-dev \
> libglusterfs-dev libiscsi-dev libjemalloc-dev libjpeg-dev libnuma-dev \
> libpci-dev libpixman-1-dev librbd-dev libsdl1.2-dev libseccomp-dev \
> libspice-protocol-dev libspice-server-dev libusb-1.0-0-dev \
> libusbredirparser-dev texi2html texinfo xfslibs-dev && \
> mk-build-deps -ir && make


Which results in the binaries "/pve-qemu-kvm-5.1.0/pbs-restore" and "pve-qemu-kvm-5.1.0/debian/pve-qemu-kvm/usr/bin/pbs-restore" (What's their difference?)

Making pve-qemu requires "libproxmox-backup-qemu0".
"libproxmox-backup-qemu0" is either supplied by installing PVE...

> echo 'deb http://download.proxmox.com/debian/pve buster pvetest' >> \
> /etc/apt/sources.list.d/proxmox-devel.list && \
> apt-get update && apt-get dist-upgrade -y && \
> apt-get install proxmox-ve


... or by building it from source. I am interested about building it myself, of course. I guess this repository is what I need:
https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary
But when I do ...

> cd ~ && git clone git://git.proxmox.com/git/proxmox-backup-qemu.git && \
> cd proxmox-backup-qemu && \
> # checkout a commit which seems to be compatible for v1.0.3, see #3173#c1 \
> git checkout 1e45883f00ca8b0dd04c84f4ae33752e69cfc5bb && \
> # need to install PVE too meet the dependencies for the next one: \
> mk-build-deps -ir && \
> # I can't meet all dependencies for the next one: \
> make


...with the "make" I get the following error message:

> error: no matching package named `cbindgen` found
> location searched: registry `https://github.com/rust-lang/crates.io-index`
> perhaps you meant: bindgen
> required by package `proxmox-backup-qemu v1.0.0 (/root/proxmox-backup-qemu)`
> make: *** [Makefile:22: all] Fehler 101


Where do I get "cbindgen" from? What's the correct way to build proxmox-backup-qemu?

[-- Attachment #2: Type: text/html, Size: 7328 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-12-02 20:11 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-30 19:11 [pbs-devel] building proxmox-backup-qemu Niko Fellner
2020-12-01 10:04 ` Fabian Grünbichler
  -- strict thread matches above, loose matches on Subject: below --
2020-12-02 19:55 Niko Fellner
2020-11-29 10:20 Niko Fellner
2020-11-30 13:00 ` Fabian Grünbichler

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal