Re: [PVE-User] Proxmox Backup Server (beta)

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

On 14.07.20 23:17, Alexandre DERUMIER wrote:
>>> Proxmox Backup Server effectively does that too, but independent from the
>>> source storage. We always get the last backup index and only upload the chunks
>>> which changed. For running VMs dirty-bitmap is on to improve this (avoids
>>> reading of unchanged blocks) but it's only an optimization - the backup is
>>> incremental either way.
> 
> What happen if a vm or host crash ? (I think on clean shutdown, the dirty-bitmap is saved, but on failure ?)
> does it need to re-read all blocks to find diff ? or make a new full backup ?

There's never a new "full backup" as long as the PBS has at least one.
But yes, it needs to re-read everything to get the diff for the first
backup after the VM process starts, from then the tracking is active again.

> 
> Is it possible to read files inside a vm backup, without restoring it first ?
> (Don't have check vma format recently, but I think it was not possible because of out of orders blocks)

There's support for block and file level backup, CTs are using a file level
backup, you can then even browse the backup on the server (if it's not encrypted)

As said, there's a block backend driver for it in QEMU, Stefan made it with
Dietmar's libproxmox-backup-qemu0 library. So you should be able to get a backup
as block device over NBD and mount it, I guess. (did not tried that yet fully
myself).

> 
> I really think it could be great to add some storage snapshot feature in the future.

The storage would need to allow us diffing from the outside between the previous
snapshot and the current state though, not sure where that's possible in such
away that it could be integrated into PBS in a reasonable way.

The ceph RBD diff format wouldn't seem to bad, though:
https://docs.ceph.com/docs/master/dev/rbd-diff/

> For ceph, the backup speed is really faster because it's done a bigger block than 64K. (I think it's 4MB object).

We use 4MiB chunks for block-level backup by default too, for file-level they're
dynamic and scale between 64KiB and 4MiB.

> and also, I really need a lot of space for my backups, and I can't fill them in a single local storage. (don't want to play with multiple datastores)

What are your (rough) space requirements?
You could always attach a big CephFS or RBD device with local FS as a storage too.
Theoretically PBS could live on your separate "backup only" ceph cluster node, or
be directly attached to it over 25 to 100G.

> Bonus, it could also be used for disaster recovery management :)

Something like that would be nice, what's in your mind for your use case?