all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"
@ 2022-07-31 15:58 DERUMIER, Alexandre
  0 siblings, 0 replies; 8+ messages in thread
From: DERUMIER, Alexandre @ 2022-07-31 15:58 UTC (permalink / raw)
  To: pve-devel

Hi,

Currently, proxmox backup are using copy-before-write to have snapshot 
backup.

This works great until you have a slow  or remote storage with higher 
latencies than the vm storage.

This can slowdown a lot the vm is a lot of writes occur when backup is 
running. (or even can hang the vm if backup storage goes down during the 
backup)


qemu backup support since 1-2 years some kind of local write cache, 
called "fleecing".

When backup is running, if a block is not yet backuped, it's going to a 
temp local disk image.

Some improvements have been made in qemu 7.0, previously only qcow2 was 
allowed for the temp disk, now it's working with any disk

https://github.com/qemu/qemu/commit/af5bcd775f3115e4c1b7715920a67b31b119de30


Could it be possible to implement same king of cache for proxmox backup ?

I think qemu implementation is a simple block filter (It's even possible 
to expose a virtual snasphot), so maybe it could be easy to implement it ?


This is really a blocker for me,I can't use pbs because I'm using nvme 
is production, and a 7200k hdd backup in a remote 200km site with 5ms 
latency.

If I could use temp nvme cache on the hypervisor, it could be wonderfull :)


I had opened a bugzilla some week ago with same kind of local cache idea 
(didn't known about qemu fleecing at this time, seem the way to go)

https://bugzilla.proxmox.com/show_bug.cgi?id=4136




^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"
@ 2022-07-31 16:19 Dietmar Maurer
  2022-07-31 16:49 ` DERUMIER, Alexandre
  0 siblings, 1 reply; 8+ messages in thread
From: Dietmar Maurer @ 2022-07-31 16:19 UTC (permalink / raw)
  To: Proxmox VE development discussion, DERUMIER, Alexandre

> This is really a blocker for me,I can't use pbs because I'm using nvme 
> is production, and a 7200k hdd backup in a remote 200km site with 5ms 
> latency.

Why don't you use a local(fast) PBS instance, then sync to the slow remote?




^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"
@ 2022-08-01  8:31 Dietmar Maurer
  2022-08-01 14:36 ` DERUMIER, Alexandre
  0 siblings, 1 reply; 8+ messages in thread
From: Dietmar Maurer @ 2022-08-01  8:31 UTC (permalink / raw)
  To: DERUMIER, Alexandre, Proxmox VE development discussion

> Can I use a small local fast PBS instance without need to keep the full 
> datastore chunks ?
> 
> I have 300TB nvme in production, I don't want to buy 300TB nvme for backup.

no, sorry.

Do you want to use Ceph as temp backup storage, or simply an additional (node local) nvme?




^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2022-08-01 14:36 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-31 15:58 [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache" DERUMIER, Alexandre
2022-07-31 16:19 Dietmar Maurer
2022-07-31 16:49 ` DERUMIER, Alexandre
2022-07-31 17:47   ` Laurent GUERBY
2022-07-31 18:24     ` DERUMIER, Alexandre
2022-08-01  8:10   ` DERUMIER, Alexandre
2022-08-01  8:31 Dietmar Maurer
2022-08-01 14:36 ` DERUMIER, Alexandre

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal