* Re: [PVE-User] PBS backup/restore issue [not found] <mailman.28.1614066435.365.pve-user@lists.proxmox.com> @ 2021-02-23 8:06 ` Dietmar Maurer 2021-02-24 9:44 ` mj 1 sibling, 0 replies; 5+ messages in thread From: Dietmar Maurer @ 2021-02-23 8:06 UTC (permalink / raw) To: Proxmox VE user list > I hit the issue 3 times, and tried creating a new VM and a new Debian > installation each time. > > Any ideas? I have the "broken" PBS backups saved. Already tried to verify the datastore? (Datastore Verify is OK?) ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PVE-User] PBS backup/restore issue [not found] <mailman.28.1614066435.365.pve-user@lists.proxmox.com> 2021-02-23 8:06 ` [PVE-User] PBS backup/restore issue Dietmar Maurer @ 2021-02-24 9:44 ` mj [not found] ` <mailman.44.1614161184.365.pve-user@lists.proxmox.com> 1 sibling, 1 reply; 5+ messages in thread From: mj @ 2021-02-24 9:44 UTC (permalink / raw) To: pve-user Hi Eneko, Just to say: that sounds scary! Do you have any update on this..? MJ ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <mailman.44.1614161184.365.pve-user@lists.proxmox.com>]
* Re: [PVE-User] PBS backup/restore issue [not found] ` <mailman.44.1614161184.365.pve-user@lists.proxmox.com> @ 2021-06-13 18:36 ` proxmox 0 siblings, 0 replies; 5+ messages in thread From: proxmox @ 2021-06-13 18:36 UTC (permalink / raw) To: Proxmox VE user list, Eneko Lacunza via pve-user, pve-user Any update on this? I had read an issue with pbs and restore of vms stored on ceph. Which is hopefully solved in actual verion of pbs and qemu... Kind regards Mehmet Am 24. Februar 2021 11:06:14 MEZ schrieb Eneko Lacunza via pve-user <pve-user@lists.proxmox.com>: >_______________________________________________ >pve-user mailing list >pve-user@lists.proxmox.com >https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <c390959d-e5b5-9a95-770c-ff6853357cbe@binovo.es>]
* Re: [PVE-User] PBS backup/restore issue [not found] <c390959d-e5b5-9a95-770c-ff6853357cbe@binovo.es> @ 2021-02-24 10:34 ` Stefan Reiter [not found] ` <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> 0 siblings, 1 reply; 5+ messages in thread From: Stefan Reiter @ 2021-02-24 10:34 UTC (permalink / raw) To: Eneko Lacunza, pve-user On 2/23/21 8:46 AM, Eneko Lacunza wrote: > Hi all, > > Yesterday I hit a problem with PBS. My sequence of actions was: > > 1.a Install debian 10 VM (updated to latest, only system utilities and > SSH server) > 1.b Add SSH access key to root > 1.c Shutdown VM > 1.d Backup to PBS and to NFS vía regular backup. > 1.e Start VM and install various packages - all OK. > 1.f Stop VM > > 2.a Restore VM from PBS backup > 2.b Start VM and install various packages - apt complains about no > newline in linux-image-amd64 file manifest, can't install new packages. > 2.c Stop VM > > 3.a Restore VM from NFS regular backup > 3.b Start VM and install various packages - all OK. > Hi, I tried to replicate your problem by following pretty much the exact steps you mention above, but to no avail. Fresh Debian 10 VM, PBS backup + VMA to local disk, restored both and both work. I compared the disk images of the restored VMs with md5sum and the hashes matched, as expected. Tested with the latest package versions, though using QEMU 5.2.0, but the backup and restore functionality shouldn't have changed there. Could you try to see what the difference is between the restored disks from the VMA and the PBS backups (if there is any)? I.e. restore them both to seperate VMIDs and then compare hashes and/or run (v)bindiff on the disk images (best restore to a directory storage in 'raw' mode) to see where they differ - of course, this only makes sense if the backups are created when the VM was shut down. Potentially something about your NFS/PBS setup is corrupting data? You mentioned in another mail that they "are the same disks", how is that configured? Also please post the exact error messages APT is giving you, and your VM config. > I hit the issue 3 times, and tried creating a new VM and a new Debian > installation each time. > > Any ideas? I have the "broken" PBS backups saved. > > VM node: > # pveversion -v [...] > > PBS node: > # pveversion -v [...] FYI, the more useful command for PBS would be: proxmox-backup-manager versions --verbose ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es>]
* Re: [PVE-User] PBS backup/restore issue [not found] ` <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> @ 2021-02-24 19:03 ` Thomas Lamprecht 0 siblings, 0 replies; 5+ messages in thread From: Thomas Lamprecht @ 2021-02-24 19:03 UTC (permalink / raw) To: Eneko Lacunza, Stefan Reiter, pve-user Hi! On 24.02.21 12:53, Eneko Lacunza wrote: > El 24/2/21 a las 11:34, Stefan Reiter escribió: >> On 2/23/21 8:46 AM, Eneko Lacunza wrote: >>> Yesterday I hit a problem with PBS. My sequence of actions was: >>> >>> 1.a Install debian 10 VM (updated to latest, only system utilities and SSH server) >>> 1.b Add SSH access key to root >>> 1.c Shutdown VM >>> 1.d Backup to PBS and to NFS vía regular backup. >>> 1.e Start VM and install various packages - all OK. >>> 1.f Stop VM >>> >>> 2.a Restore VM from PBS backup >>> 2.b Start VM and install various packages - apt complains about no newline in linux-image-amd64 file manifest, can't install new packages. >>> 2.c Stop VM >>> >>> 3.a Restore VM from NFS regular backup >>> 3.b Start VM and install various packages - all OK. >>> [snip] > My original restores were to a ceph storage (r2 on HDD with SSD journal/DB). So i have restored again both, to that ceph storage: [snip] > > Hope this info helps you to replicate the issue. FYI: Stefan could replicate it and send a patch[0] which is included now. It boiled down to caches not being flushed to disk for storage attached through user-space libraries (vs. kernel) by QEMU in the PBS restore case. In your case you probably do not have the "KRBD" checkbox ticked for the Ceph storage entry in PVE, so the user-space librbd is used for accessing that storage through QEMU, where this issue then hit on restore. If you want to try the fix out, there's a new QEMU package available on pvetest: http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/pve-qemu-kvm_5.2.0-2_amd64.deb sha256sum: 6a086e0b669447f0b8a7899f9194a26785e6b0070f3dec1be4f95067bc707c7a cheers, Thomas [0]: https://lists.proxmox.com/pipermail/pve-devel/2021-February/047131.html ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-06-13 18:42 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <mailman.28.1614066435.365.pve-user@lists.proxmox.com> 2021-02-23 8:06 ` [PVE-User] PBS backup/restore issue Dietmar Maurer 2021-02-24 9:44 ` mj [not found] ` <mailman.44.1614161184.365.pve-user@lists.proxmox.com> 2021-06-13 18:36 ` proxmox [not found] <c390959d-e5b5-9a95-770c-ff6853357cbe@binovo.es> 2021-02-24 10:34 ` Stefan Reiter [not found] ` <6120d3f8-67c8-873f-164c-1913c2016a54@binovo.es> 2021-02-24 19:03 ` Thomas Lamprecht
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox