all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH qemu] fix #1534: vma: Add extract filter for disk images
@ 2023-08-07 10:08 Filip Schauer
  2023-08-07 12:52 ` Fiona Ebner
  0 siblings, 1 reply; 2+ messages in thread
From: Filip Schauer @ 2023-08-07 10:08 UTC (permalink / raw)
  To: pve-devel

Add a filter to the "vma extract" command. A wildcard can be passed with
"-f" to match disk images that should be extracted.

Example to extract all IDE drives from vzdump.vma:

vma extract vzdump.vma -f "drive-ide*" extractdir

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 ...VE-Backup-add-vma-backup-format-code.patch | 27 ++++++++++++++-----
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/debian/patches/pve/0027-PVE-Backup-add-vma-backup-format-code.patch b/debian/patches/pve/0027-PVE-Backup-add-vma-backup-format-code.patch
index 8471a6f..b29d348 100644
--- a/debian/patches/pve/0027-PVE-Backup-add-vma-backup-format-code.patch
+++ b/debian/patches/pve/0027-PVE-Backup-add-vma-backup-format-code.patch
@@ -1738,7 +1738,7 @@ new file mode 100644
 index 0000000000..304f02bc84
 --- /dev/null
 +++ b/vma.c
-@@ -0,0 +1,878 @@
+@@ -0,0 +1,893 @@
 +/*
 + * VMA: Virtual Machine Archive
 + *
@@ -1753,6 +1753,7 @@ index 0000000000..304f02bc84
 + */
 +
 +#include "qemu/osdep.h"
++#include <fnmatch.h>
 +#include <glib.h>
 +
 +#include "vma.h"
@@ -1772,7 +1773,7 @@ index 0000000000..304f02bc84
 +        "vma list <filename>\n"
 +        "vma config <filename> [-c config]\n"
 +        "vma create <filename> [-c config] pathname ...\n"
-+        "vma extract <filename> [-r <fifo>] <targetdir>\n"
++        "vma extract <filename> [-f <filter>] [-r <fifo>] <targetdir>\n"
 +        "vma verify <filename> [-v]\n"
 +        ;
 +
@@ -1917,9 +1918,10 @@ index 0000000000..304f02bc84
 +    const char *filename;
 +    const char *dirname;
 +    const char *readmap = NULL;
++    const char *filter = NULL;
 +
 +    for (;;) {
-+        c = getopt(argc, argv, "hvr:");
++        c = getopt(argc, argv, "hvf:r:");
 +        if (c == -1) {
 +            break;
 +        }
@@ -1928,6 +1930,9 @@ index 0000000000..304f02bc84
 +        case 'h':
 +            help();
 +            break;
++        case 'f':
++            filter = optarg;
++            break;
 +        case 'r':
 +            readmap = optarg;
 +            break;
@@ -2064,12 +2069,12 @@ index 0000000000..304f02bc84
 +
 +    int i;
 +    int vmstate_fd = -1;
-+    guint8 vmstate_stream = 0;
++    uint8_t filter_bitmap[256 / 8];
++    memset(filter_bitmap, 0, sizeof(filter_bitmap));
 +
 +    for (i = 1; i < 255; i++) {
 +        VmaDeviceInfo *di = vma_reader_get_device_info(vmar, i);
 +        if (di && (strcmp(di->devname, "vmstate") == 0)) {
-+            vmstate_stream = i;
 +            char *statefn = g_strdup_printf("%s/vmstate.bin", dirname);
 +            vmstate_fd = open(statefn, O_WRONLY|O_CREAT|O_EXCL, 0644);
 +            if (vmstate_fd < 0) {
@@ -2078,6 +2083,16 @@ index 0000000000..304f02bc84
 +            }
 +            g_free(statefn);
 +        } else if (di) {
++            if (filter && (fnmatch(filter, di->devname, 0) != 0)) {
++                if (vma_reader_register_bs(vmar, i, NULL, true, true, &errp) < 0) {
++                    g_error("%s", error_get_pretty(errp));
++                }
++
++                continue;
++            }
++
++            filter_bitmap[i / 8] |= 1 << (i % 8);
++
 +            char *devfn = NULL;
 +            const char *format = NULL;
 +            uint64_t throttling_bps = 0;
@@ -2190,7 +2205,7 @@ index 0000000000..304f02bc84
 +    if (!readmap) {
 +        for (i = 1; i < 255; i++) {
 +            VmaDeviceInfo *di = vma_reader_get_device_info(vmar, i);
-+            if (di && (i != vmstate_stream)) {
++            if (di && (filter_bitmap[i / 8] & (1 << (i % 8)))) {
 +                char *tmpfn = g_strdup_printf("%s/tmp-disk-%s.raw",
 +                                              dirname, di->devname);
 +                char *fn = g_strdup_printf("%s/disk-%s.raw",
-- 
2.39.2





^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [pve-devel] [PATCH qemu] fix #1534: vma: Add extract filter for disk images
  2023-08-07 10:08 [pve-devel] [PATCH qemu] fix #1534: vma: Add extract filter for disk images Filip Schauer
@ 2023-08-07 12:52 ` Fiona Ebner
  0 siblings, 0 replies; 2+ messages in thread
From: Fiona Ebner @ 2023-08-07 12:52 UTC (permalink / raw)
  To: Proxmox VE development discussion, Filip Schauer

Am 07.08.23 um 12:08 schrieb Filip Schauer:
> Add a filter to the "vma extract" command. A wildcard can be passed with
> "-f" to match disk images that should be extracted.

The bug report doesn't mention wildcards, but a list of image names.
Restoring drive-ide2 and drive-scsi0 becomes awkward with a single
pattern and I don't see why a list of just the names shouldn't suffice.
Cases with many disks are rather rare and even in extreme cases it won't
be more than ~30 disks or so. And if users request patterns, we can
still add it on top.

> @@ -2064,12 +2069,12 @@ index 0000000000..304f02bc84
>  +
>  +    int i;
>  +    int vmstate_fd = -1;
> -+    guint8 vmstate_stream = 0;
> ++    uint8_t filter_bitmap[256 / 8];

I'd rather use a bool array (if the compiler optimizes it to a bitmap
great, otherwise it's still easier to read), but...

> ++    memset(filter_bitmap, 0, sizeof(filter_bitmap));
>  +
>  +    for (i = 1; i < 255; i++) {
>  +        VmaDeviceInfo *di = vma_reader_get_device_info(vmar, i);
>  +        if (di && (strcmp(di->devname, "vmstate") == 0)) {
> -+            vmstate_stream = i;
>  +            char *statefn = g_strdup_printf("%s/vmstate.bin", dirname);
>  +            vmstate_fd = open(statefn, O_WRONLY|O_CREAT|O_EXCL, 0644);
>  +            if (vmstate_fd < 0) {
> @@ -2078,6 +2083,16 @@ index 0000000000..304f02bc84
>  +            }
>  +            g_free(statefn);
>  +        } else if (di) {
> ++            if (filter && (fnmatch(filter, di->devname, 0) != 0)) {
> ++                if (vma_reader_register_bs(vmar, i, NULL, true, true, &errp) < 0) {
> ++                    g_error("%s", error_get_pretty(errp));
> ++                }
> ++
> ++                continue;
> ++            }

...instead of doing this, you could also just make sure not to allocate
an image when the name is not included and set the skip flag when
calling vma_reader_register_bs(), re-using the existing mechanism for
partial restore.

Does not registering all even work? I thought vma was written in such a
way that it would read everything, which is why the skip logic was
necessary IIRC. Peeking at the code, doesn't restore_write_data() call
restore_write_data() with a dev_id from the backup we didn't register
the stream for at some point otherwise?




^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2023-08-07 12:52 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-07 10:08 [pve-devel] [PATCH qemu] fix #1534: vma: Add extract filter for disk images Filip Schauer
2023-08-07 12:52 ` Fiona Ebner

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal