From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 01E737491A for ; Wed, 2 Jun 2021 13:15:05 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E0012A4DB for ; Wed, 2 Jun 2021 13:15:04 +0200 (CEST) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 25B6EA49D for ; Wed, 2 Jun 2021 13:15:01 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id s5-20020a7bc0c50000b0290147d0c21c51so1430085wmh.4 for ; Wed, 02 Jun 2021 04:15:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=odiso-com.20150623.gappssmtp.com; s=20150623; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version; bh=wGDPERYAvlqgH4P1SpZMrDBCPITRFHrz0QUzwh30nJc=; b=oXPmlJZoGXF5mxCy5Be+v/iy6ZvWhvqVbPgd9H8YhEFKv2oGY3MARk7lei6mk2fNSY QelZrjGepp0Kaa9fo9xdZflDCW5kBIfKz/AazlPKTgiM2OIVv/0OQEMY0bGowEJEWGrL IAyawBcCJm3Dq4F2uZ60TSy3ADzAfX7VT7furf43fPO2mYacUjOW0VXYKBm+SW7b8pit YwOYD/4R78EkQOYia7BGDDPQdqo7sLoc7qeh4VQnwFntubgjJgUs28GWqhQ5WEKycruy dWXETb/2tCIAJQxmNC3mZYAIF8+IW5dhpUY83XO2Wc7PlMWftTyzgDDFs1UFfXx4FB65 t7OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version; bh=wGDPERYAvlqgH4P1SpZMrDBCPITRFHrz0QUzwh30nJc=; b=W1TYRJuVq50fX2+tpqSBnsjyU73yAZYDu3OV4ELiEbzHsDZO1+yPGrXk+05MlXA0h7 +sZi0pEeKIqFBN9TrRPL4/G4ilBn4mDcupEiBFQVElKL3DSkGgs3S79BTMUuuZ4btItI eYyhgjvbzATRLgF566imZ9fygfbH4shEgZZ1ZjZnl2H9krdeDoVvgLsRdPeQ/Q8iBQ4F ONsAcMXz8orU0j/XCUJw1Hp+PlknaOP2eGupd0k7OkAykabqj63kdgO9dPATFrRtbQ42 b7ZxUeSZYZ48KJDRry68NZrBol4z3jdf3AUn2kdg5mEHcAUShiih52XLvBMifA42b8ZF u3hQ== X-Gm-Message-State: AOAM531lOyZvmVGfAYjJwKCyYDNfaYNceyVfwalTJS+H9BEfnHWQLg8T bo6iUyyJQzRc/TMikatbKw8UdQse2YO1A8Nm X-Google-Smtp-Source: ABdhPJwpoF0yBHuF3Tw5H9vVUwnptQVlr7sqrEfzBvQi42ZJdFISRIbQ35VKXAR2ISPsEwPvZstZKw== X-Received: by 2002:a7b:c8c1:: with SMTP id f1mr14314523wml.30.1622632494845; Wed, 02 Jun 2021 04:14:54 -0700 (PDT) Received: from ?IPv6:2a0a:1580:0:1::100c? (ovpn1.odiso.net. [2a0a:1580:2000::3f]) by smtp.gmail.com with ESMTPSA id o6sm7117418wre.73.2021.06.02.04.14.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jun 2021 04:14:54 -0700 (PDT) Message-ID: From: aderumier@odiso.com To: Thomas Lamprecht , Proxmox VE development discussion , pve-devel Date: Wed, 02 Jun 2021 13:14:53 +0200 In-Reply-To: <676f24c5-1645-f14e-8650-8b5cf1715566@proxmox.com> References: <30b3e3feb72fa3c4c209d1ab64e07512676910a2.camel@odiso.com> <676f24c5-1645-f14e-8650-8b5cf1715566@proxmox.com> User-Agent: Evolution 3.40.1 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.195 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature HTML_MESSAGE 0.001 HTML included in message KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [kernel.dk] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [pve-devel] qemu 6 : does proxmox backup handle parallel async chunck backup like the new backup code from qemu 6.0 ? X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2021 11:15:05 -0000 Le mercredi 02 juin 2021 à 08:58 +0200, Thomas Lamprecht a écrit : > Hi, > > On 02.06.21 08:39, aderumier@odiso.com wrote: > > I was looking for qemu 6.0 new features, > > and it seem that they have implement parallel async chunks backup > > (and > > I think for other block operations, through a new block-copy > > feature) > > > > https://github.com/qemu/qemu/commit/71eed4cebed487a4f3c9f97aba83c611bbe22f8d > > > > https://github.com/qemu/qemu/commit/de4641b46b020c5b332175f80e8bfe3d352888e8#diff-b33323044f2699244c126c6eae6c4083c3c99a16f4840030ac13238b1f569dc0 > > > > https://github.com/qemu/qemu/commit/26be9d62dd5f5268b814da24fd8e8b5c5b999ebe > > > > Is it alrealy implemented or on the roadmap for the proxmox backup > > code > > ? > > (To be honest, I didn't follow proxmox backup patches since a long > > time, so I really don't known how much they are sharing with qemu > > backup code) > > the parallel stuff works with our implementation, Stefan tried it and > with their > default the backup is much faster, so fast that the guest cannot do > anything during > that anymore :D So, IIRC, Stefan turned the default number of > parallel operations down > for a better balance of backup speed and guest impact. > ok thanks ! (I'll still don't use pbs, I'll try to test with my rbd cluster soon to compare speed vs rbd export/import) > FYI, we have basic packages for bullseye as base ready here, and > possible sync to > pvetest in the next days/weeks - in those repos there's already QEMU > 6.0. > yes, I see that.I'll try to do test soon > Something else we can use with Bullseye/PVE 7 is io_uring instead of > AIO, looks like > a much saner (and actually useful) design and there should be some, > at least slight, > improvements due to that too. > > https://kernel.dk/io_uring.pdf > yes, If I remember Wolfgang had seen some bug with kernel 5.4, but I think it's should works fine with 5.11. I'll try it too soon :) > cheers, > Thomas >