From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 2BB911FF15C for ; Fri, 25 Jul 2025 13:22:47 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9E2E016745; Fri, 25 Jul 2025 13:24:07 +0200 (CEST) To: Proxmox Backup Server development discussion Date: Fri, 25 Jul 2025 13:23:57 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Adam Kalisz via pve-devel Precedence: list Cc: Adam Kalisz , Proxmox VE development discussion X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] Chunk verification speedup discussion, similar speedup opportunities elsewhere Content-Type: multipart/mixed; boundary="===============3295243018367062377==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============3295243018367062377== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8FCC9C9F8A for ; Fri, 25 Jul 2025 13:24:05 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 796EC1672C for ; Fri, 25 Jul 2025 13:24:05 +0200 (CEST) Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 25 Jul 2025 13:24:04 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-ae0b2ead33cso359997666b.0 for ; Fri, 25 Jul 2025 04:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=notnullmakers.com; s=google; t=1753442638; x=1754047438; darn=lists.proxmox.com; h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from :subject:message-id:from:to:cc:subject:date:message-id:reply-to; bh=BHzDdJS8iS/yKTM5nxn8UgJhlzttb+qIu30uUzhpPd4=; b=VXEirTQWUYjMslIvHsXH3YxiUke3Hwp5IfQB34WVRRZMO6nSXgKWRnWmjKRktERjkE NWZpKhH/0BoMwjBDEA1RfUT06qMyMkGxku9FSBATjmxkysRGdHSUCBUkHwgi2TN7opVL 5TbQ3tsejJr4oEL2oPw5nmRogYTvUm1ZWR9dZgTypwF2Kb5VyPFTM9+kpHvcu1U+b9/d irFVlhuMGoZ2rXOy8DiUnSbiL6m9A59OadDYlv/KGe4OijzP+qovKnSc5u7MzEJZIM71 A4Rc5WH4VrEr0Z3Rghu+nfalUSaHcpclUKm8qxWGUEu8oFWpdtJXZsoKS8qR+I0g+xGf 7obg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753442638; x=1754047438; h=mime-version:user-agent:content-transfer-encoding:date:cc:to:from :subject:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=BHzDdJS8iS/yKTM5nxn8UgJhlzttb+qIu30uUzhpPd4=; b=p/ALAuFeuHlEmJgihwa59UG4XfjBhfFElSig08Q7dngkhYx/Y+h5N391bEP/ax9DvQ LB38UKlL6/KFAHXBeFAWU0JYXP7DkVXNSma18hcp2KaeKOZ+OMNMgbDG77pqGqgMeTWu cGCceNVE9AqYlV99WdmuNnMke9OGOnA+2utkWWxzeMH9G1rM0VFJkTLCleDqcrw80dbw m3qeD/DsUstGxhp/HWjdu6JnWmPQkFOeQej+tYYNb/dES1KKRpv6Os8Cqcl+qRsRdrcE oKa5kCrqzYsLdn1YCJqZ7oIyWjJikyN/ZnoxxezYKmui97r5/N/ghzClZ58ffK3T1GVT bwOA== X-Forwarded-Encrypted: i=1; AJvYcCUR31Bo9iWWMeDq2YJwII9klOFbc43siiPZm/Wn63upKlkgUQfqqqbjJDPkqDNvwwbzNUAPz7wtSRY=@lists.proxmox.com X-Gm-Message-State: AOJu0YwFyOae5cKusAH714k8/CbNGrh5FmqT5WHHzB41xc1/Ezl5EhIn KHQc7TAQsjpBLOKav3OeyTLpnRQucDiBNDk1xVHPehvNxMTaG0Z1GhcT29BffsFGtXc= X-Gm-Gg: ASbGncsCpyhEvlhsCPcBOAoZzvWNZDD+sa99hCFZVyx2VonL6O0EPwKdHSM1R6gYTWh k03NSZ+/WuksOE5UIQqK9Dpy5w84eTk6HxnoaAI6+bjTR0rwmkkOH9SSL0zAPAgpbQEc6DzCZ6u QmkXK23HvevORtcIJCTu/Y3WdtzLbZB7hbbuNLc+lLBrrDzw1sPwCE8o5wBQCCpibVCh6MS7JM9 qU38d63DbRn6+4cOZZVLIsG/MWRyGZ89SzZy095yXdeowE9BdUCkakXe+xAX1WyrwKN+4mtsXTN EFhvOZAMWt0u4IhSWoN1g5mJ8QEGtgXwv0N4NAB3XAKJP2KtrIBKV106IeY97R7QEyD7C+0yWBz IdKJH/q3yRiZkOxY7Hqt6rfdw7G7YZA9F5iTH1ESypGEpmg== X-Google-Smtp-Source: AGHT+IHbmHrTtgtb3Y3XnXQc4KjpAy8RlJxT1jL21iXPP5sCZZr5em4OEzCKlCf8nlb87PXpmqN3hw== X-Received: by 2002:a17:907:1c1b:b0:ade:4f2:9077 with SMTP id a640c23a62f3a-af61c2b4135mr197066966b.5.1753442637721; Fri, 25 Jul 2025 04:23:57 -0700 (PDT) Received: from ?IPv6:2a02:8308:299:4600::9185? ([2a02:8308:299:4600::9185]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af47cc6bc83sm251529266b.38.2025.07.25.04.23.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 04:23:57 -0700 (PDT) Message-ID: <2ac072513fdb726fd0e22fbc79425537dcf63a41.camel@notnullmakers.com> Subject: Chunk verification speedup discussion, similar speedup opportunities elsewhere From: Adam Kalisz To: Proxmox Backup Server development discussion Cc: Dominik Csapak , Proxmox VE development discussion Date: Fri, 25 Jul 2025 13:23:57 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.1-1 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.014 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Hi list, I missed whether the chunk verification speedup when loading chunks got applied or whether it was somehow included in the S3-like storage option change set. In https://forum.proxmox.com/threads/abysmally-slow-restore-from-backup.133602= /page-7 we have discussed some other opportunities for speedup using similar patterns. People mentioned LXC container restore speed and host-based backup restore, which looking at the code for the latter seems like a similar async loop pattern would bring some improvement without too much trouble: https://github.com/proxmox/proxmox-backup/blob/4940514b0f05d6cd6a5f711edfdd= 47c1fa41b537/proxmox-backup-client/src/main.rs#L1109 Similarly the sync performance between two Proxmox Backup Servers and live-migration got mentioned in various places. Btw. congrats on the 4.0 beta. Regards Adam --===============3295243018367062377== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3295243018367062377==--