From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 0B3761FF16B for ; Fri, 13 Sep 2024 16:00:12 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 635C814335; Fri, 13 Sep 2024 16:00:15 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Date: Fri, 13 Sep 2024 15:59:44 +0200 Message-Id: <20240913135944.317298-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.377 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_ASCII_DIVIDERS 0.8 Email that uses ascii formatting dividers and possible spam tricks KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [environment.rs, mod.rs, proxmox.com] Subject: [pbs-devel] [RFC proxmox-backup] fix #5710: api: backup: stat known chunks during backup X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" S25vd24gY2h1bmtzIGFyZSBleHBlY3RlZCB0byBiZSBwcmVzZW50IG9uIHRoZSBkYXRhc3RvcmUg YS1wcmlvcmksCmFsbG93aW5nIGNsaWVudHMgdG8gb25seSByZS1pbmRleCB0aGVzZSBjaHVua3Mg d2l0aG91dCB1cGxvYWRpbmcgdGhlCnJhdyBjaHVuayBkYXRhLiBUaGUgbGlzdCBvZiByZXVzYWJs ZSBrbm93biBjaHVua3MgaXMgc2VuZCB0byB0aGUKY2xpZW50IGJ5IHRoZSBzZXJ2ZXIsIGRlZHVj ZWQgZnJvbSB0aGUgaW5kZXhlZCBjaHVua3Mgb2YgdGhlIHByZXZpb3VzCmJhY2t1cCBzbmFwc2hv dCBvZiB0aGUgZ3JvdXAuCgpJZiBob3dldmVyIHN1Y2ggYSBrbm93biBjaHVuayBkaXNhcHBlYXJl ZCAodGhlIHByZXZpb3VzIGJhY2t1cApzbmFwc2hvdCBoYXZpbmcgYmVlbiB2ZXJpZmllZCBiZWZv cmUgdGhhdCBvciBub3QgdmVyaWZpZWQganVzdCB5ZXQpLAp0aGUgYmFja3VwIHdpbGwgZmluaXNo IGp1c3QgZmluZSwgbGVhZGluZyB0byBhIHNlZW1pbmdseSBzdWNjZXNzZnVsCmJhY2t1cC4gT25s eSBhIHN1YnNlcXVlbnQgdmVyaWZpY2F0aW9uIGpvYiB3aWxsIGRldGVjdCB0aGUgYmFja3VwCnNu YXBzaG90IGFzIGJlaW5nIGNvcnJ1cHQuCgpJbiBvcmRlciB0byByZWR1Y2UgdGhlIGltcGFjdCwg c3RhdCB0aGUgbGlzdCBvZiBrbm93biBjaHVua3MgdHdpY2UKZHVyaW5nIGJhY2t1cDoKLSBkdXJp bmcgcmVnaXN0cmF0aW9uIG9mIGEga25vd24gY2h1bmsKLSB3aGVuIGZpbmlzaGluZyB0aGUgYmFj a3VwCgpUaGUgZmlyc3Qgc3RhdCBpcyB0byBlYXJseSBvbiBkZXRlY3QgbWlzc2luZyBjaHVua3Ms IHRoZSBsYXR0ZXIKYXNzdXJlcyB0aGF0IGFsbCAoa25vd24gYW5kIG5ld2x5IHJlZ2lzdGVyZWQp IGNodW5rcyBhcmUgcHJlc2VudCBvbgp0aGUgZGF0YXN0b3JlIGFmdGVyIHRoZSBiYWNrdXAgcnVu LgoKSWYgYSBtaXNzaW5nIGNodW5rIGlzIGRldGVjdGVkLCB0aGUgYmFja3VwIHJ1biBpdHNlbGYg d2lsbCBmYWlsIGFuZAp0aGUgcHJldmlvdXMgYmFja3VwIHNuYXBzaG90cyB2ZXJpZnkgc3RhdGUg aXMgc2V0IHRvIGZhaWxlZC4KVGhpcyBwcmV2ZW50cyB0aGUgc2FtZSBzbmFwc2hvdCBmcm9tIGJl aW5nIHJldXNlZCBieSBhbm90aGVyLApzdWJzZXF1ZW50IGJhY2t1cCBqb2IuCgpsaW5rIHRvIGlz c3VlIGluIGJ1Z3RyYWNrZXI6Cmh0dHBzOi8vYnVnemlsbGEucHJveG1veC5jb20vc2hvd19idWcu Y2dpP2lkPTU3MTAKClNpZ25lZC1vZmYtYnk6IENocmlzdGlhbiBFYm5lciA8Yy5lYm5lckBwcm94 bW94LmNvbT4KLS0tClNlbmRpbmcgdGhpcyBhcyBSRkMgYmVjYXVzZSBvZiB0aGUgcG9zc2libGUg cGVyZm9ybWFuY2UgaW1wYWN0LAplc3BlY2lhbGx5IGZvciB1c2VycyB3aXRoIGRhdGFzdG9yZXMg bG9jYXRlZCBvbiBzdG9yYWdlcyB3aXRoIGxpbWl0ZWQKSS8wLgoKVGVzdCBvbiBteSBzaWRlIGRp ZCBub3Qgc2hvdyBhIHNpZ25pZmljYW50IHBlcmZvcm1hbmNlIGRpZmZlcmVuY2UgYXMKY29tcGFy ZWQgdG8gYW4gdW5wYXRjaGVkIHNlcnZlci4KSSBkaWQgcGVyZm9ybSB2emR1bXAgYmFja3VwcyBv ZiBhIFZNIHdpdGggYSAzMkcgZGlzayBhdHRhY2hlZCBhbmQgYQpMWEMgY29udGFpbmVyIHdpdGgg YSBEZWJpYW4gaW5zdGFsbCBhbmQgcm9vdGZzIG9mIGNhLiA0MDBNIChib3RoIG9mZiwKbm8gY2hh bmdlcyBpbiBkYXRhIGluLWJldHdlZW4gYmFja3VwIHJ1bnMpLiBQQlMgZGF0YXN0b3JlIGxvY2F0 ZWQgb24KWkZTIGJhY2tlZCBieSBhbiBOVk1FIFNTRCwgNSBydW5zIGVhY2ggYWZ0ZXIgYW4gaW5p dGlhbCBydW4gdG8gYXNzdXJlCmZ1bGwgY2h1bmsgcHJlc2VuY2Ugb24gc2VydmVyIGFuZCB2YWxp ZCBwcmV2aW91cyBzbmFwc2hvdC4KCkhlcmUgc29tZSBiYWxscGFyayBmaWd1cmVzOgoKLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KcGF0 Y2hlZCAgICAgICAgICAgICAgICAgICAgfCB1bnBhdGNoZWQKLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KVk0gICAgICAgICAgIHwgTFhD ICAgICAgICAgfCBWTSAgICAgICAgICAgfCBMWEMKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KMTUuM3MgwrEgMS40cyB8IDEuNzMgwrEg MC4wMiB8IDE0LjdzIMKxIDEuOHMgfCAxLjc3cyDCsSAwLjA1cwotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIHNyYy9hcGkyL2JhY2t1 cC9lbnZpcm9ubWVudC5ycyB8IDEyICsrKysrKysrKysKIHNyYy9hcGkyL2JhY2t1cC9tb2QucnMg ICAgICAgICB8IDQ0ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysKIDIgZmlsZXMg Y2hhbmdlZCwgNTYgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdpdCBhL3NyYy9hcGkyL2JhY2t1cC9l bnZpcm9ubWVudC5ycyBiL3NyYy9hcGkyL2JhY2t1cC9lbnZpcm9ubWVudC5ycwppbmRleCA5OWQ4 ODVlMmUuLjkwZmQ5MWQ2MyAxMDA2NDQKLS0tIGEvc3JjL2FwaTIvYmFja3VwL2Vudmlyb25tZW50 LnJzCisrKyBiL3NyYy9hcGkyL2JhY2t1cC9lbnZpcm9ubWVudC5ycwpAQCAtMjU5LDYgKzI1OSwx OCBAQCBpbXBsIEJhY2t1cEVudmlyb25tZW50IHsKICAgICAgICAgc3RhdGUua25vd25fY2h1bmtz LmdldChkaWdlc3QpLmNvcGllZCgpCiAgICAgfQogCisgICAgLy8vIFN0YXQgYWxsIGN1cnJlbnRs eSByZWdpc3RlcmVkIGNodW5rcworICAgIC8vLworICAgIHB1YiBmbiBzdGF0X3JlZ2lzdGVyZWRf Y2h1bmtzKCZzZWxmKSAtPiBSZXN1bHQ8KCksIEVycm9yPiB7CisgICAgICAgIGxldCBzdGF0ZSA9 IHNlbGYuc3RhdGUubG9jaygpLnVud3JhcCgpOworICAgICAgICBmb3IgZGlnZXN0IGluIHN0YXRl Lmtub3duX2NodW5rcy5rZXlzKCkgeworICAgICAgICAgICAgc2VsZi5kYXRhc3RvcmUKKyAgICAg ICAgICAgICAgICAuc3RhdF9jaHVuaygmZGlnZXN0KQorICAgICAgICAgICAgICAgIC5tYXBfZXJy KHxlcnJ8IGZvcm1hdF9lcnIhKCJzdGF0IGZhaWxlZCBvbiB7fSAtIHtlcnJ9IiwgaGV4OjplbmNv ZGUoZGlnZXN0KSkpPzsKKyAgICAgICAgfQorICAgICAgICBPaygoKSkKKyAgICB9CisKICAgICAv Ly8gU3RvcmUgdGhlIHdyaXRlciB3aXRoIGFuIHVuaXF1ZSBJRAogICAgIHB1YiBmbiByZWdpc3Rl cl9keW5hbWljX3dyaXRlcigKICAgICAgICAgJnNlbGYsCmRpZmYgLS1naXQgYS9zcmMvYXBpMi9i YWNrdXAvbW9kLnJzIGIvc3JjL2FwaTIvYmFja3VwL21vZC5ycwppbmRleCBlYTBkMDI5MmUuLmMz MDgwNjZiNiAxMDA2NDQKLS0tIGEvc3JjL2FwaTIvYmFja3VwL21vZC5ycworKysgYi9zcmMvYXBp Mi9iYWNrdXAvbW9kLnJzCkBAIC01NzksNiArNTc5LDggQEAgZm4gZHluYW1pY19hcHBlbmQoCiAg ICAgICAgICAgICAubG9va3VwX2NodW5rKCZkaWdlc3QpCiAgICAgICAgICAgICAub2tfb3JfZWxz ZSh8fCBmb3JtYXRfZXJyISgibm8gc3VjaCBjaHVuayB7fSIsIGRpZ2VzdF9zdHIpKT87CiAKKyAg ICAgICAgc3RhdF9rbm93bl9jaHVuayhlbnYsIHdpZCwgJmRpZ2VzdCk/OworCiAgICAgICAgIGVu di5keW5hbWljX3dyaXRlcl9hcHBlbmRfY2h1bmsod2lkLCBvZmZzZXQsIHNpemUsICZkaWdlc3Qp PzsKIAogICAgICAgICBlbnYuZGVidWcoZm9ybWF0ISgKQEAgLTY1Myw2ICs2NTUsOCBAQCBmbiBm aXhlZF9hcHBlbmQoCiAgICAgICAgICAgICAubG9va3VwX2NodW5rKCZkaWdlc3QpCiAgICAgICAg ICAgICAub2tfb3JfZWxzZSh8fCBmb3JtYXRfZXJyISgibm8gc3VjaCBjaHVuayB7fSIsIGRpZ2Vz dF9zdHIpKT87CiAKKyAgICAgICAgc3RhdF9rbm93bl9jaHVuayhlbnYsIHdpZCwgJmRpZ2VzdCk/ OworCiAgICAgICAgIGVudi5maXhlZF93cml0ZXJfYXBwZW5kX2NodW5rKHdpZCwgb2Zmc2V0LCBz aXplLCAmZGlnZXN0KT87CiAKICAgICAgICAgZW52LmRlYnVnKGZvcm1hdCEoCkBAIC03ODUsNiAr Nzg5LDEyIEBAIGZuIGZpbmlzaF9iYWNrdXAoCiApIC0+IFJlc3VsdDxWYWx1ZSwgRXJyb3I+IHsK ICAgICBsZXQgZW52OiAmQmFja3VwRW52aXJvbm1lbnQgPSBycGNlbnYuYXNfcmVmKCk7CiAKKyAg ICBpZiBsZXQgRXJyKGVycikgPSBlbnYuc3RhdF9yZWdpc3RlcmVkX2NodW5rcygpIHsKKyAgICAg ICAgZW52LmRlYnVnKGZvcm1hdCEoInN0YXQgcmVnaXN0ZXJlZCBjaHVua3MgZmFpbGVkIC0ge2Vy cn0iKSk7CisgICAgICAgIHVwZGF0ZV9wcmV2aW91c19tYW5pZmVzdF90b192ZXJpZnlfZmFpbGVk KGVudik/OworICAgICAgICBiYWlsISgic3RhdCByZWdpc3RlcmVkIGNodW5rcyBmYWlsZWQgLSB7 ZXJyfSIpOworICAgIH0KKwogICAgIGVudi5maW5pc2hfYmFja3VwKCk/OwogICAgIGVudi5sb2co InN1Y2Nlc3NmdWxseSBmaW5pc2hlZCBiYWNrdXAiKTsKIApAQCAtODcyLDMgKzg4MiwzNyBAQCBm biBkb3dubG9hZF9wcmV2aW91cygKICAgICB9CiAgICAgLmJveGVkKCkKIH0KKworLy8gQ2hlY2sg aWYga25vd24gY2h1bmsgaXMgc3RpbGwgb24gZGlzaworLy8KKy8vIElmIG5vdCwgZmFpbCBhbmQg bWFyayBwcmV2aW91cyBiYWNrdXAgc25hcHNob3QgYXMgaW52YWxpZCBzbyB0aGUgbmV4dCBiYWNr dXAgcnVuIHdpbGwgbm90IHVzZSBpdC4KK2ZuIHN0YXRfa25vd25fY2h1bmsoZW52OiAmQmFja3Vw RW52aXJvbm1lbnQsIHdpZDogdXNpemUsIGRpZ2VzdDogJlt1ODsgMzJdKSAtPiBSZXN1bHQ8KCks IEVycm9yPiB7CisgICAgaWYgbGV0IEVycihlcnIpID0gZW52LmRhdGFzdG9yZS5zdGF0X2NodW5r KCZkaWdlc3QpIHsKKyAgICAgICAgbGV0IGRpZ2VzdF9zdHIgPSBoZXg6OmVuY29kZShkaWdlc3Qp OworICAgICAgICBlbnYuZGVidWcoZm9ybWF0ISgKKyAgICAgICAgICAgICJrbm93biBjaHVuayBz dGF0IGZhaWxlZCBvbiB7ZGlnZXN0X3N0cn0gaW5kZXhlZCBieSB7d2lkfSAtIHtlcnJ9IiwKKyAg ICAgICAgKSk7CisgICAgICAgIHVwZGF0ZV9wcmV2aW91c19tYW5pZmVzdF90b192ZXJpZnlfZmFp bGVkKGVudik/OworICAgICAgICBiYWlsISgia25vd24gY2h1bmsgc3RhdCBmYWlsZWQgb24ge2Rp Z2VzdF9zdHJ9IC0ge2Vycn0iKTsKKyAgICB9CisgICAgT2soKCkpCit9CisKKy8vIFVwZGF0ZSB0 aGUgbWFuaWZlc3Qgb2YgdGhlIHByZXZpb3VzIGJhY2t1cCBzbmFwc2hvdCB0byBiZSBpbiBgVmVy aWZ5U3RhdGU6OkZhaWxlZGAsIHVzaW5nIHRoZQorLy8gY3VycmVudCBiYWNrdXAgd3JpdGVyIHRh c2tzIFVQSUQuCitmbiB1cGRhdGVfcHJldmlvdXNfbWFuaWZlc3RfdG9fdmVyaWZ5X2ZhaWxlZChl bnY6ICZCYWNrdXBFbnZpcm9ubWVudCkgLT4gUmVzdWx0PCgpLCBFcnJvcj4geworICAgIGlmIGxl dCBTb21lKGxhc3QpID0gZW52Lmxhc3RfYmFja3VwLmFzX3JlZigpIHsKKyAgICAgICAgLy8gTm8g bmVlZCB0byBhY3F1aXJlIHNuYXBzaG90IGxvY2ssIGFscmVhZHkgbG9ja2VkIHdoZW4gc3RhcnRp bmcgdGhlIGJhY2t1cAorICAgICAgICBsZXQgdmVyaWZ5X3N0YXRlID0gU25hcHNob3RWZXJpZnlT dGF0ZSB7CisgICAgICAgICAgICBzdGF0ZTogVmVyaWZ5U3RhdGU6OkZhaWxlZCwKKyAgICAgICAg ICAgIHVwaWQ6IGVudi53b3JrZXIudXBpZCgpLmNsb25lKCksIC8vIGJhY2twIHdyaXRlciBVUElE CisgICAgICAgIH07CisgICAgICAgIGxldCB2ZXJpZnlfc3RhdGUgPSBzZXJkZV9qc29uOjp0b192 YWx1ZSh2ZXJpZnlfc3RhdGUpPzsKKyAgICAgICAgbGFzdC5iYWNrdXBfZGlyCisgICAgICAgICAg ICAudXBkYXRlX21hbmlmZXN0KHxtYW5pZmVzdHwgeworICAgICAgICAgICAgICAgIG1hbmlmZXN0 LnVucHJvdGVjdGVkWyJ2ZXJpZnlfc3RhdGUiXSA9IHZlcmlmeV9zdGF0ZTsKKyAgICAgICAgICAg IH0pCisgICAgICAgICAgICAubWFwX2Vycih8ZXJyfCBmb3JtYXRfZXJyISgibWFuaWZlc3QgdXBk YXRlIGZhaWxlZCAtIHtlcnJ9IikpPzsKKyAgICB9CisgICAgT2soKCkpCit9Ci0tIAoyLjM5LjIK CgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KcGJzLWRl dmVsIG1haWxpbmcgbGlzdApwYnMtZGV2ZWxAbGlzdHMucHJveG1veC5jb20KaHR0cHM6Ly9saXN0 cy5wcm94bW94LmNvbS9jZ2ktYmluL21haWxtYW4vbGlzdGluZm8vcGJzLWRldmVsCg==