From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id BF7991FF2A1 for ; Tue, 16 Jul 2024 08:37:19 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 1BB4EC398; Tue, 16 Jul 2024 08:37:45 +0200 (CEST) Date: Tue, 16 Jul 2024 08:37:10 +0200 (CEST) From: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= To: Proxmox VE development discussion Message-ID: <875684683.19.1721111830084@webmail.proxmox.com> In-Reply-To: References: MIME-Version: 1.0 X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.6-Rev67 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.050 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pve-devel] S3 Support X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" > Tiziano Bacocco via pve-devel hat am 16.07.2024 00:20 CEST geschrieben: > Hi, to everyone, i'm interested in native S3 support on proxmox, is anyone > working on it ( to avoid double work )? None of us is, but there is an external developer that is trying to implement something like this. The whole topic might be a better fit for pbs-devel though ;) > Second question, my idea is to reuse existing pbs code but upload chunks > and index to s3, sounds good? The thing puzzling me is that a lot of code > will end up duplicated , when actually only the part uploading to S3 > instead of pbs would be different, chunking, fleeching, restoring, backup, > qemu interface would be almost identical https://bugzilla.proxmox.com/show_bug.cgi?id=2943 backing up directly to S3 will be hard for a multitude of reasons: - you don't want to write individual chunks as objects (too expensive) - you don't want to slow down the guest because of the speed of the object storage - .. instead, a similar approach to the tape backup feature would be needed: - define retention and bucket allocation policies - combine metadata (snapshots/..) and chunks of a local datastore in a meaningful way into buckets - have some sort of index/catalog that tells you which buckets you need to restore which snapshots into a datastore - .. hope this makes sense! _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel