From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id F3CD11FF15E for ; Fri, 4 Oct 2024 15:55:00 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4BE93FE83; Fri, 4 Oct 2024 15:55:22 +0200 (CEST) Date: Fri, 4 Oct 2024 16:54:39 +0300 To: pve-devel@lists.proxmox.com MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Ivaylo Markov via pve-devel Precedence: list Cc: Ivaylo Markov X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] Proposal: support for atomic snapshot of all VM disks at once Content-Type: multipart/mixed; boundary="===============3568273715691691258==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============3568273715691691258== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 21831C2B0D for ; Fri, 4 Oct 2024 15:55:21 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F2808FE3D for ; Fri, 4 Oct 2024 15:54:50 +0200 (CEST) Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 4 Oct 2024 15:54:49 +0200 (CEST) Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5c89f3e8a74so2819562a12.0 for ; Fri, 04 Oct 2024 06:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=storpool.com; s=google; t=1728050082; x=1728654882; darn=lists.proxmox.com; h=content-transfer-encoding:organization:subject:from :content-language:to:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=yule5J5q4A5Z8FdoInm4kM++QrLUXUIaWrIcLtVcn60=; b=gWyX0bzon5WA2j/oy1zAqmhyRSXbQIN1fZrWelD6pBkN9iWWiQiV2jeb8w+ac+KD9+ hZglbnxyHH8DK6P/foqQdhJiG/WlK9AmuMIfOGyAK3FuhdVl9ll/aJucPrH41cR/cHfm 1UVL6T7sin6qFwqtx1E9QtvIWGbm1kCCr9+IGDE1z+X0b1yjjP6352K+0NogtNFPU2SK fe1RXHqeU4GaexjXdlXEy2lMQZN14+MPG6a1vyap4ZKidL7c0OONO/Q7hH0XLdMIPKxJ 4Rl9qZQiRHEQu8qRfxGlzEDcB5d/KcutpKW3VctMEhYKubIB1TGbpcz8ZygGit4IOVN7 Mtcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728050082; x=1728654882; h=content-transfer-encoding:organization:subject:from :content-language:to:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yule5J5q4A5Z8FdoInm4kM++QrLUXUIaWrIcLtVcn60=; b=h32NhZWbYhM9nz7vmoj4TCYSaVYdQkAAIWsXkmNnMGwNs65mD/tpbCby6Uj+S6tZ+f 6WLDmUotItHG5X1YGCAqhBN82i57TXzmOB3mgf7hr7RZw67tjAfFLBMIA7/xG7Vst1ie Lz4Vpdaw/3p1Hd8g5zr4wE188qBRBBYiHb5eeQsuL8ovPvkWf9RZiLf20ftPeaUJqJ/9 LYl4AtY52uPCzpUSmMz3D+sNsD1wEzu85p4rCsnnmOJZ5x4lF7LgUgri1B5OTOBT/pnt 22AW7XfZ03aFWqRXlehUZq7J0OrJcdKEWCkSKwrnsqnITJSBYazGayWXGKbmE3Nu4u2S mwZA== X-Gm-Message-State: AOJu0Yz+THQEVSKV4oIC46d/ZuIsEIUlfTiLi7WdL6GGSsatW6+SLPgQ 2+WEa3xLQBngPupnUOgOeNUkEUo+jQ3T6QzQ0yB2tG1vZg9lu6pBmgYz5XjS7byjjaKgCJS0zB9 yIXU= X-Google-Smtp-Source: AGHT+IGL1EVVfhnw/3yG1/m0o8yIb/c3veXOApumjZx0vASRjTdINOyOqZc60/as0heLwZWKO/wpwQ== X-Received: by 2002:a05:6402:26c5:b0:5c8:8c35:7cb7 with SMTP id 4fb4d7f45d1cf-5c8d2e1e1c2mr2625255a12.5.1728050082092; Fri, 04 Oct 2024 06:54:42 -0700 (PDT) Received: from ?IPV6:2001:67c:21bc:e:553b:1c3a:43e1:9634? ([2001:67c:21bc:e:553b:1c3a:43e1:9634]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c8ca3dfe45sm1976336a12.38.2024.10.04.06.54.40 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 04 Oct 2024 06:54:40 -0700 (PDT) Message-ID: Date: Fri, 4 Oct 2024 16:54:39 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: pve-devel@lists.proxmox.com Content-Language: en-US From: Ivaylo Markov Subject: Proposal: support for atomic snapshot of all VM disks at once Organization: StorPool Storage Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Greetings, I am the maintainer of StorPool’s external storage plugin for PVE[0] which integrates our storage solution as a backend for VM disks. Our software has the ability to create atomic (crash-consistent) snapshots of a group of storage volumes. I’d like to use this feature in our plugin so that customers can perform whole VM snapshots, but that does not seem possible currently - the snapshot creation method is called individually for every disk. I was directed here to discuss this proposal and my implementation idea after an initial post in Bugzilla[1]. The goal is to give storage plugins the option to perform atomic crash-consistent snapshots of the virtual disks associated with a virtual machine where the backend supports it  (e.g. Ceph, StorPool, and ZFS) without affecting those without such a feature. I would add a `can_snapshot_volume_group` method to the base `PVE::Storage::Plugin` class, which would accept an array of the VM’s disks, and return a binary result whether an atomic snapshot is possible. The default implementation would return 0, but plugins with support can override it based on backend capabilities. For example, ZFS supports atomic snapshot of volume groups, but requires all volumes to be in the same pool. The actual snapshot can be performed by a `snapshot_volume_group method`, which is not expected to be called unless the driver supports this operation. In `PVE::AbstractConfig::snapshot_create` these two methods can be used to check and perform the atomic snapshots where possible, otherwise it would keep the current behavior of creating a snapshot for each disk separately. If the VM has drives with completely different storage backends (e.g. ZFS and LVM for whatever reason), the driver check can be skipped, and the current behavior used again. [0] https://github.com/storpool/pve-storpool [1] https://bugzilla.proxmox.com/show_bug.cgi?id=5752 Best regards, -- Ivaylo Markov StorPool Storage https://www.storpool.com --===============3568273715691691258== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============3568273715691691258==--