From: Fiona Ebner <f.ebner@proxmox.com>
To: "Proxmox VE development discussion" <pve-devel@lists.proxmox.com>,
"Fabian Grünbichler" <f.gruenbichler@proxmox.com>
Subject: Re: [pve-devel] [POC storage v3 14/34] add backup provider example
Date: Wed, 12 Mar 2025 14:05:19 +0100 [thread overview]
Message-ID: <525f061a-8164-4e9d-8874-fbcb4f26b459@proxmox.com> (raw)
In-Reply-To: <1731491839.jxatj6iypi.astroid@yuna.none>
Am 13.11.24 um 11:52 schrieb Fabian Grünbichler:
> didn't give this too close a look since it's an example only, but the
> hard-coded NBD indices make me wonder whether we want to have some sort
> of mechanism to "reserve" NBD slots while using them, at least for *our*
> usage?
Fixed in v5. I just copied over the corresponding method added in v4 in
qemu-server. This plugin is just a POC to test all mechanisms. I'll add
a note there to explain that actual plugins should never need to mess
with NBD block nodes, i.e. either they should speak NBD with the export
directly instead of binding to a block device node, or use the
'block-device' backup mechanism.
> On November 7, 2024 5:51 pm, Fiona Ebner wrote:
>> +my sub get_bitmap_id {
>> + my ($self, $vmid, $vmtype) = @_;
>> +
>> + return if $self->{'storage-plugin'}->get_vm_backup_mode($self->{scfg}) ne 'incremental';
>> +
>> + my $previous_info_dir = "$self->{scfg}->{path}/$vmid/";
>> +
>> + my $previous_info_file = "$previous_info_dir/previous-info";
>> + my $info = file_read_firstline($previous_info_file) // '';
>> + $self->{$vmid}->{'old-previous-info'} = $info;
>> + my ($bitmap_id, $previous_backup_id) = $info =~ m/^(\d+)\s+(\d+)$/;
>> + my $previous_backup_dir =
>> + $previous_backup_id ? "$self->{scfg}->{path}/$vmid/$vmtype-$previous_backup_id" : undef;
>
> so the backup ID is an epoch - wouldn't it be nicer to use the formatted
> one as subdir, rather than the epoch itself?
If we ever want to spin out a variant of a directory plugin for real
production support, yes sure. I didn't bother for the example here.
>> +
>> + if ($bitmap_id && -d $previous_backup_dir) {
>> + $self->{$vmid}->{'previous-backup-dir'} = $previous_backup_dir;
>> + } else {
>> + # need to start fresh if there is no previous ID or the associated backup doesn't exist
>> + $bitmap_id = $self->{$vmid}->{'backup-time'};
>> + }
>> +
>> + $self->{$vmid}->{'bitmap-id'} = $bitmap_id;
>> + make_path($previous_info_dir);
>> + die "unable to create directory $previous_info_dir\n" if !-d $previous_info_dir;
>> + file_set_contents($previous_info_file, "$bitmap_id $self->{$vmid}->{'backup-time'}");
>> +
>> + return $bitmap_id;
>> +}
>> +
>> +# Backup Provider API
>> +
>> +sub new {
>> + my ($class, $storage_plugin, $scfg, $storeid, $log_function) = @_;
>> +
>> + my $self = bless {
>> + scfg => $scfg,
>> + storeid => $storeid,
>> + 'storage-plugin' => $storage_plugin,
>> + 'log-function' => $log_function,
>> + }, $class;
>> +
>> + return $self;
>> +}
>> +
>> +sub provider_name {
>> + my ($self) = @_;
>> +
>> + return 'dir provider example';
>> +}
>> +
>> +# Hooks
>> +
>> +my sub job_start {
>> + my ($self, $start_time) = @_;
>> +
>> + log_info($self, "job start hook called");
>> +
>> + run_command(["modprobe", "nbd"]);
>
> this duplicates the modprobe in qemu-server, but without the parameter..
Since v4, qemu-server ships a modprobe config to load it during boot.
Replaced the above with a die if not already loaded. Thinking about
this, if we do go for that config, I'll also add a postinst hook to load
it during upgrade of qemu-server in v5.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2025-03-12 13:05 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-07 16:51 [pve-devel] [RFC qemu/common/storage/qemu-server/container/manager v3 00/34] backup provider API Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu v3 01/34] block/reqlist: allow adding overlapping requests Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu v3 02/34] PVE backup: fixup error handling for fleecing Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu v3 03/34] PVE backup: factor out setting up snapshot access " Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu v3 04/34] PVE backup: save device name in device info structure Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu v3 05/34] PVE backup: include device name in error when setting up snapshot access fails Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC qemu v3 06/34] PVE backup: add target ID in backup state Fiona Ebner
2024-11-12 16:46 ` Fabian Grünbichler
2024-11-13 9:22 ` Fiona Ebner
2024-11-13 9:33 ` Fiona Ebner
2024-11-13 11:16 ` Fabian Grünbichler
2024-11-13 11:40 ` Fiona Ebner
2024-11-13 12:03 ` Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC qemu v3 07/34] PVE backup: get device info: allow caller to specify filter for which devices use fleecing Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC qemu v3 08/34] PVE backup: implement backup access setup and teardown API for external providers Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC qemu v3 09/34] PVE backup: implement bitmap support for external backup access Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC common v3 10/34] env: add module with helpers to run a Perl subroutine in a user namespace Fiona Ebner
2024-11-11 18:33 ` Thomas Lamprecht
2024-11-12 10:19 ` Fiona Ebner
2024-11-12 14:20 ` Fabian Grünbichler
2024-11-13 10:08 ` Fiona Ebner
2024-11-13 11:15 ` Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC storage v3 11/34] add storage_has_feature() helper function Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC storage v3 12/34] plugin: introduce new_backup_provider() method Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC storage v3 13/34] extract backup config: delegate to backup provider for storages that support it Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [POC storage v3 14/34] add backup provider example Fiona Ebner
2024-11-13 10:52 ` Fabian Grünbichler
2025-03-12 13:05 ` Fiona Ebner [this message]
2024-11-07 16:51 ` [pve-devel] [POC storage v3 15/34] WIP Borg plugin Fiona Ebner
2024-11-13 10:52 ` Fabian Grünbichler
2025-03-11 14:15 ` Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH qemu-server v3 16/34] move nbd_stop helper to QMPHelpers module Fiona Ebner
2024-11-11 13:55 ` [pve-devel] applied: " Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [PATCH qemu-server v3 17/34] backup: move cleanup of fleecing images to cleanup method Fiona Ebner
2024-11-12 9:26 ` [pve-devel] applied: " Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [PATCH qemu-server v3 18/34] backup: cleanup: check if VM is running before issuing QMP commands Fiona Ebner
2024-11-12 9:26 ` [pve-devel] applied: " Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [PATCH qemu-server v3 19/34] backup: keep track of block-node size for fleecing Fiona Ebner
2024-11-11 14:22 ` Fabian Grünbichler
2024-11-12 9:50 ` Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC qemu-server v3 20/34] backup: allow adding fleecing images also for EFI and TPM Fiona Ebner
2024-11-12 9:26 ` Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC qemu-server v3 21/34] backup: implement backup for external providers Fiona Ebner
2024-11-12 12:27 ` Fabian Grünbichler
2024-11-12 14:35 ` Fiona Ebner
2024-11-12 15:17 ` Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [PATCH qemu-server v3 22/34] restore: die early when there is no size for a device Fiona Ebner
2024-11-12 9:28 ` [pve-devel] applied: " Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC qemu-server v3 23/34] backup: implement restore for external providers Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC qemu-server v3 24/34] backup restore: external: hardening check for untrusted source image Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH container v3 25/34] create: add missing include of PVE::Storage::Plugin Fiona Ebner
2024-11-12 15:22 ` [pve-devel] applied: " Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC container v3 26/34] backup: implement backup for external providers Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC container v3 27/34] create: factor out tar restore command helper Fiona Ebner
2024-11-12 16:28 ` Fabian Grünbichler
2024-11-12 17:08 ` [pve-devel] applied: " Thomas Lamprecht
2024-11-07 16:51 ` [pve-devel] [RFC container v3 28/34] backup: implement restore for external providers Fiona Ebner
2024-11-12 16:27 ` Fabian Grünbichler
2024-11-07 16:51 ` [pve-devel] [RFC container v3 29/34] external restore: don't use 'one-file-system' tar flag when restoring from a directory Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC container v3 30/34] create: factor out compression option helper Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC container v3 31/34] restore tar archive: check potentially untrusted archive Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC container v3 32/34] api: add early check against restoring privileged container from external source Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [PATCH manager v3 33/34] ui: backup: also check for backup subtype to classify archive Fiona Ebner
2024-11-07 16:51 ` [pve-devel] [RFC manager v3 34/34] backup: implement backup for external providers Fiona Ebner
2024-11-12 15:50 ` [pve-devel] partially-applied: [RFC qemu/common/storage/qemu-server/container/manager v3 00/34] backup provider API Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=525f061a-8164-4e9d-8874-fbcb4f26b459@proxmox.com \
--to=f.ebner@proxmox.com \
--cc=f.gruenbichler@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal