From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5297D5B4F9 for ; Tue, 7 Jul 2020 11:56:23 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4F304263B7 for ; Tue, 7 Jul 2020 11:56:23 +0200 (CEST) Received-SPF: pass (proxmox.com: 212.186.127.180 is authorized to use 'a.lauterer@proxmox.com' in 'mfrom' identity (mechanism 'mx' matched)) receiver=firstgate.proxmox.com; identity=mailfrom; envelope-from="a.lauterer@proxmox.com"; helo=proxmox-new.maurer-it.com; client-ip=212.186.127.180 Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id AD18726385 for ; Tue, 7 Jul 2020 11:56:19 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 453E2441E6 for ; Tue, 7 Jul 2020 11:49:03 +0200 (CEST) From: Aaron Lauterer To: pve-devel@pve.proxmox.com Date: Tue, 7 Jul 2020 11:48:58 +0200 Message-Id: <20200707094902.24712-2-a.lauterer@proxmox.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707094902.24712-1-a.lauterer@proxmox.com> References: <20200707094902.24712-1-a.lauterer@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [backup.pm] X-Mailman-Approved-At: Tue, 07 Jul 2020 12:27:43 +0200 Subject: [pve-devel] [PATCH v4 manager 1/5] api: backup: add endpoint to list included guests and volumes X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: PVE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2020 09:56:23 -0000 This patch adds a new API endpoint that returns a list of included guests, their volumes and whether they are included in a backup. The output is formatted to be used with the extJS tree panel. Signed-off-by: Aaron Lauterer --- The return types are `qemu`, `lxc` and `unknown`. The latter is there on purpose because it is possible that a deleted but not purged VM is still configured on a backup job. While the backup job itself will fail, I think it is good to show it in the job detail view so users can react to it. v3 -> v4: * remove the "not all permissions" field as we never show such notifications anywhere else. This makes the returned data simpler * define objects to be pushed in the return data directly in the push operation and not way ahead in the code. v2 -> v3 (hopefully I got them all): * incorporate feedback from thomas * changed double negative for permissions `not_all_permissions` to `permissions_for_all` * adapted to latest changes to return values from `get_included_guests` * define $guest only once * return VMID as int * renamed some vars to be more descriptive v1 -> v2: * simplified the code * refactored according to feedback PVE/API2/Backup.pm | 174 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+) diff --git a/PVE/API2/Backup.pm b/PVE/API2/Backup.pm index 86377c0a..6fbe2106 100644 --- a/PVE/API2/Backup.pm +++ b/PVE/API2/Backup.pm @@ -324,4 +324,178 @@ __PACKAGE__->register_method({ die "$@" if ($@); }}); +__PACKAGE__->register_method({ + name => 'get_volume_backup_included', + path => '{id}/included_volumes', + method => 'GET', + protected => 1, + description => "Returns included guests and the backup status of their disks. Optimized to be used in ExtJS tree views.", + permissions => { + check => ['perm', '/', ['Sys.Audit']], + }, + parameters => { + additionalProperties => 0, + properties => { + id => $vzdump_job_id_prop + }, + }, + returns => { + type => 'object', + description => 'Root node of the tree object. Children represent guests, grandchildren represent volumes of that guest.', + properties => { + children => { + type => 'array', + items => { + type => 'object', + properties => { + id => { + type => 'integer', + description => 'VMID of the guest.', + }, + name => { + type => 'string', + description => 'Name of the guest', + optional => 1, + }, + type => { + type => 'string', + description => 'Type of the guest, VM, CT or unknown for removed but not purged guests.', + enum => ['qemu', 'lxc', 'unknown'], + }, + children => { + type => 'array', + optional => 1, + description => 'The volumes of the guest with the information if they will be included in backups.', + items => { + type => 'object', + properties => { + id => { + type => 'string', + description => 'Configuration key of the volume.', + }, + name => { + type => 'string', + description => 'Name of the volume.', + }, + included => { + type => 'boolean', + description => 'Whether the volume is included in the backup or not.', + }, + reason => { + type => 'string', + description => 'The reason why the volume is included (or excluded).', + }, + }, + }, + }, + }, + }, + }, + }, + }, + code => sub { + my ($param) = @_; + + my $rpcenv = PVE::RPCEnvironment::get(); + + my $user = $rpcenv->get_user(); + + my $vzconf = cfs_read_file('vzdump.cron'); + my $all_jobs = $vzconf->{jobs} || []; + my $job; + my $rrd = PVE::Cluster::rrd_dump(); + + for my $j (@$all_jobs) { + if ($j->{id} eq $param->{id}) { + $job = $j; + last; + } + } + raise_param_exc({ id => "No such job '$param->{id}'" }) if !$job; + + my $vmlist = PVE::Cluster::get_vmlist(); + + my @job_vmids; + + my $included_guests = PVE::VZDump::get_included_guests($job); + + for my $node (keys %{$included_guests}) { + my $node_vmids = $included_guests->{$node}; + push(@job_vmids, @{$node_vmids}); + } + + # remove VMIDs to which the user has no permission to not leak infos + # like the guest name + my @allowed_vmids = grep { + $rpcenv->check($user, "/vms/$_", [ 'VM.Audit' ], 1); + } @job_vmids; + + my $result = { + children => [], + }; + + for my $vmid (@allowed_vmids) { + + my $children = []; + + # It's possible that a job has VMIDs configured that are not in + # vmlist. This could be because a guest was removed but not purged. + # Since there is no more data available we can only deliver the VMID + # and no volumes. + if (!defined $vmlist->{ids}->{$vmid}) { + push(@{$result->{children}}, { + id => int($vmid), + type => 'unknown', + leaf => 1, + }); + next; + } + + my $type = $vmlist->{ids}->{$vmid}->{type}; + my $node = $vmlist->{ids}->{$vmid}->{node}; + + my $conf; + my $volumes; + my $name = ""; + + if ($type eq 'qemu') { + $conf = PVE::QemuConfig->load_config($vmid, $node); + $volumes = PVE::QemuConfig->get_backup_volumes($conf); + $name = $conf->{name}; + } elsif ($type eq 'lxc') { + $conf = PVE::LXC::Config->load_config($vmid, $node); + $volumes = PVE::LXC::Config->get_backup_volumes($conf); + $name = $conf->{hostname}; + } else { + die "VMID $vmid is neither Qemu nor LXC guest\n"; + } + + foreach my $volume (@$volumes) { + my $disk = { + # id field must be unique for ExtJS tree view + id => "$vmid:$volume->{key}", + name => $volume->{volume_config}->{file} // $volume->{volume_config}->{volume}, + included=> $volume->{included}, + reason => $volume->{reason}, + leaf => 1, + }; + push(@{$children}, $disk); + } + + my $leaf = 0; + # it's possible for a guest to have no volumes configured + $leaf = 1 if !@{$children}; + + push(@{$result->{children}}, { + id => int($vmid), + type => $type, + name => $name, + children => $children, + leaf => $leaf, + }); + } + + return $result; + }}); + 1; -- 2.20.1