* [pve-devel] [PATCH common/storage/qemu-server] refactor pbs repo url generation
@ 2020-12-03 11:43 Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs Dominik Csapak
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Dominik Csapak @ 2020-12-03 11:43 UTC (permalink / raw)
To: pve-devel
moves it to pve-commons PBSClient and change qemu-server and pve-storage
to use it
this fixes issues with ipv6/ports in qemu-server pbs backups/restores
pve-common:
Dominik Csapak (1):
PBSClient: add get_repository to generate repository urls from configs
src/PVE/PBSClient.pm | 36 +++++++++++++++++++++++++++++++-----
1 file changed, 31 insertions(+), 5 deletions(-)
pve-storage:
Dominik Csapak (1):
PBSPlugin: use get_repository from PVE::PBSClient
PVE/Storage/PBSPlugin.pm | 25 +++++--------------------
1 file changed, 5 insertions(+), 20 deletions(-)
qemu-server:
Dominik Csapak (1):
use get_repository from PVE::PBSClient
PVE/QemuServer.pm | 6 ++----
PVE/VZDump/QemuServer.pm | 7 ++-----
2 files changed, 4 insertions(+), 9 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs
2020-12-03 11:43 [pve-devel] [PATCH common/storage/qemu-server] refactor pbs repo url generation Dominik Csapak
@ 2020-12-03 11:43 ` Dominik Csapak
2020-12-03 15:43 ` [pve-devel] applied: " Thomas Lamprecht
2020-12-03 11:43 ` [pve-devel] [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH qemu-server 1/1] " Dominik Csapak
2 siblings, 1 reply; 7+ messages in thread
From: Dominik Csapak @ 2020-12-03 11:43 UTC (permalink / raw)
To: pve-devel
we can use this everywhere we need to have a repository url
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/PVE/PBSClient.pm | 36 +++++++++++++++++++++++++++++++-----
1 file changed, 31 insertions(+), 5 deletions(-)
diff --git a/src/PVE/PBSClient.pm b/src/PVE/PBSClient.pm
index 44e3176..f05471c 100644
--- a/src/PVE/PBSClient.pm
+++ b/src/PVE/PBSClient.pm
@@ -10,7 +10,35 @@ use JSON;
use POSIX qw(strftime ENOENT);
use PVE::JSONSchema qw(get_standard_option);
-use PVE::Tools qw(run_command file_set_contents file_get_contents file_read_firstline);
+use PVE::Tools qw(run_command file_set_contents file_get_contents file_read_firstline $IPV6RE);
+
+# returns a repository string suitable for proxmox-backup-client, pbs-restore, etc.
+# $scfg must have the following structure:
+# {
+# datastore
+# server
+# port (optional defaults to 8007)
+# username (optional defaults to 'root@pam')
+# }
+sub get_repository {
+ my ($scfg) = @_;
+
+ my $server = $scfg->{server};
+ die "no server given\n" if !defined($server);
+
+ $server = "[$server]" if $server =~ /^$IPV6RE$/;
+
+ if (my $port = $scfg->{port}) {
+ $server .= ":$port" if $port != 8007;
+ }
+
+ my $datastore = $scfg->{datastore};
+ die "no datastore given\n" if !defined($datastore);
+
+ my $username = $scfg->{username} // 'root@pam';
+
+ return "$username\@$server:$datastore";
+}
sub new {
my ($class, $scfg, $storeid, $sdir) = @_;
@@ -116,9 +144,7 @@ my sub do_raw_client_cmd {
if ! -x $client_exe;
my $scfg = $self->{scfg};
- my $server = $scfg->{server};
- my $datastore = $scfg->{datastore};
- my $username = $scfg->{username} // 'root@pam';
+ my $repo = get_repository($scfg);
my $userns_cmd = delete $opts{userns_cmd};
@@ -144,7 +170,7 @@ my sub do_raw_client_cmd {
push @$cmd, @$param if defined($param);
- push @$cmd, "--repository", "$username\@$server:$datastore";
+ push @$cmd, "--repository", $repo;
local $ENV{PBS_PASSWORD} = $self->get_password();
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient
2020-12-03 11:43 [pve-devel] [PATCH common/storage/qemu-server] refactor pbs repo url generation Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs Dominik Csapak
@ 2020-12-03 11:43 ` Dominik Csapak
2020-12-03 16:23 ` [pve-devel] applied: " Thomas Lamprecht
2020-12-03 11:43 ` [pve-devel] [PATCH qemu-server 1/1] " Dominik Csapak
2 siblings, 1 reply; 7+ messages in thread
From: Dominik Csapak @ 2020-12-03 11:43 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
PVE/Storage/PBSPlugin.pm | 25 +++++--------------------
1 file changed, 5 insertions(+), 20 deletions(-)
diff --git a/PVE/Storage/PBSPlugin.pm b/PVE/Storage/PBSPlugin.pm
index 43bd72f..4d00472 100644
--- a/PVE/Storage/PBSPlugin.pm
+++ b/PVE/Storage/PBSPlugin.pm
@@ -13,6 +13,7 @@ use POSIX qw(strftime ENOENT);
use PVE::APIClient::LWP;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Network;
+use PVE::PBSClient;
use PVE::Storage::Plugin;
use PVE::Tools qw(run_command file_read_firstline trim dir_glob_regex dir_glob_foreach $IPV6RE);
@@ -161,18 +162,6 @@ sub print_volid {
return "${storeid}:${volname}";
}
-my sub get_server_with_port {
- my ($scfg) = @_;
-
- my $server = $scfg->{server};
- $server = "[$server]" if $server =~ /^$IPV6RE$/;
-
- if (my $port = $scfg->{port}) {
- $server .= ":$port" if $port != 8007;
- }
- return $server;
-}
-
my $USE_CRYPT_PARAMS = {
backup => 1,
restore => 1,
@@ -188,9 +177,7 @@ my sub do_raw_client_cmd {
die "executable not found '$client_exe'! Proxmox backup client not installed?\n"
if ! -x $client_exe;
- my $server = get_server_with_port($scfg);
- my $datastore = $scfg->{datastore};
- my $username = $scfg->{username} // 'root@pam';
+ my $repo = PVE::PBSClient::get_repository($scfg);
my $userns_cmd = delete $opts{userns_cmd};
@@ -216,7 +203,7 @@ my sub do_raw_client_cmd {
push @$cmd, @$param if defined($param);
- push @$cmd, "--repository", "$username\@$server:$datastore";
+ push @$cmd, "--repository", $repo;
local $ENV{PBS_PASSWORD} = pbs_get_password($scfg, $storeid);
@@ -484,12 +471,10 @@ sub path {
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
- my $server = get_server_with_port($scfg);
- my $datastore = $scfg->{datastore};
- my $username = $scfg->{username} // 'root@pam';
+ my $repo = PVE::PBSClient::get_repository($scfg);
# artifical url - we currently do not use that anywhere
- my $path = "pbs://$username\@$server:$datastore/$name";
+ my $path = "pbs://$repo/$name";
return ($path, $vmid, $vtype);
}
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] [PATCH qemu-server 1/1] use get_repository from PVE::PBSClient
2020-12-03 11:43 [pve-devel] [PATCH common/storage/qemu-server] refactor pbs repo url generation Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient Dominik Csapak
@ 2020-12-03 11:43 ` Dominik Csapak
2020-12-03 17:04 ` [pve-devel] applied: " Thomas Lamprecht
2 siblings, 1 reply; 7+ messages in thread
From: Dominik Csapak @ 2020-12-03 11:43 UTC (permalink / raw)
To: pve-devel
this fixes the issue that we did not generate the correct repository
url for pbs storages that contained an ipv6 address or a port
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
PVE/QemuServer.pm | 6 ++----
PVE/VZDump/QemuServer.pm | 7 ++-----
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 4989938..70c14ba 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -34,6 +34,7 @@ use PVE::GuestHelpers qw(safe_string_ne safe_num_ne safe_boolean_ne);
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option parse_property_string);
use PVE::ProcFSTools;
+use PVE::PBSClient;
use PVE::RPCEnvironment;
use PVE::Storage;
use PVE::SysFSTools;
@@ -6058,13 +6059,10 @@ sub restore_proxmox_backup_archive {
my ($storeid, $volname) = PVE::Storage::parse_volume_id($archive);
my $scfg = PVE::Storage::storage_config($storecfg, $storeid);
- my $server = $scfg->{server};
- my $datastore = $scfg->{datastore};
- my $username = $scfg->{username} // 'root@pam';
my $fingerprint = $scfg->{fingerprint};
my $keyfile = PVE::Storage::PBSPlugin::pbs_encryption_key_file_name($storecfg, $storeid);
- my $repo = "$username\@$server:$datastore";
+ my $repo = PVE::PBSClient::get_repository($scfg);
# This is only used for `pbs-restore`!
my $password = PVE::Storage::PBSPlugin::pbs_get_password($scfg, $storeid);
diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index 5003676..b322701 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -14,6 +14,7 @@ use PVE::Cluster qw(cfs_read_file);
use PVE::INotify;
use PVE::IPCC;
use PVE::JSONSchema;
+use PVE::PBSClient;
use PVE::QMPClient;
use PVE::Storage::Plugin;
use PVE::Storage::PBSPlugin;
@@ -473,12 +474,8 @@ sub archive_pbs {
my $starttime = time();
- my $server = $scfg->{server};
- my $datastore = $scfg->{datastore};
- my $username = $scfg->{username} // 'root@pam';
my $fingerprint = $scfg->{fingerprint};
-
- my $repo = "$username\@$server:$datastore";
+ my $repo = PVE::PBSClient::get_repository($scfg);
my $password = PVE::Storage::PBSPlugin::pbs_get_password($scfg, $opts->{storage});
my $keyfile = PVE::Storage::PBSPlugin::pbs_encryption_key_file_name($scfg, $opts->{storage});
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] applied: [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs
2020-12-03 11:43 ` [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs Dominik Csapak
@ 2020-12-03 15:43 ` Thomas Lamprecht
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Lamprecht @ 2020-12-03 15:43 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
On 03.12.20 12:43, Dominik Csapak wrote:
> we can use this everywhere we need to have a repository url
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/PVE/PBSClient.pm | 36 +++++++++++++++++++++++++++++++-----
> 1 file changed, 31 insertions(+), 5 deletions(-)
>
>
applied, thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] applied: [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient
2020-12-03 11:43 ` [pve-devel] [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient Dominik Csapak
@ 2020-12-03 16:23 ` Thomas Lamprecht
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Lamprecht @ 2020-12-03 16:23 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
On 03.12.20 12:43, Dominik Csapak wrote:
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> PVE/Storage/PBSPlugin.pm | 25 +++++--------------------
> 1 file changed, 5 insertions(+), 20 deletions(-)
>
>
applied, thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] applied: [PATCH qemu-server 1/1] use get_repository from PVE::PBSClient
2020-12-03 11:43 ` [pve-devel] [PATCH qemu-server 1/1] " Dominik Csapak
@ 2020-12-03 17:04 ` Thomas Lamprecht
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Lamprecht @ 2020-12-03 17:04 UTC (permalink / raw)
To: Proxmox VE development discussion, Dominik Csapak
On 03.12.20 12:43, Dominik Csapak wrote:
> this fixes the issue that we did not generate the correct repository
> URL for pbs storages that contained an ipv6 address or a port
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> PVE/QemuServer.pm | 6 ++----
> PVE/VZDump/QemuServer.pm | 7 ++-----
> 2 files changed, 4 insertions(+), 9 deletions(-)
>
>
applied, thanks!
Did just a successful restore with
> connecting to repository 'root@pam!holerio@[fd0f:42::38:68]:zzz'
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-12-03 17:05 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-03 11:43 [pve-devel] [PATCH common/storage/qemu-server] refactor pbs repo url generation Dominik Csapak
2020-12-03 11:43 ` [pve-devel] [PATCH common 1/1] PBSClient: add get_repository to generate repository urls from configs Dominik Csapak
2020-12-03 15:43 ` [pve-devel] applied: " Thomas Lamprecht
2020-12-03 11:43 ` [pve-devel] [PATCH storage 1/1] PBSPlugin: use get_repository from PVE::PBSClient Dominik Csapak
2020-12-03 16:23 ` [pve-devel] applied: " Thomas Lamprecht
2020-12-03 11:43 ` [pve-devel] [PATCH qemu-server 1/1] " Dominik Csapak
2020-12-03 17:04 ` [pve-devel] applied: " Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox