[pve-devel] [PATCH-SERIES v2 http-server/storage/manager] add checksum and algorithm to iso upload

[pve-devel] [PATCH-SERIES v2 http-server/storage/manager] add checksum and algorithm to iso upload

[Lorenz Stechauner]

changes to v1:
* dropped commits, that were already applied in the mean time
* better commit messages
* dropped the 'new-filename' - using 'filename' instead
* error handling for unlink
* dropped check for `new FormData()` in pve-manager
* fixed commit for pve-http-server (deleting temp file on errors)
* check file extention in front end too

note: two commits in total state, that they break/require some 
versions of other repos. please do not forget to bump those and
create the accoring 'breaks' or 'requires'.


pve-http-server:
Lorenz Stechauner (1):
  anyevent: move unlink from http-server to endpoint

 src/PVE/APIServer/AnyEvent.pm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)


pve-storage:
Lorenz Stechauner (3):
  status: move unlink from http-server to enpoint
  status: remove sleep(1) in file upload
  status: add checksum and algorithm to file upload

 PVE/API2/Storage/Status.pm | 44 +++++++++++++++++++++++++++++++-------
 1 file changed, 36 insertions(+), 8 deletions(-)


pve-manager:
Lorenz Stechauner (5):
  ui: move upload window into UploadToStorage.js
  ui: refactor UploadToStorage.js
  ui/UploadToStorage: add checksum and algorithm
  ui/UploadToStorage: add TaskViewer
  ui/UplaodToStorage: check file extension

 www/manager6/Makefile                  |   1 +
 www/manager6/storage/ContentView.js    | 197 +---------------
 www/manager6/window/UploadToStorage.js | 311 +++++++++++++++++++++++++
 3 files changed, 314 insertions(+), 195 deletions(-)
 create mode 100644 www/manager6/window/UploadToStorage.js

-- 
2.30.2

[pve-devel] [PATCH v2 http-server 1/1] anyevent: move unlink from http-server to endpoint

[Lorenz Stechauner]

any uploaded file has to be deleted by the corrosponding
endpoint. the file upload was only used by the 'upload to
storage' feature in pve.

this change allows the endpoint to delete the file itself,
making the old and racey`sleep 1` (waiting until the worker
has opened the file) obsolete.

this change breaks all pve-manager versions, in which the
worker does not unlink the temp file itself.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 src/PVE/APIServer/AnyEvent.pm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/PVE/APIServer/AnyEvent.pm b/src/PVE/APIServer/AnyEvent.pm
index cd77806..23b8d10 100644
--- a/src/PVE/APIServer/AnyEvent.pm
+++ b/src/PVE/APIServer/AnyEvent.pm
@@ -114,7 +114,7 @@ sub log_aborted_request {
 }
 
 sub cleanup_reqstate {
-    my ($reqstate) = @_;
+    my ($reqstate, $deletetmpfile) = @_;
 
     delete $reqstate->{log};
     delete $reqstate->{request};
@@ -123,7 +123,7 @@ sub cleanup_reqstate {
     delete $reqstate->{starttime};
 
     if ($reqstate->{tmpfilename}) {
-	unlink $reqstate->{tmpfilename};
+	unlink $reqstate->{tmpfilename} if $deletetmpfile;
 	delete $reqstate->{tmpfilename};
     }
 }
@@ -131,7 +131,7 @@ sub cleanup_reqstate {
 sub client_do_disconnect {
     my ($self, $reqstate) = @_;
 
-    cleanup_reqstate($reqstate);
+    cleanup_reqstate($reqstate, 1);
 
     my $shutdown_hdl = sub {
 	my $hdl = shift;
-- 
2.30.2

[pve-devel] [PATCH v2 storage 1/3] status: move unlink from http-server to enpoint

[Lorenz Stechauner]

this is the first step in which not the http server removes the
temporary file, but the worker itself.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 PVE/API2/Storage/Status.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
index 16581aa..fcf9720 100644
--- a/PVE/API2/Storage/Status.pm
+++ b/PVE/API2/Storage/Status.pm
@@ -480,6 +480,7 @@ __PACKAGE__->register_method ({
 	    print "command: " . join(' ', @$cmd) . "\n";
 
 	    eval { PVE::Tools::run_command($cmd, errmsg => 'import failed'); };
+	    unlink $tmpfilename or warn "unable to clean up temporory file '$tmpfilename' - $!";
 	    if (my $err = $@) {
 		unlink $dest;
 		die $err;
-- 
2.30.2

[pve-devel] [PATCH v2 storage 2/3] status: remove sleep(1) in file upload

[Lorenz Stechauner]

this racey sleep(1) is only there for legacy reasons: because
we don't use apache anymore and only emulate its behabiour
regarding removing temp files, this is under our own control
now and so we can improve this whole situation.

this change requires a pve-http-server version, in which the
tmpfile gets not automatically removed anymore.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 PVE/API2/Storage/Status.pm | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
index fcf9720..9cf6e40 100644
--- a/PVE/API2/Storage/Status.pm
+++ b/PVE/API2/Storage/Status.pm
@@ -488,14 +488,7 @@ __PACKAGE__->register_method ({
 	    print "finished file import successfully\n";
 	};
 
-	my $upid = $rpcenv->fork_worker('imgcopy', undef, $user, $worker);
-
-	# apache removes the temporary file on return, so we need
-	# to wait here to make sure the worker process starts and
-	# opens the file before it gets removed.
-	sleep(1);
-
-	return $upid;
+	return $rpcenv->fork_worker('imgcopy', undef, $user, $worker);
    }});
 
 __PACKAGE__->register_method({
-- 
2.30.2

[pve-devel] [PATCH v2 storage 1/1] status: add checksum and algorithm to file upload

[Lorenz Stechauner]

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 PVE/API2/Storage/Status.pm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
index 9cf6e40..4b9fda5 100644
--- a/PVE/API2/Storage/Status.pm
+++ b/PVE/API2/Storage/Status.pm
@@ -385,6 +385,19 @@ __PACKAGE__->register_method ({
 		maxLength => 255,
 		type => 'string',
 	    },
+	    checksum => {
+		description => "The expected checksum of the file.",
+		type => 'string',
+		requires => 'checksum-algorithm',
+		optional => 1,
+	    },
+	    'checksum-algorithm' => {
+		description => "The algorithm to calculate the checksum of the file.",
+		type => 'string',
+		enum => ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512'],
+		requires => 'checksum',
+		optional => 1,
+	    },
 	    tmpfilename => {
 		description => "The source file name. This parameter is usually set by the REST handler. You can only overwrite it when connecting to the trusted port on localhost.",
 		type => 'string',
@@ -474,6 +487,27 @@ __PACKAGE__->register_method ({
 	    my $upid = shift;
 
 	    print "starting file import from: $tmpfilename\n";
+
+	    eval {
+		my ($checksum, $checksum_algorithm) = $param->@{'checksum', 'checksum-algorithm'};
+		if ($checksum_algorithm) {
+		    print "calculating checksum...";
+
+		    my $checksum_got = PVE::Tools::get_file_hash($checksum_algorithm, $tmpfilename);
+
+		    if (lc($checksum_got) eq lc($checksum)) {
+			print "OK, checksum verified\n";
+		    } else {
+			print "\n";  # the front end expects the error to reside at the last line without any noise
+			die "checksum mismatch: got '$checksum_got' != expect '$checksum'\n";
+		    }
+		}
+	    };
+	    if (my $err = $@) {
+		unlink $tmpfilename or warn "unable to clean up temporory file '$tmpfilename' - $!";
+		die $err;
+	    }
+
 	    print "target node: $node\n";
 	    print "target file: $dest\n";
 	    print "file size is: $size\n";
-- 
2.30.2

[pve-devel] [PATCH v2 manager 1/5] ui: move upload window into UploadToStorage.js

[Lorenz Stechauner]

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 www/manager6/Makefile                  |   1 +
 www/manager6/storage/ContentView.js    | 195 +------------------------
 www/manager6/window/UploadToStorage.js | 192 ++++++++++++++++++++++++
 3 files changed, 194 insertions(+), 194 deletions(-)
 create mode 100644 www/manager6/window/UploadToStorage.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 75d355a5..552e842b 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -106,6 +106,7 @@ JSSRC= 							\
 	window/Snapshot.js				\
 	window/StartupEdit.js				\
 	window/DownloadUrlToStorage.js 			\
+	window/UploadToStorage.js 			\
 	window/Wizard.js				\
 	ha/Fencing.js					\
 	ha/GroupEdit.js					\
diff --git a/www/manager6/storage/ContentView.js b/www/manager6/storage/ContentView.js
index 3f5b686b..ca0ad664 100644
--- a/www/manager6/storage/ContentView.js
+++ b/www/manager6/storage/ContentView.js
@@ -1,196 +1,3 @@
-Ext.define('PVE.storage.Upload', {
-    extend: 'Ext.window.Window',
-    alias: 'widget.pveStorageUpload',
-
-    resizable: false,
-
-    modal: true,
-
-    initComponent: function() {
-        var me = this;
-
-	if (!me.nodename) {
-	    throw "no node name specified";
-	}
-	if (!me.storage) {
-	    throw "no storage ID specified";
-	}
-
-	let baseurl = `/nodes/${me.nodename}/storage/${me.storage}/upload`;
-
-	let pbar = Ext.create('Ext.ProgressBar', {
-            text: 'Ready',
-	    hidden: true,
-	});
-
-	let acceptedExtensions = {
-	    iso: ".img, .iso",
-	    vztmpl: ".tar.gz, .tar.xz",
-	};
-
-	let defaultContent = me.contents[0] || '';
-
-	let fileField = Ext.create('Ext.form.field.File', {
-	    name: 'filename',
-	    buttonText: gettext('Select File...'),
-	    allowBlank: false,
-	    setAccept: function(content) {
-		let acceptString = acceptedExtensions[content] || '';
-		this.fileInputEl.set({
-		    accept: acceptString,
-		});
-	    },
-	    listeners: {
-		afterrender: function(cmp) {
-		    cmp.setAccept(defaultContent);
-		},
-	    },
-	});
-
-	me.formPanel = Ext.create('Ext.form.Panel', {
-	    method: 'POST',
-	    waitMsgTarget: true,
-	    bodyPadding: 10,
-	    border: false,
-	    width: 300,
-	    fieldDefaults: {
-		labelWidth: 100,
-		anchor: '100%',
-            },
-	    items: [
-		{
-		    xtype: 'pveContentTypeSelector',
-		    cts: me.contents,
-		    fieldLabel: gettext('Content'),
-		    name: 'content',
-		    value: defaultContent,
-		    allowBlank: false,
-		    listeners: {
-			change: function(cmp, newValue, oldValue) {
-			    fileField.setAccept(newValue);
-			},
-		    },
-		},
-		fileField,
-		pbar,
-	    ],
-	});
-
-	let form = me.formPanel.getForm();
-
-	let doStandardSubmit = function() {
-	    form.submit({
-		url: "/api2/htmljs" + baseurl,
-		waitMsg: gettext('Uploading file...'),
-		success: function(f, action) {
-		    me.close();
-		},
-		failure: function(f, action) {
-		    var msg = PVE.Utils.extractFormActionError(action);
-                    Ext.Msg.alert(gettext('Error'), msg);
-		},
-	    });
-	};
-
-	let updateProgress = function(per, bytes) {
-	    var text = (per * 100).toFixed(2) + '%';
-	    if (bytes) {
-		text += " (" + Proxmox.Utils.format_size(bytes) + ')';
-	    }
-	    pbar.updateProgress(per, text);
-	};
-
-	let abortBtn = Ext.create('Ext.Button', {
-	    text: gettext('Abort'),
-	    disabled: true,
-	    handler: function() {
-		me.close();
-	    },
-	});
-
-	let submitBtn = Ext.create('Ext.Button', {
-	    text: gettext('Upload'),
-	    disabled: true,
-	    handler: function(button) {
-		var fd;
-		try {
-		    fd = new FormData();
-		} catch (err) {
-		    doStandardSubmit();
-		    return;
-		}
-
-		button.setDisabled(true);
-		abortBtn.setDisabled(false);
-
-		var field = form.findField('content');
-		fd.append("content", field.getValue());
-		field.setDisabled(true);
-
-		field = form.findField('filename');
-		var file = field.fileInputEl.dom;
-		fd.append("filename", file.files[0]);
-		field.setDisabled(true);
-
-		pbar.setVisible(true);
-		updateProgress(0);
-
-		let xhr = new XMLHttpRequest();
-		me.xhr = xhr;
-
-		xhr.addEventListener("load", function(e) {
-		    if (xhr.status === 200) {
-			me.close();
-			return;
-		    }
-		    let err = Ext.htmlEncode(xhr.statusText);
-		    let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
-		    if (xhr.responseText !== "") {
-			let result = Ext.decode(xhr.responseText);
-			result.message = msg;
-			msg = Proxmox.Utils.extractRequestError(result, true);
-		    }
-		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
-		}, false);
-
-		xhr.addEventListener("error", function(e) {
-		    let err = e.target.status.toString();
-		    let msg = `Error '${err}' occurred while receiving the document.`;
-		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
-		});
-
-		xhr.upload.addEventListener("progress", function(evt) {
-		    if (evt.lengthComputable) {
-			let percentComplete = evt.loaded / evt.total;
-			updateProgress(percentComplete, evt.loaded);
-		    }
-		}, false);
-
-		xhr.open("POST", `/api2/json${baseurl}`, true);
-		xhr.send(fd);
-	    },
-	});
-
-	form.on('validitychange', (f, valid) => submitBtn.setDisabled(!valid));
-
-	Ext.apply(me, {
-	    title: gettext('Upload'),
-	    items: me.formPanel,
-	    buttons: [abortBtn, submitBtn],
-	    listeners: {
-		close: function() {
-		    if (me.xhr) {
-			me.xhr.abort();
-			delete me.xhr;
-		    }
-		},
-	    },
-	});
-
-        me.callParent();
-    },
-});
-
 Ext.define('PVE.storage.ContentView', {
     extend: 'Ext.grid.GridPanel',
 
@@ -259,7 +66,7 @@ Ext.define('PVE.storage.ContentView', {
 		    text: gettext('Upload'),
 		    disabled: !me.enableUploadButton,
 		    handler: function() {
-			Ext.create('PVE.storage.Upload', {
+			Ext.create('PVE.window.UploadToStorage', {
 			    nodename: nodename,
 			    storage: storage,
 			    contents: [content],
diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
new file mode 100644
index 00000000..3c35020a
--- /dev/null
+++ b/www/manager6/window/UploadToStorage.js
@@ -0,0 +1,192 @@
+Ext.define('PVE.window.UploadToStorage', {
+    extend: 'Ext.window.Window',
+    alias: 'widget.pveStorageUpload',
+
+    resizable: false,
+
+    modal: true,
+
+    initComponent: function() {
+        var me = this;
+
+	if (!me.nodename) {
+	    throw "no node name specified";
+	}
+	if (!me.storage) {
+	    throw "no storage ID specified";
+	}
+
+	let baseurl = `/nodes/${me.nodename}/storage/${me.storage}/upload`;
+
+	let pbar = Ext.create('Ext.ProgressBar', {
+            text: 'Ready',
+	    hidden: true,
+	});
+
+	let acceptedExtensions = {
+	    iso: ".img, .iso",
+	    vztmpl: ".tar.gz, .tar.xz",
+	};
+
+	let defaultContent = me.contents[0] || '';
+
+	let fileField = Ext.create('Ext.form.field.File', {
+	    name: 'filename',
+	    buttonText: gettext('Select File...'),
+	    allowBlank: false,
+	    setAccept: function(content) {
+		let acceptString = acceptedExtensions[content] || '';
+		this.fileInputEl.set({
+		    accept: acceptString,
+		});
+	    },
+	    listeners: {
+		afterrender: function(cmp) {
+		    cmp.setAccept(defaultContent);
+		},
+	    },
+	});
+
+	me.formPanel = Ext.create('Ext.form.Panel', {
+	    method: 'POST',
+	    waitMsgTarget: true,
+	    bodyPadding: 10,
+	    border: false,
+	    width: 300,
+	    fieldDefaults: {
+		labelWidth: 100,
+		anchor: '100%',
+            },
+	    items: [
+		{
+		    xtype: 'pveContentTypeSelector',
+		    cts: me.contents,
+		    fieldLabel: gettext('Content'),
+		    name: 'content',
+		    value: defaultContent,
+		    allowBlank: false,
+		    listeners: {
+			change: function(cmp, newValue, oldValue) {
+			    fileField.setAccept(newValue);
+			},
+		    },
+		},
+		fileField,
+		pbar,
+	    ],
+	});
+
+	let form = me.formPanel.getForm();
+
+	let doStandardSubmit = function() {
+	    form.submit({
+		url: "/api2/htmljs" + baseurl,
+		waitMsg: gettext('Uploading file...'),
+		success: function(f, action) {
+		    me.close();
+		},
+		failure: function(f, action) {
+		    var msg = PVE.Utils.extractFormActionError(action);
+                    Ext.Msg.alert(gettext('Error'), msg);
+		},
+	    });
+	};
+
+	let updateProgress = function(per, bytes) {
+	    var text = (per * 100).toFixed(2) + '%';
+	    if (bytes) {
+		text += " (" + Proxmox.Utils.format_size(bytes) + ')';
+	    }
+	    pbar.updateProgress(per, text);
+	};
+
+	let abortBtn = Ext.create('Ext.Button', {
+	    text: gettext('Abort'),
+	    disabled: true,
+	    handler: function() {
+		me.close();
+	    },
+	});
+
+	let submitBtn = Ext.create('Ext.Button', {
+	    text: gettext('Upload'),
+	    disabled: true,
+	    handler: function(button) {
+		var fd;
+		try {
+		    fd = new FormData();
+		} catch (err) {
+		    doStandardSubmit();
+		    return;
+		}
+
+		button.setDisabled(true);
+		abortBtn.setDisabled(false);
+
+		var field = form.findField('content');
+		fd.append("content", field.getValue());
+		field.setDisabled(true);
+
+		field = form.findField('filename');
+		var file = field.fileInputEl.dom;
+		fd.append("filename", file.files[0]);
+		field.setDisabled(true);
+
+		pbar.setVisible(true);
+		updateProgress(0);
+
+		let xhr = new XMLHttpRequest();
+		me.xhr = xhr;
+
+		xhr.addEventListener("load", function(e) {
+		    if (xhr.status === 200) {
+			me.close();
+			return;
+		    }
+		    let err = Ext.htmlEncode(xhr.statusText);
+		    let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
+		    if (xhr.responseText !== "") {
+			let result = Ext.decode(xhr.responseText);
+			result.message = msg;
+			msg = Proxmox.Utils.extractRequestError(result, true);
+		    }
+		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
+		}, false);
+
+		xhr.addEventListener("error", function(e) {
+		    let err = e.target.status.toString();
+		    let msg = `Error '${err}' occurred while receiving the document.`;
+		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
+		});
+
+		xhr.upload.addEventListener("progress", function(evt) {
+		    if (evt.lengthComputable) {
+			let percentComplete = evt.loaded / evt.total;
+			updateProgress(percentComplete, evt.loaded);
+		    }
+		}, false);
+
+		xhr.open("POST", `/api2/json${baseurl}`, true);
+		xhr.send(fd);
+	    },
+	});
+
+	form.on('validitychange', (f, valid) => submitBtn.setDisabled(!valid));
+
+	Ext.apply(me, {
+	    title: gettext('Upload'),
+	    items: me.formPanel,
+	    buttons: [abortBtn, submitBtn],
+	    listeners: {
+		close: function() {
+		    if (me.xhr) {
+			me.xhr.abort();
+			delete me.xhr;
+		    }
+		},
+	    },
+	});
+
+        me.callParent();
+    },
+});
-- 
2.30.2

[pve-devel] [PATCH v2 manager 2/5] ui: refactor UploadToStorage.js

[Lorenz Stechauner]

this also removes the "content" selector from the window.
as far as it seems, this selector was never able to select
more than one entry, so it was useless.

the check for FormData() is also removed, because this is
supported by all major browsers for a long time. therefore
doStandardSubmit() is also not necessary.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 www/manager6/storage/ContentView.js    |   2 +-
 www/manager6/window/UploadToStorage.js | 360 ++++++++++++++-----------
 2 files changed, 209 insertions(+), 153 deletions(-)

diff --git a/www/manager6/storage/ContentView.js b/www/manager6/storage/ContentView.js
index ca0ad664..00a94f3c 100644
--- a/www/manager6/storage/ContentView.js
+++ b/www/manager6/storage/ContentView.js
@@ -69,7 +69,7 @@ Ext.define('PVE.storage.ContentView', {
 			Ext.create('PVE.window.UploadToStorage', {
 			    nodename: nodename,
 			    storage: storage,
-			    contents: [content],
+			    content: content,
 			    autoShow: true,
 			    taskDone: () => reload(),
 			});
diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index 3c35020a..fb9850b3 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -1,191 +1,247 @@
 Ext.define('PVE.window.UploadToStorage', {
     extend: 'Ext.window.Window',
     alias: 'widget.pveStorageUpload',
+    mixins: ['Proxmox.Mixin.CBind'],
 
     resizable: false,
-
     modal: true,
 
-    initComponent: function() {
-        var me = this;
+    title: gettext('Upload'),
 
-	if (!me.nodename) {
-	    throw "no node name specified";
-	}
-	if (!me.storage) {
-	    throw "no storage ID specified";
-	}
+    acceptedExtensions: {
+	iso: ['.img', '.iso'],
+	vztmpl: ['.tar.gz', '.tar.xz'],
+    },
+
+    cbindData: function(initialConfig) {
+	const me = this;
+	return {
+	    nodename: me.nodename,
+	    storage: me.storage,
+	    content: me.content,
+	    extensions: me.acceptedExtensions[me.content].join(', '),
+	};
+    },
 
-	let baseurl = `/nodes/${me.nodename}/storage/${me.storage}/upload`;
+    cbind: {
+	url: `/nodes/{nodename}/storage/{storage}/upload`,
+    },
 
-	let pbar = Ext.create('Ext.ProgressBar', {
-            text: 'Ready',
-	    hidden: true,
-	});
+    viewModel: {
+	data: {
+	    size: '-',
+	    mimetype: '-',
+	    filename: '',
+	},
+    },
 
-	let acceptedExtensions = {
-	    iso: ".img, .iso",
-	    vztmpl: ".tar.gz, .tar.xz",
-	};
+    controller: {
+	submit: function(button) {
+	    const view = this.getView();
+	    const form = Ext.getCmp('formPanel').getForm();
+	    const abortBtn = Ext.getCmp('abortBtn');
+	    const pbar = Ext.getCmp('progressBar');
+
+	    const updateProgress = function(per, bytes) {
+		let text = (per * 100).toFixed(2) + '%';
+		if (bytes) {
+		    text += " (" + Proxmox.Utils.format_size(bytes) + ')';
+		}
+		pbar.updateProgress(per, text);
+	    };
 
-	let defaultContent = me.contents[0] || '';
-
-	let fileField = Ext.create('Ext.form.field.File', {
-	    name: 'filename',
-	    buttonText: gettext('Select File...'),
-	    allowBlank: false,
-	    setAccept: function(content) {
-		let acceptString = acceptedExtensions[content] || '';
-		this.fileInputEl.set({
-		    accept: acceptString,
-		});
-	    },
-	    listeners: {
-		afterrender: function(cmp) {
-		    cmp.setAccept(defaultContent);
-		},
-	    },
-	});
+	    const fd = new FormData();
+
+	    button.setDisabled(true);
+	    abortBtn.setDisabled(false);
 
-	me.formPanel = Ext.create('Ext.form.Panel', {
+	    const contentField = form.findField('content');
+	    fd.append("content", contentField.getValue());
+	    contentField.setDisabled(true);
+
+	    const fileField = form.findField('file');
+	    const file = fileField.fileInputEl.dom.files[0];
+	    fileField.setDisabled(true);
+
+	    const filenameField = form.findField('filename');
+	    const filename = filenameField.getValue();
+	    filenameField.setDisabled(true);
+
+	    const algorithmField = form.findField('checksum-algorithm');
+	    algorithmField.setDisabled(true);
+	    if (algorithmField.getValue() !== '__default__') {
+		fd.append("checksum-algorithm", algorithmField.getValue());
+
+		const checksumField = form.findField('checksum');
+		fd.append("checksum", checksumField.getValue());
+		checksumField.setDisabled(true);
+	    }
+
+	    fd.append("filename", file, filename);
+
+	    pbar.setVisible(true);
+	    updateProgress(0);
+
+	    const xhr = new XMLHttpRequest();
+	    view.xhr = xhr;
+
+	    xhr.addEventListener("load", function(e) {
+		if (xhr.status === 200) {
+		    view.close();
+		    return;
+		}
+		const err = Ext.htmlEncode(xhr.statusText);
+		let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
+		if (xhr.responseText !== "") {
+		    const result = Ext.decode(xhr.responseText);
+		    result.message = msg;
+		    msg = Proxmox.Utils.extractRequestError(result, true);
+		}
+		Ext.Msg.alert(gettext('Error'), msg, btn => view.close());
+	    }, false);
+
+	    xhr.addEventListener("error", function(e) {
+		const err = e.target.status.toString();
+		const msg = `Error '${err}' occurred while receiving the document.`;
+		Ext.Msg.alert(gettext('Error'), msg, btn => view.close());
+	    });
+
+	    xhr.upload.addEventListener("progress", function(evt) {
+		if (evt.lengthComputable) {
+		    const percentComplete = evt.loaded / evt.total;
+		    updateProgress(percentComplete, evt.loaded);
+		}
+	    }, false);
+
+	    xhr.open("POST", `/api2/json${view.url}`, true);
+	    xhr.send(fd);
+	},
+
+	validitychange: function(f, valid) {
+	    const submitBtn = Ext.getCmp('submitBtn');
+	    submitBtn.setDisabled(!valid);
+	},
+
+	fileChange: function(input) {
+	    const vm = this.getViewModel();
+	    const name = input.value.replace(/^.*(\/|\\)/, '');
+	    const fileInput = input.fileInputEl.dom;
+	    vm.set('filename', name);
+	    vm.set('size', (fileInput.files[0] && Proxmox.Utils.format_size(fileInput.files[0].size)) || '-');
+	    vm.set('mimetype', (fileInput.files[0] && fileInput.files[0].type) || '-');
+	},
+    },
+
+    items: [
+	{
+	    xtype: 'form',
+	    id: 'formPanel',
 	    method: 'POST',
 	    waitMsgTarget: true,
 	    bodyPadding: 10,
 	    border: false,
-	    width: 300,
+	    width: 400,
 	    fieldDefaults: {
 		labelWidth: 100,
 		anchor: '100%',
             },
 	    items: [
 		{
-		    xtype: 'pveContentTypeSelector',
-		    cts: me.contents,
-		    fieldLabel: gettext('Content'),
-		    name: 'content',
-		    value: defaultContent,
+		    xtype: 'filefield',
+		    name: 'file',
+		    buttonText: gettext('Select File'),
 		    allowBlank: false,
+		    fieldLabel: gettext('File'),
+		    cbind: {
+			accept: '{extensions}',
+		    },
 		    listeners: {
-			change: function(cmp, newValue, oldValue) {
-			    fileField.setAccept(newValue);
-			},
+			change: 'fileChange',
 		    },
 		},
-		fileField,
-		pbar,
-	    ],
-	});
-
-	let form = me.formPanel.getForm();
-
-	let doStandardSubmit = function() {
-	    form.submit({
-		url: "/api2/htmljs" + baseurl,
-		waitMsg: gettext('Uploading file...'),
-		success: function(f, action) {
-		    me.close();
+		{
+		    xtype: 'textfield',
+		    name: 'filename',
+		    allowBlank: false,
+		    fieldLabel: gettext('File name'),
+		    bind: {
+			value: '{filename}',
+		    },
 		},
-		failure: function(f, action) {
-		    var msg = PVE.Utils.extractFormActionError(action);
-                    Ext.Msg.alert(gettext('Error'), msg);
+		{
+		    xtype: 'displayfield',
+		    name: 'size',
+		    fieldLabel: gettext('File size'),
+		    bind: {
+			value: '{size}',
+		    },
 		},
-	    });
-	};
-
-	let updateProgress = function(per, bytes) {
-	    var text = (per * 100).toFixed(2) + '%';
-	    if (bytes) {
-		text += " (" + Proxmox.Utils.format_size(bytes) + ')';
-	    }
-	    pbar.updateProgress(per, text);
-	};
-
-	let abortBtn = Ext.create('Ext.Button', {
+		{
+		    xtype: 'displayfield',
+		    name: 'mimetype',
+		    fieldLabel: gettext('MIME type'),
+		    bind: {
+			value: '{mimetype}',
+		    },
+		},
+		{
+		    xtype: 'progressbar',
+		    text: 'Ready',
+		    hidden: true,
+		    id: 'progressBar',
+		},
+		{
+		    xtype: 'hiddenfield',
+		    name: 'content',
+		    cbind: {
+			value: '{content}',
+		    },
+		},
+	    ],
+	   listeners: {
+		validitychange: 'validitychange',
+	   },
+	},
+    ],
+
+    buttons: [
+	{
+	    xtype: 'button',
 	    text: gettext('Abort'),
+	    id: 'abortBtn',
 	    disabled: true,
 	    handler: function() {
-		me.close();
+		const me = this;
+		me.up('pveStorageUpload').close();
 	    },
-	});
-
-	let submitBtn = Ext.create('Ext.Button', {
+	},
+	{
 	    text: gettext('Upload'),
+	    id: 'submitBtn',
 	    disabled: true,
-	    handler: function(button) {
-		var fd;
-		try {
-		    fd = new FormData();
-		} catch (err) {
-		    doStandardSubmit();
-		    return;
-		}
+	    handler: 'submit',
+	},
+    ],
+
+    listeners: {
+	close: function() {
+	    const me = this;
+	    if (me.xhr) {
+		me.xhr.abort();
+		delete me.xhr;
+	    }
+	},
+    },
 
-		button.setDisabled(true);
-		abortBtn.setDisabled(false);
-
-		var field = form.findField('content');
-		fd.append("content", field.getValue());
-		field.setDisabled(true);
-
-		field = form.findField('filename');
-		var file = field.fileInputEl.dom;
-		fd.append("filename", file.files[0]);
-		field.setDisabled(true);
-
-		pbar.setVisible(true);
-		updateProgress(0);
-
-		let xhr = new XMLHttpRequest();
-		me.xhr = xhr;
-
-		xhr.addEventListener("load", function(e) {
-		    if (xhr.status === 200) {
-			me.close();
-			return;
-		    }
-		    let err = Ext.htmlEncode(xhr.statusText);
-		    let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
-		    if (xhr.responseText !== "") {
-			let result = Ext.decode(xhr.responseText);
-			result.message = msg;
-			msg = Proxmox.Utils.extractRequestError(result, true);
-		    }
-		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
-		}, false);
-
-		xhr.addEventListener("error", function(e) {
-		    let err = e.target.status.toString();
-		    let msg = `Error '${err}' occurred while receiving the document.`;
-		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
-		});
-
-		xhr.upload.addEventListener("progress", function(evt) {
-		    if (evt.lengthComputable) {
-			let percentComplete = evt.loaded / evt.total;
-			updateProgress(percentComplete, evt.loaded);
-		    }
-		}, false);
-
-		xhr.open("POST", `/api2/json${baseurl}`, true);
-		xhr.send(fd);
-	    },
-	});
-
-	form.on('validitychange', (f, valid) => submitBtn.setDisabled(!valid));
-
-	Ext.apply(me, {
-	    title: gettext('Upload'),
-	    items: me.formPanel,
-	    buttons: [abortBtn, submitBtn],
-	    listeners: {
-		close: function() {
-		    if (me.xhr) {
-			me.xhr.abort();
-			delete me.xhr;
-		    }
-		},
-	    },
-	});
+    initComponent: function() {
+        const me = this;
+
+	if (!me.nodename) {
+	    throw "no node name specified";
+	}
+	if (!me.storage) {
+	    throw "no storage ID specified";
+	}
 
         me.callParent();
     },
-- 
2.30.2

[pve-devel] [PATCH v2 manager 3/5] ui/UploadToStorage: add checksum and algorithm

[Lorenz Stechauner]

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 www/manager6/window/UploadToStorage.js | 32 ++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index fb9850b3..0b4d991a 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -130,6 +130,18 @@ Ext.define('PVE.window.UploadToStorage', {
 	    vm.set('size', (fileInput.files[0] && Proxmox.Utils.format_size(fileInput.files[0].size)) || '-');
 	    vm.set('mimetype', (fileInput.files[0] && fileInput.files[0].type) || '-');
 	},
+
+	hashChange: function(field) {
+	    const checksum = Ext.getCmp('downloadUrlChecksum');
+	    if (field.getValue() === '__default__') {
+		checksum.setDisabled(true);
+		checksum.setValue("");
+		checksum.allowBlank = true;
+	    } else {
+		checksum.setDisabled(false);
+		checksum.allowBlank = false;
+	    }
+	},
     },
 
     items: [
@@ -184,6 +196,26 @@ Ext.define('PVE.window.UploadToStorage', {
 			value: '{mimetype}',
 		    },
 		},
+		{
+		    xtype: 'pveHashAlgorithmSelector',
+		    name: 'checksum-algorithm',
+		    fieldLabel: gettext('Hash algorithm'),
+		    allowBlank: true,
+		    hasNoneOption: true,
+		    value: '__default__',
+		    listeners: {
+			change: 'hashChange',
+		    },
+		},
+		{
+		    xtype: 'textfield',
+		    name: 'checksum',
+		    fieldLabel: gettext('Checksum'),
+		    allowBlank: true,
+		    disabled: true,
+		    emptyText: gettext('none'),
+		    id: 'downloadUrlChecksum',
+		},
 		{
 		    xtype: 'progressbar',
 		    text: 'Ready',
-- 
2.30.2

[pve-devel] [PATCH v2 manager 4/5] ui/UploadToStorage: add TaskViewer

[Lorenz Stechauner]

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 www/manager6/window/UploadToStorage.js | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index 0b4d991a..c63de16a 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -87,7 +87,21 @@ Ext.define('PVE.window.UploadToStorage', {
 
 	    xhr.addEventListener("load", function(e) {
 		if (xhr.status === 200) {
-		    view.close();
+		    view.hide();
+
+		    const result = JSON.parse(xhr.response);
+		    const upid = result.data;
+		    Ext.create('Proxmox.window.TaskViewer', {
+			autoShow: true,
+			upid: upid,
+			taskDone: view.taskDone,
+			listeners: {
+			    destroy: function() {
+				view.close();
+			    },
+			},
+		    });
+
 		    return;
 		}
 		const err = Ext.htmlEncode(xhr.statusText);
-- 
2.30.2

[pve-devel] [PATCH v2 manager 5/5] ui/UplaodToStorage: check file extension

[Lorenz Stechauner]

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
---
 www/manager6/window/UploadToStorage.js | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
index c63de16a..3f4596ce 100644
--- a/www/manager6/window/UploadToStorage.js
+++ b/www/manager6/window/UploadToStorage.js
@@ -156,6 +156,20 @@ Ext.define('PVE.window.UploadToStorage', {
 		checksum.allowBlank = false;
 	    }
 	},
+
+	filenameChange: function(field) {
+	    const view = this.getView();
+	    const filename = field.getValue();
+	    for (let ext of view.acceptedExtensions[view.content]) {
+		if (filename.endsWith(ext)) {
+		    field.setValidation();
+		    field.validate();
+		    return;
+		}
+	    }
+	    field.setValidation("wrong file extension");
+	    field.validate();
+	},
     },
 
     items: [
@@ -193,6 +207,9 @@ Ext.define('PVE.window.UploadToStorage', {
 		    bind: {
 			value: '{filename}',
 		    },
+		    listeners: {
+			change: 'filenameChange',
+		    },
 		},
 		{
 		    xtype: 'displayfield',
-- 
2.30.2

Re: [pve-devel] [PATCH v2 storage 1/3] status: move unlink from http-server to enpoint

[Thomas Lamprecht]

On 22/07/2021 15:06, Lorenz Stechauner wrote:
> this is the first step in which not the http server removes the
> temporary file, but the worker itself.
> 
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
>  PVE/API2/Storage/Status.pm | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
> index 16581aa..fcf9720 100644
> --- a/PVE/API2/Storage/Status.pm
> +++ b/PVE/API2/Storage/Status.pm
> @@ -480,6 +480,7 @@ __PACKAGE__->register_method ({
>  	    print "command: " . join(' ', @$cmd) . "\n";
>  
>  	    eval { PVE::Tools::run_command($cmd, errmsg => 'import failed'); };
> +	    unlink $tmpfilename or warn "unable to clean up temporory file '$tmpfilename' - $!";

s/temporory/temporary/

>  	    if (my $err = $@) {
>  		unlink $dest;
>  		die $err;
> 

Re: [pve-devel] [PATCH v2 storage 1/1] status: add checksum and algorithm to file upload

[Thomas Lamprecht]

Missing prefix "fix #3505: "

On 22/07/2021 15:06, Lorenz Stechauner wrote:
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
>  PVE/API2/Storage/Status.pm | 34 ++++++++++++++++++++++++++++++++++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
> index 9cf6e40..4b9fda5 100644
> --- a/PVE/API2/Storage/Status.pm
> +++ b/PVE/API2/Storage/Status.pm
> @@ -385,6 +385,19 @@ __PACKAGE__->register_method ({
>  		maxLength => 255,
>  		type => 'string',
>  	    },
> +	    checksum => {
> +		description => "The expected checksum of the file.",
> +		type => 'string',
> +		requires => 'checksum-algorithm',
> +		optional => 1,
> +	    },
> +	    'checksum-algorithm' => {
> +		description => "The algorithm to calculate the checksum of the file.",
> +		type => 'string',
> +		enum => ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512'],
> +		requires => 'checksum',
> +		optional => 1,
> +	    },
>  	    tmpfilename => {
>  		description => "The source file name. This parameter is usually set by the REST handler. You can only overwrite it when connecting to the trusted port on localhost.",
>  		type => 'string',
> @@ -474,6 +487,27 @@ __PACKAGE__->register_method ({
>  	    my $upid = shift;
>  
>  	    print "starting file import from: $tmpfilename\n";
> +
> +	    eval {
> +		my ($checksum, $checksum_algorithm) = $param->@{'checksum', 'checksum-algorithm'};
> +		if ($checksum_algorithm) {
> +		    print "calculating checksum...";
> +
> +		    my $checksum_got = PVE::Tools::get_file_hash($checksum_algorithm, $tmpfilename);
> +
> +		    if (lc($checksum_got) eq lc($checksum)) {
> +			print "OK, checksum verified\n";
> +		    } else {
> +			print "\n";  # the front end expects the error to reside at the last line without any noise
> +			die "checksum mismatch: got '$checksum_got' != expect '$checksum'\n";
> +		    }
> +		}
> +	    };
> +	    if (my $err = $@) {
> +		unlink $tmpfilename or warn "unable to clean up temporory file '$tmpfilename' - $!";
> +		die $err;
> +	    }
> +
>  	    print "target node: $node\n";
>  	    print "target file: $dest\n";
>  	    print "file size is: $size\n";
> 

Re: [pve-devel] [PATCH-SERIES v2 http-server/storage/manager] add checksum and algorithm to iso upload

[Thomas Lamprecht]

On 22/07/2021 15:06, Lorenz Stechauner wrote:
> changes to v1:
> * dropped commits, that were already applied in the mean time
> * better commit messages
> * dropped the 'new-filename' - using 'filename' instead
> * error handling for unlink
> * dropped check for `new FormData()` in pve-manager
> * fixed commit for pve-http-server (deleting temp file on errors)
> * check file extention in front end too
> 
> note: two commits in total state, that they break/require some 
> versions of other repos. please do not forget to bump those and
> create the accoring 'breaks' or 'requires'.
> 
> 
> pve-http-server:
> Lorenz Stechauner (1):
>   anyevent: move unlink from http-server to endpoint
> 
>  src/PVE/APIServer/AnyEvent.pm | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> 
> pve-storage:
> Lorenz Stechauner (3):
>   status: move unlink from http-server to enpoint
>   status: remove sleep(1) in file upload
>   status: add checksum and algorithm to file upload
> 
>  PVE/API2/Storage/Status.pm | 44 +++++++++++++++++++++++++++++++-------
>  1 file changed, 36 insertions(+), 8 deletions(-)
> 
> 
> pve-manager:
> Lorenz Stechauner (5):
>   ui: move upload window into UploadToStorage.js
>   ui: refactor UploadToStorage.js
>   ui/UploadToStorage: add checksum and algorithm
>   ui/UploadToStorage: add TaskViewer
>   ui/UplaodToStorage: check file extension
> 
>  www/manager6/Makefile                  |   1 +
>  www/manager6/storage/ContentView.js    | 197 +---------------
>  www/manager6/window/UploadToStorage.js | 311 +++++++++++++++++++++++++
>  3 files changed, 314 insertions(+), 195 deletions(-)
>  create mode 100644 www/manager6/window/UploadToStorage.js
> 

Two small nits I found when doing a shallow "smell test" review, else the direction
looks OK. I do not want to tackle a in-depth review/test _and_ the two breaks in
dependencies a day before my vacation, so I'm letting Fabian handle this, which has
a better eye for those dependencies most of the time any way. The GUI can be reviewed
by Dominik.

Re: [pve-devel] [PATCH v2 manager 2/5] ui: refactor UploadToStorage.js

[Dominik Csapak]

comments inline:

On 7/22/21 15:06, Lorenz Stechauner wrote:
> this also removes the "content" selector from the window.
> as far as it seems, this selector was never able to select
> more than one entry, so it was useless.
> 
> the check for FormData() is also removed, because this is
> supported by all major browsers for a long time. therefore
> doStandardSubmit() is also not necessary.
> 
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
>   www/manager6/storage/ContentView.js    |   2 +-
>   www/manager6/window/UploadToStorage.js | 360 ++++++++++++++-----------
>   2 files changed, 209 insertions(+), 153 deletions(-)
> 
> diff --git a/www/manager6/storage/ContentView.js b/www/manager6/storage/ContentView.js
> index ca0ad664..00a94f3c 100644
> --- a/www/manager6/storage/ContentView.js
> +++ b/www/manager6/storage/ContentView.js
> @@ -69,7 +69,7 @@ Ext.define('PVE.storage.ContentView', {
>   			Ext.create('PVE.window.UploadToStorage', {
>   			    nodename: nodename,
>   			    storage: storage,
> -			    contents: [content],
> +			    content: content,
>   			    autoShow: true,
>   			    taskDone: () => reload(),
>   			});
> diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
> index 3c35020a..fb9850b3 100644
> --- a/www/manager6/window/UploadToStorage.js
> +++ b/www/manager6/window/UploadToStorage.js
> @@ -1,191 +1,247 @@
>   Ext.define('PVE.window.UploadToStorage', {
>       extend: 'Ext.window.Window',
>       alias: 'widget.pveStorageUpload',
> +    mixins: ['Proxmox.Mixin.CBind'],
>   
>       resizable: false,
> -
>       modal: true,
>   
> -    initComponent: function() {
> -        var me = this;
> +    title: gettext('Upload'),
>   
> -	if (!me.nodename) {
> -	    throw "no node name specified";
> -	}
> -	if (!me.storage) {
> -	    throw "no storage ID specified";
> -	}
> +    acceptedExtensions: {
> +	iso: ['.img', '.iso'],
> +	vztmpl: ['.tar.gz', '.tar.xz'],
> +    },
> +
> +    cbindData: function(initialConfig) {
> +	const me = this;
> +	return {
> +	    nodename: me.nodename,
> +	    storage: me.storage,
> +	    content: me.content,

these should not be necessary, all properties of the view should 
automatically also be cbindable (maybe a default needs to be set)

> +	    extensions: me.acceptedExtensions[me.content].join(', '),

i'd prefer to check if me.content is actually a valid type,
and maybe erroring out if not (e.g 'throw ...')

> +	};
> +    },
>   
> -	let baseurl = `/nodes/${me.nodename}/storage/${me.storage}/upload`;
> +    cbind: {
> +	url: `/nodes/{nodename}/storage/{storage}/upload`,
> +    },

we normally do this in the cbindData function above

me.url = ``;

>   
> -	let pbar = Ext.create('Ext.ProgressBar', {
> -            text: 'Ready',
> -	    hidden: true,
> -	});
> +    viewModel: {
> +	data: {
> +	    size: '-',
> +	    mimetype: '-',
> +	    filename: '',
> +	},
> +    },
>   
> -	let acceptedExtensions = {
> -	    iso: ".img, .iso",
> -	    vztmpl: ".tar.gz, .tar.xz",
> -	};
> +    controller: {
> +	submit: function(button) {
> +	    const view = this.getView();
> +	    const form = Ext.getCmp('formPanel').getForm();
> +	    const abortBtn = Ext.getCmp('abortBtn');
> +	    const pbar = Ext.getCmp('progressBar');
> +
> +	    const updateProgress = function(per, bytes) {
> +		let text = (per * 100).toFixed(2) + '%';
> +		if (bytes) {
> +		    text += " (" + Proxmox.Utils.format_size(bytes) + ')';
> +		}
> +		pbar.updateProgress(per, text);
> +	    };
>   
> -	let defaultContent = me.contents[0] || '';
> -
> -	let fileField = Ext.create('Ext.form.field.File', {
> -	    name: 'filename',
> -	    buttonText: gettext('Select File...'),
> -	    allowBlank: false,
> -	    setAccept: function(content) {
> -		let acceptString = acceptedExtensions[content] || '';
> -		this.fileInputEl.set({
> -		    accept: acceptString,
> -		});
> -	    },
> -	    listeners: {
> -		afterrender: function(cmp) {
> -		    cmp.setAccept(defaultContent);
> -		},
> -	    },
> -	});
> +	    const fd = new FormData();
> +
> +	    button.setDisabled(true);
> +	    abortBtn.setDisabled(false);
>   
> -	me.formPanel = Ext.create('Ext.form.Panel', {
> +	    const contentField = form.findField('content');
> +	    fd.append("content", contentField.getValue());
> +	    contentField.setDisabled(true);

why use a separate input field for this?
you could simply access view.content, no?
would make the code here and below a bit shorter.

> +
> +	    const fileField = form.findField('file');
> +	    const file = fileField.fileInputEl.dom.files[0];
> +	    fileField.setDisabled(true);
> +
> +	    const filenameField = form.findField('filename');
> +	    const filename = filenameField.getValue();
> +	    filenameField.setDisabled(true);

why do you get the filename field here?
we already have the info in the viewModel (since we 'bind' the value)

const vm = view.getViewModel();
const filename = vm.get('filename');


> +
> +	    const algorithmField = form.findField('checksum-algorithm');
> +	    algorithmField.setDisabled(true);
> +	    if (algorithmField.getValue() !== '__default__') {
> +		fd.append("checksum-algorithm", algorithmField.getValue());
> +
> +		const checksumField = form.findField('checksum');
> +		fd.append("checksum", checksumField.getValue());
> +		checksumField.setDisabled(true);
> +	    }

it seems this belongs in the next patch?
at least here it would fail since it would not find the field

> +
> +	    fd.append("filename", file, filename);
> +
> +	    pbar.setVisible(true);
> +	    updateProgress(0);
> +
> +	    const xhr = new XMLHttpRequest();
> +	    view.xhr = xhr;
> +
> +	    xhr.addEventListener("load", function(e) {
> +		if (xhr.status === 200) {
> +		    view.close();
> +		    return;
> +		}
> +		const err = Ext.htmlEncode(xhr.statusText);
> +		let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
> +		if (xhr.responseText !== "") {
> +		    const result = Ext.decode(xhr.responseText);
> +		    result.message = msg;
> +		    msg = Proxmox.Utils.extractRequestError(result, true);
> +		}
> +		Ext.Msg.alert(gettext('Error'), msg, btn => view.close());
> +	    }, false);
> +
> +	    xhr.addEventListener("error", function(e) {
> +		const err = e.target.status.toString();
> +		const msg = `Error '${err}' occurred while receiving the document.`;
> +		Ext.Msg.alert(gettext('Error'), msg, btn => view.close());
> +	    });
> +
> +	    xhr.upload.addEventListener("progress", function(evt) {
> +		if (evt.lengthComputable) {
> +		    const percentComplete = evt.loaded / evt.total;
> +		    updateProgress(percentComplete, evt.loaded);
> +		}
> +	    }, false);
> +
> +	    xhr.open("POST", `/api2/json${view.url}`, true);
> +	    xhr.send(fd);
> +	},
> +
> +	validitychange: function(f, valid) {
> +	    const submitBtn = Ext.getCmp('submitBtn');
> +	    submitBtn.setDisabled(!valid);
> +	},
> +
> +	fileChange: function(input) {
> +	    const vm = this.getViewModel();
> +	    const name = input.value.replace(/^.*(\/|\\)/, '');
> +	    const fileInput = input.fileInputEl.dom;
> +	    vm.set('filename', name);
> +	    vm.set('size', (fileInput.files[0] && Proxmox.Utils.format_size(fileInput.files[0].size)) || '-');
> +	    vm.set('mimetype', (fileInput.files[0] && fileInput.files[0].type) || '-');
> +	},
> +    },
> +
> +    items: [
> +	{
> +	    xtype: 'form',
> +	    id: 'formPanel',
>   	    method: 'POST',
>   	    waitMsgTarget: true,
>   	    bodyPadding: 10,
>   	    border: false,
> -	    width: 300,
> +	    width: 400,
>   	    fieldDefaults: {
>   		labelWidth: 100,
>   		anchor: '100%',
>               },
>   	    items: [
>   		{
> -		    xtype: 'pveContentTypeSelector',
> -		    cts: me.contents,
> -		    fieldLabel: gettext('Content'),
> -		    name: 'content',
> -		    value: defaultContent,
> +		    xtype: 'filefield',
> +		    name: 'file',
> +		    buttonText: gettext('Select File'),
>   		    allowBlank: false,
> +		    fieldLabel: gettext('File'),
> +		    cbind: {
> +			accept: '{extensions}',
> +		    },
>   		    listeners: {
> -			change: function(cmp, newValue, oldValue) {
> -			    fileField.setAccept(newValue);
> -			},
> +			change: 'fileChange',
>   		    },
>   		},
> -		fileField,
> -		pbar,
> -	    ],
> -	});
> -
> -	let form = me.formPanel.getForm();
> -
> -	let doStandardSubmit = function() {
> -	    form.submit({
> -		url: "/api2/htmljs" + baseurl,
> -		waitMsg: gettext('Uploading file...'),
> -		success: function(f, action) {
> -		    me.close();
> +		{
> +		    xtype: 'textfield',
> +		    name: 'filename',
> +		    allowBlank: false,
> +		    fieldLabel: gettext('File name'),
> +		    bind: {
> +			value: '{filename}',
> +		    },
>   		},
> -		failure: function(f, action) {
> -		    var msg = PVE.Utils.extractFormActionError(action);
> -                    Ext.Msg.alert(gettext('Error'), msg);
> +		{
> +		    xtype: 'displayfield',
> +		    name: 'size',
> +		    fieldLabel: gettext('File size'),
> +		    bind: {
> +			value: '{size}',
> +		    },
>   		},
> -	    });
> -	};
> -
> -	let updateProgress = function(per, bytes) {
> -	    var text = (per * 100).toFixed(2) + '%';
> -	    if (bytes) {
> -		text += " (" + Proxmox.Utils.format_size(bytes) + ')';
> -	    }
> -	    pbar.updateProgress(per, text);
> -	};
> -
> -	let abortBtn = Ext.create('Ext.Button', {
> +		{
> +		    xtype: 'displayfield',
> +		    name: 'mimetype',
> +		    fieldLabel: gettext('MIME type'),
> +		    bind: {
> +			value: '{mimetype}',
> +		    },
> +		},
> +		{
> +		    xtype: 'progressbar',
> +		    text: 'Ready',
> +		    hidden: true,
> +		    id: 'progressBar',
> +		},
> +		{
> +		    xtype: 'hiddenfield',
> +		    name: 'content',
> +		    cbind: {
> +			value: '{content}',
> +		    },
> +		},
> +	    ],
> +	   listeners: {
> +		validitychange: 'validitychange',
> +	   },
> +	},
> +    ],
> +
> +    buttons: [
> +	{
> +	    xtype: 'button',
>   	    text: gettext('Abort'),
> +	    id: 'abortBtn',
>   	    disabled: true,
>   	    handler: function() {
> -		me.close();
> +		const me = this;
> +		me.up('pveStorageUpload').close();
>   	    },
> -	});
> -
> -	let submitBtn = Ext.create('Ext.Button', {
> +	},
> +	{
>   	    text: gettext('Upload'),
> +	    id: 'submitBtn',
>   	    disabled: true,
> -	    handler: function(button) {
> -		var fd;
> -		try {
> -		    fd = new FormData();
> -		} catch (err) {
> -		    doStandardSubmit();
> -		    return;
> -		}
> +	    handler: 'submit',
> +	},
> +    ],
> +
> +    listeners: {
> +	close: function() {
> +	    const me = this;
> +	    if (me.xhr) {
> +		me.xhr.abort();
> +		delete me.xhr;
> +	    }
> +	},
> +    },
>   
> -		button.setDisabled(true);
> -		abortBtn.setDisabled(false);
> -
> -		var field = form.findField('content');
> -		fd.append("content", field.getValue());
> -		field.setDisabled(true);
> -
> -		field = form.findField('filename');
> -		var file = field.fileInputEl.dom;
> -		fd.append("filename", file.files[0]);
> -		field.setDisabled(true);
> -
> -		pbar.setVisible(true);
> -		updateProgress(0);
> -
> -		let xhr = new XMLHttpRequest();
> -		me.xhr = xhr;
> -
> -		xhr.addEventListener("load", function(e) {
> -		    if (xhr.status === 200) {
> -			me.close();
> -			return;
> -		    }
> -		    let err = Ext.htmlEncode(xhr.statusText);
> -		    let msg = `${gettext('Error')} ${xhr.status.toString()}: ${err}`;
> -		    if (xhr.responseText !== "") {
> -			let result = Ext.decode(xhr.responseText);
> -			result.message = msg;
> -			msg = Proxmox.Utils.extractRequestError(result, true);
> -		    }
> -		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
> -		}, false);
> -
> -		xhr.addEventListener("error", function(e) {
> -		    let err = e.target.status.toString();
> -		    let msg = `Error '${err}' occurred while receiving the document.`;
> -		    Ext.Msg.alert(gettext('Error'), msg, btn => me.close());
> -		});
> -
> -		xhr.upload.addEventListener("progress", function(evt) {
> -		    if (evt.lengthComputable) {
> -			let percentComplete = evt.loaded / evt.total;
> -			updateProgress(percentComplete, evt.loaded);
> -		    }
> -		}, false);
> -
> -		xhr.open("POST", `/api2/json${baseurl}`, true);
> -		xhr.send(fd);
> -	    },
> -	});
> -
> -	form.on('validitychange', (f, valid) => submitBtn.setDisabled(!valid));
> -
> -	Ext.apply(me, {
> -	    title: gettext('Upload'),
> -	    items: me.formPanel,
> -	    buttons: [abortBtn, submitBtn],
> -	    listeners: {
> -		close: function() {
> -		    if (me.xhr) {
> -			me.xhr.abort();
> -			delete me.xhr;
> -		    }
> -		},
> -	    },
> -	});
> +    initComponent: function() {
> +        const me = this;
> +
> +	if (!me.nodename) {
> +	    throw "no node name specified";
> +	}
> +	if (!me.storage) {
> +	    throw "no storage ID specified";
> +	}
>   
>           me.callParent();
>       },
> 

Re: [pve-devel] [PATCH v2 manager 3/5] ui/UploadToStorage: add checksum and algorithm

[Dominik Csapak]

comments inline

On 7/22/21 15:06, Lorenz Stechauner wrote:
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
>   www/manager6/window/UploadToStorage.js | 32 ++++++++++++++++++++++++++
>   1 file changed, 32 insertions(+)
> 
> diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
> index fb9850b3..0b4d991a 100644
> --- a/www/manager6/window/UploadToStorage.js
> +++ b/www/manager6/window/UploadToStorage.js
> @@ -130,6 +130,18 @@ Ext.define('PVE.window.UploadToStorage', {
>   	    vm.set('size', (fileInput.files[0] && Proxmox.Utils.format_size(fileInput.files[0].size)) || '-');
>   	    vm.set('mimetype', (fileInput.files[0] && fileInput.files[0].type) || '-');
>   	},
> +
> +	hashChange: function(field) {
> +	    const checksum = Ext.getCmp('downloadUrlChecksum');

since we already have a controller, please use 'reference' instead
of id, and 'this.lookup' instead of Ext.getCmp

> +	    if (field.getValue() === '__default__') {

the second parameter of the change event is the value, so you
could omit the call to getValue here..

> +		checksum.setDisabled(true);
> +		checksum.setValue("");
> +		checksum.allowBlank = true;

is this correct? wouldn't it be

checksum.setAllowBlank ?

also why set that in the first place? just leave it
set to 'false'. Disabled fields will not get submitted
or calculated on form validation anyway

> +	    } else {
> +		checksum.setDisabled(false);
> +		checksum.allowBlank = false;
> +	    }
> +	},
>       },
>   
>       items: [
> @@ -184,6 +196,26 @@ Ext.define('PVE.window.UploadToStorage', {
>   			value: '{mimetype}',
>   		    },
>   		},
> +		{
> +		    xtype: 'pveHashAlgorithmSelector',
> +		    name: 'checksum-algorithm',
> +		    fieldLabel: gettext('Hash algorithm'),
> +		    allowBlank: true,
> +		    hasNoneOption: true,
> +		    value: '__default__',
> +		    listeners: {
> +			change: 'hashChange',
> +		    },
> +		},
> +		{
> +		    xtype: 'textfield',
> +		    name: 'checksum',
> +		    fieldLabel: gettext('Checksum'),
> +		    allowBlank: true,
> +		    disabled: true,
> +		    emptyText: gettext('none'),
> +		    id: 'downloadUrlChecksum',
> +		},
>   		{
>   		    xtype: 'progressbar',
>   		    text: 'Ready',
> 

Re: [pve-devel] [PATCH v2 manager 5/5] ui/UplaodToStorage: check file extension

[Dominik Csapak]

comment inline

On 7/22/21 15:06, Lorenz Stechauner wrote:
> Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
> ---
>   www/manager6/window/UploadToStorage.js | 17 +++++++++++++++++
>   1 file changed, 17 insertions(+)
> 
> diff --git a/www/manager6/window/UploadToStorage.js b/www/manager6/window/UploadToStorage.js
> index c63de16a..3f4596ce 100644
> --- a/www/manager6/window/UploadToStorage.js
> +++ b/www/manager6/window/UploadToStorage.js
> @@ -156,6 +156,20 @@ Ext.define('PVE.window.UploadToStorage', {
>   		checksum.allowBlank = false;
>   	    }
>   	},
> +
> +	filenameChange: function(field) {
> +	    const view = this.getView();
> +	    const filename = field.getValue();

as before, this is available as second parameter.

> +	    for (let ext of view.acceptedExtensions[view.content]) {
> +		if (filename.endsWith(ext)) {
> +		    field.setValidation();
> +		    field.validate();
> +		    return;
> +		}
> +	    }
> +	    field.setValidation("wrong file extension");
> +	    field.validate();
> +	},

would that not fit better in the 'validator' function?

>       },
>   
>       items: [
> @@ -193,6 +207,9 @@ Ext.define('PVE.window.UploadToStorage', {
>   		    bind: {
>   			value: '{filename}',
>   		    },
> +		    listeners: {
> +			change: 'filenameChange',
> +		    },
>   		},
>   		{
>   		    xtype: 'displayfield',
>