[pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[Daniel Herzig]

Move the queries for vCLS machines and empty datastore strings into
the already existing try-except routine to save `listvms.py` from
breaking when `vm.config` is `None`.

Do not store the result of the `any` function in a variable anymore,
do the same for the datastore query to shorten code.

Prepare more verbose logging.

Signed-off-by: Daniel Herzig <d.herzig@proxmox.com>
---
 listvms.py | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)

diff --git a/listvms.py b/listvms.py
index 89ddfb2..4995975 100755
--- a/listvms.py
+++ b/listvms.py
@@ -265,21 +265,19 @@ def main():
     with connect_to_esxi_host(connection_args) as connection:
         data = {}
         for vm in list_vms(connection):
-            # drop vCLS machines
-            vCLS = any(cfg.key == "HDCS.agent"
-                       and cfg.value.lower() == "true"
-                       for cfg in vm.config.extraConfig)
-            if vCLS:
-                continue
-            # drop vms with empty datastore
-            datastore_name, relative_vmx_path = parse_file_path(
-                vm.config.files.vmPathName
-            )
-            if not datastore_name:
-                print(f"Skipping VM (no datastore value): {vm.name}",
-                      file=sys.stderr)
-                continue
             try:
+                # drop vCLS machines
+                if any(cfg.key == "HDCS.agent"
+                       and cfg.value.lower() == "true"
+                       for cfg in vm.config.extraConfig):
+                    print(f"Skipping ESXI VM (VMware internals): {vm.name}",
+                          file=sys.stderr)
+                    continue
+                # drop vms with empty datastore
+                if not parse_file_path(vm.config.files.vmPathName)[0]:
+                    print(f"Skipping ESXI VM (no datastore value): {vm.name}",
+                          file=sys.stderr)
+                    continue
                 fetch_and_update_vm_data(vm, data)
             except Exception as err:
                 print(
-- 
2.39.5


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[Daniel Kral]

On 4/23/25 15:03, Daniel Herzig wrote:
> Move the queries for vCLS machines and empty datastore strings into
> the already existing try-except routine to save `listvms.py` from
> breaking when `vm.config` is `None`.
> 
> Do not store the result of the `any` function in a variable anymore,
> do the same for the datastore query to shorten code.
> 
> Prepare more verbose logging.
> 
> Signed-off-by: Daniel Herzig <d.herzig@proxmox.com>

Oops, I should have prefixed my patch with a "fix" at least, but I've 
sent two patches some days ago for this ;):

https://lore.proxmox.com/pve-devel/20250411150634.253684-1-d.kral@proxmox.com/T/#u

But it's also good to just extend the scope of the try-catch here 
without the additional check for whether the vm.config has a value or not.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[Daniel Herzig]

Daniel Kral <d.kral@proxmox.com> writes:

> On 4/23/25 15:03, Daniel Herzig wrote:
>> Move the queries for vCLS machines and empty datastore strings into
>> the already existing try-except routine to save `listvms.py` from
>> breaking when `vm.config` is `None`.
>> Do not store the result of the `any` function in a variable anymore,
>> do the same for the datastore query to shorten code.
>> Prepare more verbose logging.
>> Signed-off-by: Daniel Herzig <d.herzig@proxmox.com>
>
> Oops, I should have prefixed my patch with a "fix" at least, but I've
> sent two patches some days ago for this ;):
>
> https://lore.proxmox.com/pve-devel/20250411150634.253684-1-d.kral@proxmox.com/T/#u
>
Ha, I missed that one, you were too quick ;)

It turned out that the problem was on the specific ESXI-side config
(`vm.config` is there on ESXI-6.7.x as well, without it we could not do
anything anyway -- `fetch_and_update_vm_data` calls it more than
once inherently). But it's `None` for sure if there's an invalid vm config on the ESXI side.

OP on bugzilla reported that once these invalid VMs were removed from
ESXI, 'standard 0.7.3' started working as expected.

But for the earlier vCLS commit I totally missed that
possibility (shame on me) and placed the queries 'unprotected' (unlike
`fetch_and_update_vm_data`).

> But it's also good to just extend the scope of the try-catch here
> without the additional check for whether the vm.config has a value or
> not.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[Daniel Kral]

On 4/23/25 16:04, Daniel Herzig wrote:
> Daniel Kral <d.kral@proxmox.com> writes:
> 
>> On 4/23/25 15:03, Daniel Herzig wrote:
>>> Move the queries for vCLS machines and empty datastore strings into
>>> the already existing try-except routine to save `listvms.py` from
>>> breaking when `vm.config` is `None`.
>>> Do not store the result of the `any` function in a variable anymore,
>>> do the same for the datastore query to shorten code.
>>> Prepare more verbose logging.
>>> Signed-off-by: Daniel Herzig <d.herzig@proxmox.com>
>>
>> Oops, I should have prefixed my patch with a "fix" at least, but I've
>> sent two patches some days ago for this ;):
>>
>> https://lore.proxmox.com/pve-devel/20250411150634.253684-1-d.kral@proxmox.com/T/#u
>>
> Ha, I missed that one, you were too quick ;)
> 
> It turned out that the problem was on the specific ESXI-side config
> (`vm.config` is there on ESXI-6.7.x as well, without it we could not do
> anything anyway -- `fetch_and_update_vm_data` calls it more than
> once inherently). But it's `None` for sure if there's an invalid vm config on the ESXI side.

Thanks for the info!

I was wondering why it didn't work for them (in the forum) and have only 
figured that there was something undocumented in the PyVim API about 
when the OptionType is None, but with either patch it should be fine now :).

> 
> OP on bugzilla reported that once these invalid VMs were removed from
> ESXI, 'standard 0.7.3' started working as expected.
> 
> But for the earlier vCLS commit I totally missed that
> possibility (shame on me) and placed the queries 'unprotected' (unlike
> `fetch_and_update_vm_data`).
> 
>> But it's also good to just extend the scope of the try-catch here
>> without the additional check for whether the vm.config has a value or
>> not.



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-esxi-import-tools] fix #6347: do not break vm listing on invalid configs

[Daniel Herzig]

Daniel Kral <d.kral@proxmox.com> writes:

> On 4/23/25 16:04, Daniel Herzig wrote:
>>> Oops, I should have prefixed my patch with a "fix" at least, but I've
>>> sent two patches some days ago for this ;):
>>>
>>> https://lore.proxmox.com/pve-devel/20250411150634.253684-1-d.kral@proxmox.com/T/#u
>>>
>> Ha, I missed that one, you were too quick ;)
>> It turned out that the problem was on the specific ESXI-side config
>> (`vm.config` is there on ESXI-6.7.x as well, without it we could not do
>> anything anyway -- `fetch_and_update_vm_data` calls it more than
>> once inherently). But it's `None` for sure if there's an invalid vm config on the ESXI side.
>
> Thanks for the info!
>
> I was wondering why it didn't work for them (in the forum) and have
> only figured that there was something undocumented in the PyVim API
> about when the OptionType is None, but with either patch it should be
> fine now :).
>
I think so too!


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel