[PVE-User] HA migrate failing without error

[PVE-User] HA migrate failing without error

[Ralf Storm]

Hello List,

we have a cluster of 7 Nodes with ceph, all updated with subscription.

since update to 6.4 we cannot ha migrate to one single node anymore, 
without any error, it just says " start ha migrate ok" and "End HA 
Migrate  OK"

rebooted everything without success

Migrate of NON-HA vms works like a charm to this node


Anybody had a similar issue already?


best regards


Ralf

Re: [PVE-User] HA migrate failing without error

[Aaron Lauterer]

On 5/7/21 1:34 PM, Ralf Storm wrote:
> Hello List,
> 
> we have a cluster of 7 Nodes with ceph, all updated with subscription.
> 
> since update to 6.4 we cannot ha migrate to one single node anymore, without any error, it just says " start ha migrate ok" and "End HA Migrate  OK"

That is normal. What should happen a few seconds later, is that the actual migration task starts. I tested it quickly in one of my test clusters and it worked.
First there is the 'HA <VMID> - Migrate' tags with the logs as you described them. Then a few seconds later, the 'VM <VMID> - Migrate' tasks starts which does the actual migration.

Do you see that second task happening at all?

> 
> rebooted everything without success
> 
> Migrate of NON-HA vms works like a charm to this node
> 
> 
> Anybody had a similar issue already?
> 
> 
> best regards
> 
> 
> Ralf
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] HA migrate failing without error

[Ralf Storm]

I know this behavour, but the second task does not start


actually it seems like no host can migrate ha vms anymore, only without ha



Am 07/05/2021 um 13:47 schrieb Aaron Lauterer:
>
>
> On 5/7/21 1:34 PM, Ralf Storm wrote:
>> Hello List,
>>
>> we have a cluster of 7 Nodes with ceph, all updated with subscription.
>>
>> since update to 6.4 we cannot ha migrate to one single node anymore, 
>> without any error, it just says " start ha migrate ok" and "End HA 
>> Migrate  OK"
>
> That is normal. What should happen a few seconds later, is that the 
> actual migration task starts. I tested it quickly in one of my test 
> clusters and it worked.
> First there is the 'HA <VMID> - Migrate' tags with the logs as you 
> described them. Then a few seconds later, the 'VM <VMID> - Migrate' 
> tasks starts which does the actual migration.
>
> Do you see that second task happening at all?
>
>>
>> rebooted everything without success
>>
>> Migrate of NON-HA vms works like a charm to this node
>>
>>
>> Anybody had a similar issue already?
>>
>>
>> best regards
>>
>>
>> Ralf
>>
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>

Re: [PVE-User] HA migrate failing without error

[Ralf Storm]

I observed the error further and tried to restart some nodes, as I see 
under "HA" at the Master: old timestamp - dead?

after reboot quorum is aquired as usual, but the entryfor the master 
remains and no other master is elected

as i migrated some vms for the reboot - which I had to remove from HA 
before, to be able to migrate them, they stuck on "deleting" in HA

as i investigated further i find this VMs to be locked under several 
nodes and i can see ths strange entry on all nodes: lock--1.conf

can i force the master to change?

any other suggestions? I run out of ideas...


Am 07/05/2021 um 14:06 schrieb Ralf Storm:
> I know this behavour, but the second task does not start
>
>
> actually it seems like no host can migrate ha vms anymore, only 
> without ha
>
>
>
> Am 07/05/2021 um 13:47 schrieb Aaron Lauterer:
>>
>>
>> On 5/7/21 1:34 PM, Ralf Storm wrote:
>>> Hello List,
>>>
>>> we have a cluster of 7 Nodes with ceph, all updated with subscription.
>>>
>>> since update to 6.4 we cannot ha migrate to one single node anymore, 
>>> without any error, it just says " start ha migrate ok" and "End HA 
>>> Migrate  OK"
>>
>> That is normal. What should happen a few seconds later, is that the 
>> actual migration task starts. I tested it quickly in one of my test 
>> clusters and it worked.
>> First there is the 'HA <VMID> - Migrate' tags with the logs as you 
>> described them. Then a few seconds later, the 'VM <VMID> - Migrate' 
>> tasks starts which does the actual migration.
>>
>> Do you see that second task happening at all?
>>
>>>
>>> rebooted everything without success
>>>
>>> Migrate of NON-HA vms works like a charm to this node
>>>
>>>
>>> Anybody had a similar issue already?
>>>
>>>
>>> best regards
>>>
>>>
>>> Ralf
>>>
>>>
>>> _______________________________________________
>>> pve-user mailing list
>>> pve-user@lists.proxmox.com
>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] HA migrate failing without error

[[LFWD] Virgil O]

Hi Ralf,

Have you tried to check the status of the crm service ?
cf. https://forum.proxmox.com/threads/ha-mgr-old-timestamp-dead.47097/

if you have this kind of message, just remove the configuration file or
move it to the node.

Virgil

On Fri, May 7, 2021 at 3:17 PM Ralf Storm <ralf.storm@konzept-is.de> wrote:

> I observed the error further and tried to restart some nodes, as I see
> under "HA" at the Master: old timestamp - dead?
>
> after reboot quorum is aquired as usual, but the entryfor the master
> remains and no other master is elected
>
> as i migrated some vms for the reboot - which I had to remove from HA
> before, to be able to migrate them, they stuck on "deleting" in HA
>
> as i investigated further i find this VMs to be locked under several
> nodes and i can see ths strange entry on all nodes: lock--1.conf
>
> can i force the master to change?
>
> any other suggestions? I run out of ideas...
>
>
> Am 07/05/2021 um 14:06 schrieb Ralf Storm:
> > I know this behavour, but the second task does not start
> >
> >
> > actually it seems like no host can migrate ha vms anymore, only
> > without ha
> >
> >
> >
> > Am 07/05/2021 um 13:47 schrieb Aaron Lauterer:
> >>
> >>
> >> On 5/7/21 1:34 PM, Ralf Storm wrote:
> >>> Hello List,
> >>>
> >>> we have a cluster of 7 Nodes with ceph, all updated with subscription.
> >>>
> >>> since update to 6.4 we cannot ha migrate to one single node anymore,
> >>> without any error, it just says " start ha migrate ok" and "End HA
> >>> Migrate  OK"
> >>
> >> That is normal. What should happen a few seconds later, is that the
> >> actual migration task starts. I tested it quickly in one of my test
> >> clusters and it worked.
> >> First there is the 'HA <VMID> - Migrate' tags with the logs as you
> >> described them. Then a few seconds later, the 'VM <VMID> - Migrate'
> >> tasks starts which does the actual migration.
> >>
> >> Do you see that second task happening at all?
> >>
> >>>
> >>> rebooted everything without success
> >>>
> >>> Migrate of NON-HA vms works like a charm to this node
> >>>
> >>>
> >>> Anybody had a similar issue already?
> >>>
> >>>
> >>> best regards
> >>>
> >>>
> >>> Ralf
> >>>
> >>>
> >>> _______________________________________________
> >>> pve-user mailing list
> >>> pve-user@lists.proxmox.com
> >>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> >>
> >
> > _______________________________________________
> > pve-user mailing list
> > pve-user@lists.proxmox.com
> > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
> _______________________________________________
> pve-user mailing list
> pve-user@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>