public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Fabian Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: Re: [pve-devel] [RFC/PATCH qemu] PVE-Backup: avoid segfault issues upon backup-cancel
Date: Wed, 25 May 2022 10:10:43 +0200	[thread overview]
Message-ID: <a250958f-bcc5-3e2f-5fa8-386655bd8f38@proxmox.com> (raw)
In-Reply-To: <20220524113050.179182-1-f.ebner@proxmox.com>

There might still be an edge case where completion and cancel race (I
didn't run into this in practice yet, but at a first glance it seems
possible):

1. job_exit -> job_completed -> job_finalize_single starts
2. pvebackup_co_complete_stream gets spawned in completion callback
3. job finalize_single finishes -> job's refcount hits zero -> job is freed
4. qmp_backup_cancel comes in and locks backup_state.backup_mutex before
pvebackup_co_complete_stream can remove the job from the di_list
5. qmp_backup_cancel/job_cancel_bh will operate on the already freed memory

It /would/ be fine if pvebackup_co_complete_stream is guaranteed to
run/take the backup_mutex before qmp_backup_cancel. It *is* spawned
earlier so maybe it is, but I haven't looked into ordering guarantees
for coroutines yet and it does have another yield point when taking
&backup_state.stat.lock, so I'm not so sure.

Possible fix: ref jobs when adding them to di_list and unref them when
removing them from di_list (instead of the proposed ref/unref used in
this patch).


Yet another issue (not directly related, but thematically): in
create_backup_jobs_bh, in the error case, job_cancel_sync is called for
each job, but since it's a transaction, the first call will cancel and
free all jobs, also leading to segfaults in scenarios where creation of
a non-first job fails. And even if the first one fails, the job_unref
there is also wrong, since the job was freed during job_cancel_sync.




      reply	other threads:[~2022-05-25  8:11 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-24 11:30 Fabian Ebner
2022-05-25  8:10 ` Fabian Ebner [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a250958f-bcc5-3e2f-5fa8-386655bd8f38@proxmox.com \
    --to=f.ebner@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal