v3dv: fix semaphore wait from CPU job

If a CPU job comes first in a command buffer with a semaphore wait operation
we need to wait on the CPU for the semaphore to be signaled before we process
the job.

We have been doing this with a WaitForIdle operation, but that only works
if the semaphore has been submitted for signaling from the same instance
of the driver. If we have an imported payload from another instance in our
semaphore however, waitForIdle may return too early since the submission
to signal the semaphore may have been submitted by a different instance
of the driver as well, and our wait for idle checks only know about this
instance submissions.

To fix this, we always submit a noop job from our instance that waits on
the semaphores on the GPU and follow up with WaitForIdle to wait for that
to complete.

Fixes test failures and/or assert crashes in:
dEQP-VK.synchronization.cross_instance.*
(when enabling support for semaphore imports)

Reviewed-by: Melissa Wen <mwen@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15342>

diff --git a/src/broadcom/vulkan/v3dv_queue.c b/src/broadcom/vulkan/v3dv_queue.c
index ea87de5..30d5e0d 100644 (file)
--- a/src/broadcom/vulkan/v3dv_queue.c
+++ b/src/broadcom/vulkan/v3dv_queue.c
@@ -1150,8 +1150,18 @@ queue_submit_job(struct v3dv_queue *queue,
     * that any previous work has been completed, at which point any wait
     * semaphores must be signalled, and we never need to do this again for the
     * same batch.
+    *
+    * There is a corner case here when the semaphore has been imported from
+    * another instance/process. In that scenario, the Vulkan spec still requires
+    * that a signaling operation has been submitted before this semaphore wait
+    * but our wait for idle checks won't know about that submission (since they
+    * are based on the last jobs sent from our instance). To fix that we submit
+    * a noop job to "consume" the semaphores and then we wait for idle, which
+    * will ensure that our CPU job waits for the semaphores to be signaled even
+    * if they are signaled from another instance or process.
     */
    if (!v3dv_job_type_is_gpu(job) && sems_info->wait_sem_count) {
+      queue_submit_noop_job(queue, sems_info, false, false);
       v3dv_QueueWaitIdle(v3dv_queue_to_handle(&job->device->queue));
 #ifdef DEBUG
       /* Loop through wait sems and check they are all signalled */