v3d,v3dv: move TFU register definition to a common header

We are using the same definitions for both OpenGL and Vulkan, so let's
move it to common.

As we are here we are also adding versioning on the TFU register
definition. Those are basically register bit places, so really likely
to change between versions.

Adding 33 as it is the first version they got defined.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13832>

radv: simplify re-using cache entries in radv_pipeline_cache_insert_shaders()

If entry->shaders[i] is NULL, shaders[i] should be also NULL, so the
else condition is a no-op.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13823>

v3dv: fix internal bpp of D/S formats

Depth/stencil formats can, at worse (d32/d24s8), be exactly 32bpp,
which is the minimum we can program for the internal format.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13816>

llvmpipe: enable PIPE_CAP_FBFETCH_COHERENT

llvmpipe's fragment shaders are always run sequentially and
in API order for a single tile, so it's impossible to have
out of order render target writes requiring fetch barriers.

Issues fixed in previous commits were actually breaking most
piglit/deqp tests for coherent extension variant.

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

llvmpipe: remove dead args from load_unswizzled_block

They were only used in fs_fb_fetch.

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

llvmpipe: fix FB fetch with non 32-bit render target formats

Use lp_build_fetch_rgba_soa instead of lp_build_unpack_rgba_soa.
This one was failing most of deqp *framebuffer_fetch* tests.

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

llvmpipe: protect from doing FB fetch of missing buffers

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

llvmpipe: fix gl_FragColor and gl_LastFragData[0] combination

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

llvmpipe: fix wrong assumption on FB fetch shader opacity

In certain cases variant->opaque could be set to true, which
reset command list for tiles fully covered by a triangle
with this shader. This is obviously wrong in presence of
framebuffer fetch.

Signed-off-by: Pavel Asyutchenko <sventeam@yandex.ru>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13252>

zink: implement pipe_context::draw_vertex_state

rough implementation, but it should be a decent start

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13692>

lima: add more wrap modes

Using 1 bit per wrap mode looked very suspicious and after some
experiments it turns out it's 3-bit enum.

Border color is also here, it sits right after depth field. For
some reason it uses 16 bit per channel just like for clear color in RSW

GL_CLAMP mode is broken for nearest filter just as on Midgard, so add
the same workaround - use GL_CLAMP_TO_EDGE for nearest filter.

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13213>

lima: handle 1D samplers

It's just a matter of changing number of dimensions in texture
descriptor.

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13213>

lima: add support for 3D textures

It looks like MBS format used by blob doesn't distinguish sampler2D from
sampler3D, so load texture instruction is the same for 2D and 3D
textures.

So all we need to RE is texture descriptor for 3D textures, but blob
doesn't implement it, so we need to do some guesswork:

- unknown_3_1 looks like a depth since it sits after height/width and
  always set to 1
- unknown_2_2 is exactly 3 bits and it follows wrap_t, so it must be
  wrap_r
- missing part is texture type for 3D textures. By trial and error it
  seems to be 4. First bit is only set for cubemap, so it's likely a
  separate flag, and rest 2 bits look like number of tex dimensions akin
  to midgard and later (thanks, panfrost!) with 0 for 1D, 1 for 2D
  and 2 for 3D.

Put it all together and we have working 3D textures on lima!

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13213>

zink: set suballocator bo size to aligned allocation size

this is the actual memory size

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13824>

zink: block suballocator caching for swapchain/dmabuf images

these have pNext pointers which makes their memory uncacheable

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13824>

radeonsi: don't use GS SGPR6 for the small prim cull info

use a user SGPR instead. This will be needed in the future.

Also don't upload small_prim_precision because it's passed via
VS_STATE_BITS.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13811>

radeonsi: inline declare_vs_specific_input_sgprs

I think it was getting a little hard to follow.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13811>

radeonsi: cull against clip planes, clipvertex, clip/cull distances in shader

The downside is that this duplicates shader code for clip/cull distances
in both the position and parameter portions of the shader.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13811>

radeonsi: unify how ngg_cull_flags are set

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13811>

d3d12: Fix incorrect hash table usage

I'd assumed that since insert didn't take a deleter, it was
find-or-insert, not insert-or-replace. This caused a bo reference
leak if the same bo was used more than once in a batch.

Fixes: fde36d79920 ("d3d12: Don't wait for GPU reads to do CPU reads")
Reviewed By: Bill Kristiansen <billkris@microsoft.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13819>

lima: add native txp support

Currently lima uses generic TXP lowering that results in downgrading
coords precision to FP16 since we have to do some calculations with
coords instead of loading them directly from varying.

Mali4x0 has native TXP support, however coords and projector have to
come from a single source.

Add NIR lowering pass that combines coords and projector into a single
backend-specific source and use it instead of generic lowering.

Unfortunately this change regresses one test, but it also fails in blob and
disassembly is now identical.

shader-db diff:

total instructions in shared programs: 15623 -> 15603 (-0.13%)
instructions in affected programs: 877 -> 857 (-2.28%)
helped: 7
HURT: 0
helped stats (abs) min: 2 max: 8 x̄: 2.86 x̃: 2
helped stats (rel) min: 0.87% max: 10.53% x̄: 4.93% x̃: 1.85%
95% mean confidence interval for instructions value: -4.95 -0.76
95% mean confidence interval for instructions %-change: -9.31% -0.55%
Instructions are helped.

total loops in shared programs: 3 -> 3 (0.00%)
loops in affected programs: 0 -> 0
helped: 0
HURT: 0

total spills in shared programs: 136 -> 137 (0.74%)
spills in affected programs: 0 -> 1
helped: 0
HURT: 1

total fills in shared programs: 598 -> 602 (0.67%)
fills in affected programs: 0 -> 4
helped: 0
HURT: 1

Tested-by: Denis Pauk <pauk.denis@gmail.com>
Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13111>

isaspec: Add prototypes for expr evaluators

Add function prototypes for generated expr evaluators, to avoid a use-
before-declaration issue if an expression references a derived field.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13787>

freedreno/ir3: adjust condition for when to use ldib

We have to use it any time that the image is writable. Otherwise writes
from the same invocation won't have posted into the texture cache.

See: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5629
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13807>

nir: fix constant expression of ibitfield_extract

This fixes dEQP-VK.graphicsfuzz.cov-condition-bitfield-extract-integer.

For example, nir_ibitfield_extract(3, 1, 2) should return 1.

Cc: 21.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13791>

vulkan: Add a dummy sync type

This is useful in WSI scenarios where you want to trivially signal a
fence or semaphore.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan,anv: Auto-detect syncobj features

Instead of having a bunch of const vk_sync_type for each permutation of
vk_drm_syncobj capabilities, have a vk_drm_syncobj_get_type helper which
auto-detects features.  If a driver can't support a feature for some
reason (i915 got timeline support very late, for instance), they can
always mask off feature bits they don't want.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Simplify submit_simple_batch()

BO waits aren't going away any time soon so using a syncobj here doesn't
really gain us anything.  It just makes it more complicated.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Remove unnecessary syncobj wrappers

These are entirely unused except for the syncobj in submit_simple_batch.
We can use the libdrm wrappers for that as they're basically equivalent
and the core Vulkan sync code already depends on them.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add an emulated binary vk_sync type

This wraps a timeline vk_sync type and turns it into a binary one.  This
is useful for, for instance, driver layered on D3D12.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Convert to the common sync and submit framework

This is, unfortunately, a large flag-day mega-commit.  However, any
other approach would likely be fragile and involve a lot more churn as
we try to plumb the new vk_fence and vk_semaphore primitives into ANV's
submit code before we delete it all.  Instead, we do it all in one go
and accept the consequences.

While this should be mostly functionally equivalent to the previous
code, there is one potential perf-affecting change.  The command buffer
chaining optimization no longer works across VkSubmitInfo structs.
Within a single VkSubmitInfo, we will attempt to chain all the command
buffers together but we no longer try to chain across a VkSubmitInfo
boundary.  Hopefully, this isn't a significant perf problem.  If it ever
is, we'll have to teach the core runtime code how to combine two or more
VkSubmitInfos into a single vk_queue_submit.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Use helpers in util/os_time.h in the query code

These are about to be the only use of anv_gettime_ns() so lets switch
them over so the next patch can delete the helper outright.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Remove the last remnants of in/out fences

This should have been dropped as part of d44ea09e61f4 ("anv: Drop unused
sync_file and BO semaphore code") but we missed a few bits.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Add a BO sync type

This is copy+paste from the BO implementation of anv_fence

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Delete ANV_SEMAPHORE_TYPE_DUMMY

It's never set as a semaphore type.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/wsi/display: Wrap wsi_display_fence in a vk_sync

This gets rid of the bespoke vfunc interface in the WSI code.
Eventually, I'd love to get rid of wsi_display_fence entirely but
this at least makes it a lot more palatable.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

wsi/display: Rework wsi_fence a bit

Get rid of most of the guts of the base class and just leave it as a
vtable.  We can also drop some of wsi_display_fence.  One functional
change here is that we're now using VK_SYSTEM_ALLOCATION_SCOPE_INSTANCE
which is more correct anyway because, thanks to the funky reference
counting we do with destroyed and event_received, its lifetime is tied
to the physical device, at best.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/wsi: Drop wsi_common_get_current_time()

Use os_time_get_nano() instead.  At this point, it's just a wrapper
around os_time_get_nano() anyway.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/device: Log the timeline mode when lost

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add common implementations of vkQueueSubmit and vkQueueWaitIdle

This adds a new vk_queue_submit object which contains a list of command
buffers as well as wait and signal operations along with a driver hook
which takes a vk_queue and a vk_queue_submit and does the actual submit.
The common code then handles spawning a submit thread if needed, waiting
for timeline points to materialize, dealing with timeline semaphore
emulation via vk_timeline, etc.  All the driver sees are vk_queue.submit
calls with fully materialized vk_sync objects which it can wait on
unconditionally.

This implementation takes a page from RADV's book and only ever spawns
the submit thread if it sees a timeline wait on a time point that has
not yet materialized.  If this never happens, it calls vk_queue.submit
directly from vkQueueSubmit() and the thread is never spawned.

One other nicety of the new framework is that there is no longer a
distinction, from the driver's PoV, between fences and semaphores.  The
fence, if any, is included as just one more signal operation on the
final vk_queue_submit in the batch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add a common implementation of VkSemaphore

This is built on the new vk_sync primitives.  In the vk_physical_device,
the driver provides a null-terminated array of vk_sync_type pointers in
priority order.  The semaphore implementation then selects the first
type that meets the necessary criterion.  In particular, semaphores may
or may not be timelines depending on the VkSemaphoreType.  It also
auto-selects the semaphore type based on the external handle types
provided and can down-grade as needed to support a particular external
handle.

The implementation itself is mostly copy+pasted from ANV.  The primary
difference is the fact that anv_semaphore_impl has been replaced with
vk_sync.  The permanent vk_sync is still embedded (like ANV) but the
temporary one is a pointer.  This makes stealing the temporary state as
part of VkQueueSubmit a bit easier.

All of the interesting stuff around waits, signals, etc. is implemented
by the vk_sync interface.  All this code does is wrap it all in the
annoyingly detailed VkFence rules so we can provide the correct Vulkan
entrypoint behavior.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add a common implementation of VkFence

This is built on the new vk_sync primitives.  In the vk_physical_device,
the driver provides a null-terminated array of vk_sync_type pointers in
priority order.  The fence implementation then selects the first type
that meets the necessary criterion.  In particular, fences can't be
timelines and need to support reset and CPU wait.  It also auto-selects
the fence type based on the external handle types provided and can
down-grade as needed to support a particular external handle.

The implementation itself is mostly copy+pasted from ANV.  The primary
difference is the fact that anv_fence_impl has been replaced with
vk_sync.  The permanent vk_sync is still embedded (like ANV) but the
temporary one is a pointer.  This makes stealing the temporary state as
part of VkQueueSubmit a bit easier.

All of the interesting stuff around waits, resets, etc. is implemented
by the vk_sync interface.  All this code does is wrap it all in the
annoyingly detailed VkFence rules so we can provide the correct Vulkan
entrypoint behavior.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add an emulated timeline sync type

This is mostly copy+paste (and a bit of re-typing) from ANV.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

c11/threads: Re-align return values for timed waits

They're supposed to return thrd_timedout (which we mistakenly named
thrd_timeout), not thrd_busy.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add a common vk_drm_syncobj struct

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/vk_device: Add a drm_fd field

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/util: Include stdlib.h

It's needed for malloc() which is used by STACK_ARRAY

Fixes: f695171e383e ("vulkan: add common entrypoints for sparse image requirements/properties")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/meson: Re-arrange libvulkan_util deps a bit

Rename files_vulkan_runtime to vulkan_runtime_files and add a new
vulkan_runtime_deps array for dependencies.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Add a vk_sync base class

This doesn't map directly to any particular Vulkan object but is,
instead, a base class for the internal implementations of both VkFence
and VkSemaphore.  Its utility will become evident in later patches.

The design of vk_sync will look familiar to anyone with significant
experience in DRM.  The base object itself is just a pointer to a vfunc
table with function pointers providing the implementation of the various
operations.  Depending on how the vk_sync will be used some of of those
vfuncs are optional.  If it's only going to be used for VkSemaphore, for
instance, there's no need for reset().

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Wire up the new status check

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan/device: Add a check_status hook

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Use the new common device lost tracking

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

vulkan: Pull the device lost framework from ANV

It's a bit on the over-complicated side but the objective is to make the
debug log messages show up in the same thread as the first
VK_ERROR_DEVICE_LOST so we don't massively confuse the app.  It's
unknown if this is actually ever a problem but, with submit happening
off on its own thread, logging errors from threads the client doesn't
know about doesn't seem like a massively great plan.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>

anv: Move device memory maps back to anv_device_memory

This effectively partially reverts 13fe43714c1f ("anv: Add helpers in
anv_allocator for mapping BOs") where we both added helpers and reworked
memory mapping to stash the maps on the BO.  The problem comes with
external memory.  Due to GEM rules, if a memory object is exported and
then imported or imported twice,  we have to deduplicate the anv_bo
struct but, according to Vulkan rules, they are separate VkDeviceMemory
objects.  This means we either need to always map whole objects and
reference-count the map or we need to handle maps separately for
separate VkDeviceMemory objects.  For now, take the later path.

Fixes: 13fe43714c1f ("anv: Add helpers in anv_allocator for mapping BOs")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5612
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13795>

mesa: convert unsupported primtypes during display list compilation

this adds primitive type translation in before the draw reaches gallium,
which massively increases performance by avoiding any sort of buffer readback

fixes #5249

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13741>

move util/indices to core util

these are useful tools to have outside of gallium

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13741>

intel/genxml: Decode VALIGN/HALIGN values in XY_BLOCK_COPY_BLT

For easier readability in INTEL_DEBUG=bat.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13809>

intel/genxml: Fix XY_BLOCK_COPY_BLT destination tiling field type

Fixes: 2f58a63b2f1 ("intel/genxml: Add XY_BLOCK_COPY_BLT on Tigerlake and later.")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13809>

intel/genxml: Fix MI_FLUSH_DW to actually specify the length properly

Fixes: 569afd37f15 ("intel/genxml: Copy gen12.xml to gen125.xml")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13809>

intel/genxml: Collapse leading underscores on prefixed value defines

We prefix names with an underscore to make them "safe" C identifiers
when necessary.  For example, a value of "32x32" would become "_32x32".

However, when specifying something like

   <field ... prefix="BLOCK_SIZE">
     <value name="32x32" value="0"/>
   </field>

we already have a prefix that makes the field name safe.  We'd rather
generate a name with a single underscore, i.e.

    #define BLOCK_SIZE_32x32 0

rather than

    #define BLOCK_SIZE__32x32 0

This also fixes up affected defines in crocus.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13809>

intel/genxml: Simplify prefix handling for field value lists

When a <field> tag has multiple <value> children, listing symbolic names
for possible field values, we generate #defines for each value, with an
optional prefix.  I don't know why, but this code was checking whether
self.default is None.  We want to generate the same list of #defines,
with a prefix, regardless of whether the field has a default value
specified or not.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13809>

iris: Fix parameters to iris_copy_region in reallocate_resource_inplace

We had accidentally passed <x, y, z, l> instead of <l, x, y, z>.

Fixes: b8ef3271c870 ("iris: Move suballocated resources to a dedicated allocation on export")
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13815>

radv: Don't crash if VkExternalImageFormatProperties isn't provided.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5623
Stable: 21.2 21.3
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13786>

gallium/u_blitter: clean up texcoords ZW when filling up just XY

To avoid a scenario like this:
  * One blit needed the four components => XYZW filled up with 4 values
  * Following blit needing two components => ZW uses the previous values

We detected this using the v3d driver with the
arb_framebuffer_srgb-blit test, specifically:

  ./bin/arb_framebuffer_srgb-blit texture linear_to_srgb msaa enabled render -auto -fbo

The main linear to srgb with msaa (not doing the resolve yet) blit
requires the four components.

At the end (after a resolve copy), the test uses glReadPixels, and
internally it uses the blitter with two components, but the shader
still uses lod on the texel fetch, so it gets the one used for the
main blit, when it should be zero.

Right now v3d works fine even with that wrong value, and I assume that
any other driver too. But we can't ensure that would keep happening on
the future, so let's use correct values.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13753>

nir: Group per-primitive outputs at the end for driver location assign.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Don't compact per-vertex and per-primitive outputs together.

Prevent nir_compact_varyings from putting per-vertex and per-primitive
output components in the same slot.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Lower cull and clip distance arrays for mesh shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Add new option to lower invocation ID from invocation index.

Add this as an option to nir_lower_compute_system_values_options
instead of just relying on the shader's options.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir, spirv: Don't mark NV_mesh_shader primitive indices as per-primitive.

They are not per-primitive in NV_mesh_shader, but a flat array.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Fix nir_lower_io with per primitive outputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Print Mesh Shader specific info.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Rename nir_get_io_vertex_index_src and include per-primitive I/O.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

ac/rgp: Initialize clock_calibration with memset.

Fix defect reported by Coverity Scan.

Uninitialized scalar variable (UNINIT)
uninit_use_in_call: Using uninitialized value clock_calibration.
Field clock_calibration.reserved is uninitialized when calling
fwrite.

Fixes: 1ee85e8babe ("ac/rgp: add support for clock calibration")
Suggested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13783>

freedreno/a4xx: hook up sample mask/id, used to determine helper invocs

This fixes the various gl_HelperInvocation-based tests. There's a
lowering pass which converts it to (1 << sampleid) & samplemask.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: include guesses from a3xx for some of the constid's

The ones that are untested are left as comments. The ones that rename
values were tested manually.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: indicate whether outputs are uint/sint

Unclear whether this fixes anything, but the blob does seem to set
these. (Discovered while trying to determine if value clamping was
missing for non-32-bit integer formats, which fail in some tests.)

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: fix stencil-textured border colors

These are implemented with unusual sampler formats, so the usual approach
of looking at the format descriptors fails.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a6xx: re-express buffer textures more logically

Same as a5xx, move one bit into the tex type, one as a separate named
BUFFER bit.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

freedreno/a5xx: re-express buffer textures more logically

Instead of treating it as 2 bits to enable, make BUFFER a type (and
extend the bitfield width), and then add a separate BUFFER bit
(ostensibly to perform the width/height concatenation but who knows).

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

freedreno/a4xx: add proper buffer texture support

Rather than faking it as a 1d texture, add the buffer texture type, and
allow a full range of sizes.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

radeonsi: make si_llvm_emit_clipvertex non-static

it will be used in culling code

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: remove an incorrect comment at lds_byte0_accept_flag

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: improve memory instruction tracking

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: add dcc_msaa option to enable DCC for MSAA

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: unify GFX9_VSGS_NUM_USER_SGPR and GFX9_TESGS_NUM_USER_SGPR

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

ac,radeonsi: cull small lines in the shader using the diamond exit rule

It also splits clip_half_line_width into X and Y components for tighter
view culling.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: add si_state_rasterizer::ngg_cull_flags_lines and rename the others

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: set EXTRA_DX_DY_PRECISION for lines where it's supported

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: set PERPENDICULAR_ENDCAP_ENA for wide AA lines

This is more correct.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: make si_get_small_prim_cull_info static

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: use ac_build_load_to_sgpr in gfx10_emit_ngg_culling_epilogue

This is more correct because we are loading constants into an SGPR even
though there is no effect on behavior in this case.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: fix view culling for wide lines

We need to cull wide lines as quads, but only for view culling.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: fix shader culling with integer pixel centers

Only Nine was using them.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

nir: remove double-validation of src component counts

The nir_tex_instr_src_size helper already sorts this out correctly, no
need to do it twice, and validate_src takes care of it.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13781>

radv: Add more checking of cache sizes.

Hopefully prevents things.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13789>

radv: Fix memory corruption loading RT pipeline cache entries.

Oops. Forgot to account for the size here.

Fixes: ca2d96db51e ("radv: Add caching for RT pipelines.")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13789>

freedreno/ir3: fix image-to-tex flags, remove 3d -> array hack

The function would return both the 3d and array flags set for 2d array,
and would return just 3d for cubes. Fix the flags so that they are
appropriate for images.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13804>

iris: call brw_process_intel_debug_variable() earlier

We're currently only calling it after creating the screen and the
bufmgr. There are a few cases where Iris checks for the DEBUG_BUFMGR
bit before we call brw_process_intel_debug_variable(), which means
intel_debug is 0 and so we don't run the debug code. Today, these are
all related to the creation of the workaround bo and its mmap.

I found this in a custom branch after I converted to INTEL_DEBUG an
environment variable that I had.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13780>

docs: update branchpoint instructions

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13745>

lima: enable PIPE_CAP_PREFER_POT_ALIGNED_VARYINGS

Mali4x0 PP doesn't have a swizzle for load_input, so use POT-aligned
varyings to avoid unnecessary movs for vec3 and precision downgrade
in case if this vec3 is coordinates for a sampler

shader-db:

total instructions in shared programs: 15707 -> 15623 (-0.53%)
instructions in affected programs: 3906 -> 3822 (-2.15%)
helped: 47
HURT: 18
helped stats (abs) min: 1 max: 9 x̄: 3.09 x̃: 2
helped stats (rel) min: 1.49% max: 23.53% x̄: 8.20% x̃: 6.45%
HURT stats (abs)   min: 1 max: 7 x̄: 3.39 x̃: 3
HURT stats (rel)   min: 0.78% max: 20.59% x̄: 10.45% x̃: 10.97%
95% mean confidence interval for instructions value: -2.18 -0.41
95% mean confidence interval for instructions %-change: -5.70% -0.38%
Instructions are helped.

total spills in shared programs: 146 -> 136 (-6.85%)
spills in affected programs: 39 -> 29 (-25.64%)
helped: 6
HURT: 0

total fills in shared programs: 617 -> 598 (-3.08%)
fills in affected programs: 125 -> 106 (-15.20%)
helped: 6
HURT: 0

HURT shaders are vertex shaders where we may need more instructions
for non-packed vec3s. It's acceptable trade-off since we don't get
precision downgrade if this varying is coordinates for a sampler.

Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13151>

gallium: add PIPE_CAP_PREFER_POT_ALIGNED_VARYINGS

Driver should enable this cap if it prefers varyings to be aligned
to power of two in a slot, i.e. vec4 in .xyzw, vec3 in .xyz, vec2 in .xy
or .zw

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13151>