anv: flag BO for write combine when CPU visible and potentially in lmem

This should fix a performance regression with the internal kernel
branch which does not support the upstream I915_MMAP_OFFSET_FIXED.

With I915_MMAP_OFFSET_FIXED we defer the mapping flags to the kernel
since it knows better where buffers are going to end up (lmem or smem).

The internal kernel doesn´t have that and there we should use write
combined for anything that can be in lmem.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Allocate buffers with write-combined local memory

Marginally improves DG1 performance (< 1%)

v2: Only on local mem (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Use WC mapped local memory for block pool BO

Improve DG1 performance:

  Fallout: +7%
  Talos:  +15%

v2: Don't drop SNOOP (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Track BOs that need a write-combined mapping

v2: simplify logic a bit (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

aco/opt_value_numbering: use monotonic_allocator for unordered_map

This patch also changes the rename map to unordered.
Roughly halves the time spent on CSE in ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: implement allocator_traits for monotonic_allocator<T>

For easier usage, this patch also adds aliases for std::map
and std::unordered_map using this allocator.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: use monotonic_buffer_resource for instructions

As monotonic_buffer_resource is not thread-safe,
we use a thread_local instance which gets allocated once.

This change reduces the compile time spent in ACO by
approximately 10%.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: implement custom memory resource

This basic allocator implements an arena allocation strategy
and cannot free individual allocations.
It is intended for very fast memory allocations in situations
where memory is used to build up a few objects and then is
released all at once.

This class mimics std::pmr::monotonic_buffer_resource.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: simplify operands_offset calculation in create_instruction()

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

blorp: Fix typo in blorp_xy_block_copy_blt

suppotred -> supported (Thanks to Tapani for catching this.)

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

blorp: Implement blitter clears via XY_FAST_COLOR_BLT

Vulkan transfer queues need this functionality.  A lot of the code is
pretty similar to what we have for XY_BLOCK_COPY_BLT.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

blorp: Make blitter_supports_aux accessible from multiple files.

We'll want it in blorp_clear.c shortly.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

intel/genxml: Add XY_FAST_COLOR_BLT

We'll need to use this for VkCmdFillBuffer on transfer queues.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

ci: bump mold to 1.5

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>

ci: unexport local variable (and fix formatting)

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>

intel/devinfo: DG2 supports ray-tracing

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Advertise ray-tracing on DG2

Also disable ray-tracing support if with_intel_vk_rt is not set.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: Define with_intel_vk_rt based on with_intel_clc

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: Deprecate vulkan-rt-drivers intel

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/meson: Use anv_flags and anv_cpp_flags in genX compiles

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: use the right dispatch size for tracing shaders

We assumed the trampoline shader would always be SIMD8.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: bump client visible address heap to 32GiB

Some raytracing tests are allocating lots of buffer and because of our
2Mb alignment restriction on local memory, we're running our of VMA...

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: setup scratch space correctly for RT shaders

Things are a bit confusing because we use the term "scratch" for 2
different things :
  * the buffer for register allocation spilling
  * the buffer for storing live values between splitted shaders around shader calls

Here we're fixing the missing register allocation spilling buffer.

v2: update comments (Caio)
    fix scratch bo size computation with pipeline libraries (Lionel)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Build BVHs on the GPU with GRL

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/grl: Parse GRL files and generate C

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Add a helper for dispatching our pre-built kernels

v2: Use the default pipeline cache (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Build OpenCL kernels

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Add a GRL file parser

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Import GRL

GRL, or Graphics Library for Ray-tracing is a library we share with the
Windows drivers for doing BVH builds on the GPU.  It consists of a few
headers shared between CL and C code, a bunch of CL kernels, and some
GRL meta-kernels in their own format.

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: add new command buffer space allocation

To be used for acceleration structure building.

v2: fix missing u_vector_finish
    Free all BOs

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add support for OpenCL-style kernel dispatch

v2: Use brw_cs_get_dispatch_info() (Lionel)
    Merge barrier fixes (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add support for compiling OpenCL-style kernels

v2: remove unused definitions

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add extern "C" guards

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: disable SIMD16 for RT shaders

Since divergence is a lot more likely in RT than compute, it makes
sense to limit ourselves to SIMD8.

The trampoline shader defaults to SIMD16 since this one is uniform.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Set up the memory-backed FIFO buffer

v2: Fix incorrect goto (Caio)
    Comment 3DSTATE_BTD programming (Caio)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Implement VK_KHR_pipeline_library

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add an anv_address_map helper

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/formats: Advertise ACCELERATION_STRUCTURE_VERTEX_BUFFER_BIT

v2: Only expose the bit when ray tracing is supported.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: allow half GP registers for dereferencing

Some of the GRL metakernels will generate 64bit value in a register,
then use only half of that as the last operation on that value.

v2: Add comment (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: Add a helper for incrementing reference counts

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: add a way to reserve a register

Will be useful for GRL metakernels.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

genxml: add missing no duplicate anyhit flag

This mirrors the VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR
enum of VkGeometryFlagBitsKHR. Purely here for documentation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: disable split_array_vars on opencl kernels

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: disable assert on async stack id

This can be accessed from :
   - RT shaders
   - CS trampoline shader

We missed the second part here.

Fixes: 046571479028 ("intel/nir/rt: add more helpers for ray queries")
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: fix potential invalid function impl ptr usage

We keep the nir_builder::impl value around, but we've run some passes
that might have change the main function.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 96fde5518b5c ("intel/rt: Add a helper to create the raygen trampoline shader")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: fixup preserved metadata in rayquery lowering

Another case of not clearing the metadata correctly.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: c78be5da300a ("intel/fs: lower ray query intrinsics")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: take a builder arg for resolve_source_modifiers()

There will be situations where we will want to use a local builder
rather than the one associated with NIR->backend translation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: reuse rt helper

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/rt: fix procedural primitive ID access

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: SEL_EXEC uses the integer pipe for 64-bit stuff

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: Always use integer types for indirect MOVs

There's a new Gen12.5 restriction which forbids using the VxH or Vx1 on
the floating-point pipe.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/devinfo: Rename & implement num_dual_subslices

v2: Use the upper bound of dual subslices as the ID is not remapped
with fused off parts and this is what we'll use for a bunch of
computation in RT.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: bump required llvm-spirv version with intel-clc

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: remove HDC flush from invalidate bits

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: a49b145e8d59 ("anv: Replace DC Flush with HDC Pipeline Flush")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/compiler: Vectorize gl_TessLevelInner/Outer[] writes

Setting the NIR options takes care of iris thanks to the common st/mesa
linking code, and updating brw_nir_link_shaders should handle anv.

The main effort here is updating remap_tess_levels, which needs to
handle vector stores, writemasking, and swizzling.  Unfortunately,
we also need to continue handling the existing single-component
access because it's used for TES inputs, which we don't vectorize.

We could try to vectorize TES inputs too, but they're all pushed
anyway, so it wouldn't buy us much other than deleting this code.
Also, we do have opt_combine_stores, but not one for loads.

One limitation of using nir_vectorize_tess_levels is that it works
on variables, and so isn't able to combine outer/inner writes that
happen to live in the same vec4 slot (for triangle domains).  That
said, it's still better than before.

For writes, we allow the intrinsics to supply up to the full size
of the variable (vec4 for outer, vec2 for inner) even if the domain
only requires a subset of those components (i.e. triangles needs 3).

shader-db results on Icelake:

   total instructions in shared programs: 19605070 -> 19602284 (-0.01%)
   instructions in affected programs: 65338 -> 62552 (-4.26%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 6 max: 24 x̄: 10.28 x̃: 12
   helped stats (rel) min: 1.30% max: 18.18% x̄: 5.80% x̃: 7.59%
   95% mean confidence interval for instructions value: -10.71 -9.85
   95% mean confidence interval for instructions %-change: -6.17% -5.43%
   Instructions are helped.

   total cycles in shared programs: 851854659 -> 851820320 (<.01%)
   cycles in affected programs: 618749 -> 584410 (-5.55%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 69 max: 540 x̄: 126.71 x̃: 108
   helped stats (rel) min: 2.57% max: 37.97% x̄: 6.17% x̃: 5.06%
   95% mean confidence interval for cycles value: -135.89 -117.54
   95% mean confidence interval for cycles %-change: -6.72% -5.63%
   Cycles are helped.

   total sends in shared programs: 1025285 -> 1024355 (-0.09%)
   sends in affected programs: 6454 -> 5524 (-14.41%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 2 max: 8 x̄: 3.43 x̃: 4
   helped stats (rel) min: 5.71% max: 25.00% x̄: 14.98% x̃: 17.39%
   95% mean confidence interval for sends value: -3.57 -3.29
   95% mean confidence interval for sends %-change: -15.42% -14.54%
   Sends are helped.

According to Felix DeGrood, this results in a 10% improvement in
the draw call time for certain draw calls from Strange Brigade.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

st/mesa: Optionally call nir_vectorize_tess_levels()

This lets us vectorize gl_TessLevel{Inner,Outer} writes, using a pass
developed for RADV.  Not all backends are prepared to handle this, so
we make it optional.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

intel/compiler: Use an existing URB write to end TCS threads when viable

VS, TCS, TES, and GS threads must end with a URB write message with the
EOT (end of thread) bit set.  For VS and TES, we shadow output variables
with temporaries and perform all stores at the end of the shader, giving
us an existing message to do the EOT.

In tessellation control shaders, we don't defer output stores until the
end of the thread like we do for vertex or evaluation shaders.  We just
process store_output and store_per_vertex_output intrinsics where they
occur, which may be in control flow.  So we can't guarantee that there's
a URB write being at the end of the shader.

Traditionally, we've just emitted a separate URB write to finish TCS
threads, doing a writemasked write to an single patch header DWord.
On Broadwell, we need to set a "TR DS Cache Disable" bit, so this is
a convenient spot to do so.  But on other platforms, there's no such
field, and this write is purely wasteful.

Insetad of emitting a separate write, we can just look for an existing
URB write at the end of the program and tag that with EOT, if possible.
We already had code to do this for geometry shaders, so just lift it
into a helper function and reuse it.

No changes in shader-db.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

ci: disable intel-clc on debian-vulkan

We're getting a number of UBSan error while running intel-clc in that
image. It seems that we're the first ones to run into a number of code
paths with intel-clc and it shows a number of undefined behavior
operations like signed extension stuff in NIR/IntelBackend, unaligned
pointer accesses in embedded list iterators, etc...

Preparing some patches in a different MR to fix this.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci: bump llvm to 13 for some builders

Namely :
  - debian-clang
  - debian-cl
  - debian-vulkan

Seems to trigger/fix failures on llvmpipe, filed
https://gitlab.freedesktop.org/mesa/mesa/-/issues/7336

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci/debian: don't use libclc from the system

Debian (unlike Ubuntu) has a broken libclc package missing files we
would very much like to have in our image, so that intel_clc doesn't
fail. Namely :

   /usr/lib/clc/spirv-mesa3d-.spv
   /usr/lib/clc/spirv64-mesa3d-.spv

Dropping libclc from the distribution and build int the build & base
test image.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci: build our own version of the LLVM SPIRV translator

Debian stable and Fedora do not package the required version for
intel-clc.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci: disable intel-clc on fedora

Would require Fedora 35.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci: add python3-ply to debian/fedora images

This is needed by Anv to parse GRL (meta opencl kernels).

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

gallivm/nir: bitcast when non-float ptr type.

This matters more when opaque pointers are used.

Fixes: 203920d4c693 ("gallivm: add atomic 32-bit float support")

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18835>

gallivm/nir: fix fmin/fmax translation

Fixes: 203920d4c693 ("gallivm: add atomic 32-bit float support")

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18835>

lavapipe: add fmin/fmax to image lowering.

Fixes: 31695f81c925 ("lavapipe: export VK_KHR_shader_atomic_float")

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18835>

zink: also lower 64bit function temps

now that these aren't always lowered from indirects,
64bit function temp arrays may exist in shaders, and they need to use
all the same rewrite machinery

fixes #7310

SoroushIMG <soroush.kashani@imgtec.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18728>

zink: split up lower_64bit_vars pass

the component functions will be reused; no functional changes

SoroushIMG <soroush.kashani@imgtec.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18728>

zink: always run optimize_nir after lower_64bit_vars

it's otherwise possible (and likely) that optimizations won't
happen since there's no shader key data active

Fixes: 5b2f850425e ("zink: rewrite 64bit shader i/o as 32bit")

SoroushIMG <soroush.kashani@imgtec.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18728>

zink: don't flatten 64bit arrays during rewrite

dunno what I was thinking here

Fixes: 5b2f850425e ("zink: rewrite 64bit shader i/o as 32bit")

SoroushIMG <soroush.kashani@imgtec.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18728>

nir/opt_undef: add a pass to clean up 64bit undefs

somehow 64bit lowering creates patterns like

vec1 64 ssa_1 = undefined
ssa_2 = unpack_64_2x32_split_x ssa_1

and then the 64bit value is never otherwise used. for this case, rewriting
the unpack to just be a 32bit undef allows the 64bit undef to be optimized
out, avoiding spec violations

fixes #6945

SoroushIMG <soroush.kashani@imgtec.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18728>

rusticl/device: fix custom device detection

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18809>

zink: always set VK_PIPELINE_CREATE_COLOR_ATTACHMENT_FEEDBACK_LOOP_BIT_EXT sometimes

some drivers (mostly desktop) are known to ignore this pipeline flag,
thus it can be set unconditionally in pipeline state to deduplicate
pipeline variants without affecting performance

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: use feedback loop layout to correctly handle implicit feedback loops

an implicit feedback loop occurs when an app happens to bind the same image
as both a framebuffer attachment and a sampler for the same draw

an explicit feedback loop occurs when an app uses fbfetch to read data back
from the framebuffer using input attachments

fbfetch is already handled, but implicit feedback loops require more work:
* detecting them happens on-the-fly
* pipeline variants are required

this handles implicit feedback loops by detecting them at draw time during
barrier updates and then flagging pipeline state change to trigger variant creation.
the bits are then unset when the framebuffer/sampler binds are removed

fixes #7309

fixes (tu):
KHR-GL46.texture_barrier.disjoint-texels
KHR-GL46.texture_barrier.overlapping-texels
KHR-GL46.texture_barrier.same-texel-rw-multipass
KHR-GL46.texture_barrier_ARB.disjoint-texels
KHR-GL46.texture_barrier_ARB.overlapping-texels
KHR-GL46.texture_barrier_ARB.same-texel-rw-multipass

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: add a mask of fb attachment idx for resources

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: reorder zink_resource a little

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: rename fb_binds -> fb_bind_count

this is more accurate

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: remove zink_context::new_swapchain

this was my (feeble) attempt at tracking validity for swapchain images,
but it doesn't actually work right, and zink_resource::valid is better

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

zink: enable VK_ATTACHMENT_LOAD_OP_DONT_CARE for zs in renderpasses

dynamic render path already handled this

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18787>

mesa: set normalized_coords for bindless texture buffer samplers

this isn't "used", but it's the default

cc: mesa-stable

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18845>

egl: Remove a bogus restriction from eglMakeCurrent

The comment here is from the EGL_MESA_configless_context era, and maybe
that was true, but EGL_KHR_no_config_context has no such restriction.
The only restriction is that both surfaces be compatible with the
context, but a no-config context is defined to be compatible with any
surface.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18824>

egl/dri2: Fix a weird conditional in dri2_make_current

It's not valid to get here with no context but with real surfaces.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18824>

egl: Factor some common terminate cleanup up to common code

It's a little difficult to see from the diff, but this is effectively
the same calling sequence as before, and more importantly it means the
backend only cleans up backend state rather than needing to call up to
the core.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18824>

egl: Formatting fix

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18824>

mesa/fbo: Don't force both read/draw bindings to zero

... when we're only changing one of the two to zero. I could maybe
understand the old behavior here if we really did need both read/draw to
either be both from winsys or both user, but we don't. You can still
get to that half-and-half state even with the old code, you just need to
set whichever of read/draw to 0 that you want first, and then set the
other to the user fbo.

The first code to introduce a read fb to this path was dbfb375805d94,
which was trying to make sure we restored both values passed to
glXMakeCurrentRead when going back to fb 0. The commit message suggests
that "the API" doesn't allow split read/draw, but that seems to have
ignored the #ifdef EXT_framebuffer_object code immediately nearby which
did exactly that.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18824>

radeonsi/vce: using fixed value for vce 52

VCE 52 needs some hard-coded values.

Fixed: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7290

Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18823>

rusticl: Fix compilation if stdout and/or stderr aren't symbols

ISO C only guarantees that `stdout` and `stderr` are expressions of type `FILE*`. It does not
guarantee that they are symbols which can be bound to. In particular, they are macros on the BSDs,
which bindgen cannot deal with (yet?).
Add our own symbol-defining wrapper.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7269

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18810>

ci/amd: move radeonsi in amd directory

Deduplicate jobs, make easier to use GL+VK scenarios.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci/radeonsi: add traces for radeonsi Zork

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci/traces: prevent showing wine instalation dialog

Causing piglit timeouts because of an open window instead of running apitrace.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci: also handle default wine rootfs (for testing)

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci/dxvk: fix DXVK 64-bit only wine setup

Reported: https://github.com/doitsujin/dxvk/issues/2921

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci/lava: prepare wineprefix for GL and DXVK

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

ci/lava: add wine and apitrace into amd64 rootfs container

Wine is from WineHQ, with stripped i386 part, which we don't use.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>

radv: fix handling multiview with GPL

When multiview is used, the last pre-rasterization stage should export
the layer to the fragment shader. With GPL, the next stage is
implicitly FS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>

radv: fix handling primitive ID and clip/cull distances with GPL

When compiling the pre-rasterization stages with GPL, we can't know if
the fragment shader reads the primitive ID or the clip/cull distances
as inputs and we have to export them unconditionally.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>

radv: fix handling ViewportIndex with GPL

When the fragment shader reads the viewport index as input and the last
pre-rasterization stage doesn't export it, it should be implicitly
zero (ie. first viewport). When the next stage is known, it's already
lowered in NIR, but with GPL we can't because we don't know if the FS
reads it. To fix this use AC_EXP_PARAM_DEFAULT_VAL_000.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>

radv: rework multiview lowering in NIR slightly

This will be used to export multiview unconditionally with GPL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>

v3dv: expose VK_EXT_image_robustness

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18820>

broadcom/compiler: add a lowering for robust image access

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18820>