vulkan: Add more dynamic rasterizer state

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>

vulkan: Add a dynamic state for vp.depth_clip_negative_one_to_one

This is the last bit of viewport state that's not dynamic.  Making it
dynamic will help ANV out a bit.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>

vulkan: Rename viewport_state::negative_one_to_one

This makes it a bit clearer what it's for.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>

vulkan: Update the XML and headers to 1.3.230

Acked-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>

vulkan: Use altlen when available for array lengths

When the array length is a calculation, the C version of the calculation
is provided in altlen while len often contains LaTeX that we can't do
anything with.  Use altlen when available.  Also, while we're here, wrap
array lengths in parentheses in case they contain math.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>

gallium: avoid using float based conditions in loops

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12996>

zink: pass actual screen->threaded to zink_tc_context_unwrap()

using the ctx->screen pointer may not actually yield a zink_screen object

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18868>

zink: hook up VK_EXT_depth_clamp_zero_one

this is required for GL since the allowable depth range is always [0,1]

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18868>

zink: move create_pipeline_lib to zink_program.c

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18868>

zink: rework ZINK_PIPELINE_LIBRARY_FORCE into ZINK_DEBUG flag

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18868>

zink: move zink_screen_get_pipeline_cache() calls out to callers

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18868>

vulkan/wsi: Add a supports_scanout flag

This flag indicates whether or not the legacy scanout flag is supported.
It defaults to true since that has been the default assumption for the
WSI code up until now.

On NVIDIA hardware, we can't render to linear so, if we don't have
modifiers, we want to automatically fall back to the blit path.  In
theory, we could do this inside the driver but it's a giant pain and
much harder to ensure that the blit only happens as part of
vkQueuePresent().

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi: Add a typedef for memory type select callbacks

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi: Don't expose as many image configure helpers

Now that everything is going through the params-based ones, we don't
need to expose the individual ones.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi: Always configure images in wsi_swapchain_init()

Now that all the back-ends are converted, we can drop all the extra code
we had for supporting both paths.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi/win32: Configure images via params passed to wsi_swapchain_init()

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi/display: Configure images via params passed to wsi_swapchain_init()

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi/wayland: Configure images via params passed to wsi_swapchain_init()

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi/x11: Configure images via params passed to wsi_swapchain_init()

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

vulkan/wsi: Support configuring swapchain images as part of swapchain init

The eventual goal here is to move as much of the prime and blit logic
out of the individual window-system back-ends as possible.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18826>

llvmpipe/cs: move compute code to explicit pointer types

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/setup: move setup code to explicit pointers.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/fs: fix invocations access for opaque ptrs.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/cs: convert cs thread data to opaque friendly api

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/cs: convert cs context to opaque friendly api

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/fs: convert linear context to opaque pointers friendly

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/fs: convert thread data ptr to opaque ptr friendly apis

this converts the thread data code.

The cache code still isn't fixed but needs future API changes
to sampling code.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

llvmpipe/fs: start passing explicit context pointer type.

In order to support opaque pointers in the future, we need to be
more explicit with the pointer types here.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18837>

gallium: Do not include offscreen winsys if glx is disabled

Offscreen winsys introduced in Mesa 22.2 depends on glx(dri) and libswkmsdri.

The error message is:
/usr/bin/ld: src/gallium/auxiliary/libgalliumvlwinsys.a.p/vl_vl_winsys_dri_vgem.c.o: in function `vl_vgem_drm_screen_create':
vl_winsys_dri_vgem.c:(.text.vl_vgem_drm_screen_create+0x28): undefined reference to `kms_dri_create_winsys'

Fixes: 31dcb396 (gallium/vl: Add software winsys and offscreen winsys)

Cc: mesa-stable
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18843>

docs/rusticl: Document minimum required bindgen version

rusticl requires at least bindgen 0.58.0 to build.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18834>

rusticl/bindgen: Use `allowlist-*` instead of `whitelist-*` switches

bindgen 0.58.0 deprecated the `whitelist-*` switches in favor of the new `allowlist-*` switches.
Currently rusticl uses a mixture of both.

Consistently move to the newer versions.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18834>

gallivm: avoid the use of an uninitialized value

When need_derivs is false, lp_build_cube_lookup does not set the value of
derivs_out which means that the stack is not initialized but its pointer
is then used.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18635>

zink: set layouts before possibly reordering image copies

layout-setting may change which cmdbuf can be used

Fixes: 731d7be3756 ("zink: make get_cmdbuf() public")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18727>

zink: always unflag unordered_write when binding image descriptors

image descriptors have to maintain their layout, which means it's no
longer possible to reorder any operations once they use a non-transfer
layout

Fixes: ca03e35821d ("zink: expand unordered_exec")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18727>

aux/trace: dump depth_clamp for rasterizer state

cc: mesa-stable

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18783>

aux/trace: dump line_rectangular member of rasterizer state

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18783>

radv: Add and use AS and scratch layout structs

The memory layout logic is duplicated between
radv_GetAccelerationStructureBuildSizesKHR and
radv_CmdBuildAccelerationStructuresKHR. This patch adds a helper that
computes the scratch and acceleration structure memory layout for a
given build configuration.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18842>

radv: fix emitting RBPLUS state when MRTs are compacted

The RBPLUS state needs the non-compacted SPI_SHADER_COL_FORMAT value,
otherwise the state is wrongly emitted if there is holes.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7348
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7319
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7365
Fixes: 8fcb4aa0ebd ("radv: compact MRTs to save PS export memory space")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18818>

radv: Do an early check of distance in triangle culling.

Culls like 99% of the triangles that are culled at all.

Reduces VALU usage in Q2RTX traversal by ~8%, though doesn't look
like VALU is a bottleneck at this point ...

For Control we get a ~5% reduction in VALU usage, but similarly it
doesn't look like a bottleneck.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18830>

tc: don't use CPU storage for glBufferData

This fixes a performance regression with yquake2 cause by
the enablement of cpu_storage by default for radeonsi in
a5a8e197413.

Fixes: a5a8e197413 ("radeonsi: enable tc cpu_storage by default")
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18774>

tc: do a GPU->CPU copy to initialize cpu_storage

If the GPU-side storage has been written to without using cpu_storage,
then we have to initialize the CPU-side storage correctly.

This requires a sync + copy but it's a one time operation so it shouldn't
affect performance much.

I don't think it fixes any existing bug, but the next commit will need
this to behave correctly.

cc: mesa-stable

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18774>

anv: flag BO for write combine when CPU visible and potentially in lmem

This should fix a performance regression with the internal kernel
branch which does not support the upstream I915_MMAP_OFFSET_FIXED.

With I915_MMAP_OFFSET_FIXED we defer the mapping flags to the kernel
since it knows better where buffers are going to end up (lmem or smem).

The internal kernel doesn´t have that and there we should use write
combined for anything that can be in lmem.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Allocate buffers with write-combined local memory

Marginally improves DG1 performance (< 1%)

v2: Only on local mem (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Use WC mapped local memory for block pool BO

Improve DG1 performance:

  Fallout: +7%
  Talos:  +15%

v2: Don't drop SNOOP (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

anv: Track BOs that need a write-combined mapping

v2: simplify logic a bit (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>

aco/opt_value_numbering: use monotonic_allocator for unordered_map

This patch also changes the rename map to unordered.
Roughly halves the time spent on CSE in ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: implement allocator_traits for monotonic_allocator<T>

For easier usage, this patch also adds aliases for std::map
and std::unordered_map using this allocator.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: use monotonic_buffer_resource for instructions

As monotonic_buffer_resource is not thread-safe,
we use a thread_local instance which gets allocated once.

This change reduces the compile time spent in ACO by
approximately 10%.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: implement custom memory resource

This basic allocator implements an arena allocation strategy
and cannot free individual allocations.
It is intended for very fast memory allocations in situations
where memory is used to build up a few objects and then is
released all at once.

This class mimics std::pmr::monotonic_buffer_resource.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

aco: simplify operands_offset calculation in create_instruction()

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>

blorp: Fix typo in blorp_xy_block_copy_blt

suppotred -> supported (Thanks to Tapani for catching this.)

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

blorp: Implement blitter clears via XY_FAST_COLOR_BLT

Vulkan transfer queues need this functionality.  A lot of the code is
pretty similar to what we have for XY_BLOCK_COPY_BLT.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

blorp: Make blitter_supports_aux accessible from multiple files.

We'll want it in blorp_clear.c shortly.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

intel/genxml: Add XY_FAST_COLOR_BLT

We'll need to use this for VkCmdFillBuffer on transfer queues.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>

ci: bump mold to 1.5

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>

ci: unexport local variable (and fix formatting)

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>

intel/devinfo: DG2 supports ray-tracing

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Advertise ray-tracing on DG2

Also disable ray-tracing support if with_intel_vk_rt is not set.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: Define with_intel_vk_rt based on with_intel_clc

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: Deprecate vulkan-rt-drivers intel

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/meson: Use anv_flags and anv_cpp_flags in genX compiles

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: use the right dispatch size for tracing shaders

We assumed the trampoline shader would always be SIMD8.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: bump client visible address heap to 32GiB

Some raytracing tests are allocating lots of buffer and because of our
2Mb alignment restriction on local memory, we're running our of VMA...

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: setup scratch space correctly for RT shaders

Things are a bit confusing because we use the term "scratch" for 2
different things :
  * the buffer for register allocation spilling
  * the buffer for storing live values between splitted shaders around shader calls

Here we're fixing the missing register allocation spilling buffer.

v2: update comments (Caio)
    fix scratch bo size computation with pipeline libraries (Lionel)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Build BVHs on the GPU with GRL

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/grl: Parse GRL files and generate C

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Add a helper for dispatching our pre-built kernels

v2: Use the default pipeline cache (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Build OpenCL kernels

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/grl: Add a GRL file parser

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Import GRL

GRL, or Graphics Library for Ray-tracing is a library we share with the
Windows drivers for doing BVH builds on the GPU.  It consists of a few
headers shared between CL and C code, a bunch of CL kernels, and some
GRL meta-kernels in their own format.

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: add new command buffer space allocation

To be used for acceleration structure building.

v2: fix missing u_vector_finish
    Free all BOs

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add support for OpenCL-style kernel dispatch

v2: Use brw_cs_get_dispatch_info() (Lionel)
    Merge barrier fixes (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add support for compiling OpenCL-style kernels

v2: remove unused definitions

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add extern "C" guards

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: disable SIMD16 for RT shaders

Since divergence is a lot more likely in RT than compute, it makes
sense to limit ourselves to SIMD8.

The trampoline shader defaults to SIMD16 since this one is uniform.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Set up the memory-backed FIFO buffer

v2: Fix incorrect goto (Caio)
    Comment 3DSTATE_BTD programming (Caio)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Implement VK_KHR_pipeline_library

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: Add an anv_address_map helper

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv/formats: Advertise ACCELERATION_STRUCTURE_VERTEX_BUFFER_BIT

v2: Only expose the bit when ray tracing is supported.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: allow half GP registers for dereferencing

Some of the GRL metakernels will generate 64bit value in a register,
then use only half of that as the last operation on that value.

v2: Add comment (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: Add a helper for incrementing reference counts

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/mi_builder: add a way to reserve a register

Will be useful for GRL metakernels.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

genxml: add missing no duplicate anyhit flag

This mirrors the VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR
enum of VkGeometryFlagBitsKHR. Purely here for documentation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: disable split_array_vars on opencl kernels

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: disable assert on async stack id

This can be accessed from :
   - RT shaders
   - CS trampoline shader

We missed the second part here.

Fixes: 046571479028 ("intel/nir/rt: add more helpers for ray queries")
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: fix potential invalid function impl ptr usage

We keep the nir_builder::impl value around, but we've run some passes
that might have change the main function.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 96fde5518b5c ("intel/rt: Add a helper to create the raygen trampoline shader")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: fixup preserved metadata in rayquery lowering

Another case of not clearing the metadata correctly.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: c78be5da300a ("intel/fs: lower ray query intrinsics")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: take a builder arg for resolve_source_modifiers()

There will be situations where we will want to use a local builder
rather than the one associated with NIR->backend translation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/nir: reuse rt helper

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/rt: fix procedural primitive ID access

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: SEL_EXEC uses the integer pipe for 64-bit stuff

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/fs: Always use integer types for indirect MOVs

There's a new Gen12.5 restriction which forbids using the VxH or Vx1 on
the floating-point pipe.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/devinfo: Rename & implement num_dual_subslices

v2: Use the upper bound of dual subslices as the ID is not remapped
with fused off parts and this is what we'll use for a bunch of
computation in RT.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

meson: bump required llvm-spirv version with intel-clc

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

anv: remove HDC flush from invalidate bits

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: a49b145e8d59 ("anv: Replace DC Flush with HDC Pipeline Flush")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>

intel/compiler: Vectorize gl_TessLevelInner/Outer[] writes

Setting the NIR options takes care of iris thanks to the common st/mesa
linking code, and updating brw_nir_link_shaders should handle anv.

The main effort here is updating remap_tess_levels, which needs to
handle vector stores, writemasking, and swizzling.  Unfortunately,
we also need to continue handling the existing single-component
access because it's used for TES inputs, which we don't vectorize.

We could try to vectorize TES inputs too, but they're all pushed
anyway, so it wouldn't buy us much other than deleting this code.
Also, we do have opt_combine_stores, but not one for loads.

One limitation of using nir_vectorize_tess_levels is that it works
on variables, and so isn't able to combine outer/inner writes that
happen to live in the same vec4 slot (for triangle domains).  That
said, it's still better than before.

For writes, we allow the intrinsics to supply up to the full size
of the variable (vec4 for outer, vec2 for inner) even if the domain
only requires a subset of those components (i.e. triangles needs 3).

shader-db results on Icelake:

   total instructions in shared programs: 19605070 -> 19602284 (-0.01%)
   instructions in affected programs: 65338 -> 62552 (-4.26%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 6 max: 24 x̄: 10.28 x̃: 12
   helped stats (rel) min: 1.30% max: 18.18% x̄: 5.80% x̃: 7.59%
   95% mean confidence interval for instructions value: -10.71 -9.85
   95% mean confidence interval for instructions %-change: -6.17% -5.43%
   Instructions are helped.

   total cycles in shared programs: 851854659 -> 851820320 (<.01%)
   cycles in affected programs: 618749 -> 584410 (-5.55%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 69 max: 540 x̄: 126.71 x̃: 108
   helped stats (rel) min: 2.57% max: 37.97% x̄: 6.17% x̃: 5.06%
   95% mean confidence interval for cycles value: -135.89 -117.54
   95% mean confidence interval for cycles %-change: -6.72% -5.63%
   Cycles are helped.

   total sends in shared programs: 1025285 -> 1024355 (-0.09%)
   sends in affected programs: 6454 -> 5524 (-14.41%)
   helped: 271 / HURT: 0
   helped stats (abs) min: 2 max: 8 x̄: 3.43 x̃: 4
   helped stats (rel) min: 5.71% max: 25.00% x̄: 14.98% x̃: 17.39%
   95% mean confidence interval for sends value: -3.57 -3.29
   95% mean confidence interval for sends %-change: -15.42% -14.54%
   Sends are helped.

According to Felix DeGrood, this results in a 10% improvement in
the draw call time for certain draw calls from Strange Brigade.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

st/mesa: Optionally call nir_vectorize_tess_levels()

This lets us vectorize gl_TessLevel{Inner,Outer} writes, using a pass
developed for RADV.  Not all backends are prepared to handle this, so
we make it optional.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

intel/compiler: Use an existing URB write to end TCS threads when viable

VS, TCS, TES, and GS threads must end with a URB write message with the
EOT (end of thread) bit set.  For VS and TES, we shadow output variables
with temporaries and perform all stores at the end of the shader, giving
us an existing message to do the EOT.

In tessellation control shaders, we don't defer output stores until the
end of the thread like we do for vertex or evaluation shaders.  We just
process store_output and store_per_vertex_output intrinsics where they
occur, which may be in control flow.  So we can't guarantee that there's
a URB write being at the end of the shader.

Traditionally, we've just emitted a separate URB write to finish TCS
threads, doing a writemasked write to an single patch header DWord.
On Broadwell, we need to set a "TR DS Cache Disable" bit, so this is
a convenient spot to do so.  But on other platforms, there's no such
field, and this write is purely wasteful.

Insetad of emitting a separate write, we can just look for an existing
URB write at the end of the program and tag that with EOT, if possible.
We already had code to do this for geometry shaders, so just lift it
into a helper function and reuse it.

No changes in shader-db.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17944>

ci: disable intel-clc on debian-vulkan

We're getting a number of UBSan error while running intel-clc in that
image. It seems that we're the first ones to run into a number of code
paths with intel-clc and it shows a number of undefined behavior
operations like signed extension stuff in NIR/IntelBackend, unaligned
pointer accesses in embedded list iterators, etc...

Preparing some patches in a different MR to fix this.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>

ci: bump llvm to 13 for some builders

Namely :
  - debian-clang
  - debian-cl
  - debian-vulkan

Seems to trigger/fix failures on llvmpipe, filed
https://gitlab.freedesktop.org/mesa/mesa/-/issues/7336

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18788>