nir: use intrinsic builders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

glsl_to_nir: use intrinsic builders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

spirv: use intrinsic builders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: add generated intrinsic builders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: move nir_load_system_value() to nir_builder.h

To make review easier.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: remove useless nir_builder_opcodes.h include

This is included earlier in the file.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: add destination bit-size information to more intrinsics

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: add bit_size_src for when the destination bit size matches a source

If bit_size_src is not -1, then it's the index of the source the
destination bit size can be expected to match. This will be useful for
generating intrinsic builders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: use a single canonical list of intrinsic indices

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

nir: fix sampler_lod_parameters_pan indices

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Fixes: deaebc82a73 "nir: Add load_sampler_lod_paramaters_pan intrinsic"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>

freedreno/a6xx: Fix typo in height alignment calculation in a6xx layout

Fixes KHR-GL31.texture_size_promotion.functional

Fixes: e49748521ec9182e8d2eec823182cc463709123f
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7792>

freedreno/a6xx: add support for dual-source blending

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7708>

nir/lower_non_uniform: improve code with the same texture, sampler indices

NIR can't CSE the read_first_invocation intrinsics, so we can end up
creating iand(read_first_invocation(a) == a, read_first_invocation(a) == a)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3813>

anv: fix descriptor pool leak in VMA object

Quoting the spec :

   "When a pool is destroyed, all descriptor sets allocated from the
    pool are implicitly freed and become invalid. Descriptor sets
    allocated from a given pool do not need to be freed before
    destroying that descriptor pool."

This implies we might leak nodes allocated in the vma object.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 0a6d2593b8b63d ("anv: Allocate descriptor buffers from the BO cache")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7796>

aco/optimizer: Propagate scc_needed label through p_wqm.

Without this, RA is confused and adds a bunch of unnecessary moves.

Fossil-db results (Navi 10):

Totals from 7658 (5.55% of 137887) affected shaders:
SGPRs: 752488 -> 752392 (-0.01%)
CodeSize: 74046720 -> 73924368 (-0.17%); split: -0.17%, +0.00%
Instrs: 14293916 -> 14263350 (-0.21%); split: -0.21%, +0.00%
Cycles: 1455556728 -> 1438466536 (-1.17%); split: -1.17%, +0.00%
VMEM: 1096736 -> 1094962 (-0.16%); split: +0.03%, -0.19%
SMEM: 448436 -> 448418 (-0.00%); split: +0.00%, -0.01%
SClause: 485501 -> 485495 (-0.00%); split: -0.00%, +0.00%
Copies: 1383769 -> 1353029 (-2.22%); split: -2.22%, +0.00%
Branches: 568247 -> 568451 (+0.04%); split: -0.01%, +0.05%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7677>

aco/optimizer: Only set scc_needed when it is actually needed.

Not every p_cbranch uses the SCC, but our optimizer thought so.

Fixes: 8a32f57fff56b3b94f1b5589feba38016f39427c
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7677>

glsl: avoid an out-of-bound access while setting up a location for variable

It fixes the following valgrind issue:
==141996== Invalid read of size 4
==141996==    at 0x61F8806: gl_nir_link_uniforms (gl_nir_link_uniforms.c:1788)
==141996==    by 0x60F17AA: gl_nir_link_glsl (gl_nir_linker.c:672)
==141996==    by 0x5C1AEDF: st_link_nir (st_glsl_to_nir.cpp:739)
==141996==    by 0x5C15574: st_link_shader (st_glsl_to_ir.cpp:172)
==141996==    by 0x5C673B0: _mesa_glsl_link_shader (ir_to_mesa.cpp:3117)
==141996==    by 0x5E7B61C: link_program (shaderapi.c:1311)
==141996==    by 0x5E7B61C: link_program_error (shaderapi.c:1419)
==141996==    by 0x5E7CF8A: _mesa_LinkProgram (shaderapi.c:1911)
==141996==    by 0x4923D13: stub_glLinkProgram (piglit-dispatch-gen.c:33956)
==141996==    by 0x1142C0: link_and_use_shaders (shader_runner.c:1636)
==141996==    by 0x1205A6: init_test (shader_runner.c:5347)
==141996==    by 0x121555: piglit_init (shader_runner.c:5725)
==141996==    by 0x4991C84: run_test (piglit_fbo_framework.c:50)

It can be reproduced on `iris` using the following piglit test:
instance-matching-shader-storage-blocks-align-qualifier-mismatch.shader_test

Closes: #3818
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Fixes: 47c35823 ("glsl: fix up location setting for variables pointing to a UBO's base")
Signed-off-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Signed-off-by: Andrii Simiklit <andrii.simiklit@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7376>

glsl: Drop mingw -O1 workaround for GCC>=7.3

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7770>

scons: Fix build with llvm-12.

This patch fixes these SCons build errors.

/usr/bin/ld: build/linux-x86_64-debug/gallium/auxiliary/libgallium.a(lp_bld_misc.os): in function `llvm::InitializeNativeTarget()':
llvm/Support/TargetSelect.h:118: undefined reference to `LLVMInitializeX86TargetInfo'
/usr/bin/ld: llvm/Support/TargetSelect.h:119: undefined reference to `LLVMInitializeX86Target'
/usr/bin/ld: llvm/Support/TargetSelect.h:120: undefined reference to `LLVMInitializeX86TargetMC'
/usr/bin/ld: build/linux-x86_64-debug/gallium/auxiliary/libgallium.a(lp_bld_misc.os): in function `llvm::InitializeNativeTargetAsmPrinter()':
llvm/Support/TargetSelect.h:132: undefined reference to `LLVMInitializeX86AsmPrinter'
/usr/bin/ld: build/linux-x86_64-debug/gallium/auxiliary/libgallium.a(lp_bld_misc.os): in function `llvm::InitializeNativeTargetDisassembler()':
llvm/Support/TargetSelect.h:156: undefined reference to `LLVMInitializeX86Disassembler'

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7722>

radv/winsys: fix the sysmem submission path for GFX6

Oops.

Fixes: cba6ec309ab ("radv: Fix -Wshadow warnings")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7790>

radv: Const aco_compiler_statistic_info usage

Necessary for upcoming const correctness change to aco struct.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7787>

radv: fix missing initialization of the predication value

It's expected to be 0.

Fixes: 62d9ca696e0 ("radv: use 32-bit predication for conditional rendering on GFX10.3+")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7789>

v3dv: remove non-conformant warning

The driver is now Vulkan 1.0 conformant.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7758>

docs/features: update list of v3dv supported features

We have been implementing some features without updating the file, and
we even had some that were supported for a while.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
v2: include VK_KHR_wayland_surface as !7303 got merged

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7758>

ac: use bigger storage for ac_arg::arg_index / ac_shader_args::arg_count

AC_MAX_ARGS is now 384 so uint8_t isn't enough.

Fixes: 6f130342658 ("ac/llvm: prepare for passing VS->TCS IO via VGPRs")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7750>

radeonsi/gfx10: flush gfx cs on ngg -> legacy transition

with a sequence like this:

  glClear(STENCIL)
  glBeginTransformFeedback()
  ...
  glEndTransformFeedback()
  glClear(STENCIL)

The second clear sometimes may produce an unexpected result.

Calling si_flush_gfx_cs() when doing ngg -> legacy transition seems to be a
valid workaround (both for the synthetic reproducer and the real Blender bug).

Using flush flags or events (BOTTOM_OF_PIPE_TS, RESET_TO_LOWEST_VGT) didn't help.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2941
Cc: mesa-stable
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7750>

util,radv: Cross-platform monotonic condition variable

cnd_t operates on REALTIME clock, and isn't suitable for MONOTONIC use.

Clone the API, and implement using a monotonic clock.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7138>

util/os_time: Safe os_time_get_nano for Windows

Avoid small possibility of reading torn write on 32-bit platforms.

If frequency caching is desired, it's probably better to initialize from
C++ and extern "C" instead. It's not a tremendous optimization though.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7138>

c11/threads: Remove Windows XP support

Enable and remove EMULATED_THREADS_USE_NATIVE_CV. Delete legacy code.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7138>

c11/threads: Remove Win32 null checks

Nonsensical to pass null. glibc doesn't check, and neither should we.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7138>

c11/threads: Fix Win32 timed functions

mtx_timedlock and cnd_timedwait now use relative milliseconds.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7138>

frontends/va/postproc: Convert destination when deinterlacing

When the VAAPI deinterlacing filter is chained with other VAAPI
post-processing filters, the image might get deinterlaced multiple
times, as the filters after the deinterlacing filter might still see an
interlaced buffer.

Signed-off-by: Thong Thai <thong.thai@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6803>

frontends/va/postproc: Use the actual image height when blitting

Updates the height of the blitting parameter to use the actual image
height instead of the buffer height, otherwise when scaling, garbage
lines are shown in the output.

Signed-off-by: Thong Thai <thong.thai@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6803>

intel/disasm: Don't rely on FALLTHROUGHTs to print unsupported SFID

The code works but is a bit fragile if we ever add a case that has a
less strict requirement (a smaller gen) than the case above.  To avoid
having to reason about this, refactor code to use a variable to
indicate whether the SFID is supported or not.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7742>

panfrost: Fix stride calculation for Z32_S8X24/X32_S8X24 formats

Z32_S8X24 variants are actually stored in 2 planes (one per component),
we have to adjust the bytes_per_pixel value accordingly.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Calculate the row stride at resource creation time

So we don't have to calculate it again when we create a texture or
framebuffer descriptor.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Fix panfrost_needs_explicit_stride() for block-based formats

The expected stride calculation does not take the block width into
account, thus creating an expected line stride that's bigger than
required. Fix that by dividing the width by the block width.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Expose panfrost_block_dim()

So we can use it from pan_resource.c to retrieve the tile size based on
a modifier.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Enable MSAA on bifrost when deqp debug option is set

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Unconditionally align strides on 64 bytes for linear resources

If we don't do that we end up with DATA_INVALID faults when accessing
3D textures on Bifrost.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Set the layer stride

Needed for 3D textures.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Add two helpers to calculate the surface pointer and strides

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Clarify bit 2:28 meaning in the Midgard texture descriptor

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Add a minus(1) modifier to the Levels field

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Increase blit shader BO size on Bifrost

Otherwise we hit 'offset + program->compiled.size < total_size' assert.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

pan/bi: LOD is a 8.8 fixed point

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

pan/bi: Always emit a LOD/CUBE word for FETCH instructions

There's no flag/mode to reflect when a LOD is zero on FETCH instructions,
we have to emit the LOD/CUBE word unconditionally.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

pan/bi: Only update LOD mode on TEX operations

If we don't add this check we clobber fetch mode when a 0 LOD is
specified on a txf instruction.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Set sample_count when packing bifrost texture descriptors

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Set depth for 3D textures on Bifrost

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Fix decoding of texture payloads

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Get rid of the Sample Count enum

Sample count just needs a log2 modifier. While at it, rename the
"Multisample count" field "Sample count" to be consistent with
other descriptors.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Stop forcing depth to nr_samples

Those are two different things, and the fact that Midgard use the same
offset in the texture descriptor for both things is just an
implementation detail.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Fix RAW8/16/32 component replication

Fixes dEQP-GLES3.functional.fbo.msaa.4_samples.r32f

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

panfrost: Account for sample count in tib offsets

I don't know if we have tests for MRT + MSAA but that would hit this.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

nir: Fix LOD source type for txf_ms instructions

txf_ms takes an integer LOD, not a float.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7653>

aco/ngg: fix division-by-zero in assertion

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7576>

aco: fix GS with no outputs

With NGG, ngg_gs_known_vtxcnt[0] would be false and ngg_gs_finale() would
assert.

With legacy GS, I don't know why the assertion was there.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7576>

radv/llvm,aco/ngg: fix large shift exponent in ngg_gs_vertex_lds_addr

When vertices_out=0, we will try to shift 1u by UINT32_MAX.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7576>

pan/bi: Implement shader-db stats

v2: Drop register tracking since it was wrong, and meaningful accounting
is tricky for Bifrost (which wants round robin RA for at least some
registers)... we'll cross that bridge when we get there, possibly
preferring a "max liveness" estimate to the raw RA output.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/bi: Ensure TEXC src0 is not marked SSA

Prevents attempts to spill tied TEXC src/dest, resulting in a crash,
seen in shaders/tesseract/229.shader_test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/bi: Emit a combine even if we only pass one staging reg to TEXC

We need that to account for potential swizzling on the source reg.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/bi: Fix off-by-one in RA

Could result in trying to allocate R64 which is clearly wrong.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/bi: Fix varying writemask handling

Allows shaders/supertuxkart/1.shader_test and
shaders/unity/24-Tree.shader_test to compile.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/bi: Implement sampler1D

Enough to get shaders/humus-celshading/1.shader_test compiling, no idea
if it actually works but it looks reasonable.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

pan/mdg: Fix shader-db counter

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

panfrost: Enable indirect uniform indexing

It works fine, and without it, the lowering bloats GLES2 shaders (by a
factor of 20x for one shader on t-rex!)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

panfrost: Add PAN_GPU_ID debug option

I'd like to run shader-db locally without turning on the Bifrost
machine. I'm lazy, it's just for debug builds, ok? 😇

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7615>

nir/unsigned_upper_bound: decrement num_sources_left before recursing

Otherwise, search_phi_bcsel() will be called with a buf_size that is
slightly lower than it has to be.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>

nir/unsigned_upper_bound: fix buffer overflow in search_phi_bcsel

It should only recurse if there's enough space to add the phi sources.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>

aco: fix v_mul_hi_u32_u24 format

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 57c152af9ce ("aco: select v_mul_{hi}_u32_u24 for 24-bit multiplications")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3874
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7759>

Revert "util: Add helpers for various one-time-init patters"

This reverts commit bda4d6e0d01116db59a0a03b0c703a7af6e11949.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "nir: Use get_once() helper for one-time init's"

This reverts commit c9062df1d57df19a56288c1749d4b6d22d7c1418.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "freedreno/ir3: Use get_once() for one-time init"

This reverts commit b4ad27a986e1c6899cbf23355c3e9c6de345a323.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "gallium/hud: Use do_once for one-time init"

This reverts commit 2e81ec5e009e3fbeef3fe1a76f2dfee428b7c160.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "mesa/st: Use do_once for one-time init"

This reverts commit bcb2981e145f0c2a54a637e20b6c55eaf316c04f.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "util: Fix helgrind complaint about one-time init"

This reverts commit f8c7a43f33d4647c16c4892d56706a14e5d6bf17.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "mesa: Fix helgrind complaint about one-time init"

This reverts commit f7102ac376a23a394786085a2b0dffa94c13150c.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "gallium/trace: Fix helgrind complaint about one-time init"

This reverts commit 1aa055539f32fcb3c78d908d1635bb3a3d517bc2.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

Revert "tgsi: Fix helgrind complaint about one-time init"

This reverts commit d91fe7d1c65179e64a6ca294135ac1bad2b16fb9.

Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7760>

radv: always use 32-bit predication on compute queues

It seems that only gfx queue doesn't support it, except on GFX10.3
which supports all queues.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7732>

radv: use 32-bit predication for conditional rendering on GFX10.3+

It's now supported.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7732>

ac: add gpu_info::has_32bit_predication

32-bit predication is now supported with GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7732>

v3dv: Wayland WSI support

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7303>

intel/rt: Implement push constants as global memory reads

They're not really "push" anymore but that's because there is no such
thing as push constants in bindless shaders on Intel.  They should be
fast enough, though.  There is some room for debate here as to whether
we want to do the pull in NIR or push it into the back-end.  The
advantage of doing it in the back-end is that it'd be easier to use
MOV_INDIRECT for indirect push constant access rather than falling back
to a dataport message.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add support for hit attributes

For triangle geometry, the hit attributes are always two floats which
contain the barycentric coordinates of the hit.  For procedural
geometry, they're an arbitrary blob of data passed from the intersection
shader to the hit shaders.  In our implementation, we stash that data
right after the HW RayQuery in the ray stack.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add a helper to create the raygen trampoline shader

Unlike graphics and compute pipelines, Vulkan ray-tracing pipelines do
not have a single entrypoint.  Instead, the raygen shader is specified
as a one-element shader binding table in the vkCmdTraceRay call.  This
means that raygen shaders have to be bindless shaders just like any
other ray tracing shader.  To launch them, we have a tiny compute shader
that acts as a trampoline and sets up the hotzone and uses btd_spawn to
fire off the raygen shader.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add lowering for combined intersection/any-hit shaders

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add lowering for ray-walk intrinsics in any-hit shaders

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add support for shader buffer record memory

Most of the work for this is done for us by spirv_to_nir which gives us
a load_global from a memory address based on the shader_record_ptr
system values.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Implement the new ray-tracing system values

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Implement traceRay()

This is a little bit more work than executeCallable() because we also
have to set up the MemRay data structure which the ray traversal
hardware uses to keep its state.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/fs: Add and implement intel-specific ray-tracing intrinsics

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Implement support for shader call payloads

Both traceRay() and executeCallable() take a payload parameter which
gets passed from the caller to the callee and which the callee can write
to pass data back to the caller.  We implement these by passing a
pointer to the data structure in the callee to the caller as the second
QWord on its stack.  Coming out of spirv_to_nir, the incoming call
payloads get the nir_var_shader_call_data variable mode allowing us to
easily identify them.  Outgoing call payloads get assigned the
nir_var_shader_temp mode and will have been turned into function_temp by
nir_lower_global_vars_to_local.  All we have to do is crawl the shader
looking for references to the nir_var_shader_call_data variable and
rewrite those to use the passed in pointer.  nir_lower_explicit_io will
do the rest for us.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add a helper to create a trivial return shader

These are required for ray-tracing.  There are many cases where the
ray-tracing hardware may decide to execute some but not all of our
shaders.  In these cases, it needs a shader to execute at the end which
will pop the stack back to the shader which called traceRay().

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add a pass to lower shader call instructions

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add return instructions at the end of ray-tracing shaders

Each callable ray-tracing shader shader stage has to perform a return
operation at the end.  In the case of raygen shaders, it retires the
bindless thread because the raygen shader is always the root of the call
tree.  In the case of any-hit shaders, the default action is accep the
hit.  For callable, miss, and closest-hit shaders, it does a return
operation.  The assumption is that the calling shader has placed a
BINDLESS_SHADER_RECORD address for the return in the first QWord of the
callee's scratch space.  The return operation simply loads this value
and calls a btd_spawn intrinsic to jump to it.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add support for scratch in ray-tracing shaders

In ray-tracing shader stages, we have a real call stack and so we can't
use the normal scratch mechanism.  Instead, the invocation's stack lives
in a memory region of the RT scratch buffer that sits after the HW ray
stacks.  We handle this by asking nir_lower_io to lower local variables
to 64-bit global memory access.  Unlike nir_lower_io for 32-bit offset
scratch, when 64-bit global access is requested, nir_lower_io generates
an address calculation which starts from a load_scratch_base_ptr.  We
then lower this intrinsic to the appropriate address calculation in
brw_nir_lower_rt_intrinsics.

When a COMPUTE_WALKER command is sent to the hardware with the BTD Mode
bit set to true, the hardware generates a set of stack IDs, one for each
invocation.  These then get passed along from one shader invocation to
the next as we trace the ray.  We can use those stack IDs to figure out
which stack our invocation needs to access.  Because we may not be the
first shader in the stack, there's a per-stack offset that gets stored
in the "hotzone".

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add lowering functions for each ray-tracing stage

These will eventually contain per-stage lowering for various ray-tracing
things.  This is separate from brw_nir_lower_rt_intrinsics because, for
reasons that will become apparent later, brw_nir_lower_rt_intrinsics has
to be run very late in the compile process, right before brw_compile_bs.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>

intel/rt: Add a pass to lower the new ray-tracing intrinsics

The new intrinsics we added for doing address calculations are all
things we fetch from the RT_DISPATCH_GLOBALS struct.  We could emit an
RT_DISPATCH_GLOBALS load at every point we want it and trust NIR to CSE
it for us but it's easier to use intermediate intrinsics.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>