aco/ngg: Implement workgroup reduce / exclusive scan for NGG GS.

This function calculates two things at once:

1. The total number of vertices emitted by the threadgroup.
2. Exclusive scan of emitted vertex count accross the threadgroup.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Create LDS layout for NGG GS.

For NGG GS, we need to store the following in LDS:

1. The ESGS ring, similarly to legacy ESGS.
2. Emitted vertices from the GS threads.
3. Temporary space used by the workgroup scan.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Setup NGG GS.

Make it possible for ACO to recognize when to use HW NGG GS.
Also add a few notes about the various GS stages in the comments.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Allow NGG GS to create VS exports.

NGG GS need to use the same instructions to export vertex
attributes at the end.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Allow NGG GS to load per-vertex GS inputs.

They work the same way as in legacy GS, so we can reuse that.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Allow NGG GS to store ES outputs.

We can reuse the existing ES output code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Clean up and reorganize NGG VS/TES code.

Make the NGG VS/TES code easier to follow, give better names to
some functions and make ngg_nogs_early_prim_export a variable.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Make primitive export packing less prone to error.

Use lshl_or instead of lshl_add, which makes it more robust in
handling -1 and -2 indices which will now just become null
exports, which is what we want.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Refactor ngg_emit_prim_export in preparation for NGG GS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco/ngg: Refactor gs_alloc_req in preparation for NGG GS.

Previously, this function inferred the vertex and primitive counts
from the gs_tg_info shader argument, but in case of NGG GS, it will
need to be calculated in runtime.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Add wave-specific opcode for s_lshl and s_flbit.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Optimize thread_id_in_threadgroup when there is just one wave.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Use thread_id_in_threadgroup helper for ES outputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Extract thread_id_in_threadgroup to a separate function.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Extract lanecount_to_mask to a separate function.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Clarify missing export error message in assembler.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

aco: Add loop creation helpers.

Will be useful for NGG GS and probably testing. The helpers take care of
divergence but not creating correct phis.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

radv/aco: Use new GS lowering options for ACO with NGG GS.

This makes it easier for ACO to implement NGG GS:

1. No need to keep track of vertex and primitive counts.
2. No need to discard incomplete primitives.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

nir: Add ability to count primitives per stream.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

nir: Count vertices per stream.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

nir: Add ability to overwrite incomplete GS primitives.

After each end_primitive and at the end of the shader before emitting
set_vertex_and_primitive_count, we check if the primitive that is being
emitted has enough vertices or not, and we adjust the vertex and
primitive counters accordingly.

As a result, if the backend uses this option, the backend compiler
will not have to worry about discarding the unneeded vertices
and primitives.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

nir: Add ability to count emitted GS vertices per primitive.

Add an option to nir_lower_gs_intrinsics so that it can also track
the number of emitted vertices per primitive, not just the total
vertex count.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

nir: Add ability to count emitted GS primitives.

Add an option to nir_lower_gs_intrinsics which tells it to track
the number of emitted primitives, not just vertices. Additionally,
also make it per-stream.

Also rename the set_vertex_count intrinsic to
set_vertex_and_primitive_count.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>

radv,anv: use CLOCK_MONOTONIC_FAST when CLOCK_MONOTONIC_RAW is undefined

CLOCK_MONOTONIC_FAST is a similar clock from FreeBSD.

Acked-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6995>

util/os_misc: os_get_available_system_memory() for FreeBSD

Based on the OpenBSD variant.
The only difference between those two system is the sysctl mib.

Signed-off-by: Emmanuel Vadot <manu@FreeBSD.org>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6995>

radv: use the same NIR compiler options for both compiler backends

No changes, they are already similar.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6938>

radv/llvm: do not lower sub

To match ACO.

Totals from 268 (0.20% of 136420) affected shaders:
CodeSize: 1214060 -> 1214096 (+0.00%); split: -0.05%, +0.06%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6938>

lima/parser: Fix varyings decoding in RSW

Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Signed-off-by: Andreas Baierl <ichgeh@imkreisrum.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7044>

aco: dump the program if the disassembler failed

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6979>

clover: don't call validate spirv at all if not needed

Suggested on irc seems like a good plan.

Reviewed-by: Pierre Moreau <dev@pmoreau.org>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7051>

util/disk_cache: remove unused function param

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7059>

swr/rasterizer: Remove BuilderGfxMem member mpTrackMemAccessFuncTy.

mpTrackMemAccessFuncTy is not used anywhere.

Fix defect reported by Coverity Scan.

Uninitialized pointer field (UNINIT_CTOR)
uninit_member: Non-static class member mpTrackMemAccessFuncTy is not
initialized in this constructor nor in any functions that it calls.

Suggested-by: Jan Zielinski <jan.zielinski@intel.com>
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Jan Zielinski <jan.zielinski@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6930>

meson: Use more portable compiler option -std.

GCC and Clang support --std and -std options but Intel C++
Compiler only supports -std.

icpc: command line warning #10159: invalid argument for option '--std'

Fixes: 8a05d6ffc65d ("driconf: Make the driver's declarations be structs instead of XML.")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7020>

turnip: Add support for GetSwapchainGrallocUsage2ANDROID().

This is lifted straight from anv, which seems like a reasonable way to go.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7015>

turnip: Detect Qualcomm gralloc and its UBWC flag on gralloc surfaces.

And document where to find information on qcom gralloc's private handle
layout.  I chose not to #include the gralloc_priv because it seems that
there's not much we need yet, and I'm hoping we can avoid the build-time
dependency on the specific platform.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7015>

turnip/kgsl: Add support for importing dma-bufs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7015>

turnip/kgsl: Fix last minute breakage of the build.

Need to land KGSL in CI!

Fixes: 8163c818e32e ("turnip: implement timestamp fences/semaphores for kgsl backend")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7015>

Revert "anv: Add driconf option to disable compression for 16bpp format"

This reverts commit bcfec61d1ee2675b8aaaf3adec8109ce6fa6c003.

The previous patch fixed the underlying issue that the above commit was
actually working around. It turns out that the previously observed
performance regression was due to invalid aux-map entries for
multi-layer HiZ+CCS buffers.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7046>

anv: Enable multi-layer aux-map init for HIZ+CCS

Fixes rendering corruption in the shadowmappingcascade Sascha Willems
Vulkan demo. To see the corruption, I adjusted the demo options as
follows:

 1. Enable "Display depth map"
 2. Set "Split lambda" to 0.100
 3. Make "Cascade" non-zero.

Fixes: 80ffbe915fe ("anv: Add support for HiZ+CCS")
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7046>

freedreno/ir3: Don't leave holes the UBO upload plan.

Shaders may not use a particular region of a UBO in a given shader (think
UBOs shared between stages, or between shaders), and by just always
extending the existing range for a given UBO, we'd waste bandwidth
uploading it, and also waste our precious const space in storing the
unused data.

Instead, only upload exactly the ranges we can use, and merge ranges when
they're neighbors.  We may end up with more upload packets, but the
bandwidth savings is surely going to be worth it (and if find we want a
distance threshold for merging with nearby uploads, that would be easy to
add).

total instructions in shared programs: 9266114 -> 9255092 (-0.12%)
total full in shared programs: 343162 -> 341709 (-0.42%)
total constlen in shared programs: 1454368 -> 1275236 (-12.32%)
total cat6 in shared programs: 93073 -> 82589 (-11.26%)
total (ss) in shared programs: 212402 -> 206404 (-2.82%)
total (sy) in shared programs: 122905 -> 114007 (-7.24%)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7036>

freedreno/ir3: Clean up the UBO upload plan setup.

No more start > end for signaling that the slot isn't used, no more funny
setup of num_enabled.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7036>

ci/softpipe: Add another flaky GS test to the skips list.

We've seen the new GS flake once or twice a week for a long time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7070>

util/hash_table: add function for reserving size in a hash table

rehashing a populated hash table is very expensive, so for the case where
the maximum/likely table size is already known, this function allows for
pre-sizing the table to avoid ever needing a rehash

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7037>

nir/opt_deref: Fix the vector bitcast optimization

It assumes the parent is a vector or scalar so we need to fail if it
isn't.

Fixes: 9190f82d573c "nir/opt_deref: Add an optimization for bitcasts"
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7064>

clover: Stop leaking NIR shaders

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7068>

util/xmlconfig: eliminate memory leak

It fixes coverity issue: CID 1467703: (RESOURCE_LEAK):
`Variable "cp" going out of scope leaks the storage it points to.`

Fixes: 23c3eb1fe16 ("driconf: Delete disjoint range support")
Reviewed-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Andrii Simiklit <andrii.simiklit@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7021>

radv/llvm: remove dead code for 64-bit GS inputs

64-bit IO are lowered with NIR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7008>

radv/llvm: do not lower nir_op_fsat

To match ACO.

fossilds-db (Navi10):
Totals from 20869 (15.30% of 136420) affected shaders:
SGPRs: 1851128 -> 1851920 (+0.04%); split: -0.41%, +0.46%
VGPRs: 1607360 -> 1608212 (+0.05%); split: -0.20%, +0.25%
SpillSGPRs: 267331 -> 261350 (-2.24%); split: -3.67%, +1.43%
CodeSize: 155460104 -> 155303508 (-0.10%); split: -0.21%, +0.11%
MaxWaves: 179156 -> 178928 (-0.13%); split: +0.48%, -0.60%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6932>

ac/nir: implement nir_op_fsat

With fmed3 if available, otherwise fallback to fmin/fmax.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6932>

ci: Disable pm_runtime and max clocks in LAVA jobs

To get more consistent performance and results, use the performance
devfreq governor and disable PM runtime.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7011>

ci: Test Panfrost on Khadas VIM3 boards

For testing Panfrost on Bifrost GPUs, add a job for dEQP GLES2 testing.

Right now almost all tests are skipped, but as we make progress and
things show stable, we'll be running more and more tests.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7011>

ci: Update kernel for LAVA

Update to v5.9-rc5-based drm-misc-for-next, so we can run jobs on
machines with Bifrost GPUs.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7011>

freedreno/a6xx: Support PIPE_FORMAT_R8_G8B8_420_UNORM for texturing

This makes freedreno advertise support for
PIPE_FORMAT_R8_G8B8_420_UNORM on a6xx, which enables lowering NV12 to
this format.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6693>

freedreno/a6xx: Generalize pointers in struct fd6_pipe_sampler_view

Add resource pointers ptr1 and ptr2 and offsets offset1 and offset2,
and just emit relocs if the pointers are non-NULL.  This lets us move
a little more logic to the CSO building.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6693>

st/mesa: Add NV12 lowering to PIPE_FORMAT_R8_G8B8_420_UNORM

Some GPUs can sample biplanar formats like NV12 natively, returning
the YUV values. Add a lowering type that uses that for sampling and
relies on existing colorspace conversions.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6693>

util/formats: Add PIPE_FORMAT_R8_G8B8_420_UNORM

This is a planar, subsampled format. It's basically NV12, but without
colorspace conversion.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6693>

iris: Use the data cache for indirect UBO pulls

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

anv: Use the data cache for indirect UBO pulls on Gen8+

On Gen7, the data cache is pretty terrible so we'd rather avoid it
there.  On Gen8+, it should be fine and is less likely to conflict with
texturing so we should get less cache thrashing there.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

anv: Plumb the device into *bits_for_access_flags

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

anv: Use format_for_descriptor_type for descriptor buffers

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

anv: Add a device parameter to format_for_descriptor_type

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

intel/fs: Add an option to use dataport messages for UBOs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

intel/fs: Add an alignment to VARYING_PULL_CONSTANT_LOAD_LOGICAL

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

i965: Take an isl_format in emit_buffer_surface_state

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>

intel/dev: fix 32bit build issue

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7049>

intel/nir: Lower load_global_constant in lower_mem_access_bit_sizes

It's identical to nir_intrinsic_load_global except that it works on data
that's guaranteed to be constant throughout the shader invocation.

Fixes: ff2f44d86585 "intel/fs: Implement nir_intrinsic_load_global_constant"
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6872>

intel/nir: Don't try to emit vector load_scratch instructions

In 53bfcdeecf4c9, we added load/store_scratch instructions which deviate
a little bit from most memory load/store instructions in that we can't
use the normal untyped read/write instructions which can read and write
up to a vec4 at a time.  Instead, we have to use the DWORD scattered
read/write instructions which are scalar.  To handle this, we added code
to brw_nir_lower_mem_access_bit_sizes to cause them to be scalarized.
However, one case was missing: the load-as-larger-vector case.  In this
case, we take small bit-sized constant-offset loads replace it with a
32-bit load and shuffle the result around as needed.

For scratch, this case is much trickier to get right because it often
emits vec2 or wider which we would then have to lower again.  We did
this for other load and store ops because, for lower bit-sizes we have
to scalarize thanks to the byte scattered read/write instructions being
scalar.  However, for scratch we're not losing as much because we can't
vectorize 32-bit loads and stores either.  It's easier to just disallow
it whenever we have to scalarize.

Fixes: 53bfcdeecf4c9 "intel/fs: Implement the new load/store_scratch..."
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6872>

ci: enable piglit testing of clover/llvmpipe.

This adds support for building clover/llvmpipe and running the
piglit CL tests on it.

It uses the gl testing container, and add builds the libclc
spirv libraries as part of that which requires the llvm spirv
translator in the build container.

It also builds the llvm spirv translator as part of the build
root and creates a mesa build that builds clover for testing
against it. It uses llvm 10 as the baseline.

This drops bswap as it has an oob memory access with llvmpipe
which cause flaky test results. phatk also seems flaky

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6901>

CI: Don't run pixmark-piano twice on radeonsi

For some reason, the radeonsi Tracie definition had the piano traces
listed twice.

Noted by @airlied

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7052>

spirv: Move nir_lower_libclc to src/compiler/spirv

This puts it in a shared place where everyone can get at it.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

clover: Use core libclc loader

v2 (Jason Ekstrand):
 - Use the newly added nir_can_find_libclc() helper

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

spirv: Add a shared libclc loader

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

meson: generalize libclc usage

So that it's not tied directly to clover.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

clover/meson: use dep.get_variable instead of deprecated get_pkgconfig_variable

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

util/xxd.py: Add an option for binary files

If -b is specified, we don't add a null to the end of the char array.
If -b is not specified, we assert that there are no nulls in the middle.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

util: Make xxd.py output char array instead of string

This keeps us from hitting the 65k string limit on MSVC

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

util: Move xxd.py to util

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

glsl/xxd.py: fix imports

sys and string are unused, os is needed but not imported

fixes: 412472da5cb30b603e218b34893936cc70039ded
       ("glsl: Add utility to convert text files to C strings")

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7034>

gallium/pipe: Add a GALLIUM_PIPE_SEARCH_DIR override env var

This can be useful if you rsync an install between two machines and the
paths don't perfectly match up.  OpenGL drivers already work fine but
anything which uses pipe-loader has a compile-time path.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7047>

iris: Fill out compute caps and enable clover support

This commit enables clover support for iris.  It is intended as a
compiler developer tool and not as a new OpenCL implementation from
Intel.  If you want competent OpenCL, we have a different open-source
driver for that built on our LLVM-based IGC compiler stack.  However,
using clover with iris is becoming increasingly useful as a compiler
development tool and I'm getting tired of carrying the patches in a
private branch.

By default, clover will not initialize on iris.  To enable clover, set
the IRIS_ENABLE_CLOVER environment variable to "1" or "true".  As we've
done with the semi-sketchy platform support in ANV, it dumps a very loud
WARNING to stderr when enabled.  Use at your own risk.

NOTE: To anyone intending to benchmark this, the performance is going to
be terrible and that is expected.  This is in no way representative of
the Intel/NIR compiler stack.  As it currently stands, clover passes
-O0 to clang when compiling OpenCL C to make SPIRV-LLVM-Transator work.
When compiling the SPIR-V, clover currently doesn't run any NIR
optimizations before it lowers memory access so any NIR optimizations
iris attempts to do are severely hampered.  One day, clover will get a
NIR optimization loop or the ability to hand things off to the driver
per-lowering but today is not that day.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7047>

iris: Add support for load_work_dim as a system value

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7047>

iris: Handle runtime-specified local memory size

The value specified in pipe_compute_state is in addition to the implicit
value computed by NIR.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7047>

iris: Add pipe-loader support

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7047>

broadcom/compiler: add V3D_DEBUG_RA option

To ask to debug a registr allocation failure
(V3D_DEBUG_REGISTER_ALLOCATION seemed too long to me).

When a fallback register allocation algorithm was added, if the
register allocation fails, it only dumpg the current vir with the
register pressure info with the failed fallback. But if we want do
debug the problem, we would be interested on both.

Additionally, it was strange that we got the full vir dump with the
failure even if no debug option was set.

Additionally we add shaderdb like stats for those failures, to make
easier to compare one and the other.

v2: keep a small warning message in case both register allocation
    algorithms fails (Neil)

Reviewed-by: Neil Roberts <nroberts@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6999>

broadcom/common: increase V3D_MAX_TEXTURE_SAMPLERS, add specific OpenGL limit

This is needed due Vulkan because by spec (31.1. Limit Requirements)
the minimum value for the following limits are the following ones:
  maxPerStageDescriptorSampledImages 16
  maxPerStageDescriptorStorageImages  4
  maxPerStageDescriptorInputAttachments 4

And we are using v3d textures for all of them, so current limit would
not be enough for some cases.

Note that as the current comment explains there is not exactly a HW
limit for it, so we could bump to 32 for example, but let's just be
conservative and ask the minimum required.

It is worth to note that we needed to maintain the same value for the
OpenGL case, as it gets a register allocation failure on some GL
cases. We tried to fix that with small changes on the nir scheduler,
but we found that it would require some non-trivial effort to get it
done (that eventually we would need to).

Fixes tests like:
dEQP-VK.binding_model.descriptorset_random.sets16.constant.ubolimitlow.sbolimitlow.imglimitlow.noiub.uab.comp.noia.0

v2: keep the previous limit for Opengl (Eric)

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6999>

aco/isel: Fix out-of-bounds write in visit_load_input

Shaders may read out components past the attributes provided by the
application, so the read mask can indicate a larger component count than
were actually reserved in the array.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

radv: Respect alignment requirements in descriptor set layouts

Previously, ycbcr samplers were tightly packed with 4-byte alignment,
but the structure requires 8-byte alignment. These samplers are now padded
to 8-byte boundaries instead.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

radv: Clean up CreateDescriptorSetLayout

"max_bindings + 1" was repeatedly used throughout this function,
so talking about the binding *count* is more natural here.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

radv: Fix unaligned memory access when writing specialization map entries

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

radv: Avoid calling memcpy with null pointers

Vulkan allows for these input pointers to be null when the respective
object count is zero. Calling memcpy with null pointers is undefined,
so they are guarded with a check for the legit use pattern now.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

nir: Fix unaligned pointer access

This was observed with the intel vulkan driver when running
dEQP-VK.spirv_assembly.instruction.compute.float32.comparison_1.modfstruct
with ubsan enabled.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

nir: Fix undefined behavior due to signed integer multiplication overflows

Notably this happened when applying constant folding on the intermediate
computations generated from nir_lower_idiv.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6728>

ci: move to using clang 10 for meson + clover

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7017>

st/mesa: pass inlinable uniforms to drivers if they requested it

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6955>

gallium: add pipe_context::set_inlinable_constants

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6955>

nir: add new helper passes that lower uniforms to literals

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6955>

util: add _mesa_set_create_u32_keys where keys are not pointers

the only limitation is that key=0 is not allowed

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6955>

panfrost: Fix fixed-function blend on Mali v6

Mali v6 (G72) doesn't support constants in blend equations, let's use
a shader in that case.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6980>

panfrost: XML-ify the blend descriptors

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6980>

panfrost: Rework the render target layout to use overlapping structs

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6980>