anv/allocator: Use list->u64 in free_list_push

Since we always modify this structure with atomics on the u64, it seems
better to use the u64 here too.  I have no idea if this fixes a bug.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11857>

llvmpipe: Handle NULL views in llvmpipe_cleanup_stage_sampling

This is the only barrier to lavapipe fully working in RenderDoc.

Fixes: 21864bdaaef6 ("llvmpipe: unmap display target of shader image/sampler")

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11856>

freedreno+turnip: Add a6xx gen4 support

This adds support for a660 and a635.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno+turnip: Get device name from device-info table

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno+turnip: Add has_8bpp_ubwc

Newer a6xx devices seem to drop 8b/pixel UBWC support.

The turnip part was adapted from Jonathans patch on !10892

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno+turnip: Add has_cp_reg_write

Newer a6xx devices drop this packet from the sqe firmware, and use
direct (pkt4) register writes instead for the few cases that previously
used CP_REG_WRITE.

The turnip part was adapted from Jonathans patch on !10892

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Get has_sample_locations from fd_dev_info

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Get has_tex_filter_cubic from fd_dev_info

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Get indirect_draw_wfm_quirk from fd_dev_info

At some point we might want to change this to minimum fw version, but
for now it can be a bool.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Get storage_16bit from fd_dev_info

Removing more gpu_id checks that will become bogus as we add more a6xx.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Drop unused vshs_workgroup param

Unused since d968995c67f5e3a48e49015184f9ea1f143251ec, and this gets rid
of one more gpu_id check.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno/ir3: Get reg_size_vec4 from fd_dev_info

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno/ir3: Get tess_use_shared from fd_dev_info

A step towards getting rid of checks for gpu_id sprinkled around.
Checking major generation is ok, but checking for == or >= a specific
gpu_id is going to start getting messy as we add more a6xx.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

turnip: Convert fd_dev_info to const pointer

Split out from earlier patch to reduce churn.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno: Convert fd_dev_info to const pointer

Split out from previous patch to reduce churn.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno: Generate device-info tables at build time

This way we can make the tables const.  At the same time, for a6xx, this
introduces a "sub-generation template" to reduce the copy/paste for
parameters which are keyed to the sub-generation.  It also explicitly
lists every supported GPU, to get rid of duplicate lists of supported
gpus between the device-info and drivers.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno: Rename *_dev_info

Everywhere else symbols/types/etc are shortend to "fd_*", so lets do the
same here for consistency.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno/common: unhardcode CCU color cache offset

Replace it with a calculation which works for all current GPUs.

Duplicated the calculation in both drivers because freedreno_dev_info isn't
meant for derived parameters (and drivers might want to just calculate on
the fly instead).

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

freedreno/a6xx: larger gmem_page_align from tile align instead of gpu id

The larger page alignment is directly related to the 96 tile alignment,
so check for that instead of a specific gpu id.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

tu: remove workaround for conditional rendering + hw binning

- It hurts users with newer firmware who don't need the workaround
- Kernel now rejects older firmware due to security issues, so this will
  prevent users from using older firmware anyway.
- Only whitelisting 650 enables the workaround by default for any new GPUs

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11790>

iris: Use simple_mtx in the bufmgr.

This avoids having to call out through the PLT just to lock/unlock.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11858>

panfrost: Initialise the blend equation in create_blend_state

Valgrind still complains about uninitialised values, but tests don't
flake anymore.

Fixes flakes in dEQP-GLES3.functional.fragment_out.*

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11862>

crocus: use simple_mtx in the bufmgr

anything you can do I can do better^W^Wadapt for crocus

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11859>

radv, aco, ac/nir: Tweak position export scheduling for NGG culling.

The result is about +5-ish fps in Doom Eternal.

It turns out that the location of position exports matters more
than we thought, and it's actually better to keep them at the bottom
for culling shaders rather than schedule it up to the top.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Reuse uniforms from top part of culling shaders.

Uniforms have the same value in all invocations, therefore they
can safely be reused by invocations even after repacking.
This saves several instructions from culling shaders, mainly
UBO loads and such.

We exclude uniform floats, because those would harm the VGPR
usage of the shaders too much.

Fossil DB results on Sienna Cichlid (with NGG culling on):

Totals from 55379 (43.05% of 128647) affected shaders:
VGPRs: 1926472 -> 1925360 (-0.06%); split: -0.07%, +0.01%
SpillSGPRs: 139 -> 330 (+137.41%)
CodeSize: 159472988 -> 157462856 (-1.26%); split: -1.27%, +0.00%
MaxWaves: 1571492 -> 1571412 (-0.01%)
Instrs: 30665685 -> 30302076 (-1.19%); split: -1.21%, +0.02%
Latency: 127385148 -> 126723891 (-0.52%); split: -0.55%, +0.03%
InvThroughput: 21096298 -> 20773069 (-1.53%); split: -1.53%, +0.00%
VClause: 514792 -> 511231 (-0.69%); split: -0.83%, +0.13%
SClause: 713959 -> 679556 (-4.82%); split: -4.84%, +0.02%
Copies: 2975106 -> 2828185 (-4.94%); split: -5.39%, +0.45%
Branches: 1201921 -> 1152766 (-4.09%)
PreSGPRs: 1753786 -> 1892848 (+7.93%); split: -0.00%, +7.93%
PreVGPRs: 1590522 -> 1583574 (-0.44%); split: -0.44%, +0.00%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Analyze culling shaders to remember which inputs are used when.

These will be useful for some optimizations.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Reuse the repacked output positions of culling shaders.

The position outputs are stored into LDS and reloaded after
repacking, therefore the repacked position values can be
reused in the bottom part of the shader.

Fossil DB results on Sienna Cichlid (with NGG culling on):

Totals from 9016 (7.01% of 128647) affected shaders:
VGPRs: 372472 -> 347560 (-6.69%); split: -6.82%, +0.13%
SpillSGPRs: 437 -> 87 (-80.09%)
CodeSize: 32359340 -> 30441692 (-5.93%); split: -5.93%, +0.00%
MaxWaves: 222030 -> 238970 (+7.63%); split: +7.83%, -0.20%
Instrs: 6207833 -> 5834149 (-6.02%); split: -6.02%, +0.00%
Latency: 27626263 -> 27890632 (+0.96%); split: -5.34%, +6.29%
InvThroughput: 4792958 -> 4361336 (-9.01%); split: -9.01%, +0.00%
VClause: 144385 -> 139586 (-3.32%); split: -9.29%, +5.97%
SClause: 141350 -> 129875 (-8.12%); split: -8.57%, +0.45%
Copies: 580017 -> 568916 (-1.91%); split: -3.60%, +1.68%
Branches: 209067 -> 209154 (+0.04%); split: -0.24%, +0.28%
PreSGPRs: 281320 -> 277814 (-1.25%)
PreVGPRs: 290040 -> 273861 (-5.58%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

radv: Run algebraic optimizations before NGG lowering.

This makes culling shaders more efficient because they split the
shader in two parts. It is better to optimize before this split
happens.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

radv: Support NGG culling with new perftest environment variable.

Currently we don't enable it on any chip by default, but
we plan to enable it soon on GFX10.3 when we are comfortable
with its performance.

RADV_PERFTEST=nggc environment variable enables it on GFX10+ GPUs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

aco: Implement NGG culling related intrinsics.

These are very straightforward as they just copy data from
the newly added shader arguments.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

radv: New shader args for NGG culling settings and viewport.

Add new shader arguments in RADV for:
- NGG culling settings
- Viewport transform

These will be used by NGG culling shaders.

Additionally, some tweaks are made to some config registers
in order to make culling shaders more efficient.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

radv: Expose radv_get_viewport_xform in radv_private.h

We need to emit viewport transform information for culling shaders.
This is used for small primitive culling.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Implement NGG deferred attribute culling in NIR.

Culling is traditionally done by the rasterizer, but that
can be a bottleneck when an app creates a large number
of primitives. Eg. a lot of tiny triangles reduce the
rasterziation efficiency.

NGG makes it possible for the shader to check primitives
and delete those that it can prove are not needed.

After this is done, we have to repack the surviving invocations
so they remain compact. This also saves bandwidth, because
some memory loads are only executed by those vertices that
survived the culling.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Use a ballot that matches the wave size during NGG lowering.

This generates slightly more efficient code in Wave32 mode.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ac/nir: Add a NIR port of ac_llvm_cull.

The algorithms were originally implemented by Marek Olšák,
hence the copyright to AMD.

This commit just ports the LLVM based implementation to NIR,
using the new intrinsics added earlier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

nir: Add AMD specific intrinsics for NGG shader based culling.

The new intrinsics fall into the following categories:

1. New viewport intrinsics:
For missing components that we need.
RADV will emit new SGPR arguments which will contain the
viewport information for culling shaders. These are used to
compute the screen space coordinates for small primitive culling.

2. load_cull_xxx:
Load the culling settings in runtime.
These will be a new SGPR argument in RADV.

3. overwrite_xxx:
These are needed because system values such as vertex and
instance ID are not writeable, but we need to change them
after repacking shader invocations of VS and TES.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>

ci: Enable testing of i915g in the the debian -Werror release build.

Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11852>

i915g: Fix release build compiler warnings.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11852>

freedreno: Optimize duplicate obj-obj ring relocs.

No need to include the same BO multiple times in the long-lived ringbuffer
object's list of relocs to be added to the submit.

Improves non-TC drawoverhead -test 9 (8 tex updates) throughput by 1.4901%
+/- 0.8705% (n=20)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>

freedreno/a6xx: Allocate just enough memory for SO state, only if we do SO.

Continuing to improve our suballocation packing.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>

freedrneo/a6xx: Reduce the size of the long-lived texture stateobj.

It's just a few commands to upload the sampler/texconst data.  Improves
the efficiency of suballocation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>

freedreno/a6xx: Reduce the size of the config stateobj allocation.

Improves the efficiency of suballocation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>

freedreno: Suballocate our long-lived ring objects.

On drawoverhead -test 9 (8 texture changes), this saves us 172kb of
memory.  That's only ~1% of the GEM memory while the test is running, but
more importantly it saves us 29% of the gem BO allocations.

non-TC drawoverhead -test 9 (8 texture change) throughput 0.449019% +/-
0.336296% (n=100), but this gets better as we get better suballocation
density.

Note that this means that all fd_ringbuffer_new_object calls can now
return data aligned to 64 bytes, instead of 4k.  We may find that we need
to increase it if some of our objects (tex consts, sampler consts, etc.)
require more alignment than that.  But, this may help non-drawoverhead
perf if any of our RB objects have a cache in front of them (indirect
consts?) and we don't have most of our data in the same cache set any
more.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>

lima: Take offset in account when checking BO size

BO resources imported from a handle may have an offset provided, which
reduces the available size within the BO. Take this in account when
checking that the size is sufficient in lima.

Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11076>

driconf: Add more workarounds for Teardown

Enable radeonsi_no_infinite_interp for Teardown to fix hangs.

Based on comments from #3714.

Tested-by: Joshua Ashton <joshua@froggi.es>
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11814>

anv: Implement VK_EXT_acquire_drm_display

Signed-off-by: Simon Zeni <simon@bl4ckb0ne.ca>
Reviewed-by: Simon Ser <contact@emersion.fr>
Tested-by: Simon Ser <contact@emersion.fr>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11735>

aco: Remove deprecated Operand constructors

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

aco: Clean up unneeded literal casts

These were only needed to select the appropriate Operand constructor before.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

aco: Remove use of deprecated Operand constructors

This migration was done with libclang-based automatic tooling, which
performed these replacements:
* Operand(uint8_t) -> Operand::c8
* Operand(uint16_t) -> Operand::c16
* Operand(uint32_t, false) -> Operand::c32
* Operand(uint32_t, bool) -> Operand::c32_or_c64
* Operand(uint64_t) -> Operand::c64
* Operand(0) -> Operand::zero(num_bytes)

Casts that were previously used for constructor selection have automatically
been removed (e.g. Operand((uint16_t)1) -> Operand::c16(1)).

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

aco: Remove use of deprecated Operand constructors in aco_builder.h

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

aco: Remove use of deprecated Operand constructors in test_to_hw_instr.cpp

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

aco: Replace Operand literal constructors with factory member functions

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>

ci: Make sure that we build the piglit dmabuf tests.

Force the option rather than relying on autodetection -- ARM runners were
apparently finding the necessary deps, but the x86 rootfs (radeonsi, iris)
and x86_test-gl container (i915g) were not.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11834>

zink: simplify modifier ifdefs

these are the only two defines referenced, so they can be defined to 0
for platforms that don't support modifiers in order to remove a ton of
ifdefs and make the code more readable

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11847>

freedreno/ir3: Add float immed "FLUT" support

We can encode a limited set of float immeds into cat2 instructions,
using hw's float lookup table (FLUT) feature.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/36
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8705>

freedreno/computerator: Add script to probe FLUT values

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8705>

turnip: Split tu6_emit_xs()

Emit all the state layout config (such as push-const CONSTLEN) first,
before emitting anything that depends on that state.  This fixes an
issue that was showing up when FLUT is enabled in ir3 (which results
in higher probability of not having any immediats lowered to push-
consts).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8705>

turnip: avoid some UB

Reduce a bit of extra noise that makes diffing cmdstream traces more
annoying.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8705>

glsl: Delete lower_texture_projection

This is only used by i965 and we've been getting it through
nir_lower_tex since forever.  Get rid of the GLSL IR pass.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11827>

ci: only trigger gallium_core_file_list jobs from dri and glx frontend changes

these are the only frontends which may be used by gallium drivers in ci,
so stop triggering all driver jobs when other frontends are changed since
those changes can never affect ci

<MrCooper> Not that simple unfortunately. E.g. the llvmpipe-piglit-cl job hits
           src/gallium/frontends/clover & possibly src/gallium/targets/opencl,
           many jobs hit src/gallium/{frontends,targets}/dri and probably
           src/gallium/targets/pipe-loader, lavapipe jobs hit src/gallium/{frontends,targets}/lavapipe.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11832>

ci: add vulkan files to lavapipe rules

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11833>

pan/bi: Create a nop clause when the shader starts with ATEST

Otherwise there would be no clause with the dependencies needed for
ATEST set, so the GPU would get stuck.

Not needed on v7, as there shader_wait_dependency in the RSD will wait
for the dependencies before the shader starts.

Explicitly create a NOP instruction, as it is assumed that clauses
have a non-zero count of instructions in various places.

Fixes GPU timeouts in many applications, such as SuperTuxKart and
GZDoom.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11842>

panfrost: Fix full_threads calculation on v6

Fixes: 8ba2f9f6985 ("panfrost: Create a blitter library to replace the existing preload helpers")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11842>

gbm/dri: Fix leaking bo memory on failure path

In gbm_dri_bo_create, when modifiers are requested but not supported, do
not return NULL immediately, but first go to cleanup section to free
already allocated buffer object.

Fixes: cb9ae4273d68 ("dri: add loader_dri_create_image helper")
Signed-off-by: Heinrich Fink <hfink@snap.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Simon Ser <contact@emersion.fr>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11844>

pps: Panfrost documentation

Add documentation for the Panfrost Perfetto datasource.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10215>

pps: Panfrost pps driver

Add the Panfrost pps driver.

v2: Human readable names for counter blocks and use `unreachable`.
v3: Use libpanfrost_perf to collect counter values.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10215>

mesa: clear shader_info::is_lowered in prog_to_nir

This needs to be resetted each time prog_to_nir is called because it
turns st_nir_assign_vs_in_locations into a no-op when set.

Fixes: 81d106d6eca ("radeonsi: lower IO intrinsics - complete rewrite of input/output scanning")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5001
Reviewed-by: Isaac Bosompem <mrisaacb@google.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11636>

v3dv: use defines for push constant offsets used by texel buffer copy shaders

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11843>

v3dv: allow batching texel buffer copies for 3D images

For these we only need to check that the depth extent we are
copying is the same across regions in the batch, since we use
that to specify the number of layers in the framebuffer used
for the copy.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11843>

v3dv: implement layered texel buffer copies using a geometry shader

Instead of specifying a separate framebuffer per layer which is expected
to be much slower.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11843>

v3dv: fix push constant range for texel buffer copy pipelines

As per get_texel_buffer_copy_fs(), we load 24 bytes of data.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11843>

zink: remove duplicate format-mapping on little-endian

Doing *both* of thse ends up rewriting the previous mapping. Since this
doesn't seem to have lead to issues, it seems like the new mapping works
just as well.

Fixes: a22a1c0324e ("zink: Fix VK_FORMAT_A8B8G8R8_SRGB_PACK32 mapping on big-endian")
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11417>

loader: Handle failure to load DRI driver library

I factored out the chunk of loader code that dlopen()s
libraries from the rest of the DRI driver loader function
in this commit:

  commit bc343154f816b637b200933e1c1a35db19f478ef
  Author: James Jones <jajones@nvidia.com>
  Date:   Thu Apr 22 23:17:08 2021 -0700

  loader: Factor out driver library loading code

However, I failed to adjust the DRI loader function that
now uses the new helper function to handle the case where
the requested DRI library is not found.

This change restores the prior behavior, and also ensures
loader_open_driver() consistently returns NULL in the
out_driver_handle parameter on failure.

Fixes: bc343154f816 ("loader: Factor out driver library loading code")
Signed-off-by: James Jones <jajones@nvidia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11807>

aco: fix extract_vector optimization

If the allocated_vec map contains a different RegType
for the elements, ensure that the size matches exactly.

Otherwise, it could happen that extracting a dword
element matched with a subdword element.

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11823>

aco: fix self-intersecting register swaps

Splitting self-intersecting register swaps into
3 sections was unnecessary and only worked because
the middle section was always empty for full dword
swaps.

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11823>

lavapipe: add the separate depth/stencil layout enable.

This was missing.

Fixes: ac07ad8414cd ("lavapipe: implement KHR_separate_depth_stencil_layouts")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11840>

panfrost: Fix format swizzles on G72

Careless typo. Cleaner fix suggested by Icecream95.

Reported-by: macc24
Tested-by: macc24
Fixes: 531d80de7c7 ("panfrost: Specialize blendable_formats for v6")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11837>

relnotes: add some missing zink/lavapipe updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11838>

features: mark off line rasterization for lavapipe

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11838>

zink: add a pipe_screen::resource_create_with_modifiers hook

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: add fallback for linear modifier use

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: don't pass modifier count to first image create

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: explicitly disallow using the modifier image create for non-linear images

the first call is just for the base image, the second call is the one that needs
modifiers

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: use VkImageDrmFormatModifierListCreateInfoEXT for creating from modifier array

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: add a pipe_screen::resource_get_param hook

this is used for querying image properties

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: store modifier aspect to resource

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: start storing modifiers to the base resource struct

need these for rebinds

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: populate modifier props onto screen object during init

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

zink: implement support for non-planar DRM modifiers

Planar DRM modifiers are a bit more tricky, so let's kick that ball down
the road. For now, this should help a bit.

Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10782>

mailmap: Add two more lines for Alyssa Rosenzweig

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11788>

Convert a few files to UTF-8

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11788>

mailmap: Update for Emma's new e-mail address

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11788>

pan/bi: Add a bundling heuristic

Pick instructions locally that are likely to minimize nop's globally.
Note the obvious greedy solution is ineffective. Instead the principle
is to choose instructions which will increase _future_ scheduler
freedom, on the assumption that means fewer nop's overall will be
needed.

There is no concern about register pressure and little concern about
message timing, since this is post-RA and message ordering was fixed.

total tuples in shared programs: 125304 -> 123770 (-1.22%)
tuples in affected programs: 88301 -> 86767 (-1.74%)
helped: 548
HURT: 198
helped stats (abs) min: 1.0 max: 31.0 x̄: 3.43 x̃: 2
helped stats (rel) min: 0.33% max: 16.67% x̄: 3.09% x̃: 2.44%
HURT stats (abs)   min: 1.0 max: 12.0 x̄: 1.73 x̃: 1
HURT stats (rel)   min: 0.27% max: 11.11% x̄: 2.82% x̃: 2.35%
95% mean confidence interval for tuples value: -2.39 -1.72
95% mean confidence interval for tuples %-change: -1.77% -1.27%
Tuples are helped.

total clauses in shared programs: 26059 -> 25655 (-1.55%)
clauses in affected programs: 10821 -> 10417 (-3.73%)
helped: 311
HURT: 42
helped stats (abs) min: 1.0 max: 6.0 x̄: 1.44 x̃: 1
helped stats (rel) min: 0.48% max: 16.00% x̄: 5.26% x̃: 5.00%
HURT stats (abs)   min: 1.0 max: 2.0 x̄: 1.05 x̃: 1
HURT stats (rel)   min: 2.13% max: 33.33% x̄: 7.49% x̃: 6.25%
95% mean confidence interval for clauses value: -1.26 -1.02
95% mean confidence interval for clauses %-change: -4.28% -3.21%
Clauses are helped.

total cycles in shared programs: 12154.79 -> 12114.83 (-0.33%)
cycles in affected programs: 1907.63 -> 1867.67 (-2.09%)
helped: 227
HURT: 99
helped stats (abs) min: 0.041665999999999315 max: 1.2083360000000027 x̄: 0.22 x̃: 0
helped stats (rel) min: 0.36% max: 20.00% x̄: 4.14% x̃: 3.85%
HURT stats (abs)   min: 0.041665999999999315 max: 0.5 x̄: 0.09 x̃: 0
HURT stats (rel)   min: 0.28% max: 9.09% x̄: 2.75% x̃: 2.63%
95% mean confidence interval for cycles value: -0.15 -0.09
95% mean confidence interval for cycles %-change: -2.51% -1.59%
Cycles are helped.

total arith in shared programs: 4658.13 -> 4603.42 (-1.17%)
arith in affected programs: 3449.83 -> 3395.12 (-1.59%)
helped: 509
HURT: 249
helped stats (abs) min: 0.041665999999999315 max: 1.2083360000000027 x̄: 0.14 x̃: 0
helped stats (rel) min: 0.36% max: 20.00% x̄: 3.56% x̃: 2.63%
HURT stats (abs)   min: 0.041665999999999315 max: 0.5 x̄: 0.07 x̃: 0
HURT stats (rel)   min: 0.28% max: 25.00% x̄: 3.32% x̃: 2.27%
95% mean confidence interval for arith value: -0.09 -0.06
95% mean confidence interval for arith %-change: -1.64% -0.97%
Arith are helped.

total quadwords in shared programs: 111394 -> 110114 (-1.15%)
quadwords in affected programs: 78074 -> 76794 (-1.64%)
helped: 503
HURT: 204
helped stats (abs) min: 1.0 max: 43.0 x̄: 3.33 x̃: 2
helped stats (rel) min: 0.41% max: 13.79% x̄: 3.02% x̃: 2.44%
HURT stats (abs)   min: 1.0 max: 26.0 x̄: 1.94 x̃: 1
HURT stats (rel)   min: 0.48% max: 7.55% x̄: 2.44% x̃: 2.22%
95% mean confidence interval for quadwords value: -2.16 -1.46
95% mean confidence interval for quadwords %-change: -1.67% -1.21%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Calculate dependency graph when bundling

Code is ported from Midgard, modified to be scalar, post-RA, and to put
the arrays on the worklist instead of the instruction to save memory.
This enables out-of-order scheduling.

total tuples in shared programs: 128691 -> 125304 (-2.63%)
tuples in affected programs: 114091 -> 110704 (-2.97%)
helped: 844
HURT: 377
helped stats (abs) min: 1.0 max: 150.0 x̄: 4.88 x̃: 3
helped stats (rel) min: 0.30% max: 26.42% x̄: 5.56% x̃: 4.35%
HURT stats (abs)   min: 1.0 max: 8.0 x̄: 1.94 x̃: 1
HURT stats (rel)   min: 0.20% max: 33.33% x̄: 6.84% x̃: 3.23%
95% mean confidence interval for tuples value: -3.16 -2.38
95% mean confidence interval for tuples %-change: -2.19% -1.27%
Tuples are helped.

total clauses in shared programs: 27579 -> 26059 (-5.51%)
clauses in affected programs: 20606 -> 19086 (-7.38%)
helped: 941
HURT: 39
helped stats (abs) min: 1.0 max: 21.0 x̄: 1.66 x̃: 1
helped stats (rel) min: 0.69% max: 44.44% x̄: 10.48% x̃: 9.09%
HURT stats (abs)   min: 1.0 max: 2.0 x̄: 1.15 x̃: 1
HURT stats (rel)   min: 1.89% max: 10.00% x̄: 4.73% x̃: 4.55%
95% mean confidence interval for clauses value: -1.63 -1.47
95% mean confidence interval for clauses %-change: -10.27% -9.48%
Clauses are helped.

total cycles in shared programs: 12262.54 -> 12154.79 (-0.88%)
cycles in affected programs: 2210.54 -> 2102.79 (-4.87%)
helped: 374
HURT: 56
helped stats (abs) min: 0.041665999999999315 max: 6.25 x̄: 0.30 x̃: 0
helped stats (rel) min: 0.42% max: 26.00% x̄: 6.90% x̃: 7.14%
HURT stats (abs)   min: 0.041665999999999315 max: 0.5833319999999986 x̄: 0.11 x̃: 0
HURT stats (rel)   min: 0.16% max: 100.00% x̄: 55.17% x̃: 50.00%
95% mean confidence interval for cycles value: -0.29 -0.21
95% mean confidence interval for cycles %-change: -1.37% 3.73%
Inconclusive result (%-change mean confidence interval includes 0).

total arith in shared programs: 4852.29 -> 4658.13 (-4.00%)
arith in affected programs: 4525.17 -> 4331 (-4.29%)
helped: 1112
HURT: 166
helped stats (abs) min: 0.041665999999999315 max: 6.25 x̄: 0.19 x̃: 0
helped stats (rel) min: 0.42% max: 33.33% x̄: 6.59% x̃: 5.36%
HURT stats (abs)   min: 0.041665999999999315 max: 0.5833319999999986 x̄: 0.07 x̃: 0
HURT stats (rel)   min: 0.16% max: 100.00% x̄: 25.05% x̃: 2.40%
95% mean confidence interval for arith value: -0.17 -0.14
95% mean confidence interval for arith %-change: -3.44% -1.51%
Arith are helped.

total quadwords in shared programs: 117141 -> 111394 (-4.91%)
quadwords in affected programs: 104390 -> 98643 (-5.51%)
helped: 1245
HURT: 76
helped stats (abs) min: 1.0 max: 69.0 x̄: 4.74 x̃: 4
helped stats (rel) min: 0.28% max: 35.00% x̄: 7.88% x̃: 6.45%
HURT stats (abs)   min: 1.0 max: 8.0 x̄: 2.01 x̃: 1
HURT stats (rel)   min: 0.20% max: 10.00% x̄: 3.52% x̃: 4.25%
95% mean confidence interval for quadwords value: -4.61 -4.09
95% mean confidence interval for quadwords %-change: -7.56% -6.88%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Handle 4-src instructions in scheduler

Spill to a move. This allows us to emit SHADDX and general CSEL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Add bi_before_tuple convenience method

For constructing instructions during scheduling.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Handle multiple destinations in scheduler

Needed for correct handling of pseudo ops like SHADDX and CUBEFACE

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Do helper termination analysis on clauses

Unlike the dependency analysis for the skip bits which is a function of
the data flow graph, the thread termination analysis is dependent on the
actual sequence of instructions. As such, it must be done after
scheduling to be correct in the presence of out-of-order scheduling.

Furthermore it's specified in terms of clauses, not instructions.
Reflecting this in our code gets a nice simplification. As a side effect
this puts extra td flags on subsequent clauses, which matches the DDK's
behaviour. (Maybe td is just a hint?)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Update ins->link after scheduling

Otherwise foreach_instr and friends will be subtly wrong. None of our
current passes rely on this but it's a footgun.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>

pan/bi: Make bi_foreach_instr_in_tuple safer

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10961>