radv: Refactor some nir_channels usage to use nir_channel.

cleanup, nir_channels wasn't needed as these were only accessing a
single channel.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

aco: Implement call scope.

Since we do no repacking yet, just use invocation.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Modify load_sbt_amd intrinsic to get the descriptor.

That way we can get the address to the entry, which is needed for
some nir builtins because extra data in the entry can be used as
shader input.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Add bvh node definitions to a header.

So that we can avoid some magic numbers in the pipeline creation.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Add optimized CPU BVH builds.

This trivial sorting helps speed up bad cases (like in CTS significantly).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Add more acceleration structure formats.

These are required ...

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Add accel struct build support for the object-to-world matrix.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Fix arrayOfPointers for instances in accel struct build.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

radv: Fix CPU AABB build.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>

meson: introduce option vmware-mks-stats controlling the instrumentations of gallium svga driver

The new boolean option controls the instrumentations of gallium svga driver for mksGuestStats.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Reviewed-by: Zack Rusin <zackr@vmware.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
(cherry picked from commit 57760b7fe4eeb80acc8d6cd8bf6ec609a11a11dc)
(cherry picked from commit 12aed00f08bd95afd605cab34c833e81a4957dbd)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12873>

svga: enable DRM mks-stats via hooking to the corresponding DRM ioctls

SVGA DRM stat calls were situated but did not actually register with the mks-stats
mechanism due to absence of corresponding ioctls. The employed new ioctls in vmwgfx
are DRM_VMW_MKSSTAT_ADD and DRM_VMW_MKSSTAT_REMOVE, subject to version check.

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
(cherry picked from commit be47c077cc927c27a8c36342b47697aa81719677)
(cherry picked from commit 0388afc67b830f6ab916d0839c33eb1d91d6353f)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12873>

build: fix nine compilation with only zink enabled as a gallium driver

fixes #5360

Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12943>

zink: fix regex syntax from previous ci commit

argh

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12948>

zink: disable miplevel tests in ci completely for now

I still can't repro, and I'm running these regularly on multiple platforms,
so they're not going to get any worse for the time being

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12947>

etnaviv: add mov for direct depth store output from load input

If the fragment shader writes fragment depth from an ALU instruction,
the register allocator makes sure that the instruction is pointed to the
correct register and write mask (t0.__z_).
If there is no instruction emitted because the source is an input
load intrinsic, or if the source instruction does not support swizzle
and write mask, we have to add a mov instruction for this to work.

Fixes piglit test spec@glsl-1.10@execution@glsl-1.10-fragdepth.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12891>

nir_to_tgsi: Remove the abs on fcsel's bool src.

While the nir fcsel opcode specifies src0 != 0.0, as the comment says,
it's only ever used on bools-as-floats, so we know that src0 is
non-negative.  This saves an instruction per CMP on i915.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12921>

ci/freedreno: Update restricted trace sha1s.

Rendering changed slightly recently, but the diffs look fine.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12939>

ci/baremetal: Retry if our network device spontaneously fails.

Seen in https://gitlab.freedesktop.org/mesa/mesa/-/jobs/13824132.  It's
unlikely that graphics would kill the network, so just assume it's not our
fault and keep going.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12939>

ci/freedreno: Add some cubearray piglit flakes on a630 I noticed.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12939>

freedreno: Assert to check for the previous regression.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12942>

freedreno: Fix autotune regression since batch-cache rework.

I freed the key that autotune needed a little early.

Fixes: b2349a46715e ("freedreno: Move the batch cache to the context.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12942>

turnip: Fix a6xx gen4 compute shaders

Port of 74d10525374 freedreno/a6xx: ("Fix a6xx gen4 compute shaders")

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5354
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12856>

turnip: Rast updates for a6xx gen4

Port of 219e12b7f39 ("freedreno/a6xx: Rast updates for a6xx gen3")

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12856>

turnip: Fix unitialized cs->device

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12856>

freedreno: Add info->a6xx.has_shading_rate

@flto noticed these registers seem to be related to GL_QCOM_shading_rate

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12856>

turnip: consider multiview_mask when clearing depth-stencil attachment

Otherwise only first layer is being cleared.

Would fix several VK_EXT_multi_draw tests:
 dEQP-VK.draw.multi_draw.overlapping.normal.max_draws.*.vert_only.multiview.no_offset

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12913>

nir: Exclude non-generic patch variables from get_variable_io_mask.

These are I/O variables which are not going to be removed anyway.
However, get_variable_io_mask handles their location incorrectly.

Found using the GCC undefined behavior sanitizer.
Fixes the following error:

runtime error:
shift exponent 4294967258 is too large
for 64-bit type 'long unsigned int'

Closes: #5319
Fixes: cf5f8f55c3e25508fb975b263d6430a93442247a
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12719>

ac/nir: Fix match_mask to work correctly for VS outputs.

match_mask checks the intrinsic type and decides whether it's
per-patch or not. VS don't have per-patch outputs,
so this causes wrong behaviour there.

Found using the GCC undefined behavior sanitizer.
Fixes the following error:

runtime error:
shift exponent 18446744073709551584 is too large
for 64-bit type 'long unsigned int'

Closes: #5319
Fixes: bf966d1c1dd968116b8b547ca2739f5113caccb5
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12719>

mesa/st: Allow loops in GLSL when NIR is enabled, even if the HW can't.

The jump lowering enabled by EmitNoLoops breaks GLSL's loop unrolling on
various obviously unrollable loops, resulting in a lot of deqp-gles2 and
piglit failures.  NIR will help unroll whatever GLSL doesn't, so we can
trust the driver to apply that after GLSL's unrolling, so no need to ask
GLSL to lower all loops.

Fixes: #4979
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12917>

ci/i915g: Clarify failure happening in fbo-fragcoord2.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12917>

etnaviv: fix dirty bit check for baselod emission

Since baselod is stored in sampler state, not sampler view, we should
check the ETNA_DIRTY_SAMPLERS bit instead of ETNA_DIRTY_SAMPLER_VIEWS.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12916>

X11: Ensure that VK_SUBOPTIMAL_KHR propagates to user code

Commit 0245b825 switched from returning the error code VK_ERROR_OUT_OF_DATE_KHR
to returning the success code VK_SUBOPTIMAL_KHR. Prior to that commit, the error
code caused all code paths to fail immediately, but the success code does not.

Currently the success code is not recorded in some scenarios, resulting in a
result of VK_SUCCESS instead. This breaks applications that rely on the
result (per the spec) to trigger resizes.

This commit ensures that the proper VK_SUBOPTIMAL_KHR success code is set as a
sticky status (as comments indicate was intended), ensuring that it is
propagated to user code.

Fixes #5331

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12782>

radv: keep depth/stencil images compressed for TRANSFER_DST on compute

Only if the image is TC-compat HTILE because it can be decompressed
on compute for partial copies.

This should remove few depth/stencil decompressions for RAGE2 and Red
Dead Redemption 2 because they declare all images as concurrent but
never use the compute queue for them.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>

radv: add support for copying compressed depth/stencil images on compute

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>

radv: implement depth/stencil expand on compute

This works as long as the image is TC-compatible HTILE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>

radv: rename radv_decompress_depth_stencil()

To radv_expand_depth_stencil() for consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>

panfrost/ci: Skip the indirect_draw+XFB tests

We lack a dependency between the vertex job filling the indirect draw
buffers and the indirect draw compute job reading from these buffers,
leading to unreliable results (the tests pass if the vertex job is
done before the compute job starts, and fail otherwise). Let's disable
those tests until we sort it out.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12589>

panfrost: Fix collision in the indirect draw shader table

Min/max index search shaders are different for the !primitive_restart
and primitive_restart. We need to add entries for the primitive restart
cases otherwise we might retrieve a wrong shader from the cache.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12589>

panfrost: Fix indirect draws when vertex or instance count is 0

In that case we should just skip the vertex/tiler jobs as done in the
direct draw path.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12589>

panfrost: fix null deref when no color buffer is attached

Do not dereference color buffer #0 in the SFBD code path if no color buffer is
attached, as with depth-only attachments. Fixes a crash running glmark2 -b
shadow on Mali T720.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Fixes: c746747cb82 ("panfrost: fix GL_EXT_multisampled_render_to_texture regression")
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12927>

panfrost: Prepare shader helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare texture helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare pan_encoder.h to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare scoreboard helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare pandecode to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Move panfrost_major_version() to gen_macros.h

So we can use this function in decode_common.c when transitioning to
per-gen XML. While at it rename the function pan_arch().

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare pan_cs helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare blend helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: Prepare blitter helpers to per-gen XML

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12935>

panfrost: RGB332_UNORM is not a valid texture format on v6+

Cc: mesa-stable
Fixes: c6bdd976e611 ("panfrost: Split out v6/v7 format tables")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12508>

panfrost: Drop the R and T flags on SCALED formats

Sampling from SCALED textures / rendering to SCALED FBOs is a bit tricky
(requires extra int <-> float conversions in a few places).
mesa/st only use SCALED formats as vertex formats, and those formats
are optional in Vulkan, so let's drop the RENDER/TEXTURE flags to keep
things simple.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Suggested-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12508>

panfrost: RGB10_A2_SNORM is not a valid texture format on v6+

Cc: mesa-stable
Fixes: c6bdd976e611 ("panfrost: Split out v6/v7 format tables")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12508>

panfrost: Fix the Z32_S8X24 and X32_S8X24 definitions

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12508>

panfrost: Patch Z32_S8X24 format when creating a sampler view

The gallium driver always stores Z32_S8X24 textures on 2 different
planes. Let's fix the create_sampler_view() logic so we can support
single-planar Z32_S8X24 in the vulkan driver.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12508>

ac/nir: Remove byte permute from prefix sum of the repack sequence.

The byte-permute instruction v_perm_b32 is not exposed by older
LLVM releases (only available on LLVM 13 and later), therefore a new
sequence is needed which we can use with these LLVM versions too.

The prefix sum is replaced by two alternatives:

1. For GPUs that support v_dot, we shift 0x01 to the wanted byte
positions and then use v_dot to sum the results.

2. For older GPUs (Navi 10), we simply shift out the unwanted bytes
and use v_sad_u8 to produce the sum.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>

aco/isel: Fix emit_vop2_instruction to apply 16/24-bit flags properly.

Previously it used a builder function but didn't use the return
value from that function, so the flags were not applied.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>

aco: Add ability to optimize v_lshl + v_sub into v_mad_i32_i24.

Also change combine_add_lshl to use check_vop3_operands instead
of its own checks of the operands.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>

mesa: fix timestamp enum with EXT_disjoint_timer_query

Extension implementation missed GL_TIMESTAMP_EXT for Get* functions,
commit 5d58fea660c added GetInteger64vEXT support but obviously we need
to support the enum as well ...

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5361
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12890>

bin/khronos-update.py: add upstream for vulkan_directfb.h & vulkan_screen.h

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12877>

bin/khronos-update.py: update the branch name (s/master/main/)

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12877>

radeonsi: fix ps SI_PARAM_LINE_STIPPLE_TEX arg

This arg size should be 1 instead of 3. It does not affect functionality
because we does not enable it in SPI_PS_INPUT_ADDR. But it does affect
the VGPR number that LLVM produce when LLVM still count with all PS
function arguments.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12922>

freedreno/computerator/a4xx: Fix enum mismatch warning

Fixes: fb5deb2b4a0 ("a4xx/computerator: add initial backend")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12923>

freedreno/ir3: Fix generation check

Fixes: fb5deb2b4a0 ("a4xx/computerator: add initial backend")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12923>

freedreno/ir3: Cleanup liveness lifetime

I'm going to want to use this in other passes, so lets let the
allocation hang off the pass's context.  Also, while we're at it,
fix the error path leak in ir3_ra().

Fixes: 0ffcb19b9d9 ("ir3: Rewrite register allocation")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12923>

freedreno/ir3: Fix sched debug msgs

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12923>

fdno/resource: Rewrite layout selection for allocation

The previous code had a number of errors, the most glaring of which was
forcing linear when it was one of the possible layouts requested.

When modifiers are being used, a list of _acceptable_ modifiers is
supplied; it's up to the driver to then make a decision as to which it
thinks is most optimal.

Normally we would select between linear/tiled/UBWC in ascending order of
preference according to what's possible, however we can't use a tiled
layout with explicit modifiers as there is no modifier token defined for
it.

Rewrite the layout-selection mechanism to always try to do the most
optimal thing. If the use flags force us to, or we have a shared
resource without explicit modifiers, we use linear. Failing that, we use
UBWC wherever possible; if this is not possible, we use tiled for
internal resources only or linear for shared resources.

v2 (Rob): respect FD_FORMAT_MOD_QCOM_TILED; do not print perf warning on
user choice of disabling UBWC;

v3: fix several issues breaking CI tests: revert removal of using
MOD_INVALID in various places, and assume implicit modifiers if present;
do not attempt to set UBWC flags when screen->tile_mode(prsc) falls back
to LINEAR (e.g. for small mip-maps levels); use TILED for implicit
modifier case with non-shared resources

v4: fix unintended demotion of UBWC, i.e. only check QCOM_COMPRESSED
modifier and demote UBWC to less optimal format when using explicit
modifiers

Signed-off-by: Daniel Stone <daniels@collabora.com>
Tested-by: Heinrich Fink <hfink@snap.com>
Signed-off-by: Heinrich Fink <hfink@snap.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12595>

lima: fix leak of the screen hash table

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12902>

docs/panfrost: Remove obsolete note on Android.mk

Android.mk was removed, so remove a reference to it from Panfrost
documentation. We should document building for Android, but it'll be
through meson. This note as-is adds more confusion than note.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12914>

aco: cleanup assignment of unique_ptrs

Reviewed-by: Joshua Ashton <joshua@froggi.es>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12903>

radv: Expose modifiers that support DCC image stores with STORAGE_IMAGE_BIT

Some games, ie. Doom Eternal, present from compute following compute post-fx and would benefit from having DCC image stores available.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12862>

ac/surface: Add ac_modifier_supports_dcc_image_stores helper

Helper function to check if a modifier supports DCC image stores.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12862>

ac/surface: Add modifiers capable of DCC image stores

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12862>

iris: Add finalize_nir

Improves performance of SynMark OglDrvShComp by +241.879%±1.01366% (n=5)
on a random KBL desktop that I have.  That seems to put it at about the
same performance as i965, but I did not test that in a statistically
sound way.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

iris: Move iris_set_max_shader_compiler_threads and iris_is_parallel_shader_compilation_finished

There's going to be at least one more shader function set in
pipe_screen, so it makes more sense to do it in iris_program.c.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

iris: Eliminate iris_uncompiled_shader::needs_edge_flag

Use the flag that was set by nir_lower_passthrough_edgeflags.  The
lowering passes will soon be moved to a finalize_nir hook, so there
won't be any choice.  Ideally we'd like to eliminate iris_fix_edge_flags
completely, and this is a first step.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

nir/edgeflags: Add a flag to indicate the edge flag input is needed

Most modern hardware needs the edge flag added as a hidden vertex input
and needs code added to the vertex shader to copy the input to an
output.  Intel hardware is a little different.  Gfx4 and Gfx5 hardware
works in the previously described mannter.  Gfx6+ hardware needs the
edge flag as a specific vertex shader input, and that input is magically
processed by fixed-function hardware without need for extra shader code.

This flag signals only that the vertex shader input is needed.  It would
be nice if we could decouple adding the vertex shader input from
generating the copy-to-output code, but that has proven to be
challenging.  Not having that code causes other passes to want to
eliminate that shader input.

v2: Convert conditional to assertion.  This pass is only called for
vertex shaders.  Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

iris: Calculate uses_atomic_load_store after all lowering

The lowering passes will soon be moved to another function, so there
won't be any choice.

As a side benefit, this allows eliminating the uses_atomic_load_store
**pointer** parameter from brw_nir_lower_storage_image.  For some reason
crocus was passing false instead of NULL.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

iris: crocus: Use shader_info::is_arb_asm flag

...instead of looking for "ARB" in the name of the shader.  This matches
the behavior of i965.  Using "ARB" was added in a1ebac3750e ("iris:
Implement ALT mode for ARB_{vertex,fragment}_shader"), but there's no
explanation of why that method was used.

v2: Just use shader_info::is_arb_asm everywhere instead of
iris_uncompiled_shader::use_alt_mode.  Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

freedreno/a6xx: Pre-bake SO-disable stateobj

No need to re-create this every time we transition from stream-out
enabled to disabled.

Creation of streamout_disable_stateobj is deferred until we create
a program state using streamout to avoid creating it unnecessarily
and because fd6_prog_init() is called before ctx->pipe is created.
(Changing that ordering is complicated by the fact that u_blitter
copies pctx->bind_fs_state(), and friends.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12918>

zink: don't leak drm fd on drmPrimeFDToHandle failure

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12911>

freedreno: Add perf warning for WC readback

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno: Used cached coherent for staging resources

These are really only accessed by the GPU once, so CPU access speed is
more important.  Especially for PIPE_MAP_READ.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Use cached-coherent for control bo

Userspace frequently reads the elapsed fence, but the GPU only writes it
once per submit.  So this should be another useful place for cached-
coherent.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Use cached-coherent cmdstream buffers

Some more extreme examples, like gl_driver2_off, can be bottlenecked on
writes to cmdstream.  OTOH the CP is pretty pipelined in how it slurps
in memory, so the penalty of using coherent buffers should not be so
much.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Add cached-coherent bo support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Don't return shared/control bo's to cache

They can never be allocated from the cache, as fd_bo_state() would
return FD_BO_STATE_UNKNOWN

Fixes: 7dabd624649 ("freedreno/drm: Userspace fences")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Consider allocation flags in bo-cache

It hasn't really mattered until now, as we keep a separate cache for
cmdstream (which is FD_BO_GPU_READONLY), and the only other flag so
far is FD_BO_SCANOUT (which the bo cache probably messes up, but it
does not matter on most hw, and on hw where it does the scanout buffer
will be imported (and therefore won't end up in the bo cache).

But when we add cached-coherent (or if we wanted to use GPU_READONLY
more) it starts to matter.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Rename bo->flags to bo->reloc_flags

Next patch adds alloc_flags, lets rename bo->flags first to make it
clear *which* flags these are.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Garbage collect unused bo_cache

The ring_cache that is actually used is in fd_device.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

ci/bare-metal: add etnaviv

Add deqp gles2 CI run for GC2000.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci/bare-metal: add support for eth008 power relay

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci/bare-metal: add telnet based serial

Makes it possible to use e.g. a ser2net server in the lan.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci: update kernel

Switch to v5.13-rc5-for-mesa-ci-2bb5d9ffd79c branch, which includes
etnaviv MMU patches.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci: include etnaviv support in ARMHF container.

Build the kernel with CONFIG_DRM_ETNAVIV=y and include
imx6q-cubox-i.dtb.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

nir/algebraic: distribute fmul(fadd(a, b), c) when b and c are constants

This allows for more MAD/FMA instructions to be created.

fossil-db (Sienna Cichlid):
Totals from 50134 (33.46% of 149839) affected shaders:
VGPRs: 2436536 -> 2436000 (-0.02%); split: -0.05%, +0.03%
SpillSGPRs: 13136 -> 13135 (-0.01%); split: -0.02%, +0.02%
CodeSize: 206621424 -> 206278292 (-0.17%); split: -0.23%, +0.07%
MaxWaves: 1116804 -> 1117448 (+0.06%); split: +0.07%, -0.01%
Instrs: 38977460 -> 38862886 (-0.29%); split: -0.33%, +0.04%
Latency: 832425389 -> 827432260 (-0.60%); split: -0.63%, +0.03%
InvThroughput: 184193457 -> 183563350 (-0.34%); split: -0.37%, +0.03%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7458>

freedreno: Remove dead fd_batch_reset().

Unused since 58f5605124a7 ("freedreno: Handle full blit discards by
invalidating the resource.")

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Use a BO bitset for faster checks for resource referenced.

When moving the batch cache to the context, I added hash table lookups
from batch to rsc for "is this resource in use" because we could no longer
store data in the rsc bo under the batch cache's lock.

We can save that cost by tracking a bitfield of resources referenced by
the batch, which gives us very cheap checks in the draw path at a minor
cost in memory.  We can just use the GEM BO handle, since it's a nice
small integer already (we can't use the TC buffer ID, because the frontend
changes that, and we're in the driver thread).

This required moving the !pending() assert up in resource shadowing, since
the BO swap meant we were checking pending on the wrong resource.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Remove the submit lock locking.

I think the whole submit lock thing should be possible to remove now, but
just getting rid of the lock since we're no longer sharing batches between
ctxes means another several percent draw overhead improvement.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Move the batch cache to the context.

Our draw call rate was significantly limited by the atomics we had to do
to manage access to the batch cache.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Use TC's flag for whether get_query is in the driver thread.

In moving batch cache to the context, the check for whether there's
pending work being done to this resources ends up accessing the context,
so we can't do it outside of the fd_context_access_begin().  This flag
lets us do the driver-thread asserts before we've decided whether we need
to flush.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>