panfrost: Don't skip the test with a 4k shader

Commit bfcdc8f1747e ("pan/bi: Add some zero bytes after shaders on
Bifrost") should have fixed the problem.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8470>

nir/loop_unroll: unroll more aggressively if it can improve load scheduling

Significantly improves performance of a Control compute shader. Also seems
to increase FPS at the very start of the game by ~5% (RX 580, 1080p,
medium settings, no MSAA).

fossil-db (Sienna):
Totals from 81 (0.06% of 139391) affected shaders:
SGPRs: 3848 -> 4362 (+13.36%); split: -0.99%, +14.35%
VGPRs: 4132 -> 4648 (+12.49%)
CodeSize: 275532 -> 659188 (+139.24%)
MaxWaves: 986 -> 906 (-8.11%)
Instrs: 54422 -> 126865 (+133.11%)
Cycles: 1057240 -> 750464 (-29.02%); split: -42.61%, +13.60%
VMEM: 26507 -> 61829 (+133.26%); split: +135.56%, -2.30%
SMEM: 4748 -> 5895 (+24.16%); split: +31.47%, -7.31%
VClause: 1933 -> 6802 (+251.89%); split: -0.72%, +252.61%
SClause: 1179 -> 1810 (+53.52%); split: -3.14%, +56.66%
Branches: 1174 -> 1157 (-1.45%); split: -23.94%, +22.49%
PreVGPRs: 3219 -> 3387 (+5.22%); split: -0.96%, +6.18%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6538>

freedreno/ir3: Remove legacy packed-struct encoding

Note that we can't actually remove the packed structs themselves yet,
because tu still uses them in some hand-coded blit shaders.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3/decode: Switch over to new disasm

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Realign disasm shader stats

To better match up with what mesa shader-db stats look like, for easier
comparision.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Better sstall estimation

1) Take into account repeat/nop cycles
2) Clear sfu_delay after an (ss) sync

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Small resinfo disasm tweak

Add the 'type' field.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Switch over to new encoder/decoder

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3/tests: Switch disasm test over to new decoder

Also, uncomment the `stc` test vectors (since the new decoder decodes
these properly) and comment out an instruction which looks suspiciously
like -6.0 in hex.

This also switches the parser back to `atomic.b.op` from `atomic.op.b`
which was a short-term workaround to make it easier for the legacy
disassembler.

Also switch the binary encoding for ldib to clear b0, because the new
disassembler warns about unexpected dontcare bits (which cases the
disasm to not match).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/hw/isa: Add expression caching

Drops decoding an ~850KB collection of instructions from ~4min to ~1sec.

Granted for normal sized shaders, this probably doesn't matter.. but it
at reduces my cycle time for fixing things to match existing disasm
syntax using this massive collection of unique instructions.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/hw/isa: Add description of ir3 ISA

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/hw: Add isaspec mechanism for documenting/defining an ISA

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Add some new "logical" opcodes

Once we switch over to the xml based ir3 ISA definition, the opcodes
will be decoupled from instruction encoding.  Which will let us better
handle cases where a single "opcode" (from instruction encoding stand-
point) means different things on different generations.  And also cases
like the different variations of `b`ranch instructions, which share a
single hw "opcode" plus a separate "brtype" field.  When we start using
these in ir3, we'd like to treat them as separate instructions and not
have to care about the details of how they are encoded.

For now, these are only used internally within the new xml generated
instruction encoding, but once the existing "packed struct" encoding/
decoding is replace, we'll update ir3 to start using the new opcode
enums directly (except for the `mov` variants).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Decouple ir3_info collection from assembler

We'll want to re-use this when cutting over to the new XML based
instruction encoding.  So untangle it from instruction packing.

Also, move handling of the appended constant data out of the
assembler, since this isn't much related to instruction encoding.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

freedreno/ir3: Fix ldg decoding/parsing

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7997>

aco/ra: Use PhysRegInterval for count_zero

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use PhysRegInterval for collect_vars parameters

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use PhysReg when indexing into RegisterFile's containers

This gets rid of a lot of implicit/explicit conversions from PhysReg to
unsigned.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use PhysReg for member functions of PhysRegInterval

This replaces the various PhysReg{lb} casts that had been all over the place.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Remove unused function parameter

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use std::all_of to simplify a loop

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Add helpers to test for intersection/containment of reg intervals

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Move commonly repeated code to a helper function

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Conservatively refactor get_reg_specified to use PhysRegInterval

All expressions have been replaced by their closest equivalent. No major
simplification efforts have been made to minimize risk of regressions.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use std::all_of to simplify a loop

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Use std::find_if(_not) to clean up get_reg_simple

This makes for a more self-describing iteration behavior, and it gets rid
of the need for the duplicated "final check" at the bottom.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Add iterator interface for PhysRegInterval

This enables various loops to use range-based for.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Remove always-false conditions

All code paths that set "found" to true either break or return before the
loop header is reached again, so the checks are unnecessary.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Conservatively refactor existing code to use PhysRegInterval

All expressions have been replaced by their closest equivalent. No major
simplification efforts have been made to minimize risk of regressions.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Introduce PhysRegInterval helper class

This mainly clarifies the semantics of register bounds (inclusive vs
exclusive), and further groups related varaibles together to clarify
sliding-window-style loops.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco/ra: Update register use bounds before recursing in get_regs_for_copies

Delaying the call to adjust_max_used_regs until after get_regs_for_copies
returns puts the RA context into a state where registers past max_used_gpr
may be blocked. This isn't an issue on its own, but it adds a surprising
corner case to get_reg_simple that is easily avoided now.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7799>

aco: remove divergent branches which only jump over very few instructions

Totals from 18436 (13.23% of 139391) affected shaders (NAVI10):
CodeSize: 138428504 -> 138172588 (-0.18%)
Instrs: 26605127 -> 26541176 (-0.24%)
Cycles: 1624994088 -> 1622461620 (-0.16%)
VMEM: 3689892 -> 3689102 (-0.02%)
SMEM: 1131767 -> 1131761 (-0.00%)
Branches: 851796 -> 787852 (-7.51%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7814>

aco: propagate swizzles when optimizing packed clamp & fma

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: optimize v_pk_fma_f16 -> v_pk_fmac_f16 on GFX10

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: optimize packed fneg

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: optimize packed clamp

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: optimize packed mul+add to v_pk_fma_f16

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: simplify multiply-add combining

When both operands of a v_sub (same apply for v_add) are mul and one
already uses clamp/omod, pick the other operand to get a chance to
combine to a MAD.

No fossils-db changes.

Co-authored-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

radv: vectorize 16bit instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: emit packed 16bit instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: create helpers to emit vop3p instructions

Also make get_alu_src() capable to return
unswizzled multi-component SGPR sources.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: change usesModifiers() considering opsel_hi on packed instructions

opsel_hi == 1 means that the high operand selects the
high bits of the input, which is the normal behavior.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: allow SGPRs on every src position for VOP3P

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: allow constants/literals on every src position for VOP3P

and prevent literals on VOP3P pre-GFX10.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco/RA: fix subdword operands on VOP3P instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

aco: fix VOP3P assembly, VN and validation

aco/opcodes: rename v_pk_fma_mix* -> v_fma_mix*
and add modifier capabilities for VOP3P.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>

VERSION: bump for 21.1.0 cycle

radv: enable DCC for MSAA on GFX10+

It should work fine now.

This gives +1-2% improvements with Control MSAA (2x and 4x)
on Sienna.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8413>

pan/bi: Fix the !immediate case in bi_emit_store_vary()

The base offset was ignored, take it into account.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8469>

nouveau: trigger the current fence's work on destroy explicitly

Otherwise the delete yells at us that there's still work pending. This
isn't an actual problem, but annoying to see each time.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8462>

ci: Add an expected failures list for Oland (GFX6)

This is a copy of the expected failures list of Pitcairn (also GFX6)
with some Oland specific failures added.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8473>

frontends/va: Return an error if non-interlaced buffer is not supported

Add a check to vaDeriveImage to see if a non-interlaced buffer was
created successfully. Otherwise, return an error, since we won't be able
to derive an image from the interlaced buffer.

Prevents a null pointer dereference from occuring on some nVidia cards,
reported by Alexander Kapshuk.

v2: Check for PIPE_VIDEO_CAP_SUPPORTS_PROGRESSIVE support (Ilia)

Fixes: fcb558321e6 ("frontends/va: Derive image from interlaced buffers")
Signed-off-by: Thong Thai <thong.thai@amd.com>
Tested-by: Alexander Kapshuk <alexander.kapshuk@gmail.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8320>

radv: Use L2 coherency on GFX9+.

Especially on GFX10 we can avoid pretty much all L2 flushes.

However, instead of that we have to do L2_METADATA invalidations. We
do that every time we could possibly be reading new DCC/HTILE info
from the L2 cache in shaders.

Benchmark results, basemark on high preset with a navi10 on profile_standard
(which is slower than a navi10 on default settings, please don't compare
 to random navi10 results you find)

before:
  5932
  5928
  5937

after:
  6011
  6013
  6009

So this looks like a >1% increase.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

radv: Use L2 for CP DMA on GFX9+.

This enables assuming that the L2 is always up to date for barriers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

radv: Use access helpers for flushing with meta operations.

This way we're properly using the vulkan barrier paradigm instead
of adhoc guessing what caches need to be flushed. This is more robust
for cache policy changes as we now don't have to revisit all the meta
operations all the time.

Note that a barrier has both a src and dst part though. So

barrier:
   flush src
   meta op
   flush dst

becomes

barrier:
  flush barrier src
  flush meta op dst
  meta op
  flush meta op src
  flush barrier dst

And there are some places where we've been able to replace a CB flush
with a shader flush because that is what we'd need according to vulkan rules
(and it turns out that in the cases the CB flush mattered the app will set the
bit in one of the relevant flushes or it was needed as a result of an optimization
that we counter-acted in the previous patch.)

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

radv: Do dst invalidations for write accesses.

For write-after-write hazards.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

radv: Invalidate CB on SHADER_WRITE for meta operations.

To cancel the optimization in radv_dst_access_flush if these helpers
get used by meta operations.

We could also remove that optimization but I think this triggers less
often as all SHADER_WRITE flushes on images not supporting STORAGE should
be meta

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

radv: Remove redundant WB_L2 flush.

INV_L2 already does that.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7202>

panfrost: Implement alpha testing natively

On Midgard, we still have to lower on v6+. Passes Piglit
./fbo-mrt-alphatest (saving a cycle in the fragment shader to
compare/discard).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8447>

panfrost: Add alpha reference to XML

Midgard only, v6 dropped support.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8447>

panfrost: Handle explicit primitive restart

Don't fall back. Passes piglit ./bin/primitive-restart on Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8447>

radv: disable VK_EXT_sample_locations again on GFX10+

I attempted to enable it for 21.0, only 2x and 4x were supported
but there is new failures if DCC+MSAA is enabled.

Disable it again because DCC is more important than this feature and
no Mesa releases have it on GFX10+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8472>

panfrost: Fix panfrost_afbc_format_needs_fixup()

This function returns true for PIPE_FORMAT_R8G8B8X8_UNORM, which is
wrong.

Fixes: 44217be92134 ("panfrost: Adjust the format for AFBC textures on Bifrost v7")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8466>

radv: enable DCC for mipmaps on GFX10+

Seems to work fine.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8468>

radv: do not enable DCC for 3D images with mipmaps on GFX10+

This is broken for some reasons, and probably rare enough to
care for now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8468>

radv: add support for fast-clearing DCC levels on GFX10+

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8468>

radv: prevent fast-clearing uncompressed DCC levels

When size is 0, this means the level can't be compressed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8468>

ac/surface: store DCC mip info into the surface

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8468>

pan/bi: Implement TEXS for cube maps

Saves a few instructions in the common case. Requires refactoring the
TEXS check.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8287>

aco/tests: don't rely on argument evaluation order

The argument evaluation order is implementation-defined and affects the
order the instructions are inserted.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3938
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7945>

v3d: add fast-path tile-based blit for depth/stencil buffers

This extends the TLB based blit to support both depth and stencil
buffers.

v2:
 - Ammend comment for further clarification (Iago)
 - Remove parenthesis (Iago)
 - Remove condition so separate stencil blit is done (Iago)

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8304>

v3d: check blit mask inside blit subpaths

Move the blit mask check (RGBA, Depth/Stencil) inside the blit paths
(stencil, TFU, TLB and render blit paths).

v2:
 - Add missing Fixes tag (Iago)

Fixes: 1c76f6e755a ("v3d: implement tile-based blit operation")
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8304>

radv: skip fast-clear eliminate for CMASK based on a predicate

If we have CMASK, we can also skip FCE like we do for DCC.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8332>

radv: update the FCE predicate for fast clears using CMASK

Fast clearing with CMASK should always be eliminated.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8332>

radv: allocate and initialize the FCE predicate value for CMASK too

In case we don't have DCC, we can still predicate FCE with CMASK.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8332>

radv: only use predication if the FCE value is allocated

The FCE predicate value is only allocated if DCC is enabled.
We only want to use predication for DCC decompressions and for FCE
but not having FMASK doesn't mean the predicate is allocated.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4075
Fixes: 6e7008e94ba ("radv: do not predicate FMASK decompression when DCC+MSAA is used")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8441>

turnip: implement indirect dispatch

Vulkan guarantees only 4 byte alignment of offset for vkCmdDrawIndirect,
while CP_LOAD_STATE.EXT_SRC_ADDR requires 16 byte alignment which
makes us copy indirect parameters to a correctly aligned buffer.

Blob does essentially the same but emits indirect CP_LOAD_STATE
with src = SS6_UBO and EXT_SRC_ADDR = 0xe0000, and only for a
first dispatch.

Fixes:
dEQP-VK.compute.indirect_dispatch.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8444>

turnip: remove unused IR3_DP_LOCAL_GROUP_SIZE_* from cs params

In Turnip local group size is lowered in NIR via
nir_lower_compute_system_values.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8444>

st/mesa: use the correct src format in ReadPixels

If reading from an FBO that uses a texture view src->format will
be the format of the original texture, not from the view.

Acked-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8387>

mesa/fbo: don't check_end_texture_render on fb read change

Otherwise this resets is_rtt to false in st/mesa, and then
breaks fbo + texture view.

This change also aligns the code with the comment above:
    * Note that if the ReadBuffer has texture attachments we don't consider
    * that a render-to-texture case.

Acked-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8387>

st/mesa: consider texture view format for fbo blits

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4034
Acked-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8387>

r600/nir: use "unreachable" instead of "assert"

In release builds the assert goes away resulting in build failures
because no return value was specified.

Fixes 165fb5117bf70402e66d34538d4085e060f57fea
   r600/sfn: add lowering passes to get 64 bit ops lowered to 32 bit vec2

Closes #4089

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8465>

radv: fix clearing DCC on GFX9

dcc_slice_size is in DWORD on GFX9... Also, layers aren't supported
because they might be interleaved. Fix this by clearing the entire
DCC buffer.

Fixes: 5e8f6967b1d ("radv: add support for fast-clearing DCC layers on GFX9+")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8443>

v3d: mark some variables static const

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8438>

v3d: update fallthrough comments

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8438>

v3d: drop not use function parameter

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8438>

nv50/ir: Initialize CodeEmitterGM107 members in constructor.

Fix defects reported by Coverity Scan.

uninit_member: Non-static class member progType is not initialized
in this constructor nor in any functions that it calls.
uninit_member: Non-static class member insn is not initialized in
this constructor nor in any functions that it calls.
uninit_member: Non-static class member data is not initialized in
this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7390>

panfrost: Fix size assertion in bi_alu_src_index

Shifting by the bitsize was not only wrong, the shift is undefined
behavior when bitsize is 32, causing the assertion to fire on AArch32.

Fixes: 95d62ee7cfa ("pan/bi: Add bi_alu_src_index helper")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8460>

ci: include nouveau in shader-db runs

This should include coverage of the whole pipeline including the nouveau
codegen compiler across the "interesting" chips which should generate
sufficiently different code.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8432>

dri: Restrict glthread for CS:GO to radeonsi

Fixes a ~12% performance regression in iris.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8448>

pan/bi: Add some zero bytes after shaders on Bifrost

Bifrost will prefetch bytes after the end of shaders, so make sure
these bytes are allocated and zeroed.

Fixes GPU faults in Xonotic.

Suggested-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8428>

pan/bi: Add a define for the Bifrost shader prefetch size

Found by adding NOPs to the start of a shader and checking dmesg to
see at what sizes the GPU faulted trying to read the following
non-executable page.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8428>

radeonsi: Only set modifier creation function for GFX9+ & with kernel support.

Fixes: c786150dfa5 ("radeonsi: Add modifier support.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3963
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8407>

panfrost: Skip an XFB test that's passing/failing randomly

transform_feedback.array_element.interleaved.triangles.mediump_mat2x4
seems to pass/fail randomly, skip it for now.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8449>

panfrost: Re-enable AFBC on 3D, 2D arrays

Things have now been fixed and AFBC on 3D/2D-arrays seems to work fine.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8449>

panfrost: Fix estimate_texture_payload_size() on Bifrost

Bifrost mandates manual stride usage.

Fixes: a3d2936a8e9e ("panfrost: The texture descriptor has a pointer to a trampoline")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8449>

panfrost: Pass the resource dimension to panfrost_compression_tag()

The reload surface logic creates 2D image views pointing to a specific
3D texture layer, but panfrost_compression_tag() cares about the resource
dimension, not the image view one.

Fixes: 4dd7991422ce ("panfrost: Add a pan_image_layout object")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8449>

panfrost: Get layer stride of level 0 on staging resources

Staging resources have one level, we shouldn't query the stride of
level > 0.

Fixes: 3c92abe35937 ("panfrost: Use panfrost_get_layer_stride() instead of open-coding it")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8449>

vulkan/overlay: fix vkGetInstanceProcAddr self-resolving

vkGetInstanceProcAddr(instance, "vkGetInstanceProcAddr") should return our
vkGetInstanceProcAddr not the next in the chain.

CC: mesa-stable
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8285>

docs/features: Fix missing close paranthesis

My bad. This is what I get for writing documentation at midnight.

Fixes: bd697652a73 ("docs: Document extensions exposing GL3.0")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8429>