pan/bi: Add pack_format helper

Packs a single quadword of a clause with a particular format and
parameters.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add subword 4 or 7 pack

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add subword 5/6 pack

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add tuple/embedded constant pack

Used for the first 4 subwords.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_pack_sync

The type/sync byte, also known as the tag.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_pack_tuple_bits

More general than the top 3 bit special case. There's some serious
complication around 78-bit shifting but I digress.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_pack_upper

Top 3-bits, found e.g. in the tag.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_pack_literal

Identifies formats.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Move bi_packed_tuple to compiler.h

To be used for pack prototype.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add clause encodings as a table

We would rather not type out all of the packs by hand (that's error
prone), so declaratively specify the encodings as a table corresponding
to the bit patterns.

This is all formats, except for format 12 which just encodes constants.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Pack multiple tuples in-memory

There's still some silly clause-level packing required, but this way the
register slots are at least assigned correctly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Amend misleading comment

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Clarify tuple comment

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Use enum bifrost_message_type

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Use canonical terminology for tuple

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Move bi_constants to bifrost.h

Although it's software-defined, this stems from architectural traits and
is useful in both the disasm and the compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Test read predicates

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Test bi_must_message

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Test bi_must_last

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Unit test bi_can_{fma, add}

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add various read predicates

Correpsonding to source masks in the XML.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Emit branch and table bits in opcode table

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Label table instructions

Table instructions are a subset of those scheduled to the ADD unit. They
typically involve lookup tables. Table (TBL) instructions have
additional restrictions on their sources which the scheduler must
respect.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_must_message predicate

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_must_last predicate

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Pipe last flag into opcode tables

Only ADD unit for now.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Annotate ISA.xml with 'last' parameter

If this instruction must be the last in the clause.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_can_{fma, add} predicates

Stubs due to some edge cases, for the scheduler.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Factor nir_function_impl out of the context

Unnecessary and complicates unit testing.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Stub out scheduler unit test

Someone who understands meson and gtest could do something much nicer,
but for now let's just stuff some assertions into debug builds of the
standalone compiler and call it a day...

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add "word equivalence" relation for index

Takes offset *but not swizzle* into account, so the scheduler can
predict whether indices will end up mapped to the same scalar register.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Move init_builder to common code

Needs to take a cursor to be applicable outside NIR->BIR of course.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Print multiple destinations if needed

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add CUBEFACE pseudoinstruction

Abstracts over *CUBEFACE1/+CUBEFACE2, takes the sources of the former
and outputs two destinations.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Don't fill garbage

If an index is an SSA form and we haven't even written to it yet, there
is absolutely no value in filling it, it'd just be uninitialized garbage
that won't get used. Saves some fills in STK.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Implement spilling at the clause-level

We use essentially the same logic, but we need to treat entire clauses
as large instructions, and spill on clause boundaries instead of
instruction boundaries. So factor out the code a bit, using the new
iterators, removing bi_unwrap_singleton.

A few specific fixes are needed to adapt. One is simple: rewriting
destinations needs to preserve clamps and such. The other is a much more
subtle bug. Consider the clause

   {
      ADD 0, ...
      ---unrelated code---
      MUL 1, 0, ...
   }

Suppose we spill 0. The naive spill code would generate an SSA temporary to
spill to and another SSA temporary to fill from, generating:

   {
      LOAD.tl 10
   }
   {
      ADD 11, ...
      ---unrelated code---
      MUL 1, 10, ...
   }
   {
      STORE.tl 11
   }

But this is wrong; the MUL now reads stale (and for SSA, likely
undefined/uninitialized) data. The simplest fix, employed here, is to
spill/fill within a clause simultaneously, which means the temporary
can't be SSA, generating correct code:

   {
      LOAD.tl r0
   }
   {
      ADD r0, ...
      ---unrelated code---
      MUL 1, r0, ...
   }
   {
      STORE.tl r0
   }

This is suboptimal, since the LOAD is still likely reading garbage that
is unused. But it is still correct, and the next commit will avoid
generating the load in this case.

To make the bug even more subtle, if ADD/MUL are within 2-3 instructions
of each other, the scheduler will optimize the load to a
temporary/passthrough, so the bug isn't noticed. It only happens if they
are 3+ instructions apart yet still in the same clause (<=16
instructions).

Special thanks to macc24 for reporting this bug and to Jason Ekstrand
for allowing me to rubberduck.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add interference per clause

With new helpers.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Permit multiple destinations in RA

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Don't open code bi_foreach_dest

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add destination iterator macro

Convenience.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_foreach_instr_in_clause iterators

These are convenient for post-sched passes, in particular register
allocation. They work by noting the underlying linked list of
instructions in the block must be preserved by scheduling. It isn't
necessary iterate the clause structure directly, it can simply be used
to bound a iteration within the block by recalling clauses are strictly
contained in a single block.

   <alyssa> I don't think I'm enough of a C ninja to write fancy iterator macros yet.
   <zmike> sometimes those can get pretty mind-bendy

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_foreach_instr_in_tuple helper

Written in a funny way but easy to convince yourself of correctness by
considering cases.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_foreach_clause_in_block_rev

Trivial absense.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add bi_{before,after}_clause cursors

Will be needed to insert spill code after scheduling once we have
multiple instructions in a clause.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add "soft" mode to DCE

We would like to reuse the DCE logic to eliminate register writes
without eliminating instructions, as a post-sched pass. This type of
operation will eventually generalize to intrinsics that write a register
*and* have side effects (just atomics, I think).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add dead branch elimination pass

Ported from Midgard due to the same quirk of our code generation.
Additional validation, though.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Pass through wait_{6, 7} flags

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Move bi_next_clause to bir.c

Not really packing specific anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Pull out bi_count_read_registers helper

I want to transition away from the ad hoc masks anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Fix M1/M2 decoding in disassembler

C's definition of the % operator has a footgun around sign conversion.
Avoid it and just use bitwise arithemtic instead like the hardware
would, fixing the disassembly and making buggy assembly more obvious.

Fixes: 08a9e5e3e89 ("pan/bi: Decode M values in disasm")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Fix dependency wait calculation

Unconditional branches have a successor in the first slot only.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Fix staging register packing

Writes are from the previous tuple, not the current one, otherwise we
incorrectly write to "two" places at once and raise an INSTR_INVALID_ENC
fault. While we're at it, fix the weird spacing.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Fix IDLE register mode packing

Was incorrectly returning zero. Special case like IDLE_1.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Print disasm/stats with DEBUG=internal

Arguably more important than the IR prints.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Lint for infinite loops

I would make this unconditional, but conditionally branching to the same
clause in a tight loop is (disturbingly) legal, as far as I know.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Refactor PC-relative printing

Let's get the offset in a named variable for validation.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Print FAU index in verbose mode

Even if we're not loading a uniform, this is useful information. The
uniform pretty-printing didn't correspond well to the hardware anyway so
this is a net win, although if somebody really wanted pretty-printing
could be added in here.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Validate format 12 tuple count in disasm

We were throwing away this information. Let's just use a lookup table
and add an assertion. Would have caught a bug in this series resulting
in INSTR_INVALID_ENC faults.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

pan/bi: Add internal debug flag

Since 31864017510 ("pan/bi: Suppress disassembly for internal shaders"),
we haven't had a good way to debug blit shaders. I keep rewriting this
patch manually, let's just a debug flag for it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

panfrost: Allow waiting on slots 6/7 during preload

I don't understand the underlying uarch details but ATEST needs to wait
on slot 6 and BLEND needs to wait on both, so these bits are used if
ATEST/BLEND are in the first clause, which happens if e.g. a constant
colour is written, or if the input is preloaded.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

panfrost: Fix TLS sizing if cores are missing

I have no idea if there are any implementations we care about that have
missing shader cores (a mask of 1101 or something like that), but if one
crops up, this would be a royal pain to debug so let's just get it
right...

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>

zink: move tess/geom shader info to vs shader key

now that there exists a shader key for vertex stages, we can stop modifying
the zink_shader values and instead use this as a more reliable method of detecting
the state

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8682>

zink: flag shaders as needing update when clip_halfz changes

this means we may or may not need to run the nir pass in the shader,
so force this to go back through the update path using the shader key

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8682>

zink: add shader key for vs shaders

we're reusing these for tes/gs for now too

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8682>

zink: flag previous vertex stages as dirty when toggling a later stage

this ensures that the correct variant is used for streamout and halfz

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8682>

zink: improve barrier helper for buffer resources and add check for barrier need

now we've got the ability to add fine-grained barriers for buffer resources, so we
can also have a utility function to check whether we need to use barriers and
then skip them when we don't

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8669>

zink: add helper function for checking if access flags include write access

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8669>

zink: add a stage param for buffer resource barriers

it's useful to be able to set this more granularly when doing resource
barriers

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8669>

zink: add barrier helper for buffer resources

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8669>

radv: fix centroid with VRS coarse shading

Ported from RadeonSI.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8775>

radv: re-disable TC-compat HTILE for D32S8 on all generations

This actually introduced some VRS related regressions and some others.

Fixes: cc5b6a0e897 ("radv: enable TC-compat HTILE with D32S8 and MSAA on GFX9+")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8777>

intel/compiler: cache computed register pressure benefit

This halves the number of calls to get_register_pressure_benefit
and decreases shader-db CPU time by ~1.5%.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8741>

radeonsi/sqtt: forward string markers to sqtt

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8746>

radeonsi/sqtt: allow AMD_THREAD_TRACE_TRIGGER to be a frame number

This makes it easier to capture the exact same frame (for instance from an
apitrace replay).

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8746>

radeonsi/sqtt: fix SQTT bo size overflow

Ported from c40ea24ee009d8c9816ff6327f65be3fbd45deb7

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8746>

radeonsi/sqtt: use more event identifier

Using event identifiers allows to add a bit more context to the RGP trace.
Without this all draw calls are identified as vkCmdDraw.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8746>

ci: split src/mesa/**/* matching rule

Split the rule to avoid running useless tests when touching the driver specific
sources.

v2: removed src/mesa/drivers/x11/**/*

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Eric Anholt <eric@anholt.net> (v2)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8735>

iris: Enable PIPE_CAP_SHAREABLE_SHADERS.

Now that we store shader variants in the objects themselves rather
than a per-context hash table, they are actually global across
contexts.  We can enable this feature.

This makes shaders shared across contexts, so apps can compile in
one and use it in another.  This has always been allowed by GL,
but in the past we've simply recompiled the shaders in every context,
which is slow and painful.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>

iris: Store a list of shader variants in the shader itself

We've traditionally stored shader variants in a per-context hash table,
based on a key with many per-stage fields.  On older hardware supported
by i965, there were potentially quite a few variants, as many features
had to be emulated in shaders, including things like texture swizzling.

However, on the modern hardware targeted by iris, our NOS dependencies
are much smaller.  We almost always guess the correct state when doing
the initial precompile, and so we have maybe 1-3 variants.  iris NOS
keys are also dramatically smaller (4 to 24 bytes) than i965's.

Unlike the classic world, Gallium also provides a single kind of object
for API shaders---pipe_shader_state aka iris_uncompiled_shader.  We can
simply store a list of shader variants there.  This makes it possible
to access shader variants across contexts, rather than compiling them
separately for each context, which better matches how the APIs work.

To look up variants, we simply walk the list and memcmp the keys.
Since the list is almost always singular (and rarely ever long),
and the keys are tiny, this should be quite low overhead.

We continue storing internally generated shaders for BLORP and
passthrough TCS in the per-context hash table, as they don't have
an associated pipe_shader_state / iris_uncompiled_shader object.
(There can also be many BLORP shaders, and the blit keys are large,
so having a hash table rather than a list makes sense there.)

Because iris_uncompiled_shaders are shared across multiple contexts,
we do require locking when accessing this list.  Fortunately, this
is a per-shader lock, rather than a global one.  Additionally, since
we only append variants to the list, and generate the first one at
precompile time (while only one context has the uncompiled shader),
we can assume that it is safe to access that first entry without
locking the list.  This means that we only have to lock when we
have multiple variants, which is relatively uncommon.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>

iris: Refcount shader variants

There is a small gap of time where the currently bound uncompiled
shaders, and compiled shader variant, are out of sync.  Specifically,
between pipe->bind_*_state() and the next draw.

Currently, shaders variants live entirely within a single context,
and when deleting an iris_uncompiled_shader, we check if any of its
variants are currently bound, and defer deleting those until the next
iris_update_compiled_shaders() hook runs and binds new shaders to
replace them.  (This is due to the time gap between binding new
uncompiled shaders, and updating variants at draw time when we have
the required NOS in place.)

This works pretty well in a single context world.  But as we move to
share compiled shader variants across multiple contexts, it breaks down.
When deleting a shader, we can't look at all contexts to see if its
variants are bound anywhere.  We can't even quantify whether those
contexts will run a future draw any time soon, to update and unbind.

One fairly crazy solution would be to delete the variants anyway, and
leave the stale pointers to dead variants in place.  This requires
removing any code that compares old and new variants.  Today, we do
that sometimes for seeing if the old/new shaders toggled some feature.
Worse than that, though, we don't just have to avoid dereferences, we'd
have to avoid pointer comparisons.  If we free a variant, and quickly
allocate a new variant, malloc may return the same pointer.  If it's
for the same shader stage, we may get a new different program that has
the same pointer as a previously bound stale one, causing us to think
nothing had changed when we really needed to do updates.  Again, this
is doable, but leaves the code fragile - we'd have to guard against
future patches adding such checks back in.

So, don't do that.  Instead, do basic reference counting.  When a
variant is bound in a context, up the reference.  When it's unbound,
decrement it.  When it hits zero, we know it's not bound anywhere and
is safe to delete, with no stale references.  This ends up being
reasonably cheap anyway, since the atomic is usually uncontested.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>

microsoft: Fix comma in variadic macro for MSVC

New preprocessor seems to be enabled by default when C17 mode is active.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8772>

gallium/tessellator: Fix warning suppression

Single-line version of MSVC warning suppression does not extend beyond
the #endif directive. Use push/disable/pop instead.

Also suppress 26452, which is a similar analysis warning.

This could also be fixed with constexpr if, but C++17 would be required.

Fixes: 790516db0bf ("gallium/swr: fix gcc warnings")
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8093>

panfrost: Add the tiler heap to fragment jobs

In some cases the GPU reads from the tiler heap in fragment jobs, so
always add it to GPU jobs.

Fixes faults in many applications that use multiple windows
(e.g. Firefox, plasmashell).

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4157
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8773>

glapi: guard against invalid XML definitions for glthread

This would have prevented the bug that the previous commit fixes.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8753>

glthread: fix glVertexAttribDivisor calls not being tracked by non-VBO uploads

marshal_call_after is ignored if the function is an alias of another
function. Move it to the right place.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8753>

glthread: fix interpreting vertex size == GL_BGRA for vertex attribs

Fixes: c9c9f57b022 - glthread: track pointers and strides for Pointer & EXT_dsa attrib functions
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4116

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8753>

ci: Update baremetal kernel to 5.11-rc5 plus patches.

The dr_mode hack is now folded into the git tree.  The uprev brings in a
shrinker fix for msm and a fix for the GPU_SET OOB messages on cheza
(possibly involved in piglit flakes).

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8768>

iris: Move VS draw parameter dirty flagging to iris_bind_vs_state

Now that we're looking at shader info system values rather than
vs_prog_data, there's no reason we have to do this when updating
the shader variants.  We can simply check it when binding a new
shader from the API point of view.

Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8759>

iris: Minor code restyling in iris_bind_vs_state

We'll be adding more code here shortly, this will make it easier.

Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8759>

iris: Use shader_info rather than vs_prog_data for draw parameter checks

brw_compile_vs sets the vs_prog_data fields based on the NIR program's
system values read info field.  We can use that directly, enabling more
cleanups in the next patches.

Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8759>

radv: Expose VK_KHR_workgroup_memory_explicit_layout.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8752>

freedreno: Remove duplicate bc invalidate on flush_write_batch().

The fd_batch_flush() internals already do the invalidate at the end to
clean up the bc's references to the batch.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8729>

freedreno: Early-out from the resource write path when we're the writer.

No need to do the other checks in this case, because then we know that
we've done the UBWC clears and recursed on stencil and added deps on read
batches.

Done as a separate patch to reduce behavior changes in my upcoming move of
the batch cache to the context.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8729>

freedreno: Use a real type instead of void * for the fd_batch->key.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8729>

mapi: Undefine MemoryBarrier

Reviewed-by: Marek Ol\9aák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8661>

glapi: Undefine MemoryBarrier

Reviewed-by: Marek Ol\9aák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8661>

anv: Support multiple engines with DRM_IOCTL_I915_GEM_CONTEXT_CREATE_EXT

v2 (Jason Ekstrand):
 - Separate the anv_gem interface from anv_queue internals
 - Rework on top of the new anv_queue_family stuff

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>

anv: Add anv_gem_count_engines

v2 (Jason Ekstrand):
 - Take a drm_i915_query_engine_info

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>

anv: Gather engine info from i915 if available

v2 (Jason Ekstrand):
 - Don't take an anv_physical_device in anv_gem_get_engine_info()
 - Return the engine info from anv_gem_get_engine_info()
 - Free the engine info in anv_physical_device_destroy()

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>

anv: Support i915 query (DRM_IOCTL_I915_QUERY) from Linux v4.17

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>