radv: fix CmdSetColorWriteEnableEXT(attachmentCount==MAX_RTS)

cc: mesa-stable

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15508>

radv,aco: lower color exports in NIR

fossils-db (Sienna Cichlid):
Totals from 27108 (20.09% of 134913) affected shaders:
VGPRs: 1260608 -> 1261424 (+0.06%); split: -0.00%, +0.07%
CodeSize: 112795868 -> 112785892 (-0.01%); split: -0.05%, +0.04%
MaxWaves: 628608 -> 628448 (-0.03%); split: +0.00%, -0.03%
Instrs: 20750003 -> 20749314 (-0.00%); split: -0.01%, +0.00%
Latency: 288088081 -> 288015865 (-0.03%); split: -0.06%, +0.04%
InvThroughput: 53944847 -> 53961693 (+0.03%); split: -0.01%, +0.04%
VClause: 396463 -> 396467 (+0.00%); split: -0.02%, +0.02%
SClause: 842088 -> 842150 (+0.01%); split: -0.03%, +0.04%
Copies: 1244982 -> 1259026 (+1.13%); split: -0.01%, +1.14%
PreSGPRs: 1251949 -> 1251909 (-0.00%)
PreVGPRs: 1099647 -> 1100879 (+0.11%); split: -0.03%, +0.14%

fossils-db (Polaris10):
Totals from 23928 (17.60% of 135960) affected shaders:
SGPRs: 1751792 -> 1751024 (-0.04%); split: -0.05%, +0.01%
VGPRs: 1098964 -> 1098556 (-0.04%); split: -0.13%, +0.09%
CodeSize: 99893472 -> 99837940 (-0.06%); split: -0.06%, +0.00%
MaxWaves: 138322 -> 138306 (-0.01%); split: +0.03%, -0.04%
Instrs: 19213995 -> 19211980 (-0.01%); split: -0.02%, +0.01%
Latency: 273026926 -> 273109402 (+0.03%); split: -0.01%, +0.04%
InvThroughput: 111160907 -> 111195187 (+0.03%); split: -0.04%, +0.07%
VClause: 343058 -> 343097 (+0.01%); split: -0.02%, +0.03%
SClause: 802756 -> 802884 (+0.02%); split: -0.04%, +0.06%
Copies: 1729387 -> 1739208 (+0.57%); split: -0.04%, +0.61%
PreSGPRs: 1090264 -> 1090303 (+0.00%); split: -0.00%, +0.01%
PreVGPRs: 959490 -> 960600 (+0.12%); split: -0.04%, +0.15%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15263>

vulkan-device-select: Don't leak xcb_query_extension_reply_t

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15600>

broadcom/compiler: allow ldunifa with indirect uniform loads

We handle uniforms by copying them into the uniform stream to be
consumed with ldunif when they have a constant offset. Otherwise
we fallback to general TMU access, which has more latency.

However, just like we did for UBOs and read-only SSBOs, we can
also try to use the unifa mechanism to handle indirect accesses
in certain cases instead of the TMU fallback.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15575>

v3dv: implement VK_EXT_inline_uniform_block

Inline uniform blocks store their contents in pool memory rather
than a separate buffer, and are intended to provide a way in which
some platforms may provide more efficient access to the uniform
data, similar to push constants but with more flexible size
constraints.

We implement these in a similar way as push constants: for constant
access we copy the data in the uniform stream (using the new
QUNIFORM_UNIFORM_UBO_*) enums to identify the inline buffer from
which we need to copy and for indirect access we fallback to
regular UBO access.

Because at NIR level there is no distinction between inline and
regular UBOs and the compiler isn't aware of Vulkan descriptor
sets, we use the UBO index on UBO load intrinsics to identify
inline UBOs, just like we do for push constants. Particularly,
we reserve indices 1..MAX_INLINE_UNIFORM_BUFFERS for this,
however, unlike push constants, inline buffers are accessed
through descriptor sets, and therefore we need to make sure
they are located in the first slots of the UBO descriptor map.
This means we store them in the first MAX_INLINE_UNIFORM_BUFFERS
slots of the map, with regular UBOs always coming after these
slots.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15575>

radv: Add more RT pipeline stubs.

Entry points have to be provided even if the features are not supported.

Helps Doom Eternal.

Fixes: f1095260a41 ("radv: Experimentally enable RT extensions.")
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15573>

radv: Add a vkCmdBuildAccelerationStructuresIndirectKHR stub.

Since this entry point is provided by VK_KHR_acceleration_structure, radv has
to implement it even if it doesn't support the indirect build feature.

Helps Doom Eternal.

Fixes: 82de184c3a2 ("radv: Enable VK_KHR_acceleration_structure with RADV_PERFTEST=rt.")
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15573>

aco: fix RA validation of 16-bit fma_mix operands

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15562>

glsl/nir/linker: fix shader_storage_blocks_write_access

shader_storage_blocks_write_access was computed using the buffer indices
in the program but ShaderStorageBlocksWriteAccess is used with the shader
buffers.

So if a VS had 3 SSBOs and a FS had 4, the mask for VS was 0x3 (correct) but
the mask for the FS was 0x78 instead of 0x15.

Fix this by substracting the index of the first shader buffer in the program's
buffers.

Fixes: 79127f8d5be ("glsl: set ShaderStorageBlocksWriteAccess in the nir linker")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6184
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15552>

glsl/nir/linker: update shader_storage_blocks_write_access for SPIR-V

Most of the code inside the "!prog->data->spirv" blocks shouldn't be
executed for SPIR-V except the part updating the writable mask.

See https://gitlab.freedesktop.org/mesa/mesa/-/issues/6184

Cc: mesa-stable
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15552>

aco: use branch definition as scratch register for SSA lowering

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15505>

zink: use z24_in_z32f support and radv ci updates

This uses the new transfer helper codepath in zink and fixes a bunch of fail on radv.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15516>

u_transfer_helper: add a new option for handling z24 stored in z32

It might be possible to combine this with the other merge to avoid
the overheads of making a temp copy.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15516>

u_transfer: refactor out code to check interleave/deinterleave path.

The checks were reproduced making adding another one not so fun.

rework the deinterleave path code to match the interleave path code.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15516>

util/format: add new z24/s8 packing helper to pack z32/s8.

If zink runs on top of a vulkan impl with no 24-bit float support
it needs support to pack into 24-bit for GL.

To avoid having to make a temp copy, add a new helper to convert
and pack.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15516>

intel/compiler: Use nir_opt_uniform_atomics()

In general, an atomic intrinsic may perform separate atomics for every
enabled SIMD channel, as each channel may operate on different memory.

However, an extremely common case is for all channels to access the same
memory location.  In this case, we can simply perform a reduction/scan
across the subgroup, and perform one atomic for the whole subgroup,
rather than one per channel.  For example, if an intrinsic says to take
the minimum value of the existing memory and the value in each channel,
we can do a thread-local minimum of all enabled channels, then do a
single atomic to take the minimum of that and the existing memory.

Our hardware doesn't optimize the case where multiple channels ask for
atomics on the same memory location; it assumes the compiler will do so.

nir_opt_uniform_atomics() uses divergence analysis to detect this case,
adds the necessary subgroup operations, and moves the atomic inside a
conditional that disables all but a single invocation.  It even detects
cases where the shader code already performs this kind of optimization,
and avoids doing it a second time.

This may not be the optimal solution for us.  In the backend, we could
detect this case and emit send(1) instructions with NoMask, rather than
generating if...send(16)...endif, and a lot of unnecessary ALU ops.  But
it's simple to do, reuses the same path as ACO, and still provides most
of the benefit by cutting up to 16x atomics down to a single atomic,
which is more merciful to the memory bus.

Improves performance of Shadow of the Tomb Raider by 5.5% on XeHP.

Improves performance of a customer-internal benchmark on XeHP at
3840x2160 and low settings by approximately 30%.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>

intel/compiler: Convert to LCSSA and use divergence analysis.

We'll use this more shortly.  For now, enable it to separately in case
anything bisects to this.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>

intel/compiler: Set divergence analysis options

Although we don't use divergence analysis yet, we've had several
work-in-progress series that make use of it.  We may as well set
our options so that those series can assume they're in place.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>

intel/compiler: Implement nir_intrinsic_last_invocation

We haven't exposed this intrinsic as it doesn't directly correspond to
anything in SPIR-V.  However, it's used internally by some NIR passes,
namely nir_opt_uniform_atomics().

We reuse most of the infrastructure in brw_find_live_channel, but with
LZD/ADD instead of FBL.  A new SHADER_OPCODE_FIND_LAST_LIVE_CHANNEL is
like SHADER_OPCODE_FIND_LIVE_CHANNEL but from the other side.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>

nir: Teach nir_divergence_analysis about Intel-specific intrinsics

- load_reloc_const is just an immediate constant load, it's convergent.

- nir_intrinsic_load_global_const_block_intel should be convergent,
  it says the address must be uniform, and we uniformize the predicate

- Lowered image intrinsics: image_deref_load_param_intel just reads
  information about an image, as long as the image variable is
  convergent it should be too.  load_raw_intel...if the address we
  come up with is convergent, it ought to be as well.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>

zink: add another radv flake

literally no idea

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15589>

intel/compiler: Inline TUE map computation into TUE Input lowering

Refactor since the TUE compute function is simpler now and the
comments make sense being near the lowering.

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15022>

intel/compiler: Use nir_var_mem_task_payload

Instead of reusing the in/out slot mechanism, use a separated NIR
variable mode.  This will make easier later to implement staging the
output in shared memory (and storing all at the end to the URB).

Note to get 64-bit type support we currently rely on the
brw_nir_lower_mem_access_bit_sizes() pass.

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15022>

aco: remove 'high' parameter from can_use_opsel()

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15551>

aco/optimizer: fix call to can_use_opsel() in apply_insert()

The definition index is -1.

Fixes: 54292e99c7844500314bfd623469c65adef954c5 ('aco: optimize 32-bit extracts and inserts using SDWA ')
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15551>

wsi/x11: xcb_wait_for_special_event failure is an error

The only ways that function can return NULL are:

- the xcb connection was closed
- the window for the swapchain was destroyed
- the special event listener was unregistered from another thread
- malloc failure

All of these are permanent errors, the swapchain is no longer in a
usable state, so we should treat this as VK_ERROR_SURFACE_LOST_KHR.

Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15558>

pan/va: Lower BLEND to call blend shaders

Do this as late as possible.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add packing unit tests

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add shader-db support

Reports the common subset from Bifrost, as well as Mali offline compiler
style normalized cycle counts.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Implement the cycle model

Will feed into shader-db reporting, and maybe other things eventually.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Test instruction selection lowerings

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Lower branch offsets

Logic is lifted from bi_layout.c, adapted to work on instructions (not
clauses) and for Valhall's off-by-one semantic which is annoyingly
different than Bifrost. (But the same as Midgard -- Bifrost was
annoyingly different than Midgard!)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add instruction selection lowering pass

Valhall removes certain instructions from Bifrost, requiring a canonical
lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add constant lowering pass

Valhall has a lookup table for common constants. Add a pass to take
advantage of it, lowering away immediate indices.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Validate FAU before packing

These are pre-conditions required for packing.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add FAU validation

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add unit tests for ADD_IMM optimizations

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Optimize add with imm to ADD_IMM

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add packing routines

Mostly manual since Valhall is regular.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add helpers for swapping bitwise sources

Annoyingly different from Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Generate header containing enums

We already collect enums in the ISA description XML. Export them for use in the
compiler backend, particularly the packing code.

Usually we'd use Mako for templating. In this case, the script is so trivial a
template engine didn't seem worth it. (The obvious version with Mako was about
10 lines longer than just prints and f-strings used here.)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Suggested-by: Icecream95 <ixn@disroot.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Build opcode info structures

Filled out the new structures from XML.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Permit encoding more flags

Missed the first time around.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Unify flow control

Group together dependency waits and flow control into a single enum. This
simplifies the code, clarifies some detail, and ensures consistency moving
forward.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add Bifrost-style LD_VAR instructions

For use in the legacy non-MALLOC_IDVS flow. Especially useful in blit shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add LD_VAR_BUF instructions

Like LD_VAR_BUF_IMM but indirect.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/va: Add ST_TILE instruction

Encoded like LD_TILE, required for some MSAA blend shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Rename I->action to I->flow

For consistency with the Valhall ISA.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Model LD_VAR_BUF instructions

These are indirect versions of LD_VAR_BUF_IMM, taking their index in bytes. Used
for indirect varying loads (the NIR lowering is inefficient).

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Augment ST_TILE with register format

To model its Valhall incarnation.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Check return addresses in blend shaders

Required on Valhall, where jumping to 0x0 doesn't automatically terminate the
program. Luckily the check is free there too.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Allow branch_offset on BLEND

Required to model BLEND accurately on Valhall, where it encodes a special
relative branch... Midgard style!

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Model Valhall-style A(CMP)XCHG

Handled consistently with computational atomics.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Add ATOM_RETURN pseudo-instruction

Allows modeling Valhall's atomics better.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Rename PATOM_C to ATOM

This is basically what's native on Valhall. Use the Valhall naming for the
pseudo-instruction on Bifrost for consistency.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Gate late DCE/CSE on "optimize"

Otherwise we can end up with unlowered ATOM.i32 on Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

pan/bi: Use consistent modifier lists in packing

If there are modifiers only used by pseudo instructions, not the real
instructions, bi_packer can get out-of-sync with bi_opcodes, causing
hard-to-debug issues. Do the stupid-simple thing to ensure this doesn't happen.

This may be a temporary issue, depending whether ISA.xml and the IR get split
out for better Valhall support.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15223>

nouveau/nir: Enable nir_opt_move/sink.

NIR load_consts/inputs tend to happen together at the top of the program.
In the TGSI backend the loads got emitted at use time, while the NIR
backend was emitting the loads at load intrinsic time.  By sinking the
intrinsics, we can greatly reduce register pressure.

nv92 NIR results:

total local in shared programs: 2024 -> 2020 (-0.20%)
local in affected programs: 4 -> 0
total gpr in shared programs: 790424 -> 735455 (-6.95%)
gpr in affected programs: 215968 -> 160999 (-25.45%)
total instructions in shared programs: 6058339 -> 6051208 (-0.12%)
instructions in affected programs: 410795 -> 403664 (-1.74%)
total bytes in shared programs: 41820104 -> 41660304 (-0.38%)
bytes in affected programs: 7147296 -> 6987496 (-2.24%)

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15542>

pvr: Use vk_common_GetDeviceQueue API.

Removes pvr_GetDeviceQueue implementation. As we are now
using the common vk_queue structure as a base for pvr_queue,
we can use vk_common_GetDeviceQueue implementation instead.

Signed-off-by: Rajnesh Kanwal <rajnesh.kanwal@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15574>

dzn: Compile-test the driver

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766>

microsoft: Initial vulkan-on-12 driver

This is Dozen, the Vulkan on DirectX 12 driver. Not to be confused with
DirectEggs.

This is an early prototype, and not meant to be upstreamed as-is.

Co-Authored-by: Boris Brezillon <boris.brezillon@collabora.com>
Co-Authored-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Co-Authored-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Co-Authored-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766>

microsoft/spirv_to_dxil: Add missing ralloc_free

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766>

vulkan/util: Make STACK_ARRAY() work for arrays of pointers

And add an explicit cast on the pointer returned by malloc() to
make C++ happy.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766>

vulkan/image: Make MSVC C++ compiler happy

Fix 'error C4576: a parenthesized type followed by an initializer
list is a non-standard explicit type conversion syntax' errors by
declaring an actual variable and returning it in
vk_image_view_subresource_range().

All those MSVC/c++ related-constraints are quite annoying to be honest,
but it looks like the D3D12 headers have been updated to plain C
recently, which will allow us to write the driver in C, and hopefully
get all this sort of issues behind us.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766>

zink: add anv cts skips from waiver

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15578>

zink: update radv fails

this should be the exact current baseline

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15576>

pvr: Implement vkCreateSampler and vkDestroySampler APIs.

Implements vkCreateSampler and vkDestroySampler APIs.

Also fixes maxSamplerLodBias value from 15.0f to 16.0f
as it's the max supported by our hardware.

Also changing maxSamplerAnisotropy from 16.0f to 1.0f to
temporarily disable anisotropy as we are missing software
support for it.

Signed-off-by: Rajnesh Kanwal <rajnesh.kanwal@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15557>

vulkan/util: Get rid of VK_OUTARRAY_MAKE()

Get rid of VK_OUTARRAY_MAKE() so people don't get tempted to
use it and produce code that doesn't compile with MSVC, which
doesn't support typeof().

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

vulkan/wsi: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

vulkan/device_select: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

venus: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

panvk: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

anv: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

pvr: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

turnip: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

v3dv: Stop using VK_OUTARRAY_MAKE()

We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>

intel/fs: Fix IsHelperInvocation for the case no discard/demote are used

Use emit_predicate_on_sample_mask() helper that does check where to
get the correct mask depending on whether discard/demote was used or
not.

Fixes: 45f5db5a84a ("intel/fs: Implement "demote to helper invocation"")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15400>

intel/fs: Initialize the sample mask in flags register when using demote

Without this change, a check for "is helper invocation" could read
uninitialized values.

Fixes: 45f5db5a84a ("intel/fs: Implement "demote to helper invocation"")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15400>

nir: Use const for nir_shader_get_entrypoint(..)

nir_shader_get_entrypoint(..) should not modify the passed nir_shader
object. Enforce this by marking shader paramenter as const.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15362>

mesa/st: add special casing for pointsize constant updating during validate

the previous method of using affected_states to trigger constant updates
was ineffectual in the scenario where a ubo pointsize was needed on
the first time a non-precompiled shader was used after being the not-last
vertex stage:

* have vs+gs -> gs precompiles with pointsize lowering -> gs constants get updated
* remove gs -> vs was precompiled without pointsize lowering -> vs constants broken

now just do a quick check as in st_atom_shader.c and set the flag manually to
ensure the update is done correctly every time

cc: mesa-stable

fixes #6207

fixes (radv):
KHR-GL46.texture_cube_map_array.image_op_fragment_sh
KHR-GL46.texture_cube_map_array.sampling
KHR-GL46.texture_cube_map_array.texture_size_fragment_sh
KHR-GL46.constant_expressions*

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15570>

zink: update anv icl ci list

praise be to dynamic state fixes!

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15569>

zink: flush clears before toggling color write

ensure these sync up onto the expected buffers

Fixes: 3892c133811 ("zink: add an alternate path for EXT_color_write_enable usage")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15568>

zink: fix up color_write_enable workaround

this needs to only swizzle to dummy surfaces if it's the workaround,
not just if color_write_enable is active

Fixes: 3892c133811 ("zink: add an alternate path for EXT_color_write_enable usage")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15568>

freedreno/ci: Update a306 expectations

These have started to flakey UnexpectedPass somewhere along the way.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

pipe-loader: Try loading freedreno for virtgpu device

Freedreno will check if the virtgpu supports the pass-thru context, and
if not will bail, falling back to virgl.

TODO this requires that virgl is also enabled in the mesa build, even if
it is not needed.. maybe there is a better way to handle this?

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Add virtio backend

Add a new backend to enable using native driver in a VM guest, via a new
virtgpu context type which (indirectly) makes host kernel interface
available in guest and handles the details of mapping buffers to guest,
etc.

Note that fence-fd's are currently a bit awkward, in that they get
signaled by the guest kernel driver (drm/virtio) once virglrenderer in
the host has processed the execbuf, not when host kernel has signaled
the submit fence.  For passing buffers to the host (virtio-wl) the egl
context in virglrenderer is used to create a fence on the host side.
But use of out-fence-fd's in guest could have slightly unexpected
results.  For this reason we limit all submitqueues to default priority
(so they cannot be preepmted by host egl context).  AFAICT virgl and
venus have a similar problem, which will eventually be solveable once we
have RESOURCE_CREATE_SYNC.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Reorder device destroy

Call backend specific cleanup fxn earlier.  This is needed if the
backend has things like bo's to delete, otherwise the handle_table
will already be destroyed causing problems in bo_del()

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Extract out "softpin" submit/ringbuffer base class

We are going to want basically the identical thing, other than
flush_submit_list, for virtio backend.  Now that we've moved various
other dependencies into the base classes, extract out an abstract base
class for submit/ringbuffer.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Move ring_pool slab parent to base

Prep to move most of sp submit/ringbuffer to something that can be
re-used by virtio backend.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Move bo idx to base

The virtio backend will want this too, and it will make it easier to
share most of the submit/ringbuffer implementation with the virtio
backend.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Move submit_queue to base

The virtio backend will want this too.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Avoid CPU_PREP ioctl if bo is idle

With userspace fences, if we know definitely that the buffer is idle
(which implies that it is not shared with other processes, etc), then
skip the ioctl.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Add fd_bo_upload()

There are some buffers that we mmap just to write to them a single time.
Add the possibility of the drm backend to provide an alternate upload
path to avoid these mmap's.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Add FD_BO_SHARED hint

With the virtio backend we will need to pass an extra flag when
allocating buffers that will be shared cross-device (such as with
virtio-wl for passing between host and guest)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Add FD_BO_NOMAP hint

Add a hint for buffers that we won't need to mmap.  With the virtio
backend, virglrenderer needs to create a dmabuf fd for mapping into
the host, which we want to avoid when possible.

Low hanging fruit is to use this hint for anything tiled/ubwc.  There
are probably more bo's that can be flagged as such.

TODO add fd_bo_upload() for memcpy to bo.. this would be useful for
uploads, for example, shaders which we just write once and never touch
again.. for virtio this could be implemented with a TRANSFER_TO_HOST
ioctl.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Rework bo creation path

Decoupling handle and fd_bo creation simplifies things for "normal" drm
drivers, avoiding duplication for the create vs import paths.  But this
is awkward for the virtio backend when wants to do multiple things in
the same guest<->host round trip.

So instead, split the paths in the interface backend and move the code
sharing for the two different paths into the msm backend itself.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Add fd_device_open() helper

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

freedreno/drm: Split msm backend into subdir

Let's keep things a bit better organized when we add a new backend.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14900>

zink: update radv ci

bunch of these I just removed by mistake

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15567>

venus: update protocol to remove redundant decoders

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15537>