panfrost: Assume that nir_tex_instr::dest_type is sized

Get rid of some now-redundant code, and cleanup the is-float check in
the bifrost compiler.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

nir/lower_tex: Assume that nir_tex_instr::dest_type is sized

This reverts the code back to the form it was before, but with an
explicitly sized float32 instead of float, now that all producers are
switched over.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

nir: Validate nir_tex_instr::dest_type bitsize

In theory, we could also verify this against the sampler type for
sampler derefs, but there are a number of complications there:

- SPIR-V 1.4 lets you override the signedness of integer samplers
per-instruction. So the base type may not match.
- mediump/RelaxedPrecision samplers may get lowered to f16 in the
instruction or may not. So the bitsize may not match.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

ptn: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

vtn: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

glsl/nir: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

st/atifs: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

ttn: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

gallium/nir: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

st/mesa: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

nir: Use sized types for nir_tex_instr::dest_type

Revieweeviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

d3d12/blit: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

panfrost/blit: Use sized types for nir_tex_instr::dest_type

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

dxil: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

anv: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

intel/blorp: Use sized types for nir_tex_instr::dest_type

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

v3dv/meta: Use sized types for nir_tex_instr::dest_type

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

radv/meta: Use sized types for nir_tex_instr::dest_type

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

brw/vec4: Don't convert tex dest type to glsl_type

We were using nir_tex_instr::dest_type to a glsl_type, then passing it
to emit_texture(), only to just check the number of components. Just
pass the number of components directly. This lets us delete
brw_glsl_base_type_for_nir_type, which was asserting with
nir_texop_all_samples_equal because it didn't handle bool32.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

nir/lower_bool: Rewrite dest_type for boolean destinations

This happens with nir_texop_samples_identical, and we need to keep
things consistent and (soon) keep the validator happy when expanding
booleans once we switch that to having a dest_type of bool1.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

ntt: Handle sized tex destination types

I believe this code doesn't handle 16-bit destination types so we only
need to handle float32, etc.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

freedreno/ir3: Handle sized tex destination types

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

nir/lower_tex: Handle sized tex destination types

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>

intel: silence unused var warnings in release builds

v2: Use ASSERTED

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4162
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8681>

vk-device-select: add device group support

This just sorts CPU device groups last in case an app tries to
use the first device group.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8551>

lima/ppir: fix creation of mov node for non-ssa tex dest

In ppir when a texture node has only a single successor, it is used
directly to output the texture lookup value, in order to save the
insertion of a mov.
However, a sequence like this can happen:

  r0 = (float)tex r8 (coord), 0 (texture), 0 (sampler)
  r1 = mov r0.z

In this case, even if the mov is a single successor, the assumption
that only the elements needed by the successor node cannot be made.
The target register can also be read or written elsewhere and so the
simplification cannot be made. Add an exception to cover this case.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8517>

zink: fix streamout for clipdistance

the assert added here during review broke this since clipdistance is 8 components

Fixes: 9eec52c67e6 ("zink: tweak xfb slot mapping in ntv")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8673>

ci/freedreno/a6xx: Skip vs-output-array-vec2-index-wr-before-gs

This one test seems to be destroying the world, and causing massive
flakeyness, and is generally more a recovery/GMU stress test than
anything.

See #4159

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>

freedreno: Add perf_warn() for missed UBWC opportunities

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>

freedreno/a6xx: Add helper to check if UBWC is supported

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>

freedreno: Add fmt/args macros for pipe_resource

Deduplicate some long debug prints before adding more.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>

freedreno: Add perf_warn() trace helper

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>

glx: Provide glvnd wrapper for glXSwapIntervalEXT

When using glvnd, this function needs to be exposed through
getDispatchAddress or libglvnd will not find it.

Signed-off-by: Keith Packard <keithp@keithp.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Fixes: 60ebeb4608a8 "glx: Implement GLX_EXT_swap_control for DRI2 and DRI3"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8621>

vc4/ci: Replace expect script by python script

Replace the expect-based script to turn on/off the Raspberry Pi devices
using a python-based script.

v2:
 - Fix small nitpicks (Juan)
 - Limit line length (Andres)

v3:
 - Bump image tags (Eric, Andres)

v4:
 - Bump image tags (Eric)

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Acked-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8362>

spirv: handle NoContraction in GLSL450 alu ops

we were dropping this when it was set, leading to incorrect algebraic
optimizations that broke various types of tests, e.g., running
spec@arb_gpu_shader5@execution@precise@fs-fract-of-nan in zink

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6116>

zink: don't force a renderpass start when setting framebuffer state

this breaks compute and isn't strictly necessary since we'll be starting
a renderpass during draw anyway; we just need to flush here to update the
state

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8662>

zink: add function for waiting on a specific batch's fence

previously we only had zink_fence_wait(), which just waits on the
current batch to finish, but it may be the case that we don't want to
wait on all batches up to that point, so we can optimize a bit by only
waiting as long as we have to

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8662>

gallium/u_inlines: add helper for simplifying pipe_context::resource_copy_region

this hook has too many params, and I only need like half of them

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8624>

zink: enable PIPE_CAP_SAMPLER_VIEW_TARGET

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8629>

virgl: add support for VIRGL_CAP_V2_UNTYPED_RESOURCE

An untyped resource is a blob resource that contains only raw bytes
without type information (e.g., width, height, format, etc.).  virgl
supports only typed resources, and when it encounters untyped resources,
it fails silently in the host.

This cap enables virgl to assign type information to untyped resources.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-By: Isaac Bosompem <mrisaacb@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8584>

virgl: update headers

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-By: Isaac Bosompem <mrisaacb@google.com>
Acked-By: Gurchetan Singh <gurchetansingh@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8584>

docs/zink: add GL 4.2 requirements

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: add GL 4.1 requirements

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: add GL 4.0 requirements

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: fix phrasing of GL 3.3 requirements

These requirements aren't all about VkPhysicalDeviceFeatures, so let's
make the text reflect that.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: document the independentBlend requirement for GL3

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: add two missing required features

If these features aren't supported, we'll start doing illegal stuff, so
let's document it.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

docs/zink: add missing colon

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8648>

nv50/ir: Initialize RegAlloc member func in constructor.

Fix defect reported by Coverity Scan.

Uninitialized pointer field (UNINIT_CTOR)
uninit_member: Non-static class member func is not initialized in
this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8568>

zink: clone shader before lowering clip_halfz

If we don't clone the shader before lowering clip_halfz, we risk ending
up performing the same lowering multiple times, each time we compile a
new variant.

This fixes rendering in Neverball.

Fixes: 15f478fe840 ("zink: only run nir_lower_clip_halfz for last vertex processing stage")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4147
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8652>

zink: check correct caps for PIPE_CAP_IMAGE_LOAD_FORMATTED

we actually need more features here than just the one which was checked to
enabled this

Fixes: 2f6f4b613c3 ("zink: export shader image caps using features")

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8656>

ci: Enable process isolation for softpipe & freedreno piglit jobs

Disabling process isolation causes a random set of tests to be
spuriously skipped. The set of skipped tests can change when piglit is
rebuilt (even from the same Git commit), which can make docker image
rebuilds painful. (Not to mention the reduced testing coverage due to
the skipped tests)

One downside of this change is that the arm64_a630_piglit_shader job
now takes almost 10 minutes.

v2:
* Change arm64_a530_piglit_shader job as well, but make it run manually
  on branches of forked repositories only, since it takes almost 20
  minutes now. (Eric Anholt)

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8611>

anv: Advertise shaderInt64 on Gen11+

On Gen11, they took away our hardware int64 support.  We have lowering
for all of it in NIR except for subgroup ops.  Now that all the subgroup
ops are implemented, we can enable the feature.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Implement umin/umax shuffle

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Refactor our shuffle emit code

This adds an emit_scan_step helper which gives us a place to do
something a bit more interesting than emitting a single op.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Properly lower 64-bit MUL on 64-bit-incapable platforms

There are two problems this commit solves:  First, is that the 64x64 MUL
lowering generates a Q MOV which, because of how late it runs in the
compile pipeline, it never gets removed.  Second, it generates 32x32
MULs and we have to run it a second time to lower those.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Support 64-bit CLUSTER_BROADCAST on Gen11+

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Support 64-bit SHUFFLE on Gen11+

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: Support 64-bit SEL_EXEC on Gen11+

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/fs: QUAD_SWIZZLE requires packed data

We could probably support some strides if we tried hard enough but the
whole point of this opcode is to accelerate things with crazy Align16 or
crazy regions.  It's ok if we have to emit an extra MOV to get a packed
source.

Fixes: 8b4a5e641bc3 "intel/fs: Add support for subgroup quad operations"
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/reg,fs: Handle immediates properly in subscript()

Just returning the original type isn't what we want in basically any
case.  Mask and shift the immediate as needed.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/compiler: Move brw_reg_type_for_bit_size to brw_reg_type.h

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

intel/compiler: Return 1 for immediates in regs_read

Previously, we were returning 2 whenever the source was a Q type.  As
far as I can tell, the only reason why this hasn't blown up before is
that it was only ever used for VGRFs until the SWSB pass landed which
uses it for everything.  This wasn't a problem because Q types generally
aren't a thing on TGL.  However, they are for a small handful of
instructions.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

nir/lower_int64: Lower 64-bit vote_ieq

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

nir/lower_int64: Add lowering for 64-bit iadd shuffle/reduce

Lowering iadd is a bit trickier because we have to deal with potential
overflow but it's still not bad to do in NIR.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

nir/lower_int64: Add lowering for some 64-bit subgroup ops

These are all pretty trivial because we can just split the op into one
subgroup op per half of the value.  There's some question as to whether
these belong in lower_int64 or lower_subgroups but, on Intel, they key
decider of whether or not we need the lowering is based on whether or
not we have hardware int64 support.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

nir/lower_int64: Fix lowering of f2[ui]64 for 16-bit float

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

nir/lower_int64: Add a level of wrapper functions

We're about to start lowering a few intrinsics so we need support more
than just ALU.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7329>

radv: only decompress the depth/stencil aspect that needs to be resolved

For depth/stencil images, the driver was decompressing both aspects
while it should be enough to only decompress the one that's going
to be resolved.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8561>

util: When building 'ARM64EC', don't use x64 intrinsics which need to be emulated

ARM64EC is a new build target for Windows ARM64 devices for x64 support.
These binaries can be loaded in x64 processes, but don't need to be emulated. For
code that's heavily used, avoiding the emulation can be a huge perf win.

Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8619>

main: Undefine MemoryBarrier for Windows

In winnt.h, MemoryBarrier can be defined in one of 4 ways, depending
on which architecture is being targeted.
- For x86, it's an inline function.
- For x64, it's an object-like macro, which means that the MemoryBarrier
  function in the table actually ends up being called __faststorefence.
- For arm and arm64, it's a function-like macro, and the preprocessor fails
  because the function table entry doesn't call it.

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8485>

anv: Early-exit from cmd_buffer_flush_state

If we don't have any dynamic state, pipeline, or descriptor changes,
we can do a very quick early-exit instead of checking for a bunch of
stuff bit-by-bit.

Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8594>

anv: Only flush descriptors used by the pipeline

Previously, if we had a pipeline transition from something which used,
say, tessellation to something which didn't and we ended up with
tessellation descriptors dirty, we could end up re-emitting far more
than necessary.  With this commit, we mask off unused stages so we only
update when necessary.

Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8594>

anv: Take the set of stages to flush in flush_descriptor_sets

Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8594>

anv: Exit early from cmd_buffer_apply_pipe_flushes

Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8594>

radv: use a workgroup size of 8x8 for FMASK color expand

Probably better for access patterns.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8642>

radv: use the range aspect mask in FMASK color expand

Instead of harcoding it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8642>

radv: add multi-layer support to FMASK color expand

For better performance for layered MSAA images.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8642>

radeonsi: iterate from draw 1 for total/min_direct_count computation

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: enable accidentally disabled fast launch with non-indexed tri strips

Only *indexed* triangle strips hang.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: skip some code for ALLOW_PRIM_DISCARD_CS if tess or GS is enabled

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: rename SI_SGPR_RW_BUFFERS to SI_SGPR_INTERNAL_BINDINGS

They are just internal buffers and images.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: move if (sctx->vertex_buffers_dirty) into the upload function

This looks unnecessary, but the next commit will build upon it and add
more stuff into the function.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: don't set vertex buffer dirty flags when they don't do anything

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: move y_inverted out of si_viewports

for better packing

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: inline si_blend_color and si_clip_state structures

better packing

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: simplify determining whether render condition is enabled at draw time

Read one bool instead of reading one bool and one pointer.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: add internal blitter_running flag

to skip the indirection in si_decompress_textures

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: don't use rasterizer_discard to validate draws, only check ps_shader

Let's just say that PS can never be NULL here.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: optimize translating index_size to index_type

gcc generated a lookup table for the switch. This replaces it with
arithmetics.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: don't mark NULL states as dirty in si_pm4_reset_emitted

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: clear dirty_states if si_pm4_bind_state is unbinding or no-op

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radeonsi: add new possibly faster command submission helpers

This decreases the release libgallium_dri.so size without debug symbols
by 16384 bytes. The CPU time spent in si_emit_draw_packets decreased
from 4.5% to 4.1% in viewperf13/catia/plane01.

The previous code did:
    cs->current.buf[cs->current.cdw++] = ...;
    cs->current.buf[cs->current.cdw++] = ...;
    cs->current.buf[cs->current.cdw++] = ...;
    cs->current.buf[cs->current.cdw++] = ...;

The new code does:
    unsigned num = cs->current.cdw;
    uint32_t *buf = cs->current.buf;
    buf[num++] = ...;
    buf[num++] = ...;
    buf[num++] = ...;
    buf[num++] = ...;
    cs->current.cdw = num;

The code is the same (radeon_emit is redefined as a macro) except that
all set and emit functions must be surrounded by radeon_begin(cs) and
radeon_end().

radeon_packets_added() returns whether there has been any new packets added
since radeon_begin.

radeon_end_update_context_roll(sctx) sets sctx->context_roll = true
if there has been any new packets added since radeon_begin.

For now, the "cs" parameter is intentionally unused in radeon_emit and
radeon_emit_array.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8653>

radv: fix separate depth/stencil layout in render pass

We used to select the stencil layout even if we should have selected
the depth/stencil one.

Fixes: e4c8491bdf8 ("radv: implement VK_KHR_separate_depth_stencil_layouts")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8552>

zink: respect fragment-shader depth-layout

This is required by GLSL 1.30 and later, so we should also respect the
setting.

Fixes: 6785d8c4601 ("zink: expose GLSL 1.30")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8644>

zink: make all xfb caps depend on extension

Without this, we'll expose GL_ARB_transform_feedback2 and
GL_ARB_transform_feedback3 even without VK_EXT_transform_feedback,
because these caps are directly wired up without checking the pervious
extensions.

Fixes: e8ad52f7b0c ("zink: enable xfb extension in screen creation")
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8640>

zink: require vulkan memory model for tesselation

We enable the KHR_vulkan_memory_model extension whenever we use
tesselation, so right now this is a defacto requirement. So let's make
this requirement explicit.

Fixes: f815b87e188 ("zink: export tess shader pipe caps")
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8640>

zink: check for extension instead of function

This shouldn't change any behavior, it just seems a bit more to the
point to check for the extension that decides if these are NULL or not
than to check both of them.

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8640>

zink: respect feature-cap for multi-draw indirect

Even in the presence of VK_KHR_draw_indirect_count, we still technically
need to respect the feature-cap when using a Vk 1.0 core-function.

Fixes: cef876910a0 ("zink: enable PIPE_CAP_MULTI_DRAW_INDIRECT(_PARAMS) caps")
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8640>

zink: respect feature-cap for sample-shading

Vulkan has a cap to enable this, we should check that one rather than
always claiming support.

Fixes: 0c70268ff73 ("zink: mark ARB_sample_shading as supported")
Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8640>