gallivm: Fix LLVMAtomicRMWBinOpFMax spelling

Fixes build with LLVM >= 15.

Fixes: 203920d4c693 ("gallivm: add atomic 32-bit float support")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18701>

ci/nouveau: Update gm20b xfails.

Similar set of skips as gk20a, so we can find any remaining flakes given
the firehose of SSBOs and geom/tess flakes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18674>

ci/nouveau: fix up Jetson Nano

The updated board has a stabilized GPU and now I just need to decide if
I'm building a farm of them or not.  The new firmware flash needs a
reminder to the kernel of how to do NFS (no v2, thanks).  Also, the full
run is long and we need the TEST_PHASE_TIMEOUT variable to go past 20
minutes now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18674>

ci/nouveau: Rearrange job setup variables.

Now there's "generic stuff for nouveau with bare metal", "the two board
types and how to use them", and "the specific jobs for those boards."

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18674>

ci/nouveau: Drop BM_POE_TIMEOUT.

Unused since 5f09b1ebe988 ("ci/bare-metal: Add test phase timeouts to all
boards.").

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18674>

egl: Return EGL_BAD_MATCH for invalid share_list

From the eglspec.1.5: "An EGL_BAD_MATCH is generated if [...]
share context was created on a different display than
the one referenced by config."

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6414
Signed-off-by: Mykhailo Skorokhodov<mykhailo.skorokhodov@globallogic.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16368>

radv: Correct accel struct header size

The size was changed when adding metadata but not updated here.

Fixes: 07eceb4f ("radv: Add metadata to acceleration structures")
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18680>

r300: reduce CPU overhead in IF transformation pass

Right now there is a call to rc_get_variables, which performs a global
analysis of the whole shader, for every IF encountered. As a result,
shaders with a lot of IFs are compiled very slowly. The patological
cases are shaders using relative adressing, where the lowered array
access can result in tens of IFs.

This patch restructures the pass to call the rc_get_variables just once
at the beginning and later reuse the gathered info. We can do this,
because even though we transform the shader in the meantime (like for
example adding extra MOVs) the transformations are not siginificant
enough to influence the relevant variable info we are using.

This reduces CPU time for my shader-db by more than a half. I also
checked that the generated code for all shaders in shader-db is
identical.

Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Filip Gawin <filip@gawin.net>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18678>

radv,aco: do not compact MRTs if the pipeline uses a PS epilog

We can't detect color attachment without exports when compiling a PS
epilog, so we can't compact MRTs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18514>

v3dv: fix VK_EXT_texel_buffer_alignment

This extension was promoted to Vulkan 1.3 so we should be setting its
properties directly in the VkPhysicalDeviceVulkan13Properties struct
which the common mesa code will use to populate outgoing properties.

Apparently, only the properties struct was promoted and not the features
struct.

Reviewed-by: Eric Engestrom <eric@igalia.com>
Tested-by: Eric Engestrom <eric@igalia.com>
Fixes: ee62a4c751c ('v3dv: implement VK_EXT_texel_buffer_alignment')
Fixes: dEQP-VK.api.info.get_physical_device_properties2.properties.basic
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18697>

aco: DCE ra_ctx::defs_done

This was used to distinguish definitions fixed before and during RA, but
it seems it isn't used anymore.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18547>

radv: allow to build the main FS in a graphics pipeline library

Corner cases like implicit gl_PrimitiveID are currently broken and
will be fixed later, but the general case should work.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18516>

radv: do not link shaders when the next stage is unknown

With GPL, it's possible to build the pre-rasterization stages separately
from the fragment stage. Implicit IO (like gl_PrimitiveID) between the
last pre-rast stage and the FS will be addressed later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18516>

nir, anv, hasvk, radv: pull uses_wide_subgroup_intrinsics into shader_info

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18504>

anv: small cleanup of anv_graphics_pipeline_compile

Extract variables for things that are computed multiple times.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18504>

anv: add support for anv_assume_full_subgroups to task & mesh stages

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18504>

nir: add uses_wide_subgroup_intrinsics to task/mesh shader_info

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18504>

radv,aco: lower barycentric_at_sample in NIR

fossils-db (NAVI21):
Totals from 158 (0.12% of 134913) affected shaders:
CodeSize: 569456 -> 568824 (-0.11%)

Only Control seems affected.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18615>

radv: run nir_opt_cse before lowering FS intrinsics

Otherwise, there might be redundant barycentric_at_sample intrinsics
that will be lowered and this will increase code size.

No fossils-db changes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18615>

radv: add nir_intrinsic_load_sample_positions_amd in the ABI

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18615>

nir: add nir_intrinsic_load_sample_positions_amd

This will be used to lower barycentric_at_sample in NIR for RADV.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18615>

ac/surface: Fix some warnings.

../mesa/src/amd/common/ac_surface.c:2324:48: warning: implicit conversion from enumeration type 'AddrResourceType' (aka 'enum _AddrResourceType') to different enumeration type 'enum gfx9_resource_type' [-Wenum-conversion]
   surf->u.gfx9.resource_type = AddrSurfInfoIn.resourceType;
                              ~ ~~~~~~~~~~~~~~~^~~~~~~~~~~~
../mesa/src/amd/common/ac_surface.c:3046:38: warning: implicit conversion from enumeration type 'const enum gfx9_resource_type' to different enumeration type 'AddrResourceType' (aka 'enum _AddrResourceType') [-Wenum-conversion]
   input.resourceType = surf->u.gfx9.resource_type;
                      ~ ~~~~~~~~~~~~~^~~~~~~~~~~~~
../mesa/src/amd/common/ac_surface.c:3069:38: warning: implicit conversion from enumeration type 'const enum gfx9_resource_type' to different enumeration type 'AddrResourceType' (aka 'enum _AddrResourceType') [-Wenum-conversion]
   input.resourceType = surf->u.gfx9.resource_type;

The enums are compatible so lets just add some casts.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18694>

clover: Fixes use of designated initializers requires in c++ that doesn't support by MSVC

../src/gallium/frontends/clover/nir/invocation.cpp(400): error C7555: use of designated initializers requires at least '/std:c++20'

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18683>

v3dv: optimize ldunif load into unifa write

If we emit a ldunif to load the ubo/ssbo base address and
then we are immediately moving it to the unifa register we
can have the ldunif write directly to unifa and avoid the mov
in between, which won't be done by copy propagation because that
only works with temp registers.

Also, since we can't read from unifa we must be careful to disallow
reuse of the ldunif result for a future ldunif of the same base address.
We do that by only reusing ldunif results from temp registers.

total instructions in shared programs: 12468943 -> 12455139 (-0.11%)
instructions in affected programs: 1661233 -> 1647429 (-0.83%)
helped: 8307
HURT: 3994

total uniforms in shared programs: 3704532 -> 3704522 (<.01%)
uniforms in affected programs: 339 -> 329 (-2.95%)
helped: 7
HURT: 0

total max-temps in shared programs: 2148158 -> 2148290 (<.01%)
max-temps in affected programs: 9320 -> 9452 (1.42%)
helped: 175
HURT: 295

total spills in shared programs: 2202 -> 2202 (0.00%)
spills in affected programs: 0 -> 0
helped: 0
HURT: 0

total fills in shared programs: 3059 -> 3057 (-0.07%)
fills in affected programs: 27 -> 25 (-7.41%)
helped: 1
HURT: 0

total sfu-stalls in shared programs: 21167 -> 21056 (-0.52%)
sfu-stalls in affected programs: 497 -> 386 (-22.33%)
helped: 209
HURT: 127

total inst-and-stalls in shared programs: 12490110 -> 12476195 (-0.11%)
inst-and-stalls in affected programs: 1662875 -> 1648960 (-0.84%)
helped: 8312
HURT: 3987

total nops in shared programs: 316563 -> 313553 (-0.95%)
nops in affected programs: 24269 -> 21259 (-12.40%)
helped: 2158
HURT: 1006

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18667>

broadcom/compiler: check signal writes to magic regs when updating scoreboard

We have only been checking magic writes from ADD and MUL ports, but signals
can potentially write to magic registers too.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18667>

broadcom/compiler: detect unifa write from signal

It is possible for some signals to write to unifa directly. We will
enable this from ldunif shortly so we should check for it here.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18667>

radv: Use count_tes_user_sgprs return value.

Fix defect reported by Coverity Scan.

Useless call (USELESS_CALL)
side_effect_free: Calling count_tes_user_sgprs(key) is only useful for
its return value, which is ignored.

Fixes: 8253ec3855a ("radv: add shader arguments for dynamic patch control points")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18659>

radeonsi: print out remove_streamout shader key

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17456>

radeonsi: implement nir_intrinsic_load_streamout_buffer_amd

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17456>

radeonsi: implement nir_intrinsic_load_num_vertices_per_primitive_amd

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17456>

radeonsi: set VS_OUT_MISC_SIDE_BUS_ENA=1 for clip distance exports on gfx10.3

This should improve performance of clip distances.

Reviewed-by: Pierre-eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18639>

anv: remove vk_sample_locations_state from emit_multisample

State for sample locations is not used within this function.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18669>

frontends/va: enable sao in hevc encoding

enable sao feature from config attribute. by default
from vcn2, hevc encoding enables sao.

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18626>

freedreno/a6xx: Simplify fd6_build_user_consts()

Get rid of the table indirects.  Cuts in half the time spent in this fxn
in drawoverhead test 31 ("many uniforms / 1 changemany uniforms / 1
change")

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/a6xx: Move user const upload to bind

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno: Drop unused arg

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/a6xx: Pre-calculate user const state size

We can do this when we construct the program state object, rather than
at draw time.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/a6xx: Skip IBO state when unused

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/drm: Simplify emit_reloc_common

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/a6xx: Drop "hardpin" support

The upstream kernel supported everything needed to stop doing
kernel-side relocs before the first things with a6xx were fully
supported in upstream kernel.  Take advantage to drop some extra
overhead in OUT_RELOC() and equiv in the pack macros.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/drm: Add fd_ringbuffer_attach_bo()

Which does only the bo bookkeeping and skips all the extra legacy reloc
related overhead.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/drm: Inline fd_bo_get_iova()

The struct body was originally hidden to avoid it being part of the ABI
between libdrm_freedreno and mesa.  But that is no longer a problem.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno/ir3: GC unused macro

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

freedreno: Update github wiki links

The github wiki isn't really maintained anymore.  Update references to
point to the gitlab wiki instead.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18646>

radv: Add scratch stack to reduce LDS stack in RT traversal.

The current stack size is a significant limiter for occupancy, and
hence we need smaller stacks in LDS.

Rhys earlier had a patch that just put the N entries closest to the
root in LDS and the rest in scratch. However, this is not ideal for
performance as most of the activity is happening away from the root,
near the leaves. Of course we can't just switch it around, as the
leaf activity likely isn't happening all the way at the end of the
stack.

So what we do is make the LDS stack kinda a ringbuffer by always
accessing it using the stack index modulo the buffer size (always
a power of two so we can efficiently mask). If we then do not have
free space in this buffer we evict the entries closest to the root
to scratch and if we hit the "bottom" of the LDS space we load from
scratch.

Some rough perf numbers for indication with Q2RTX:

| evicting | LDS entries | perf |
|----------|-------------|------|
|       no |          76 |  55% |
|       no |          32 | 100% |
|       no |          24 | 105% |
|      yes |          32 |  95% |
|      yes |          16 | 100% |
|      yes |           8 |  90% |
|      yes |           4 |  75% |

(For the case with 4 entries we need to do some extra accounting as
 a full batch may not be available to evict)

So an obvious choice is to use a stack of 16 entries.

One might wonder if Q2RTX perf is mainly good due to BVHs with very
little geometry and hence low depth, so I also did some profiling
with control. This is done with RGP instruction timing, so this is
instructions executed not weighted for enabled masks, i.e. divergence
effects included.

| game    | LDS entries | scratch action | fraction of iterations |
|---------|-------------|----------------|------------------------|
| Control |           8 |          store |                  10.3% |
| Control |           8 |          load  |                  34.8% |
| Control |          16 |          store |                  0.58% |
| Control |          16 |          load  |                  2.62% |
| Q2RTX   |          16 |          store |                  1.00% |
| Q2RTX   |          16 |          load  |                  3.07% |

So Q2RTX doesn't seem like an unreasonably good case for this
algorithm.

On the implementation side, we can always place the scratch stack at
address 0 by just reserving the scratch space, and in the case of fixed
callstack size moving that up. In the dynamic case the dynamic stack
base already takes any reserved scratch space into account.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18541>

radv: fix dynamic RT stack size with VGPR spilling

VGPR spilling might cause VGPRs to be spilled at scratch offset 0, so we
can't use that.

fossil-db (Sienna Cichlid, Q2RTX and Control):
Totals from 4 (0.26% of 1524) affected shaders:
Instrs: 8734 -> 8737 (+0.03%)
CodeSize: 48492 -> 48504 (+0.02%)
Latency: 384375 -> 384369 (-0.00%)
InvThroughput: 256250 -> 256246 (-0.00%)
Copies: 1312 -> 1313 (+0.08%)
Branches: 256 -> 258 (+0.78%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18541>

docs: add new llvmpipe/lavapipe atomic float extensions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18636>

lavapipe: export VK_KHR_shader_atomic_float

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18636>

llvmpipe: export GL_NV_shader_atomic_float

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18636>

gallivm: add atomic 32-bit float support

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18636>

zink: disable GPL when sample shading is in use

this seems impossible to handle as near as I can tell

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18671>

radv: Use nested ifs for pushing child nodes in traversal loop.

Avoids a bunch of overhead costs if the previous child was empty
already.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18538>

radv: Use constant for ray traversal exit condition.

Make the stack base ssa def dead in the loop, can save a register.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18538>

Revert "radv/rt: use derefs for the traversal stack"

This reverts commit 3750663c72b01d8668c13eed44f40863af588ba4.

Doing things with derefs adds extra instructions for multiplying the
index with the element size, e.g.

BBF0_13:
   s_waitcnt vmcnt(0)
   v_mov_b32_e32 v27, v55
   s_mov_b32 s23, exec_lo
   v_cmpx_ne_i32_e32 -1, v27
   s_cbranch_execz _L14
BBF0_14:
   v_lshlrev_b32_e32 v48, 2, v46  <--
   ds_write_b32 v48, v27
   v_add_nc_u32_e32 v46, 32, v46
_L14:
   s_mov_b32 exec_lo, s23
   v_mov_b32_e32 v27, v54
   s_mov_b32 s23, exec_lo
   v_cmpx_ne_i32_e32 -1, v27
   s_cbranch_execz _L15
BBF0_15:
   v_lshlrev_b32_e32 v48, 2, v46 <--
   ds_write_b32 v48, v27
   v_add_nc_u32_e32 v46, 32, v46

On Q2RTC indirect lighting this saves about 2.3 VALU instructions
per loop iteration, which is ~4% of VALU instructions (we're at
58 per iteration now according to RGP).

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18538>

radv: Store top of stack in a register.

Saves a bunch of processing and a lot of LDS traffic.

Improves perf of the indirect lighting RT pass in Q2RTX by ~3%. This
is mostly due to the -5% VALU instructions and -25% LDS instructions.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18538>

radv: Don't flatten bottom AS exit if statement.

The flattening by ACO is more efficient than the nir condmask.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18538>

fix: frontends/va: unused variable ‘max_pipe_hevc_slices’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: zink: unused variable ‘intr’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: iris: unused variable ‘devinfo’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sfn: unused variable ‘spi_sid’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sfn: variable ‘fail’ set but not used [-Wunused-but-set-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sfn: unused variable ‘{splitpos,param}’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sfn: unused variable ‘splitpos’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sb: warning: unused variable ‘{b,nl}’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sfn: variable ‘opinfo’ set but not used [-Wunused-but-set-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sb: unused variable ‘r’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sb: unused variable ‘fop’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: r600/sb: unused variable ‘repdep2’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: gallivm: variable ‘type_kind’ set but not used [-Wunused-but-set-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: mesa/st: unused variable ‘src’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: mesa/st: variable ‘texobj’ set but not used [-Wunused-but-set-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: mesa: unused variable ‘ret’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: ac/llvm: unused variable ‘offset’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: nir: unused variable ‘else_block’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

fix: util/format: unused variable ‘desc’ [-Wunused-variable]

Only used in debug builds.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18643>

st/drawpixels: use normalized coords in samplers with lowered rects

cc: mesa-stable

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18614>

st/bitmap: use normalized coords in samplers with lowered rects

cc: mesa-stable

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18614>

zink: fix atomic ssbo fadd offsets

Fixes: 99a4a9c6ff0 ("zink: fix atomic ssbo indexing with non-32bit values")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18684>

zink: defer acquire semaphore destruction

these have noticeable overhead, so handle them in the submit thread

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18364>

zink: fix/relax resolve geometry check

there's no requirement in the spec that the geometry for resolves must match,
only that the geometry must be positive (i.e., no flipped extents)

this avoids major perf issues for scaled resolves

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18364>

radeonsi/vcn: Add ability to encode with ltr

reads flags field from CurrPic struct in pps for VA_PICTURE_H264_LONG_TERM_REFERENCE. If found, Curr_pic.frame_idx wil be used for the long term reference index
In get_picture_storage, check if current frame is ltr, and whether its ref frame is ltr.
In radeon_enc_slice_header, adds the ref_pic_list_modification_flag_l0 and long_term_reference_flag for ltr

v2: fix code formatting issues

Reviewed-by: Ruijing Dong ruijing.dong@amd.com
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18219>

pan/bi: Implement unpack_64_2x32

This duplicates the lowering from nir_lower_packing. However, nir_lower_packing
also lowers a pile of other instructions that we do implement natively, and this
is easier than adding a bunch of knobs to nir_lower_packing to get just what we
need.

Fixes test-printf address_space_4.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Stub out scoped_barrier

Implement like other workgroup barriers. No subgroup barriers yet, but that
doesn't seem needed yet.

Fixes test_basic.async_copy_global_to_local and a pile of other OpenCL tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Fix 1D array indexing on Valhall

Array index always goes in the fourth 16-bit component on Valhall. I'm unsure
whether that should also apply to Bifrost. f256ec2a881 ("pan/bi: Fix 1DArray
image coordinate retrieval") says that it should be in the third component on
Bifrost, but I can't remember why that would be the case.

Fixes OpenCL test image_streams.write.1darray on Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Use .auto for image stores

Works around LLVM/SPIR-V stupidity. In effect this means we always use typeless
image stores, which is good enough for both CL and GL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Call nir_lower_64bit_phis

Fixes test_basic.local_kernel_scope

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Scalarize phis before the opt loop

Scalarizing phis results in vector constructions (nir_op_vec) of the same size
as the phi, so a wide phi (>128-bit) will result in a wide vector op that the
backend can't handle. These wide vector ops can always be copypropped away, but
that relies on running NIR copy/prop after scalarizing phis, which was not
always happening before. By scalarizing phis before the opt loop instead of
after, we guarantee that copyprop and DCE run to completion and we get
appropriately lowered code in the backend.

Fixes parts of integer_ops.integer_divideAssign with longs.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Lower fisnormal

Fixes test_bruteforce.isnormal. We don't implement fisnormal in the backend, but
actually lower_bool_to_bitsize was failing earlier since there's no fisnormal32
to lower to either.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Lower <32-bit bit_count

While we have a POPCOUNT.i32 instruction, we do not have v2i16/v4i8 variants.
The code generated by lower_to_bitsize doesn't seem any better than what we
could do ourselves, so let's use that.

While we're at it, give bitfield_reverse the same treatment as we have only
BITREV.i32. I don't think we can get <32-bit bitfield_reverse in either GL or
CL, but that seems likely to change in the future. (It looks to be valid SPIR-V,
at least.)

Fixes integer_ops.popcount.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Handle swizzles in unpack_64_2x32_split_{x,y}

No known fixes but this would still be wrong for OpenCL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Allow selecting from an 8-bit vec8

The word offset is already handled by the above code, there's no need to
restrict the further restrict the swizzle. This pattern can come up with OpenCL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Remove bogus assert for pack_32_2x16

The following IR is valid NIR:

   vec1 16 ssa_0 = ...
   vec1 32 ssa_1 = pack_32_2x16 ssa_0.xx

In this case, pack_32_2x16 takes in a two component vector, but the source
itself ssa_0 has only a single component. This is fine due to the shuffle, but
will fail the assert. Remove the assert and all is well.

Fixes test_relational.shuffle_copy.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Lower f2i8, f2u8

These need a simple two-instruction lowering regardless of the size of float
involved. Fixes integer_ops.integer_divideAssign

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Lower 8-bit min/max to bcsel+comparison

We don't have an 8-bit CSEL, so this is the best we can do. It's easier to write
the lowering as an algebraic rule since we don't need to do anything clever.
Fixes integer_ops.integer_clamp.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/va: Add 8-bit integer max assembler case

This needs to be lowered to a two instruction sequence because there is no
CSEL.v4s8.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Add HADD.v4s8.rhadd packing test cases

To confirm the XML is right.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/va: Pack .rhadd bit

Fixes integer_ops.integer_rhadd.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/bi: Handle uhadd, urhadd opcodes

Fixes integer_ops.integer_hadd.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/va: Fix v4s8 form of R2 opcodes

The XML had a typo which was copypasted (incorrectly) into various instructions.
Fixes a pile of integer_ops subtests.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>

pan/va: Pack IADD.sat bit

Fixes 32-bit portion of integer_ops integer_add_sat.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18656>