nv50/ir: Convert to new-style NIR registers

Shader-db results on Turing:

    total inst in shared programs : 11121531 -> 11121458 (-0.00%)
    total gpr in shared programs : 1848287 -> 1848425 (0.01%)
    total ugpr in shared programs : 0 -> 0 (0.00%)
    total local in shared programs : 27200 -> 27200 (0.00%)
    total shared in shared programs : 236476 -> 236476 (0.00%)
    total bytes in shared programs : 177944496 -> 177943328 (-0.00%)
    total cached in shared programs : 0 -> 0 (0.00%)

               inst      gpr     ugpr    local   shared    bytes   cached
    helped      470       50        0        0        0      470        0
      hurt      327      197        0        0        0      327        0

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24110>

venus: refactor query feedback cmd record

Now copy and reset are similar enough to unify.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24130>

venus: add a missing barrier before copying query feedback

The 1st sync scope of vkCmdCopyQueryPoolResults is not sufficient to
cover transfer writes against query feedback buffer. We must ensure
ordering against prior query reset cmd where the feedback buffer fill
gets injected.

Fixes: de4593faa193 ("venus: add query pool feedback cmds")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24130>

vulkan/util: Use ycbcr_info for multiplane helpers in vk_format.c

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24096>

vulkan/util: Support VK_EXT_ycbcr_2plane_444_formats color formats in vk_format.c

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24096>

vulkan/util: Support 10-bit and 12-bit color formats in ycbcr_info in vk_format.c

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24096>

venus: ensure consistency of query overflow behavior

Fixes: e6cffa1f0e4e ("venus: use feedback for vkGetQueryPoolResults")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24123>

venus: handle query feedback creation failure

Fixes: e6cffa1f0e4e ("venus: use feedback for vkGetQueryPoolResults")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24123>

radv/ci: Set DRIVER_NAME in LAVA raven vkcts jobs

Some CTS tests work with RADV, but take a very long time, making
deqp-runner trigger timeout failures. These tests are supposed to be
skipped, so they're contained in radv-skips.txt. But without setting
DRIVER_NAME to "radv", deqp-runner.sh won't pick up that file.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24095>

ci: avoid running hardware jobs if lint fails - now on Windows too!

I missed this in !23774.

Fixes: a1c1cce9dfc2d8400a67 ("ci: avoid running hardware jobs if there are already trivial issues")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24121>

ci: avoid running hardware jobs if lint fails - now on LAVA too!

I missed this in !23774.

Fixes: a1c1cce9dfc2d8400a67 ("ci: avoid running hardware jobs if there are already trivial issues")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24121>

zink: Switch to register intrinsics

SPIR-V does not have anything like nir_register natively, so we were already
inserting loads/stores for register sources/destinations. That means it's easy
to switch to register intrinsics, getting explicit load_reg/store_reg intrinsics
in the NIR and translating those to the SPIR-V load/stores, dropping the
handling for nir_register. There's no need to use any of the chasing helpers for
coalescing the load/stores, like a hardware backend would. (In
fact, the underlying Vulkan driver will probably turn this back into SSA.)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24111>

VERSION: bump to 23.3.0-devel

For further development

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24124>

api/icd: drop static lifetime from `get_ref` return type

This was never correct as the object pointed to can be destroyed at any
moment.

Signed-off-by: Karol Herbst <git@karolherbst.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24061>

rusticl/device: make it &'static

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24061>

rusticl: Replace &Arc<Device> with &Device

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24061>

rusticl/kernel: silence newer clippy warning

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24061>

ci: avoid running hardware jobs if there are already trivial issues

Suggested-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Eric Engestrom <eric@igalia.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23774>

freedreno/regs: Document a7xx CP_BV_BR_COUNT_OPS

Fully tested on HW. Credits to Connor Abbott for finding out how
it works.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Properly document a7xx CP_EVENT_WRITE, CP_WAIT_TIMESTAMP

Event write is changes so much in a7xx that it makes sense to
create a new event CP_EVENT_WRITE7.

All credits to Connor Abbott for finding out what different flags
in these commands are doing.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

tu: Use reg usage tables for stale reg dbg option

Defining regs to stomp as ranges in a separate header is a mistake
from maintenance standpoint. Now we have this information at the
point where reg is defined.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

tu: Allow reg stomping of compute related registers

We don't use draw states for dispatches, so the bound pipeline
could be overwritten by reg stomping in a renderpass or blit.

The solution is to re-emit pipeline's IB on every dispatch if
reg stomping is used.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Define usage for all a6xx/a7xx regs

Could be used for knowing which regs to stomp and to verify that
only appropriate regs are emitted.

Each register that is actually being used by driver should have "usage"
defined, currently there are following usages:
- "cmd" - the register is used outside of renderpass and blits,
roughly corresponds to registers used in ib1 for Freedreno
- "rp_blit" - the register is used inside renderpass or blits
(ib2 for Freedreno)

It is expected that register with "cmd" usage may be written into only at
the start of the command buffer (ib1), while "rp_blit" usage indicates that
register is either overwritten by renderpass/blit (ib2) or not used if not
overwritten by a particular renderpass/blit.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Generate per-gen reg usage tables

"reg" and "array" now could have `usage="a,b,c"` attribute, for each
usage a separate array is generated.

Would be used for register stomping debug option.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Fix a7xx SP_FS_PREFETCH definition

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Add more a7xx regs and reg fields

Deduced from a740 cmdtraces from running CTS on prop driver.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Add some new a7xx events

There are many more a7xx events but they are left for later.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Add 2 new a7xx modes to CP_COND_REG_EXEC

Also reworked how CP_COND_REG_EXEC is defined to print
less irrelevant fields.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: a7xx has a new source type CP_REG_TEST

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Add a7xx pseudo-regs to CP_SET_PSEUDO_REG

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Clarify polling on a7xx for CP_WAIT_REG_MEM/CP_COND_WRITE5

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Document a7xx CP_MODIFY_TIMESTAMP

Clears, adds to local, or adds to global timestamp

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Document CP_MEM_TO_SCRATCH_MEM

Best guess is that it is a faster way to fetch all the VSC_STATE registers
and keep them in a local scratch memory instead of fetching every time
when skipping IBs.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Document a7xx CP_FIXED_STRIDE_DRAW_TABLE

Executes an array of fixed-size command buffers where each
buffer is assumed to have one draw call, skipping buffers with
non-visible draw calls.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: More CP commands are the same on a7xx as on a6xx

These ones are seen to be used by blob in CTS, the rest a6xx commands
were not seen beeing used.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Change a7xx regs to have open range for generation

Until proven otherwise regs stay the same between gens.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/rnn: Take into account array's variant for regs

Otherwise even if array only exists in one generation the code for
its registers is generated for all gens.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/rnn: Make addvariant work for fields in the same reg

Previously if addvariant was processed after other fields in the reg
these fields would never get matched. Fix this by moving bitfields that
add variant to the beginning of the list.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/rnn: Fix addvariant being set effectively once

Each time addvariant was added it was added to the end of ctx->vars
list, without previous variant being removed. While the check for
variant tests only the first one that has expected enum name.

Fix this by updating `variant` instead of appending new one if variant
with such enum already exists.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

freedreno/regs: Print xml validation error if validation fails

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23881>

meson: clarify what "off-screen rendering" means

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24114>

meson: clarify description of `opengl` option

There was some confusion from users as to whether disabling this option
disables OpenGL ES as well, so let's remove the confusing "all versions"
note and specify this affects "desktop OpenGL" only.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24113>

anv: limit stack usage for anv_surface_state

Each one is 136 bytes.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24109>

anv: pass anv_surface_state using a pointer

It's 136 bytes, so passing it by stack is wasteful.

CID: 1531860

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24109>

anv: fix how NULL buffer_view is handled in anv_descriptor_set_write_buffer_view

CID: 1531855

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24109>

aco: Add MESA_SHADER_KERNEL to instruction selection setup.

Treat it the same way as MESA_SHADER_COMPUTE.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24112>

broadcom/ci: add the renderonly folder to things that can affect v3d & vc4

Also, move the v3d/vc4 lines together so it's clearer.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24105>

nir/lower_tex: optimize offset lowering for has_texture_scaling

Generates much better code and even helps to beat a blob driver.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24054>

nir: rename has_txs to has_texture_scaling

Convert it to an opt-in for backends to prefer and use nir_load_texture_scale
instead of txs for nir lowerings.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Suggested-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24054>

nir: rename intrinsic to have a more generic nameing

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24054>

etnaviv/ci: drop duplicate line in etnaviv files list

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24106>

nvc0: backport fp helper invocation fix to 2nd gen Maxwell+

Ben prefers that we use the firmware method where possible.

Cc: mesa-stable
Signed-off-by: Karol Herbst <git@karolherbst.de>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23802>

anv: Adds a workaround for HEVC decoding on some old platforms.

HEVC support on Gfx9 is only available on VCS0. So limit the number of video queues
to the first VCS engine instance.

We should be able to query HEVC support from the kernel using the engine query uAPI,
but this appears to be broken : https://gitlab.freedesktop.org/drm/intel/-/issues/8832

When this bug is fixed we should be able to check HEVC support to determine the
correct number of queues.

Closes: mesa/mesa#9172, mesa/mesa#9314

Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24065>

ci: update fails for fixed tests due to llvmpipe linear changes.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

llvmpipe/linear: add support for rgba color buffers.

This adds support to the linear rast for rgba outputs.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

llvmpipe/linear: add support for sampling when cbuf order is different.

This rewrites bgra sampling when the output is rgba,
and vice-versa.

It allows to skip swaps if the sampling and cbuf match.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

llvmpipe/linear: add sample routines for swapping r/b channels

This lets rgba textures be sampled in linear mode.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

llvmpipe/linear/tgsi: calculate num_texs properly for nir.

This is a bit hacky, but it does the right thing and counts the number
of textures instructions so the linear path can work for multiple textures.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

llvmpipe/linear: refactor linear samplers into templated code.

Before adding new copies of all of these for swapping start by
refactoring into macro templated code.

I avoided using inline functions because I want to test with
opts turned down, and this will kill perf.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24066>

freedreno/ci: Update pixmark piano checksums

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

pan/mdg: Ingest new-style registers

Switch to register intrinsics, using the helpers. Since our backend copyprop
chokes on non-SSA moves, we get better coalescing with this approach, hence the
small improvements to instruction count / cycle count in shader-db. Changes to
register pressure seem to be noise from iteration order. I'm not too worried.

   total instructions in shared programs: 1508444 -> 1508193 (-0.02%)
   instructions in affected programs: 42581 -> 42330 (-0.59%)
   helped: 482
   HURT: 41
   Inconclusive result (value mean confidence interval includes 0).

   total bundles in shared programs: 643023 -> 643136 (0.02%)
   bundles in affected programs: 16318 -> 16431 (0.69%)
   helped: 230
   HURT: 85
   Inconclusive result (value mean confidence interval includes 0).

   total quadwords in shared programs: 1125992 -> 1125600 (-0.03%)
   quadwords in affected programs: 125366 -> 124974 (-0.31%)
   helped: 507
   HURT: 351
   Quadwords are helped.

   total registers in shared programs: 90632 -> 90554 (-0.09%)
   registers in affected programs: 669 -> 591 (-11.66%)
   helped: 114
   HURT: 31
   Registers are helped.

   total threads in shared programs: 55607 -> 55600 (-0.01%)
   threads in affected programs: 20 -> 13 (-35.00%)
   helped: 1
   HURT: 7
   Inconclusive result (value mean confidence interval includes 0).

   total spills in shared programs: 1371 -> 1437 (4.81%)
   spills in affected programs: 44 -> 110 (150.00%)
   helped: 0
   HURT: 2

   total fills in shared programs: 5133 -> 5273 (2.73%)
   fills in affected programs: 84 -> 224 (166.67%)
   helped: 0
   HURT: 2

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

gallivm: Switch to reg intrinsics

This is pretty straightforward, since we don't try to "coalesce" register access
the way a GPU backend would. In the old path, we generated register load/store
instructions internally when hitting register sources/destinations. In the new
path, we just translate the register load/store intrinsics to the LLVM
loads/stores and we're back where we started. It's a bit more code, but it's
more straightforward.

Notably, although this continues to use registers, this does NOT use the chasing
helpers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Remove nir_register-based unit tests

Non-SSA functionality will become obsolete after nir_register is removed, so
there's no need to keep the tests around, and they will interfere with the
nir_register de-clawing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Remove nir_lower_regs_to_ssa

It is now unused, as all internal producers of registers have been switched over
to intrinsics and no drivers call it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir/lower_shader_calls: Convert to register intrinsics

Yet another internal use of nir_register that gets lowered back to SSA after the
pass. Easy enough to replace with intrinsic-based registers instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

mesa: Convert PTN to register intrinsics

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

mesa: Return SSA defs from PTN ALU helpers

Mostly a big simplifcation. Some noise on Haswell shader-db:

   total instructions in shared programs: 2978203 -> 2978161 (<.01%)
   instructions in affected programs: 9812 -> 9770 (-0.43%)
   helped: 61
   HURT: 39
   helped stats (abs) min: 1 max: 5 x̄: 1.44 x̃: 1
   helped stats (rel) min: 0.27% max: 7.69% x̄: 1.76% x̃: 1.18%
   HURT stats (abs)   min: 1 max: 4 x̄: 1.18 x̃: 1
   HURT stats (rel)   min: 0.55% max: 16.67% x̄: 4.49% x̃: 3.45%
   95% mean confidence interval for instructions value: -0.71 -0.13
   95% mean confidence interval for instructions %-change: -0.11% 1.46%
   Inconclusive result (%-change mean confidence interval includes 0).

   total cycles in shared programs: 45346214 -> 45346684 (<.01%)
   cycles in affected programs: 519970 -> 520440 (0.09%)
   helped: 157
   HURT: 157
   helped stats (abs) min: 2 max: 2970 x̄: 166.80 x̃: 6
   helped stats (rel) min: 0.05% max: 40.38% x̄: 5.01% x̃: 1.42%
   HURT stats (abs)   min: 2 max: 1922 x̄: 169.80 x̃: 10
   HURT stats (rel)   min: 0.04% max: 44.00% x̄: 6.28% x̃: 2.46%
   95% mean confidence interval for cycles value: -49.93 52.92
   95% mean confidence interval for cycles %-change: -0.49% 1.76%
   Inconclusive result (value mean confidence interval includes 0).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

mesa: Simplify ptn_log() a bit

Using fdiv lets us drop the fneg. nir_opt_algebraic will re-optimize
this if the driver implements fdiv using fmul and frcp.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

gallium: Convert TTN to register intrinsics

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

gallium: Return SSA values from TTN ALU helpers

This is a lot simpler!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add lower_vec_to_regs pass

This is a variant of nir_lower_vec_to_movs that produces register intrinsics
(store_reg with write masks) instead of masked moves.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add intrinsics version of locals_to_regs

This isn't so bad. I still duplicated the pass because it makes a lot easier to
have them coexist, switch users over one by one, and then garbage collect the
old when we're done.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir/from_ssa: Support register intrinsics

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir/from_ssa: Make additional assumptions in coalescing

At this point, everything is SSA.  Also, NIR no longer allows different
numbers of components on the two sides of a phi so we can just assert
rather than trying to gracefully handle mismatches.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Produce intrinsics in lower_{phis,ssa_defs}_to_regs

A number of passes lower SSA partially to registers, do work that would be
invalid in SSA, and then go back into SSA with nir_lower_regs_to_ssa. As a step
towards replacing nir_register with intrinsics,
the nir_lower_{phis,ssa_defs}_to_regs passes are changed to produce intrinsics
instead of nir_registers, and their callers are updated to call
nir_lower_reg_intrinsics_to_ssa instead of nir_lower_regs_to_ssa to compensate.

Jointly authored with Faith.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add a reg_intrinsics flag to nir_convert_from_ssa

It doesn't do anything yet. We leave that to the subsequent patches so we can
keep the tree-wide refactor as simple as possible.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add new version of lower_regs_to_ssa

in the sense of operating on register intrinsics instead of nir_registers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add legacy data structures & helpers

These are registerful versions of core nir_src/nir_dest which will become
SSA-only soon enough, and modifierful versions of nir_alu_src/nir_alu_dest.
The latter will let us remove modifiers from nir_alu_instr finally.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add pass for trivializing register access

After running the pass, all register access intrinsics are guaranteed to be
"trivial" in the sense that the program is free of hazards preventing
propagating them away without inserting any copies.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add helpers for walking register uses

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

nir: Add intrinsics for register access

Note the writemask handling is chosen for consistency with the rest of NIR. In
every other instance, writemask=w requires a vec4 source. This is hardcoded into
nir_validate and nir_print as what it means to have a writemask.

More importantly, consistency with how register writemasks currently work.
nir_print hides it, but r0.w = fneg ssa_1.x is actually a vec4 instruction with
source ssa_1.xxxx. As a silly example nir_dest_num_components(that) = 4 in the
old model. I realize this is quite strange coming from a scalar ISA, but it's
perfectly natural for the class of vec4 hardware for which this was designed. In
that hardware, conceptually all instructions are vec4`, so the sequence "fneg
ssa_1 and write to channel w" is implemented as "fneg a vec4 with ssa_1.x in the
last component and write that vec4 out but mask to write only the w channel".

Isn't this inefficient? It can be. To save power, Midgard has scalar ALUs in
addition to vec4 ALUs. Those details are confined to the backend VLIW scheduler;
the instruction selection is still done as vec4. This mechanism has little in
common with AMD's SALUs. Midgard has a wave size of 1, with special hacks for
derivatives.

As a result, all backends consuming register writemasks are expecting this
pattern of code. Changing the store to take a vec1 instead of a vec4 would
require changing every backend to reswizzle the sources to resurrect the vec4. I
started typing a branch to do this yesterday, but it made a mess of both Midgard
and nir-to-tgsi. Without any good reason to think it'd actually help
performance, I abandoned the idea. Getting all 15 backends converted to the
helpers is enough of a challenge without forcing 10 backends to reswizzle their
sources too.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23089>

ci: split farm rules out of test-source-dep.yml

That file has become a bit of the new `.gitlab-ci.yml` with just about
everything in there, but a lot of its content doesn't need to be in the
same file anymore now that `!reference` exists.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24090>

llvmpipe/linear: don't allow linear path for shader output with location frac

This has been broken for a while, but we weren't hitting the linear paths in CI

This fixes:
tests/spec/arb_enhanced_layouts/execution/component-layout/fs-output.shader_test.

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24083>

ci: fix .valve-farm-manual-rules

28667995e4c3437868ee ("ci: create manual farm rules") was clearly written before
9a8a7aaf1d17c4586ca7 ("ci: split valve farm in two") and landed after but missed
this change when rebasing.

Fixes: 28667995e4c3437868ee ("ci: create manual farm rules")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24097>

nir/opt_dead_cf: Clarify comment

Make it obvious that the comment is about the block stitching behavior
of nir_cf_node_remove.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22064>

nir/opt_dead_cf: Run dead_cf_block while it makes progress

Previously, nir_opt_dead_cf could skip dead CF nodes because overwriting
cur after dead_cf_block is not enough to cover the whole CF list.
foreach_list_typed would select the next node, skipping the node that
previously made progress:

block 1
if (true) {}
block 2
if (true) {}
block 3
if (true) {}

Would turn into:

block 1, then, block 2
if (true) { }
block 3, then

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22064>

nir/lower_shader_calls: Remat derefs after shader calls

This avoids spilling deref instructions by wrapping shader calls inside
dummy blocks, rematerializing derefs in their use blocks and removing
the dummy blocks.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22064>

freedreno/drm/virtio: Trigger host side wait boost

Let the host know that we'll be waiting for a fence via an asynchronous
WAIT_FENCE command.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23955>

anv: fix utrace signaling with Xe

utrace submits can either have a batch or not.

When there is a batch, the utrace vk_sync is signaled by the utrace
batch (because utrace does a timestamp buffer copy using its own
batch). When there is no batch, the utrace vk_sync should be signaled
by the application batch (no timestamp copy required, utrace can read
the timestamps when the application batch has completed).

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: fdea48df5e ("anv: Implement Xe version of anv_queue_exec_locked() and queue_exec_trace()")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24085>

gallium: Remove `PIPE_CAP_RGB_OVERRIDE_DST_ALPHA_BLEND`

Since the mesa state tracker can promote RGB texture formats
to RGBA texture formats (among other formats) without exposing
any of that information to a driver, it is more desirable to
have the behaviour of `PIPE_CAP_RGB_OVERRIDE_DST_ALPHA_BLEND`
be the default. This avoids rendering bugs where an application
sets `DST_ALPHA` blending on a format where there is no alpha
channel, that has been promoted to a format that actually has an
alpha channel. The driver can instead rely on the common code
in the state tracker to convert the blending parameter to one
that reflects the limitations of the application requested format,
as long as `PIPE_CAP_INDEP_BLEND_FUNC` is supported.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24044>

tu: Fix zombie VMAs array not initialized when first BOs may be freed

First BOs were allocated before zombie_vmas was initialized so on
failure their clean up paths used uninitialized zombie_vmas.

Fixes
 dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
 dEQP-VK.api.object_management.alloc_callback_fail.device_group

See https://gitlab.freedesktop.org/mesa/mesa/-/issues/9247

Fixes: 63904240f21b192a5fb1e79046a2c351fbd98ace
("tu: Re-enable bufferDeviceAddressCaptureReplay")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24086>

ci: set priority:low tag only on non-Marge pipelines

This allows dynamically setting the priority to avoid starving Marge.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23976>

ci: document workflow rules

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23976>

radeonsi/gfx11: fix a regression with PAIRS packets due to shader changes

When the vertex shader switches from hw GS to hw HS and vice versa, we need
to re-emit all draw user SGPRs.

Fixes: 1753b321f876 - radeonsi/gfx11: use SET_SH_REG_PAIRS_PACKED for gfx by buffering reg writes

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24087>

amd: Do shader binary alignment for prefetch at memory allocation time.

This makes it consistent between drivers and compilers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23799>

amd: move end-of-code marker padding to ACO.

This makes it consistent between drivers and compilers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23799>

radeonsi: fix gfx9 regression causing GPU hangs

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Fixes: 283be8ac3b8610a77b2 - radeonsi: handle GE_CNTL and IA_MULTI_VGT_PARAM as a tracked register
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2651
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9249
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24082>

etnaviv: linker: clean up etna_link_shader(..)

There is no case that etna_link_shader(..) can fail now.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24030>

etnaviv: linker: handle scenario where there are FS inputs without matching VS output

If there is a FS input but no VS output the behavior is undefined
but okay. Use a register 0 (position) for such cases.

glsl-routing test triggers it with e.g. the following subtest.

Test: VS(C0 -- T0 -- T2 -- T4 T5)
      FS(C0 C1 T0 T1 T2 T3 T4 T5)

This will now end with following linker debug output:

link result:
  vs  -> fs  comps use     pa_attr
  t1  -> t1  xyzw  0,0,0,0 0x000002f1
  t2  -> t2  xyzw  0,0,0,0 0x000002f1
  t0  -> t3  xyzw  0,0,0,0 0x000002f1
  t3  -> t4  xyzw  0,0,0,0 0x000002f1
  t0  -> t5  xyzw  0,0,0,0 0x000002f1
  t4  -> t6  xyzw  0,0,0,0 0x000002f1
  t5  -> t7  xyzw  0,0,0,0 0x000002f1

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24030>

ci/freedreno: another batch of a530 flakes

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24081>