aco: Stylistic changes to emit_gfx10_wave64_bpermute.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20293>

aco: Split opcodes for GFX6 and GFX10 emulated bpermute.

Different sequences are emitted for these, so it makes sense to
have different opcodes too.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20293>

aco: Don't accept constants on p_bpermute.

The sequence emitted for this pseudo instruction is not ready
to handle constants or literals at all.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20293>

ci/venus: add a VKCTS mapping test to the flakes list

Seen on https://gitlab.freedesktop.org/mesa/mesa/-/jobs/33483156.

Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20312>

iris: Enable compression for image load/store in more cases

We were calling iris_resource_texture_aux_usage here, which disables
auxiliary support if color happens to already be resolved.  This makes
sense for read only images, where if we know ahead of time that aux
doesn't contain any useful information, we can just tell the hardware
to not bother looking at it.  However, it makes no sense for mutable
images, as even if the aux currently has no useful data, we want to
produce that data when doing our image writes.

Import the bits of logic we need from there and shed the rest.  We don't
need to consider HiZ, MCS, or MC, nor do we need to do format-based
CCS compatibility checks on Gfx12+, so it's actually very little code.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Allow fast clears on compressed image load/store access

While I haven't found documentation saying definitively that HDC
supports fast clear blocks, it seems to work just fine, even on
Tigerlake.  I have found several issues (atomics and HDC support
for linear compression) that both call out fast clears as an issue
in those corner cases, which suggests that fast clears do actually
work outside of those corners (which we already disable).

The previous commit implemented actual aux state updates for image
views.  With ISL_AUX_USAGE_GFX12_CCS_E, this means that we update
the aux state to COMPRESSED_CLEAR after writes.  But because we
weren't supporting fast clears, this meant that any such images
would need partial resolves to remove the clear color on next use.
Supporting fast clears allows us to drop all these resolves.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Update aux state tracking for image views after draws/dispatches

On Tigerlake and later, we enable compression for image views.  However,
we never actually added any code to update the aux state, which meant
that if it ever changed, things would break, badly.

We managed to avoid catastrophic effects in most cases because of
two other issues which papered over the problem: if compression wasn't
already enabled for an image, we'd leave it disabled.  And, we avoided
writing via the CPU to buffers with auxiliary.  So in most cases, CCS
remained disabled, or got enabled (say by glTexImage()) then stayed on
permanently.  There were still issues, but they managed to remain more
hidden than one would expect given the severity of the bug.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Drop disable_rb_aux_buffer handling for image views

The goal here is to support OpenGL 4.6 section 9.3, "Feedback Loops
Between Textures and the Framebuffer" (from GL_ARB_texture_barrier)
where you can bind an image as both a framebuffer attachment and a
texture, and simultaneously sample-from and render-to it.

I'm not aware of any spec language that requires us to handle
simultaneously accessing something as a framebuffer attachment and an
image load/store resource.  GL_ARB_shader_image_load_store tends to
make flushing and synchronization something the app has to handle
explicitly rather than something the driver needs to do implicitly.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Drop 'isl_' prefix from 'formats_are_fast_clear_compatible'

Every time I see this function I think it's part of isl.  But it's not,
it's just a static function in an iris file.  The point of the name was
that the function checks two isl_format enums...but the prefix is just
confusing.  Just drop the prefix as it's a static function.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Pin the clear color BO in use_image()

Images with the RC_CCS modifier store the clear color in a separate BO,
which we also need to pin when using an image view.

Most images store the clear color in the same BO so it works anyway.

Thanks to Nanley Chery for catching this!

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

iris: Drop batch parameter from iris_update_postdraw_resolve_tracking

Eventually the resolve code started making everything take ice instead
of batch, and at some point this ceased to be used.  It's always render.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19060>

zink: Fix reversed cap declarations for ImageBuffer

Fixes validation fails on
KHR-GLES31.core.texture_buffer.texture_buffer_texture_buffer_range.

Fixes: f55a4407ef97 ("zink: more accurately set {Sampled,Image}Buffer caps")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20308>

radv/ci: bump most jobs to the kernel to 6.1 + latest firmwares

Unfortunately, not all jobs can be using Linux 6.1 right now, as
NAVI10 hits __vm_enough_memory errors then hangs in VKCTS. So for
this job, we will keep Linux 5.17 until this gets fixed.

Reference: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7888
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16835>

anv: assert when number of primitives is higher than max

Such cases can lead to memory corruptions.

Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20279>

anv: handle mesh shaders with max primitives == 0

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20279>

radv: disable more NIR opts in radv_postprocess_nir() with DISABLE_OPTIMIZATIONS

To make fast-linking with GPL hopefully a bit faster.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20244>

radv: move a conditional check to radv_remove_color_exports()

Better to have all restrictions inside the function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20244>

radv: advertise VK_AMD_shader_early_and_late_fragment_tests

Pass all dEQP-VK.*early_and_late* tests on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19738>

radv: implement AMD_shader_early_and_late_fragment_tests

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19738>

spirv: add support for AMD_shader_early_and_late_fragment_tests

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19738>

radeonsi/vcn: add support for 10bit input and enc 8bit output

This change is to support 10bit YUV input in addition to
original H264/HEVC 8bit output case. It adds
rvcn_enc_input_format_t and rvcn_enc_output_format_t for
picture input format and output format separately.

Signed-off-by: David (Ming Qiang) Wu <David.Wu3@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20284>

nir: Eliminate nir_op_i2b

There are a lot of optimizations in opt_algebraic that match ('ine', a,
0), but there are almost none that match i2b.  Instead of adding a huge
pile of additional patterns (including variations that include both ine
and i2b), always lower i2b to a != 0.

At this point in the series, it should be impossible for anything to
generate i2b, so there /should not/ be any changes.

The failing test on d3d12 is a pre-existing bug that is triggered by
this change.  I talked to Jesse about it, and, after some analysis, he
suggested just adding it to the list of known failures.

v2: Don't rematerialize i2b instructions in dxil_nir_lower_x2b.

v3: Don't rematerialize i2b instructions in zink_nir_algebraic.py.

v4: Fix zink-on-TGL CI failures by calling nir_opt_algebraic after
nir_lower_doubles makes progress.  The latter can generate b2i
instructions, but nir_lower_int64 can't handle them (anymore).

v5: Add back most of the hunk at line 2125 of nir_opt_algebraic.py. I
had accidentally removed the f2b(bf2(x)) optimization.

v6: Just eliminate the i2b instruction.

v7: Remove missed i2b32 in midgard_compile.c. Remove (now unused)
emit_alu_i2orf2_b1 function from sfn_instr_alu.cpp. Previously this
function was still used. :shrug:

No shader-db changes on any Intel platform.

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 141165875 -> 141165873 (-0.0%)
Instructions helped: 2

Cycles in all programs: 9098956382 -> 9098956350 (-0.0%)
Cycles helped: 2

The two Vulkan shaders are helped because of the "new" (('b2i32',
('ine', ('ubfe', a, b, 1), 0)), ('ubfe', a, b, 1)) algebraic pattern.

Acked-by: Jesse Natalie <jenatali@microsoft.com> [earlier version]
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev> [earlier version]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/builder: Handle i2b conversions specially in nir_type_convert

The shaders affected here are ones that were previously affected when
i2b was unconditionally lowered in opt_algebraic. There are a few places
where some transformations happen in a different order, so some
algebraic patterns are missed.

All Broadwell and newer Intel platforms had similar results. (Ice Lake shown)
total instructions in shared programs: 19914369 -> 19914566 (<.01%)
instructions in affected programs: 92375 -> 92572 (0.21%)
helped: 0 / HURT: 90

total cycles in shared programs: 853851470 -> 853867215 (<.01%)
cycles in affected programs: 12400663 -> 12416408 (0.13%)
helped: 28 / HURT: 69

Haswell and Ivy Bridge had similar results. (Haswell shown)
total instructions in shared programs: 16710721 -> 16710700 (<.01%)
instructions in affected programs: 108010 -> 107989 (-0.02%)
helped: 57 / HURT: 103

total cycles in shared programs: 884299412 -> 884306546 (<.01%)
cycles in affected programs: 12986423 -> 12993557 (0.05%)
helped: 87 / HURT: 102

total spills in shared programs: 14937 -> 14925 (-0.08%)
spills in affected programs: 12 -> 0
helped: 9 / HURT: 0

total fills in shared programs: 17569 -> 17557 (-0.07%)
fills in affected programs: 12 -> 0
helped: 9 / HURT: 0

Sandy Bridge
total instructions in shared programs: 13902341 -> 13902347 (<.01%)
instructions in affected programs: 7311 -> 7317 (0.08%)
helped: 3 / HURT: 8

total cycles in shared programs: 741795500 -> 741792266 (<.01%)
cycles in affected programs: 273308 -> 270074 (-1.18%)
helped: 9 / HURT: 2

No shader-db changes on any other Intel platform.

No fossil-db changes on any Intel platform.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

intel/fs: Use nir_type_convert instead of nir_type_conversion_op

In a future commit, nit_type_conversion_op won't be able to handle i2b
(and in a much later commit f2b), so switch many users to the fully
featured function.

No shader-db or fossil-db changes on any Intel platform.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

microsoft/compiler: Use nir_type_convert instead of nir_type_conversion_op

In a future commit, nit_type_conversion_op won't be able to handle i2b
(and in a much later commit f2b), so switch many users to the fully
featured function.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

spirv: Use nir_type_convert instead of nir_type_conversion_op

In a future commit, nit_type_conversion_op won't be able to handle i2b
(and in a much later commit f2b), so switch many users to the fully
featured function.

No shader-db or fossil-db changes on any Intel platform.

v2: Use the actual bit size of the source to determine the conversion
op.  With mediump, the "planned" bit size and the actual bit size might
be different.  Fixes many, many Vulkan CTS assertion failures on any
platform that sets mediump_16bit_alu (e.g., Freedreno).

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> [v1]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir: Use nir_type_convert instead of nir_type_conversion_op

In a future commit, nit_type_conversion_op won't be able to handle i2b
(and in a much later commit f2b), so switch many users to the fully
featured function.

No shader-db or fossil-db changes on any Intel platform.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

glsl: Use nir_type_convert instead of nir_type_conversion_op

In a future commit, nit_type_conversion_op won't be able to handle i2b
(and in a much later commit f2b), so switch many users to the fully
featured function.

In gl_nir_lower_packed_varyings, all of the type conversions are between
int32 and uint32 types.  In NIR, those are just moves, so elide them.

No shader-db or fossil-db changes on any Intel platform.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/builder: Add rounding mode parameter to nir_type_convert

Later changes will use this.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

glsl_to_nir: Fix NIR bit-size of ir_triop_bitfield_extract and ir_quadop_bitfield_insert

Previously these would return result->bit_size of 32 even though the
type might have been int16_t or uint16_t.  This prevents many assertion
failures in "glsl: Use nir_type_convert instead of
nir_type_conversion_op" on zink.

Fixes: 5e922fbc160 ("glsl_to_nir: fix bitfield_extract with 16-bit operands")
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

microsoft/compiler: Simplify nir_intrinsic_load_front_face handling

It is invalid to have Boolean variables as either shader inputs or
outputs, so there is no point to try to lower them in general.  The only
use for this was some two-phase lowering of
nir_intrinsic_load_front_face that could be done in a single phase.
Create the SYSTEM_VALUE_FRONT_FACE as a uint and compare it with zero at
the same time.

No shader-db or fossil-db changes on any Intel platform.

v2: Remove dxil_nir_lower_bool_input from dxil_nir.h and drop it from
the other caller in the spirv_to_dxil codepath.  Noticed by Jesse.  Fix
setting bit size when loading SYSTEM_VALUE_FRONT_FACE.  Caught by CI.

v3: Use nir_ine_imm.  Change type of gl_FrontFacing GS output in
d3d12_nir_passes from Boolean to integer.  Both suggested by Jesse.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/builder: Emit x != 0 for nir_i2b

There are a lot of optimizations in opt_algebraic that match ('ine', a,
0), but there are almost none that match i2b.  Instead of adding a huge
pile of additional patterns (including variation that include both ine
and i2b), just emit a != 0 instead of i2b(a).

I think that the changes to the unit tests weaken them slightly, but
perhaps that's okay?

No shader-db changes on any Intel platform.  The GLSL paths use other
means to generate i2b operations, but the SPIR-V paths use nir_i2b.
Presumably since 4676b3d3dd9 (nir: Use nir_test_mask instead of
i2b(iand)), no fossil-db changes either.

v2: Use nir_ine_imm.  Suggested by Jesse.

Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir: Use nir_i2b wrapper everywhere instead of using nir_i2b1 directly

No shader-db or fossil-db changes on any Intel platform.

v2: Add missed i2b1 in ir3_nir_opt_preamble.c.

v3: Add missed i2b1 in ac_nir_lower_ngg.c.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Optimize some b2i involved in masking operations

v2: Remove the ineg from the b2i in the ior pattern.  Suggested by
Jason.

All Ivy Bridge and newer Intel platforms had similar results. (Ice Lake shown)
total instructions in shared programs: 19914441 -> 19914369 (<.01%)
instructions in affected programs: 63507 -> 63435 (-0.11%)
helped: 24 / HURT: 0

total cycles in shared programs: 853869766 -> 853851470 (<.01%)
cycles in affected programs: 10551542 -> 10533246 (-0.17%)
helped: 24 / HURT: 0

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 141163061 -> 141092683 (-0.0%)
Instructions helped: 14103
Instructions hurt: 55

Cycles in all programs: 9132376195 -> 9133183045 (+0.0%)
Cycles helped: 13775
Cycles hurt: 380

Spills in all programs: 18286 -> 18284 (-0.0%)
Spills helped: 1

Fills in all programs: 30647 -> 30643 (-0.0%)
Fills helped: 1

Gained: 133
Lost: 130

Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Eliminate unary op on src of integer comparison w/ zero

This helps because it enables cmod propagation to do more.

The removed patterns involving b2i will be handled by other existing
patterns after the unary operations are removed.

All Intel platforms had similar results. (Ice Lake shown)
total instructions in shared programs: 19914458 -> 19914441 (<.01%)
instructions in affected programs: 5456 -> 5439 (-0.31%)
helped: 17 / HURT: 0

total cycles in shared programs: 855302118 -> 853869766 (-0.17%)
cycles in affected programs: 327354347 -> 325921995 (-0.44%)
helped: 291 / HURT: 81

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 141205979 -> 141205961 (-0.0%)
Instructions helped: 4
Instructions hurt: 3

SENDs in all programs: 7466919 -> 7466913 (-0.0%)
SENDs helped: 1

Cycles in all programs: 9133387327 -> 9133384475 (-0.0%)
Cycles helped: 3
Cycles hurt: 12

In the shader that was helped for sends, it appears that a NIR pass that
moves code out of loops was able to move 3 send operations outside a
loop after this change.  I did not investigate further.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Simplify min and max of b2i

This prevents ~400 shader-db regresssions and a handful of fossil-db
regressions after i2b is always lowered.

All Ivy Bridge and newer Intel platforms had similar results. (Ice Lake shown)
total cycles in shared programs: 855301494 -> 855302118 (<.01%)
cycles in affected programs: 52787 -> 53411 (1.18%)
helped: 4 / HURT: 5

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 141206055 -> 141205979 (-0.0%)
Instructions helped: 14

Cycles in all programs: 9133376616 -> 9133387327 (+0.0%)
Cycles helped: 13
Cycles hurt: 3

Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Reassociate some iand to eliminate an operation

No shader-db changes on any Intel platform.

All of the helped shaders were presumably regressed by 4676b3d3dd9 (nir:
Use nir_test_mask instead of i2b(iand)).

v2: Add some comments explaining why specific replacements are used.  In
the umin pattern, only markup the first usage of 'b' in the source
pattern.

Tiger Lake, Ice Lake, and Skylake had similar results. (Ice Lake shown)
Instructions in all programs: 141384970 -> 141200966 (-0.1%)
Instructions helped: 45842

Cycles in all programs: 9133648977 -> 9133282672 (-0.0%)
Cycles helped: 26812
Cycles hurt: 6025

Gained: 23
Lost: 135

Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Remove redundant i2b(b2i(x)) patterns

A loop below already adds all the permutations... including the 1-bit
version that isn't included in this group.

No shader-db or fossil-db changes on any Intel platform.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Remove redundant i2b(-x) pattern

The exact same pattern appears later (around line 1323).

No shader-db or fossil-db changes on any Intel platform.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

nir/algebraic: Catch some kinds of copy-and-paste bugs in algebraic patterns

A later commit adds a pattern

   (('umin', ('iand', a, '#b(is_pos_power_of_two)'),
             ('iand', c, '#b(is_pos_power_of_two)')),
    ('iand', ('iand', a, b), ('iand', c, b))),

When I originally made that pattern, I copied and pasted the search to
the replacement as

  (('umin', ('iand', a, '#b(is_pos_power_of_two)'),
            ('iand', c, '#b(is_pos_power_of_two)')),
   ('iand', ('iand', a, '#b(is_pos_power_of_two)'),
            ('iand', c, '#b(is_pos_power_of_two)'))),

The caused the variables in the replacement to be marked is_constant,
and that resulted in an assertion failure deep inside nir_search.

    src/compiler/nir/nir_search.c:530: construct_value: Assertion `!var->is_constant' failed.

These extra validation rules catch this kind of error at compile time
rather than at run time.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>

gallium/pp: typedef and use pp_st_invalidate_state_func to avoid cast

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20042>

gallium/hud: typedef and use hud_st_invalidate_state_func to avoid cast

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20042>

intel: Allow CCS_E on R11G11B10_FLOAT for TGL+

We now support blorp_copy with this format.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19937>

intel/isl: Bump format_info entries from 100 to 110

The new format support is only tested on Ice Lake and onward. Makes the
next patch clearer.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19937>

intel/isl: Lessen CCS_E-compatibility checks for TGL+

Tiger Lake and onward allow drivers to specify a compression format
independently from the surface format. So, even if the surface format
changes, hardware is still able to determine how to access the CCS.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19937>

intel: Hook up RENDER_SURFACE_STATE::DecompressInL3

The sampler's decompressor seems to lack support for some types of
format re-interpretation. Use the more capable decompressor for these
cases. This will be needed to avoid regressing piglit's
arb_texture_view-rendering-formats in later commits.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19937>

v3dv: skip some invalid tests

Cc: mesa-stable
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20305>

asahi: unbind samplers and fix sampler_count if state is NULL

When states is NULL, unbind samplers (to avoid dangling pointers) and
set sampler_count to the highest non-null samplers[] entry instead of
setting it to 0.

This is ported from a similar fix in panfrost:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20285

Signed-off-by: Iago López Galeiras <iaguis@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20306>

radv/rt: Handle no-null shader flags

If those flags are set, we can assume that idx is not 0.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20286>

nir: validate that store_buffer_amd doesn't use a non-trivial writemask

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

ac/llvm: implement ACCESS_USE_FORMAT_AMD as buffer_load/store_format

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

ac/llvm: unify load_buffer_amd and store_buffer_amd code

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

nir: add ACCESS_USES_FORMAT_AMD for typed buffer opcodes

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

nir: replace IS_SWIZZLED flag with ACCESS_IS_SWIZZLED_AMD

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

nir: remove redundant SLC_AMD in favor of ACCESS_STREAM_CACHE_POLICY

ACCESS_STREAM_CACHE_POLICY was added to map to SLC for AMD.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

nir: add nir_texop_sampler_descriptor_amd

We'll use it to query the min/mag filter in the shader.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>

anv: move functions around to plan for generated draws

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

genxml: add gen12/12.5 CS prefetch disable bits

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: move draw command helpers to their own file

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: move total_batch_size to anv_batch

We'll want 2 batches :

   * the main one

   * another to contain dispatch commands to generate stuff in the
     main batch

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: add helper to get current address of a batch

And allow the function to get the very first address in the batch.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: add a utility function to ensure command buffer space

Ensuring the space is contiguous.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: document utrace field

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv: Use extended parameters on Gen11+

Gen11 added a nifty feature where we have three custom system-generated
values called extended parameters that we can set to any 32-bit values
we want.  These work just like vertex and instance ID and are controlled
in the pipeline by the 3DSTATE_SGVS_2 packet.  They are provided to the
draw call either by extra DWORDs on the end of 3DSTATE_PRIMITIVE or by
storing values to more state registers.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

intel/blorp: Emit a dummy 3DSTATE_VF_SGVS_2

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

intel/genxml: Add 3DPRIMITIVE_EXTENDED opcodes on Gen11+

On Gen11 and above, the 3DPRIMITIVE command takes an optional additional
three DWORDs of data as "extended parameters".  These extended
parameters only exist in the packet if "Extended Parameters Present" is
set.  Because our packing code doesn't handle variable-length commands
well, this commit adds a second version of the command which isn't real
but is just a copy of 3DPRIMITIVE with the additional dwords where the
"Extended Parameters Present" defaults to true and "DWord Length" is
adjusted by 3 as needed.  The 3DPRIMITIVE command is then the gen4-9
version which still works fine but doesn't have the new parameters.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

intel/genxml: Make some fields Booleans

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

anv/utrace: only record the emitted pipecontrol flags

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

intel/ds: trace a couple of more pipe control flags

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20295>

cso: fix a regression from draw_vbo rework

Fixes: c9b13a93385 - cso: remove cso_draw_vbo from all draws, call the driver or u_vbuf directly

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20288>

panfrost: fix sampler_count and unbind samplers in bind_sampler_states

1. Old approach did not support unbind (set to NULL) samplers because
it only copied memory if sampler is not empty. New approach checks
if sampler is empty - it will set NULL.

2. Old approach just set sampler_count to 0 if sampler is empty.
That's wrong and we need to find highest non-null samplers[] entry.
It was done in new approach.

3. Gallium dosc says:
```
NOTE: at this time, start is always zero ...
This may change in the future.
```
It's better to take into consideration start parameter in new approach.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20285>

anv: disable Wa_1806565034 when robustImageAccess is enabled

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5711
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7859
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20280>

intel/fs: make Wa_1806565034 conditional to non robust access

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20280>

isl: make Wa_1806565034 conditional to non robust access

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20280>

frontends/va: fix crash during grayscale rendering

crash is observed as format is assumed to be yuv and buffer plane
order is undefined for monochrome. luma only format is not to be
considered yuv format. It has to be rendered with rgb=rrr.

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20165>

radeonsi/vcn: fix regression in yuv422 jpeg decode

- yuv422 decode was blocked in 12acee17fad5, enable it back.
- nv12 yuv422 and grayscale decode is supported on all versions.
- JPEG2 and higher versions supports 444p decode.
- add l8_unorm to supported formats, can be used for grayscale.

Fixes: 12acee17fad5 (frontends/va: reallocate surface for yuv400/yuv444 picture)

v2: indent the switch case correctly (Thong Thai)

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20165>

radv: do not remove the PointSize built-in for polygon mode as points

Determine if it can be removed when generating the graphics pipeline
key because of dynamic states.

Fixes new CTS dEQP-VK.rasterization.polygon_as_large_points.* and also
spec@!opengl 1.1@polygon-offset with Zink/RADV.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20073>

intel/compiler/mesh: extract emit_urb_direct_vec4_write

No functional changes.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

intel/compiler: optimize away local_inv_index and local_inv_id if workgroup size is 1

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

intel/compiler: split lower_cs_intrinsics_convert_block

No functional changes.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

intel/compiler/mesh: extract shared code for offset adjustment

No functional changes.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

anv: simplify EXT_mesh_shader lowering

No functional changes.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

anv,intel/compiler/mesh: drop lowering of gl_Primitive*IndicesEXT

Until U888X index format lands this change shouldn't have any impact on performance.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20292>

intel: fix the wrong function name

pipe not create_vertex_elements fuction

Signed-off-by: cheyang <cheyang@bytedance.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20291>

anv: fixup descriptor copies

I did not properly understood that we cannot access the views written
to the descriptor sets because they might have been destroyed after
the write operation and the copy operation is allowed to copy what is
invalid data. The shader just can't access it.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 03e1e19246 ("anv: Refactor descriptor copy")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20222>

r600: Lower all bitfield ops on pre-EG

Fixes:  4dff3ff005b47befd3e4a903b08d5b4bdbef6ae3
    nir/opt_algebraic: Optimize open coded bfm.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20283>

r600/sfn: Use variable length DOT on Evergreen and Cayman

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20283>

r600/sfn: prepare alu split for use of variable length dot

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20283>

r600/sfn: check used channels when evaluating allowed mask

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7878

v2: Fix ws in comment (iorn10)

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20283>

ci/test-vk: uprev vkd3d

This commit uprevs vkd3d, which flags
test_primitive_restart_list_topology_stream_output as being a flake on
RADV / NAVI21.

Suggested-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20278>

ac/nir: add ac_nir_lower_legacy_gs

For legacy (non-NGG) GS to lower outputs to memory stores and add
shader query when required.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>

ac/nir: move ngg_gs_shader_query to a common function

To be shared by NGG GS and legacy GS. Legacy GS need this when
GFX10 which mix use NGG and legacy GS. For example when streamout
is enabled, it uses legacy GS, otherwise uses NGG GS. So legacy
GS also need to update query emulation which is a sum of NGG and
legacy GS results.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>

nir,ac/llvm,radv: add stream id index to nir_load_ring_gsvs_amd

For used by legacy GS to store output to different ring according
to stream id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>

ac/llvm,radv,radeonsi: pass instruction to intrinsic_load abi

For simple intrinsic which also need other fields to translate
to LLVM like stream_id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>

nir: add nir_load_ring_gs2vs_offset_amd

Used by legacy GS output lowering.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>

ac/nir,radeonsi: replace llvm ngg gs invocation query with nir

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>

nir,radv,radeonsi: add nir_atomic_add_gs_invocation_count_amd

For shader query emulation.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>

freedreno/registers: Add some docs for PM4 events.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19733>

doc/freedreno: Add a bunch of docs of the hardware and drivers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19733>

freedreno: Clarify that CACHE_FLUSH is pre-5xx.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19733>