nir: Add new option to lower invocation ID from invocation index.

Add this as an option to nir_lower_compute_system_values_options
instead of just relying on the shader's options.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir, spirv: Don't mark NV_mesh_shader primitive indices as per-primitive.

They are not per-primitive in NV_mesh_shader, but a flat array.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Fix nir_lower_io with per primitive outputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Print Mesh Shader specific info.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

nir: Rename nir_get_io_vertex_index_src and include per-primitive I/O.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>

ac/rgp: Initialize clock_calibration with memset.

Fix defect reported by Coverity Scan.

Uninitialized scalar variable (UNINIT)
uninit_use_in_call: Using uninitialized value clock_calibration.
Field clock_calibration.reserved is uninitialized when calling
fwrite.

Fixes: 1ee85e8babe ("ac/rgp: add support for clock calibration")
Suggested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13783>

freedreno/a4xx: hook up sample mask/id, used to determine helper invocs

This fixes the various gl_HelperInvocation-based tests. There's a
lowering pass which converts it to (1 << sampleid) & samplemask.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: include guesses from a3xx for some of the constid's

The ones that are untested are left as comments. The ones that rename
values were tested manually.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: indicate whether outputs are uint/sint

Unclear whether this fixes anything, but the blob does seem to set
these. (Discovered while trying to determine if value clamping was
missing for non-32-bit integer formats, which fail in some tests.)

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a4xx: fix stencil-textured border colors

These are implemented with unusual sampler formats, so the usual approach
of looking at the format descriptors fails.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13806>

freedreno/a6xx: re-express buffer textures more logically

Same as a5xx, move one bit into the tex type, one as a separate named
BUFFER bit.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

freedreno/a5xx: re-express buffer textures more logically

Instead of treating it as 2 bits to enable, make BUFFER a type (and
extend the bitfield width), and then add a separate BUFFER bit
(ostensibly to perform the width/height concatenation but who knows).

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

freedreno/a4xx: add proper buffer texture support

Rather than faking it as a 1d texture, add the buffer texture type, and
allow a full range of sizes.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13805>

radeonsi: make si_llvm_emit_clipvertex non-static

it will be used in culling code

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: remove an incorrect comment at lds_byte0_accept_flag

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: improve memory instruction tracking

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: add dcc_msaa option to enable DCC for MSAA

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: unify GFX9_VSGS_NUM_USER_SGPR and GFX9_TESGS_NUM_USER_SGPR

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

ac,radeonsi: cull small lines in the shader using the diamond exit rule

It also splits clip_half_line_width into X and Y components for tighter
view culling.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: add si_state_rasterizer::ngg_cull_flags_lines and rename the others

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: set EXTRA_DX_DY_PRECISION for lines where it's supported

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: set PERPENDICULAR_ENDCAP_ENA for wide AA lines

This is more correct.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: make si_get_small_prim_cull_info static

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: use ac_build_load_to_sgpr in gfx10_emit_ngg_culling_epilogue

This is more correct because we are loading constants into an SGPR even
though there is no effect on behavior in this case.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: fix view culling for wide lines

We need to cull wide lines as quads, but only for view culling.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

radeonsi: fix shader culling with integer pixel centers

Only Nine was using them.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>

nir: remove double-validation of src component counts

The nir_tex_instr_src_size helper already sorts this out correctly, no
need to do it twice, and validate_src takes care of it.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13781>

radv: Add more checking of cache sizes.

Hopefully prevents things.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13789>

radv: Fix memory corruption loading RT pipeline cache entries.

Oops. Forgot to account for the size here.

Fixes: ca2d96db51e ("radv: Add caching for RT pipelines.")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13789>

freedreno/ir3: fix image-to-tex flags, remove 3d -> array hack

The function would return both the 3d and array flags set for 2d array,
and would return just 3d for cubes. Fix the flags so that they are
appropriate for images.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13804>

iris: call brw_process_intel_debug_variable() earlier

We're currently only calling it after creating the screen and the
bufmgr. There are a few cases where Iris checks for the DEBUG_BUFMGR
bit before we call brw_process_intel_debug_variable(), which means
intel_debug is 0 and so we don't run the debug code. Today, these are
all related to the creation of the workaround bo and its mmap.

I found this in a custom branch after I converted to INTEL_DEBUG an
environment variable that I had.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13780>

docs: update branchpoint instructions

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13745>

lima: enable PIPE_CAP_PREFER_POT_ALIGNED_VARYINGS

Mali4x0 PP doesn't have a swizzle for load_input, so use POT-aligned
varyings to avoid unnecessary movs for vec3 and precision downgrade
in case if this vec3 is coordinates for a sampler

shader-db:

total instructions in shared programs: 15707 -> 15623 (-0.53%)
instructions in affected programs: 3906 -> 3822 (-2.15%)
helped: 47
HURT: 18
helped stats (abs) min: 1 max: 9 x̄: 3.09 x̃: 2
helped stats (rel) min: 1.49% max: 23.53% x̄: 8.20% x̃: 6.45%
HURT stats (abs)   min: 1 max: 7 x̄: 3.39 x̃: 3
HURT stats (rel)   min: 0.78% max: 20.59% x̄: 10.45% x̃: 10.97%
95% mean confidence interval for instructions value: -2.18 -0.41
95% mean confidence interval for instructions %-change: -5.70% -0.38%
Instructions are helped.

total spills in shared programs: 146 -> 136 (-6.85%)
spills in affected programs: 39 -> 29 (-25.64%)
helped: 6
HURT: 0

total fills in shared programs: 617 -> 598 (-3.08%)
fills in affected programs: 125 -> 106 (-15.20%)
helped: 6
HURT: 0

HURT shaders are vertex shaders where we may need more instructions
for non-packed vec3s. It's acceptable trade-off since we don't get
precision downgrade if this varying is coordinates for a sampler.

Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13151>

gallium: add PIPE_CAP_PREFER_POT_ALIGNED_VARYINGS

Driver should enable this cap if it prefers varyings to be aligned
to power of two in a slot, i.e. vec4 in .xyzw, vec3 in .xyz, vec2 in .xy
or .zw

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13151>

docs/submittingpatches: mention use of the `-x` flag of `git cherry-pick` when backporting a commit

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13746>

docs/submittingpatches: add formatting around the release branches names

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13746>

docs/submittingpatches: add link to section describing how to make a backport MR

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13746>

freedreno/a6xx: Fix a bunch of 3D texture layout to match blob behavior.

This doesn't get all of the texelfetch sampler3d testcases working, but
it's sure a lot more.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno/cffdump: Handle the TILE_ALL flag in unit test generation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno/cffdump: Fix up formatting of texturator unit test script output.

Now I don't need to re-clang-format as I generate testcases.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno/fdl: Add support for unit testing 3D texture array strides.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno/cffdump: Fix 64-bit reg decode in script mode.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno: Fix the texturator unit test script.

We no longer have reg defs for the HI fields, so all we can access from
lua is the low 32 bits.  LUA has only double-precision floats for numbers,
so we can't fix that.  However, the high bits are almost always the same,
so it's not that big of a deal to be ignoring them for this script.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

freedreno/fdl: Dump the generated layout when a layout test fails.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13733>

util/vl: move gallium vl_vlc.h and vl_rbsp.h to shared code.

For vulkan video I need these to parse slice headers, so move
them somewhere easier to get at them.

drops pointer_to_uintptr in favour of a cast.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13768>

intel/dev: Add platform enum with DG2 G10 & G11

Based on Lionel's "intel/devinfo: store the different kind of DG2".

Ref: 361b3fee3c5 ("intel: move away from booleans to identify platforms")
Ref: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9e22cfc5e9b92556a56d8a564cdab31045f29010
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13797>

zink: always add VK_IMAGE_CREATE_2D_ARRAY_COMPATIBLE_BIT for 3D images

there's no way to know what an image will be used for, so this bit needs
to always be added

fixes KHR-GL46.packed_pixels.varied_rectangle.compressed_rgb

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13798>

zink: stop running discard_if in generated tcs

just embarrassing smh

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13798>

zink: skip one GLES31 subset to avoid GPU hangs on Navi10

Weird bug... I will figure out later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13796>

intel/genxml: generate video headers

This just generates the video engine pieces.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: fix gen6 LD->VLD typo.

Pointed out by Ilia

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: capitalize decoder mode select properly

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: fix Picure->Picture typo

Ilia pointed this out.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: align QM field names across gens.

This just picks a consistent name.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: fix some missing address from the 75 xml

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

intel/genxml: cleanup video xml collisions.

When you enable video genxml, lots of warnings about redefined things
appear, just clean those up before things get started.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13788>

aco: consider pseudo-instructions reading exec in needs_exec_mask()

No matter the format, this should return true if the instruction has an
exec operand.

Otherwise, eliminate_useless_exec_writes_in_block() could remove an exec
write in a block if it's successor begins with:
 s2: %3737:s[8-9] = p_parallelcopy %0:exec
 s2: %0:exec,  s1: %3738:scc = s_wqm_b64 %3737:s[8-9]

Totals from 3 (0.00% of 150170) affected shaders (GFX10.3):
CodeSize: 23184 -> 23204 (+0.09%)
Instrs: 4143 -> 4148 (+0.12%)
Latency: 98379 -> 98382 (+0.00%)
Copies: 172 -> 175 (+1.74%)
Branches: 95 -> 97 (+2.11%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: bc130497472 ("aco: Eliminate useless exec writes in jump threading.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5620
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13776>

radv: use nir_fold_16bit_sampler_conversions()

for now only for texture dest and if there is no rounding mode required.

Totals from 2 (0.00% of 150170) affected shaders: (GFX10.3)
CodeSize: 7980 -> 7948 (-0.40%)
Instrs: 1441 -> 1422 (-1.32%)
Latency: 7703 -> 7626 (-1.00%)
InvThroughput: 2336 -> 2302 (-1.46%)
VClause: 34 -> 36 (+5.88%)
Copies: 57 -> 58 (+1.75%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13592>

aco: implement D16 texture loads

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13592>

aco: workaround GFX9 hardware bug for D16 image instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13592>

aco: add more D16 load/store instructions to RA and validator

This enables correct handling for
buffer_load/store_format_d16_x and
D16 Image instructions.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13592>

nir/fold_16bit_sampler_conversions: skip sparse residency tex instructions

The residency return value mismatches between NIR and Radeon.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13592>

freedreno: caps for clover

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12500>

freedreno: Add PIPE_SHADER_IR_NIR_SERIALIZED support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12500>

a5xx: remove astc srgb workaround logic

This was copied from a4xx, which only needs it on one chip model (A420).

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13782>

zink: update the CI lists for RADV

Lot of GPU hangs fixed lately.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13792>

microsoft/clc: Add a test for arg metadata

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13759>

clc: Use kernel_arg_type_qual string to add const type qualifier to arg metadata

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13759>

v3d,v3dv: move tile size calculation to a common helper

We had this code replicated in 3 places across both drivers.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13790>

ac/spm: fix determining the counter slot

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: e928f475ccf ("ac: add initial SPM support")
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13758>

ac/spm: fix determing the SPM wire

One SPM wire holds two 16-bit counters.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: e928f475ccf ("ac: add initial SPM support")
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13758>

radv: fix a sync issue on GFX9+ by clearing the upload BO fence

If the same cmdbuf is submitted more than once, they were waiting on
the same fence value. Fix this by clearing the value when beginning
a new command buffer.

This might fix spurious GPU hangs, especially on GFX9.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5401
Cc: 21.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13777>

mesa: fix buffer overrun in SavedObj texture obj array

Fixes: 3be42f9ca19d ("mesa: rewrite glPushAttrib/glPopAttrib to get rid of malloc")

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5621

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13773>

llvmpipe: fix compressed image sizes.

VK CTS just added some new tests to write to a compressed image
from a compute shader, which was overrunning memory.

The image width/height need to be sized according to the block
sizes to avoid overwriting memory.

dEQP-VK.image.sample_texture.*bit_compressed*

Cc: mesa-stable
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13618>

llvmpipe: disable 64-bit integer textures.

This fixes some crashes in VK-GL-CTS where it doesn't deal with these.

Cc: mesa-stable
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13617>

aco: Fix how p_is_helper interacts with optimizations.

p_is_helper doesn't have any operands, so ACO's value numbering and/or
the pre-RA optimizer could incorrectly recognize two such instructions
as the same.

This patch adds exec as an operand to p_is_helper in order to achieve
correct behavior.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5570
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13577>

ci/freedreno: Add a link to the issue for color_depth_attachments.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

ci/freedreno: Add notes explaining the KHR-GL* failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

freedreno/a6xx: Do sparse setup of the TFB program.

We don't need to init the whole program RAM, just the locations we are
actually writing from.  Syncs this code up with tu a bit more.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

ci/freedreno: Enable the tes-input/tcs-input tests.

They seem to be mostly passing these days.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

freedreno/ir3: Fix an off-by-one in so->outputs_count safety assert.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

freedreno/a6xx: Add some notes about piglit failures.

Hopefully this helps others save time looking at piglit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13747>

aco: optimize load_local_invocation_index with single-wave workgroups

fossil-db (Sienna Cichlid):
Totals from 668 (0.52% of 128647) affected shaders:
CodeSize: 2201912 -> 2193336 (-0.39%)
Instrs: 403124 -> 402325 (-0.20%)
Latency: 4510940 -> 4510214 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 681057 -> 679453 (-0.24%); split: -0.24%, +0.00%
VClause: 6470 -> 6467 (-0.05%)
SClause: 12759 -> 12755 (-0.03%)
Copies: 26348 -> 26218 (-0.49%); split: -0.50%, +0.00%
PreSGPRs: 26140 -> 26101 (-0.15%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel-schuermann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13757>

radv: lower load_local_invocation_index with 1D workgroups

For 1D workgroups, we can just load from an input VGPR.

fossil-db (Sienna Cichlid):
Totals from 226 (0.18% of 128647) affected shaders:
CodeSize: 1200476 -> 1195696 (-0.40%); split: -0.49%, +0.09%
Instrs: 223817 -> 223328 (-0.22%); split: -0.29%, +0.07%
Latency: 2552394 -> 2549606 (-0.11%); split: -0.15%, +0.04%
InvThroughput: 533989 -> 532670 (-0.25%); split: -0.27%, +0.02%
VClause: 5191 -> 5188 (-0.06%)
SClause: 7637 -> 7636 (-0.01%)
Copies: 18165 -> 18182 (+0.09%); split: -0.22%, +0.31%
Branches: 10446 -> 10442 (-0.04%)
PreSGPRs: 8049 -> 8041 (-0.10%); split: -0.17%, +0.07%
PreVGPRs: 7785 -> 7767 (-0.23%); split: -0.32%, +0.09%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel-schuermann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13757>

nir/lower_system_values: replace local_invocation_id components with zero

fossil-db (Sienna Cichlid):
Totals from 360 (0.28% of 128647) affected shaders:
VGPRs: 7912 -> 7272 (-8.09%); split: -8.59%, +0.51%
CodeSize: 542456 -> 544688 (+0.41%); split: -0.32%, +0.73%
MaxWaves: 10866 -> 10952 (+0.79%)
Instrs: 95973 -> 96010 (+0.04%); split: -0.34%, +0.38%
Latency: 4366023 -> 4344664 (-0.49%); split: -0.90%, +0.41%
InvThroughput: 19656659 -> 18297185 (-6.92%); split: -6.92%, +0.00%
VClause: 3242 -> 3116 (-3.89%); split: -4.04%, +0.15%
SClause: 3422 -> 3504 (+2.40%); split: -0.20%, +2.60%
Copies: 8854 -> 9376 (+5.90%); split: -0.89%, +6.79%
Branches: 2329 -> 2326 (-0.13%); split: -0.39%, +0.26%
PreSGPRs: 7620 -> 7841 (+2.90%); split: -0.43%, +3.33%
PreVGPRs: 5765 -> 5504 (-4.53%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel-schuermann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13757>

tu: Expose Vulkan 1.2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13756>

tu: Add VK_KHR_buffer_device_address stubs

dEQP-VK.api.version_check.entry_points requires us to return a function
pointer, even though the feature is optional in Vulkan 1.2.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13756>

tu: Enable subgroupBroadcastDynamicId

It's a Vulkan 1.2 only feature, but it's trivially supported.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13756>

freedreno/a[345]xx: add R8/RG8 SRGB formats

These enable the GL_EXT_texture_sRGB_R8 / GL_EXT_texture_sRGB_RG8
extensions.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13765>

freedreno: prefer float immediates when float values are involved

Using double immediates can cause a natively-float value to have to get
upgraded to a double unnecessarily. Use float immediates where possible.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13764>

pan/mdg: Remove duplicate compiler option

Noted by clang.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Enable dual texture fusing pass

Everything is in place for it now -- ship it! Our Bifrost cycle model is
coarse, so take the shader-db results with a massive spoonful of salt:

total instructions in shared programs: 107504 -> 107252 (-0.23%)
instructions in affected programs: 39692 -> 39440 (-0.63%)
helped: 191
HURT: 1
helped stats (abs) min: 1.0 max: 20.0 x̄: 1.32 x̃: 1
helped stats (rel) min: 0.11% max: 9.52% x̄: 1.21% x̃: 0.98%
HURT stats (abs)   min: 1.0 max: 1.0 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 2.04% max: 2.04% x̄: 2.04% x̃: 2.04%
95% mean confidence interval for instructions value: -1.60 -1.02
95% mean confidence interval for instructions %-change: -1.37% -1.01%
Instructions are helped.

total tuples in shared programs: 89864 -> 89664 (-0.22%)
tuples in affected programs: 27787 -> 27587 (-0.72%)
helped: 146
HURT: 6
helped stats (abs) min: 1.0 max: 17.0 x̄: 1.54 x̃: 1
helped stats (rel) min: 0.14% max: 15.38% x̄: 1.83% x̃: 1.25%
HURT stats (abs)   min: 1.0 max: 11.0 x̄: 4.17 x̃: 2
HURT stats (rel)   min: 0.54% max: 3.55% x̄: 1.29% x̃: 0.87%
95% mean confidence interval for tuples value: -1.64 -0.99
95% mean confidence interval for tuples %-change: -2.06% -1.36%
Tuples are helped.

total clauses in shared programs: 18253 -> 18044 (-1.15%)
clauses in affected programs: 5127 -> 4918 (-4.08%)
helped: 164
HURT: 1
helped stats (abs) min: 1.0 max: 19.0 x̄: 1.28 x̃: 1
helped stats (rel) min: 0.78% max: 28.57% x̄: 6.73% x̃: 5.88%
HURT stats (abs)   min: 1.0 max: 1.0 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 2.94% max: 2.94% x̄: 2.94% x̃: 2.94%
95% mean confidence interval for clauses value: -1.50 -1.04
95% mean confidence interval for clauses %-change: -7.42% -5.91%
Clauses are helped.

total cycles in shared programs: 8118.54 -> 8103.88 (-0.18%)
cycles in affected programs: 414.96 -> 400.29 (-3.53%)
helped: 43
HURT: 27
helped stats (abs) min: 0.041665999999999315 max: 4.375 x̄: 0.41 x̃: 0
helped stats (rel) min: 0.24% max: 50.00% x̄: 11.49% x̃: 5.26%
HURT stats (abs)   min: 0.041665999999999315 max: 1.1666639999999973 x̄:
0.10 x̃: 0
HURT stats (rel)   min: 0.43% max: 4.71% x̄: 1.42% x̃: 1.28%
95% mean confidence interval for cycles value: -0.35 -0.07
95% mean confidence interval for cycles %-change: -9.50% -3.53%
Cycles are helped.

total arith in shared programs: 3375.67 -> 3376.42 (0.02%)
arith in affected programs: 345.29 -> 346.04 (0.22%)
helped: 24
HURT: 32
helped stats (abs) min: 0.041665999999999315 max: 0.5833329999999997 x̄:
0.09 x̃: 0
helped stats (rel) min: 0.24% max: 14.29% x̄: 2.82% x̃: 1.50%
HURT stats (abs)   min: 0.041665999999999315 max: 1.1666639999999973 x̄:
0.09 x̃: 0
HURT stats (rel)   min: 0.43% max: 4.71% x̄: 1.42% x̃: 1.28%
95% mean confidence interval for arith value: -0.04 0.07
95% mean confidence interval for arith %-change: -1.19% 0.39%
Inconclusive result (value mean confidence interval includes 0).

total texture in shared programs: 1275 -> 1157 (-9.25%)
texture in affected programs: 725.50 -> 607.50 (-16.26%)
helped: 192
HURT: 0
helped stats (abs) min: 0.5 max: 10.0 x̄: 0.61 x̃: 0
helped stats (rel) min: 2.86% max: 50.00% x̄: 25.20% x̃: 25.00%
95% mean confidence interval for texture value: -0.72 -0.51
95% mean confidence interval for texture %-change: -27.12% -23.27%
Texture are helped.

total vary in shared programs: 537.88 -> 536.12 (-0.33%)
vary in affected programs: 2.75 -> 1 (-63.64%)
helped: 1
HURT: 0

total quadwords in shared programs: 79762 -> 79681 (-0.10%)
quadwords in affected programs: 10261 -> 10180 (-0.79%)
helped: 59
HURT: 18
helped stats (abs) min: 1.0 max: 14.0 x̄: 1.88 x̃: 1
helped stats (rel) min: 0.38% max: 8.20% x̄: 1.95% x̃: 1.43%
HURT stats (abs)   min: 1.0 max: 4.0 x̄: 1.67 x̃: 1
HURT stats (rel)   min: 0.46% max: 8.89% x̄: 2.22% x̃: 1.21%
95% mean confidence interval for quadwords value: -1.57 -0.53
95% mean confidence interval for quadwords %-change: -1.59% -0.37%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Test dual texture fusing

These patterns are quite tricky, so let's make sure we're testing
adequately.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Add dual texture fusing pass

Bifrost supports a special "dual texture" instruction, sampling from two
textures at once at the same coordinate. Each subinstruction is highly
restricted (a subset of TEXS_2D); together, they are represented by TEXC
with a special dual texture operation descriptor. Add an optimization
pass to fuse these instructions.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Fix up dual texturing registers

This must be done after RA. How delightful. Use the GenXML strategy to
just OR the birds.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Add bi_dual_tex_as_u32 helper

Type safe cast, making dual texture descriptors easier to manipulate.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Support dual texture scheduling

Teach the scheduler about dual texturing to avoid an artifical "must not
last" constraint causing suboptimal scheduling like

clause_1:
ds(0) nbb tex ncph dwb(0)
{
    *NOP t0
    +TEXC.skip t1, r0, r1, 0xf1e00144, @r4
    *NOP t0
    +NOP t1
}

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Use BIFROST_TEXTURE_OPERATION_SINGLE enum

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Add bifrost_dual_texture_operation struct

This is the other state of the texture operation descriptor. We must
pack it in the compiler when fusing dual texturing.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>

pan/bi: Add bifrost_texture_operation_mode enum

Differentiates single/dual texturing.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13723>