util/disk_cache: fix crash in fossilize_db

We don't need to close the file here check_files_opened_successfully()
already does that for us if needed.

This fixes a crash when invalid filename/paths are provided to the
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS environment variable.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9251>

zink: rename zink_context::buffers -> vertex_buffers (and usage mask)

this is more descriptive

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9272>

zink: use dynamic vertex buffer strides

this removes another case of needing to rehash/create a new pipeline

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9272>

zink: make dynamic state usage in pipeline creation more explicit/flexible

we'll be expanding this, so we want to have it very easy to update and read

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9272>

zink: setup CmdBindVertexBuffers2EXT member in screen for dynamic state

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9272>

nouveau_vieux: use align_calloc for the context to fix m32 crashes

Fixes: 3175b63a0dfa290 "mesa: don't allocate matrices with malloc"

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9220>

radeon,r200: use align_calloc for the context to fix m32 crashes

Fixes: 3175b63a0dfa290 "mesa: don't allocate matrices with malloc"

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9220>

i915: use align_calloc for the context to fix m32 crashes

Fixes: 3175b63a0dfa290 "mesa: don't allocate matrices with malloc"
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4295

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9220>

zink: set VK_IMAGE_LAYOUT_PRESENT_SRC_KHR on fb resources at eof flush

this should maybe fix garbled images on amd?

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: handle VK_IMAGE_LAYOUT_PRESENT_SRC_KHR barriers

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: handle PIPE_FLUSH_DEFERRED

these flushes queue the fence to be submitted on the next finish call, so
we have to store the context to the fence to be able to handle threaded calls

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: add util function for returning previous batch

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: add a define for compute batch count

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: only flush batches in pipe_context::flush if they actually have work

no need to submit command buffers to the queue which have no commands, so we
can just proceed to give back an inactive fence object that always returns success

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: fix destroy batch

Extract batch destruction in its own function, then make sure to
release objects.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: create a single fence per batch on startup and then reuse

previously we would be creating a new fence per batch for every submit,
but we can just reset and reuse them if we're careful with it

based on original patch by Antonio Caggiano

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9243>

zink: reorder zink_bind_vertex_buffers()

we should be making this a no-op if we have no bindings (somehow)

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9241>

zink: move samplerview referencing around in update_descriptors()

no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9241>

zink: avoid memset during update_descriptors() for resources refs

we can simplify this a bit to further reduce cpu usage here

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9241>

lavapipe: fix depth texturing swizzle

This is pretty much derived from what works and passes
both GL via zink and VK-GL-CTS test suites.

The zink multisample regressions tests rendered black, so
"passed" before, now they render more junk.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9260>

freedreno/a5xx+: Drop the unused no_decode_srgb flag.

This was a flag used for a4xx GMEM draws.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9267>

freedreno/a5xx+: Stop recompiling on texture samples changes.

The shader only looks at this bit of the key on a3xx, and a4xx already
didn't set it.  This will help give us precompile hits for MSAA texture
sampling.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9267>

freedreno: Use the mesa/st frontend lowering of GL_CLAMP.

350 lines of code for this stupid feature, and we weren't even doing it
right for CS/GS/tess.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9267>

zink: ci updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9115>

zink: handle point sprite

this needs custom yinvert handling and also prim mode detection, which
gallium can't do on its own

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9115>

zink: store prim mode to context during draw

we need to be able to access this for shader rewrites

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9115>

nir/texcoord_replace: add a yinvert param

vulkan needs to invert the y coord in order to handle PIPE_SPRITE_COORD_LOWER_LEFT

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9115>

zink: no-op descriptor updating for draws without descriptors

this is a valid case that we can trivially shortcut to cut down on calls

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9155>

freedreno: Drop custom driver lowering of GL's color clamping.

The mesa/st frontend can do it for us now that we don't need to worry
about breaking precompiles.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8997>

freedreno: Drop custom driver lowering of two-sided color.

The GL frontend can do it for us now, so just use their code instead of
our own shader variants.  In the past we had to do hide the GL shader
variants in the driver to get precompiles from st, but no longer as of
!8601.

I tested with drawoverhead -test 6 (shader program change, n=30) and -test
1 (no statechanges, n=43) and saw no change in driver overhead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8997>

freedreno: Remove uniform variables after finalizing NIR.

mesa/st optimizes the uniform storage if you have the finalize hook in
place, causing the uniforms declared to potentially not have storage in
the ParameterValues list any more.  If you leave your uniforms around in
the NIR, then a later finalization after variant creation will re-add the
uniforms to parameters, defeating the optimization and likely reallocating
the uniform storage (causing use-after-free).  So, we have to do this
before we can start using variants in mesa/st.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8997>

mesa: Always make sure uniform storage doesn't get reallocated.

Every caller of associate_uniform_storage was doing this to safety-check
that the uniform storage didn't get reallocated, except for
st_deserialise_ir_program().  This ended up leaving an opening for
use-after-free without hitting the assert in the hot-cache case (and I
found it on freedreno).  Having added it, it also reveals use-after-frees
in the drawpixels shader variant cases on llvmpipe and zink.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8997>

android: mesa: Move the FXT1 compressor/decompressor to util/

After commit a46b73e in order to build mesa for Android
we need to generate $(intermediates)/util/format/u_format_pack.h
instead of $(intermediates)/format/u_format_pack.h

Fixes the following building error in Android build:

external/mesa/src/util/format/u_format_fxt1.c:28:10: fatal error: 'util/format/u_format_pack.h' file not found
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Fixes: a46b73e ("mesa: Move the FXT1 compressor/decompressor to util/")
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9240>

scons: Add u_format_pack.h include path.

Fixes MSVC SCons build.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9263>

zink: use 0 as default for spec constants

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9258>

ci/iris: Move the traces yml file to the driver-specific dir.

Since the job is manual, I missed it in the move and it got dropped from
the artifacts.

Fixes: 60d413b89455 ("ci: Move the piglit expectations lists to the per-driver CI dirs.")
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9226>

ci/piglit: Stop including the test counts at the end of expectations.

It's just a ton of fuss for driver developers fixing piglit tests.  This
makes the trace expectation files pretty silly (empty expectation, but
you'll get a diff to a non-empty result when something fails)

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Acked-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9226>

lavapipe: fix descriptor set layout freeing.

Have to store the used allocator otherwise ends up being free wrong.

Fix for
dEQP-VK.api.object_management.alloc_callback_fail.descriptor_set_layout*

Fixes: f94a5f30e0ebd ("lavapipe: add reference counting to descriptor set layout")

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9247>

iris: Pin surface state buffers after possibly updating the clear color

On Gen8, updating the clear color will end up allocating new
SURFACE_STATE entries.  These might end up living in a different BO
than the original copies, which means that we have to pin _after_
updating the clear color, not before.

Found by inspection.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9257>

v3d: Replace driver lowering of GL_CLAMP with mesa/st's.

Mesa core can do this logic for us now.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9228>

v3d: Drop foreach_bit() macro

Now that we have a global one.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

etnaviv: Drop foreach_bit() macro

Now that we have a global one.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

freedreno: Drop foreach_bit() macro

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

freedreno/ir3: Drop foreach_bit() macro

Now that there is a global one in util/bitscan.h

Note this version had an extra assert which is not really suitable to a
generic foreach_bit().. just move the assert to the two usages of the
iterator macro.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

anv: for_each_bit -> foreach_bit

Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

freedreno/vulkan: for_each_bit -> foreach_bit

Reviewed-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

radv: for_each_bit -> foreach_bit

Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

v3dv: remove for_each_bit() macro

this was unused

Reviewed-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

util/bitscan: add u_foreach_bit macros

this is a standardized (and very slightly improved for usability) version
of the macro that has been copied into every vulkan driver

includes fixup from Rob Clark <robclark@freedesktop.org>

Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9191>

zink: add more ci flakes

wtf

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9242>

zink: support gl_LocalGroupSize

this requires setting up a spec constant on the pipeline state which can
then propagate to the shader and be used like a regular constant

all ARB_compute_variable_group_size should pass now

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9242>

zink: add some spirv builder functions for spec constants

we'll be using these soon

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9242>

zink: force 128 fs input components on intel drivers

the hardware supports it, the driver supports it, but the driver reports
a lower value due to subtracting some usage that we shouldn't exceed anyway

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9198>

zink: set PIPE_CAP_TEXTURE_BORDER_COLOR_QUIRK

non-intel platforms need border colors pre-swizzled

this is an internal khronos spec bug that will (someday) be resolved in
a more detectable manner

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9136>

xmlconfig: Add static driconfig support

For platforms which do not have support for parsing driconf from xml
files on the filesystem, build in driconf tables generated from
00-mesa-defaults.conf at compile time and use that for option matching.

This allows us to have game/engine specific overrides built in to mesa.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9179>

xmlconfig: Reshuffle to keep attr processing

For the static-table alternative to WITH_XMLCONFIG, we are going to want
to re-use the element attribute processing, to avoid duplicating things
like engine name regexp matching and version range matching.  This just
shuffles things around a bit so we can re-use useful parts in the next
patch.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9179>

driconf: Generate a static table when no xmlconfig

For builds without runtime xmlconfig parsing, generate a static table
from 00-mesa-defaults.conf.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9179>

util: add mesa_glthread for Half Life 2 and Black Mesa.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4149
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4092
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9252>

glapi: keep declspec(thread) msvc-specific

gcc ignores with a warning:
glapi.h:83:1: warning: ‘thread’ attribute directive ignored [-Wattributes]

Fixes: ba141b95

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9245>

zink: correct inaccurate comment

PIPE_SHADER_CAP_MAX_CONST_BUFFER_SIZE translate into
GL_MAX_*_UNIFORM_COMPONENTS, all of which are allowed to be as
low as 1024 by the GL 4.6 spec.

PIPE_CAP_MAX_SHADER_BUFFER_SIZE translate into
GL_MAX_SHADER_STORAGE_BLOCK_SIZE, which has different minimum values in
different versions of the GL spec. In the GL 4.6 spec for instance, it
is required to be 2^27, the same as what Vulkan requires.

But what these limits are in GL is irrelevant at this level of
abstraction. The OpenGL state-tracker cares, but the Gallium driver
shouldn't have to. So let's just delete those parts of the comments.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9216>

nir/dce: perform DCE for unlooped instructions in a single pass

It's unnecessary to iterate twice for instructions outside loops.

Compile-time (nir_opt_dce):
Difference at 95.0% confidence
-630.64 +/- 6.18761
-27.0751% +/- 0.223134%
(Student's t, pooled s = 7.30785)

Compile-time (entire run):
Difference at 95.0% confidence
-749.54 +/- 48.8272
-1.82644% +/- 0.117838%
(Student's t, pooled s = 57.6672)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7691>

nir: inline nir_foreach_{src,dest}

Compile-time (nir_opt_dce):
Difference at 95.0% confidence
-319.51 +/- 5.67632
-12.0627% +/- 0.208076%
(Student's t, pooled s = 6.70399)

Compile-time (overall):
Difference at 95.0% confidence
-385.025 +/- 42.1124
-0.929489% +/- 0.10139%
(Student's t, pooled s = 49.7367)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7691>

nir/dce: replace instruction worklist with ssa def bitset

Instead of a keeping a worklist of live instructions, use a bitset of live
ssa defs and iterate over instructions in reverse.

Compile-time (nir_opt_dce):
Difference at 95.0% confidence
-931.911 +/- 4.41383
-26.0263% +/- 0.105781%
(Student's t, pooled s = 5.21293)

Compile-time (overall):
Difference at 95.0% confidence
-882.245 +/- 28.3492
-2.08541% +/- 0.0665121%
(Student's t, pooled s = 33.4818)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7691>

ci/vc4/v3d: Parallelize piglit jobs

Split the piglit jobs in multiple parallel executions to speed up the
runtime.

v2:
 - Set parallel in V3D piglit jobs.

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9022>

ci/piglit: allow parallel piglit jobs

This allows to split a piglit job in several parallel jobs, to speed up
the execution.

Due piglit restrictions, this only works for single profiles. Otherwise
an error will be shown in the runner.

Also, a new gitlab job variable `PIGLIT_TESTS` is introduced that
contains the excluded/included tests with `-x` or `-n`. The rest of the
piglit options go to `PIGLIT_OPTIONS` (like `--timeout n`).

v2 (Andres):
 - Replay profile is supported in parallel jobs.
 - Bail out inmediately if parallel jobs is tried with multiple
profiles.
 - Use testlist only when doing parallel jobs.
 - Do not drop pass tests when filtering executed tests.
 - Get rid of PIGLIT_FRACTION.

v4:
 - uncommit unrelated change (Andres).

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9022>

broadcom/compiler: use nir_opt_sink

total instructions in shared programs: 14072341 -> 14062334 (-0.07%)
instructions in affected programs: 1996685 -> 1986678 (-0.50%)
helped: 3038
HURT: 2432
Instructions are helped.

total uniforms in shared programs: 3797720 -> 3794523 (-0.08%)
uniforms in affected programs: 191711 -> 188514 (-1.67%)
helped: 831
HURT: 449
Uniforms are helped.

total max-temps in shared programs: 2340632 -> 2335124 (-0.24%)
max-temps in affected programs: 113632 -> 108124 (-4.85%)
helped: 2728
HURT: 436
Max-temps are helped.

total spills in shared programs: 6050 -> 5931 (-1.97%)
spills in affected programs: 2869 -> 2750 (-4.15%)
helped: 14
HURT: 4

total fills in shared programs: 13970 -> 13371 (-4.29%)
fills in affected programs: 8831 -> 8232 (-6.78%)
helped: 14
HURT: 4

total inst-and-stalls in shared programs: 14103668 -> 14093712 (-0.07%)
inst-and-stalls in affected programs: 2004035 -> 1994079 (-0.50%)
helped: 3009
HURT: 2426
Inst-and-stalls are helped.

LOST:   0
GAINED: 10

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9209>

anv/gen12: Implement programming of pixel pipe hashing tables.

See previous commit "iris/gen12: Implement programming of pixel pipe
hashing tables." for the rationale.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

anv/gen11+: Calculate pixel hashing tables instead of hardcoding.

See commit "iris/gen11+: Calculate pixel hashing tables instead of
hardcoding." for the rationale.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

iris/gen12: Implement programming of pixel pipe hashing tables.

Straightforward by using the pixel hashing table computation helper
previously introduced, assuming we know the fraction of work that
needs to be submitted to each pixel pipe.  Note that AFAIA the
hardware maps indices in the table to pixel pipes from largest to
smallest, so it shouldn't be necessary to permute indices based on the
physical IDs of the pixel pipes as we are doing on Gen11.

Improves performance of most non-trivial graphics workloads I've tried
on an 80 EU TGL.  E.g. the following testcases improve performance
significantly with sample size 27 and statistical significance 1%:

  gputest/pixmark_piano:      62.89% ±0.10%
  gputest/pixmark_volplosion: 61.51% ±0.06%
  unigine/valley:             26.72% ±0.25%
  gfxbench/gl_5_high:         24.70% ±0.19%
  unigine/heaven:             23.54% ±0.17%
  steam/csgo:                 22.75% ±4.36%
  gfxbench/gl_manhattan31:    22.43% ±0.29%
  gfxbench/gl_4:              20.92% ±0.35%
  warsow/benchsow:            19.15% ±2.53%
  gfxbench/gl_trex_off:       18.84% ±0.27%

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

iris/gen11+: Calculate pixel hashing tables instead of hardcoding.

Pixel hashing tables are a pain to type in, review and maintain IMHO.
In order to obtain satisfactory load balancing on all Gen12 parts
currently in production this series would need to add 5 different
additional tables.  Instead this introduces a simple algorithm able to
calculate a table on the fly based on a handful of parameters.

Note that the Gen11 tables generated with this algorithm are not
identical to the hardcoded ones, however the only difference should be
a phase shift that isn't expected to have any effect on performance,
since it shouldn't change the fraction of work submitted to each pixel
pipe.

The CPU overhead from this change is negligible since the tables only
need to be programmed once at context init time.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

intel/dev: Implement pixel pipe subslice counting for Gen12+.

Unlike Gen11, Gen12 hardware supports up to three pixel pipes per
slice.

Unfortunately the kernel interface is somewhat inconsistent between
Gen11 and Gen12: I915_PARAM_SUBSLICE_MASK returns a mask of enabled
*dual* subslices since TGL, so there is half the number of bits per
pixel pipe in the mask.  This is worked around here so we're able to
calculate the correct size of each pixel pipe, but the result is
returned in dual subslice units, inheriting the inconsistency from the
kernel -- Reason is that as of now all our Gen12 subslice counts
returned by gen_device_info.c are really dual subslice counts, and the
num_eu_per_subslice counts are also scaled accordingly, so it seems
like it would only make the matter worse if I fixed the units of this
field only without also fixing the rest.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

intel/genxml: Define 3DSTATE_SUBSLICE_HASH_TABLE command for Gen12 and Gen12.5.

This command allows programming custom pixel hashing tables
controlling the balancing of load across pixel pipes.  Rather
confusingly 3DSTATE_SLICE_TABLE_STATE_POINTERS was serving the same
purpose on Gen11: A pixel is mapped to the pixel pipe with index
specified by the entry in the table corresponding to the LSBs of the
pixel coordinates [Yes you read right the entries are neither subslice
nor slice indices!].  Either a 2-way or a 3-way table can be
programmed based on whether the platform has two or three pixel pipes
per slice.  In addition the 16x8 tables defined below can hold two
separate 8x8 tables when in DUAL_TABLE mode (which AFAIA is only
useful for platforms with multiple asymmetric slices -- I.e. no
production platforms as of today to my knowledge).

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

intel/genxml: Fix pixel hashing 3DSTATE_3D_MODE field definitions for Gen12 and Gen12.5.

The former "Subslice Hashing Mode" field is no longer used by the
hardware, Gen12 parts always do 16x16 subslice pixel hashing -- Remove
it since it's no longer useful.  In addition add a couple of bits that
will be useful in order to make some adjustments to the default pixel
pipe hashing behavior.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8749>

zink: ci updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9244>

zink: support stencil-export

Enable pipe capability of exporting stencil from shader when Vulkan
extension is available.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9244>

zink: check shader stencil output

Enable SPV_EXT_stencil_export and SpvCapabilityStencilExportEXT and
mark output with FragStencilRefEXT when fragment shader writes to
reference stencil value.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9244>

zink: add X32_S8X24 format

Signed-off-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9244>

zink: improve descriptor set oom handling

this attempts to dynamically establish an upper bound for per-batch descriptor
use, flushing all batches and resetting the pools on alloc failure in
an attempt to be more robust about it

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9117>

zink: ci changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: always do full-fb clears in renderpass begin when possible

previously if any of the pending clears required an explicit clear then
we'd clear them explicitly, but with this patch we're shifting the first
pending clear into the renderpass begin if possible and then applying the
remaining clears on top of that in order to reduce gpu operations

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: move all the clear stuff to zink_clear.h

this was getting way too crowded in zink_context.h

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: optimize the remaining read cases of applying pending clear calls

we have src regions for all the blit/copy/map calls, so we can use those to
verify whether we actually need to apply the clears now or if we can keep
sitting on them a while longer

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: add yet another clear helper, this time for applying overlap regions

if we know we're going to be reading from a region then we can examine the
pending clears to see if there's any overlap, which helps us decide whether
we need to apply them immediately

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: discard pending clears during blit/copy if we'll overwrite the data

these operations are redundant, so there's no point in forcing them through
since they'll never be visible

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: add helper for applying/discarding clears based on a rect

this lets us test whether we're going to overwrite a scissored clear
region in order to correctly no-op it

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: break out fb clear apply into helper function

we'll be reusing this shortly...

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: add another helper for checking whether one rect covers another

this is going to be used for no-oping writes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: add helper for converting pipe_box -> u_rect

how is this not a thing already

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: break out region overlap testing function into helper

this is useful for a lot of copying operations

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: also defer fb clears when conditional render is active

this just requires adding an explicit flush for the deferred clears in
case conditional rendering is disabled before a draw happens

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: defer pipe_context::clear calls when not currently in a renderpass

instead, we can attach the clear to the next renderpass start and even add it to
the renderpass cache for reuse

also add handling for flushing clears on map or fb switching to avoid brekaing behavior

this should save us a lot of time with potentially beginning/ending renderpasses as well
as allowing drivers to do better batching of clears by passing in all the buffers at
once

this doesn't handle deferring conditional renders yet in a futile attempt to try and keep
the size of the patch down

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: create framebuffer and renderpass objects just before vkCmdBeginRenderPass()

this lets us remove a flush and accumulate clears to perform in the renderpass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

zink: ralloc the main context

we can start sticking stuff onto this now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9206>

freedreno: Workaround for UNSYNC+DISCARD_RANGE

Dead-cells (and perhaps others) does MapBufferRange(UNSYNC|DISCARD_RANGE)
to update single buffered VBOs every frame in the opening menu screen,
and because we were considering UNSYNC with higher priority than
DISCARD_RANGE, this would result in the game racing VBO updates between
binning and tile passes, causing visibility stream inconsistency between
the two passes, resulting in "tile flicker".

The letter of the gl spec implies this is undefined behavior (at least
to my reading of it).  But we already hand DISCARD_RANGE in the !UNSYNC
case, so just go down this path instead.  It means we could potentially
end up invalidating (and back-blit) in cases where the app really knows
what it is doing, but oh well.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4337
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>

freedreno: Add perf_debug logging for bo stalls

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>

freedreno: Handle InvalidateBufferData() case

We already had all the mechanism, it just wasn't wired up.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>

lavapipe: enable KHR_maintenace3

At this stage I can't think of any reason a swrast would need to
limit descriptor sets at this point. I'm sure there are some in the
future.

Otherwise it just fixes up a lookup that really is a noop, but
may as well make it correct.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9238>

lavapipe: enable KHR_maintenance2

This seems to pass all the CTS tests at least. I think
most of the details are handled already.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9238>

lavapipe: handle tessellation domain winding

Fixes: dEQP-VK.tessellation.winding.*

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9238>

intel/compiler: Relax some conditions in try_copy_propagate

Previously can_do_source_mods was used to determine whether a value with
a source modifier or a value from a scalar source (e.g., a uniform)
could be copy propagated.  The former is a superset of the latter, so
this always produces correct results, but it is overly restrictive.  For
example, a BFI instruction can't have source modifiers, but it can have
scalar sources.

This was originally authored to prevent a small number of shader-db
regressions in a commit that marked SHR has not being able to have
source modifiers.  That commit has since been dropped in favor of a
different method.

v2: Refactor register region restriction detection to a helper function.
Suggested by Jason.

No fossil-db changes on any Intel platform.

All Gen7+ platforms had similar results. (Ice Lake shown)
total instructions in shared programs: 20039111 -> 20038943 (<.01%)
instructions in affected programs: 31736 -> 31568 (-0.53%)
helped: 104
HURT: 0
helped stats (abs) min: 1 max: 9 x̄: 1.62 x̃: 1
helped stats (rel) min: 0.30% max: 0.88% x̄: 0.45% x̃: 0.42%
95% mean confidence interval for instructions value: -2.03 -1.20
95% mean confidence interval for instructions %-change: -0.47% -0.42%
Instructions are helped.

total cycles in shared programs: 980309750 -> 980308897 (<.01%)
cycles in affected programs: 591078 -> 590225 (-0.14%)
helped: 70
HURT: 26
helped stats (abs) min: 2 max: 622 x̄: 23.94 x̃: 4
helped stats (rel) min: <.01% max: 2.85% x̄: 0.33% x̃: 0.12%
HURT stats (abs)   min: 2 max: 520 x̄: 31.65 x̃: 6
HURT stats (rel)   min: 0.02% max: 2.45% x̄: 0.34% x̃: 0.15%
95% mean confidence interval for cycles value: -26.41 8.64
95% mean confidence interval for cycles %-change: -0.27% -0.03%
Inconclusive result (value mean confidence interval includes 0).

No shader-db changes on earlier Intel platforms.

Reviewed-by: Anuj Phogat anuj.phogat@gmail.com [v1]
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9237>