nir: handle negatives in ffma reassociation optimisation

shader-db results Iris (BDW):

total instructions in shared programs: 16632076 -> 16631057 (<.01%)
instructions in affected programs: 48010 -> 46991 (-2.12%)
helped: 47
HURT: 6

total cycles in shared programs: 915266726 -> 915263622 (<.01%)
cycles in affected programs: 1182283 -> 1179179 (-0.26%)
helped: 18
HURT: 27

total loops in shared programs: 4929 -> 4929 (0.00%)
loops in affected programs: 0 -> 0
helped: 0
HURT: 0

total spills in shared programs: 18834 -> 18801 (-0.18%)
spills in affected programs: 525 -> 492 (-6.29%)
helped: 3
HURT: 0

total fills in shared programs: 23008 -> 22981 (-0.12%)
fills in affected programs: 435 -> 408 (-6.21%)
helped: 3
HURT: 0

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8608>

lavapipe: use clear interface for renderpass clears

For llvmpipe this means clears happen in the raster threads
rather than in the single cpu thread.

This path can't be used for scissored or clears with different
clear colors per mrt.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9041>

llvmpipe: zs clear use 64-bit memset

Use the newly added memset64 path

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9041>

u_surface: use optimised memset64

Use the newly added optimised memset64 path

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9041>

util: add optimised memset64

This just adds a memset64 along the lines of the previously
added memset32.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9041>

util/fossilize_db: Fix compile error with clang

../src/util/fossilize_db.c:178:58: error: use of GNU empty initializer extension [-Werror,-Wgnu-empty-initializer]
         char hash_str[FOSSILIZE_BLOB_HASH_LENGTH + 1] = {};

Fixes: eca6bb9540d ("util/fossilize_db: add basic fossilize db util to read/write shader caches")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9180>

lavapipe: reset object base on recycled command buffers

The loader_set_dispatch overwrites the magic with the dispatch
pointer, however when cmd buffers get recycled, and the loader
is in debug mode, it asserts that the magic isn't set anymore.

When recycling command buffers, reset the object base.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9145>

tu: reset object base on recycled command buffers

The loader_set_dispatch overwrites the magic with the dispatch
pointer, however when cmd buffers get recycled, and the loader
is in debug mode, it asserts that the magic isn't set anymore.

When recycling command buffers, reset the object base.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9145>

radv: reset object base on recycled command buffers

The loader_set_dispatch overwrites the magic with the dispatch
pointer, however when cmd buffers get recycled, and the loader
is in debug mode, it asserts that the magic isn't set anymore.

When recycling command buffers, reset the base object.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9145>

vulkan/util: add api to reset object magic + private data.

This will be used to for reissuing dispatchable objects like
command buffers.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9145>

gallium/tessellator: Rename D3D11 defines

We're Gallium, c'mon!

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9118>

gallium/tessellator: Remove unused includes

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9118>

gallium/tessellator: Remove XBOX 360 code

Unused.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9118>

lima: fix max sampler views

If this is not defined, mesa may not deallocate sampler views,
which can result in memory leaks.
Just define it to the same as max texture samplers, like other
mesa drivers do.

Cc: mesa-stable
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9172>

docs/features: add lima features

This was mostly generated through scripts parsing glxinfo and then some
manual adjustments.

Note that lima is mostly a GLES 2.0 driver so its usefulness on this
list is a bit limited. But there was a long standing user request to
include it here and I guess it's better to be on the radar.

The extension list might not be fully correct, hopefully putting it here
for comparison also helps tracking if something is wrong or missing.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9031>

util/disk_cache: make use of single file cache when env var set

When the MESA_DISK_CACHE_SINGLE_FILE environment variable is set
we make use of the new single file shader cache implementation.

The new cache uses the following directory structure based on the
first defined name as follows:

 $MESA_GLSL_CACHE_DIR/driver_id/gpu_name/foz_cache.foz
 $MESA_GLSL_CACHE_DIR/driver_id/gpu_name/foz_cache_idx.foz

 $XDG_CACHE_HOME/mesa_shader_cache_sf/driver_id/gpu_name/foz_cache.foz
 $XDG_CACHE_HOME/mesa_shader_cache_sf/driver_id/gpu_name/foz_cache_idx.foz

 <pwd.pw_dir>/.cache/mesa_shader_cache_sf/driver_id/gpu_name/foz_cache.foz
 <pwd.pw_dir>/.cache/mesa_shader_cache_sf/driver_id/gpu_name/foz_cache_idx.foz

Where foz_cache_idx.foz is a database of offsets pointing to the location of
the shader cache entries in foz_cache.foz

This initial implementation doesn't have any max cache size handling and is
initially intended to be use by applications such as steam that will handle
cache management for us.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

util/fossilize_db: add basic fossilize db util to read/write shader caches

My benchmarking shows no significant change in cache load times with a
single shader cache file vs the existing cache implementation of many
small files (tested with my spinning rust HDD).

However this new single file cache implementation does reduce the total
size on disk used by the shader cache. We have a problem with the existing
cache where writing tiny files to disk causes more disk space to be used than
is actually needed for the files due to the minimum size required for a file.
In pratice this tends to inflate the size of the cache on disk to over 3x
larger.

There are other advantages of using a single file for shader cache entries
such as allowing better removal of cache entries once we hit the max cache
size limit (although we don't implement any max cache size handling in this
initial implementation).

The primary reason for implementing a single file cache for now is to allow
better performance and handling by third party applications such as steam
that collect and distribute precompiled cache entries.

For this reason we also implement a new environment variable
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS which allows a user to pass in a path
to a number of external read only shader cache dbs. There is an initial
limit of 8 dbs that can be passed to mesa like so:

MESA_DISK_CACHE_READ_ONLY_FOZ_DBS=/full_path/filename1, ... ,/full_path/filename8

Where the filename represents the cache db and its index file e.g.
filename1.foz and filename1_idx.foz

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

util/mesa_sha1: add helper to reconvert sha1 hex strings

Converts the sha1 hex string representation back into its original
more compact format.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

util/disk_cache: use a new cache dir for the single file cache feature

This allows us to guarantee the different cache implementations will not
interfere with each other and should make it more clear that the max cache
size limits are applied separately for each cache implementation.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

util/disk_cache: move cache path strdup call back into disk_cache.c

This code is already OS agnostic and moving it here enables code
sharing in the following patches.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

util/disk_cache: do crc32 check on compressed data for ZSTD

This will be faster and avoids checking for errors with the
compression implementation which we shouldn't need to do. Instead
we trust the compression library does the correct thing and simply
error check the data loaded from disk.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7725>

intel: check setuid before writing output file in INTEL_MEASURE

Without this check, the INTEL_MEASURE environment variable could be
misused to overwrite arbitrary files.

Fixes: 0f4143ec378 ("intel: Print GPU timing data based on INTEL_MEASURE")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9166>

nir/algebraic: Add some max/min optimizations with 3 variables

Specifically, ARB assembly shaders with code like

    SLT    r0, r0, c[0].xxxx;
    ...
    KIL    r0.xyzx;

can result in this pattern.  The other cases (e.g., 'KIL r0.xxxx' and
'KIL r0.xyxx') are handled by existing patterns.

Reviewed-by: Matt Turner <mattst88@gmail.com>
All Intel platforms had similar results. (Tiger Lake shown)
total instructions in shared programs: 21050098 -> 21050065 (<.01%)
instructions in affected programs: 2062 -> 2029 (-1.60%)
helped: 31
HURT: 1
helped stats (abs) min: 1 max: 3 x̄: 1.10 x̃: 1
helped stats (rel) min: 1.14% max: 4.35% x̄: 1.89% x̃: 1.69%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.65% max: 0.65% x̄: 0.65% x̃: 0.65%
95% mean confidence interval for instructions value: -1.23 -0.84
95% mean confidence interval for instructions %-change: -2.12% -1.50%
Instructions are helped.

total cycles in shared programs: 855105466 -> 855105055 (<.01%)
cycles in affected programs: 50136 -> 49725 (-0.82%)
helped: 33
HURT: 0
helped stats (abs) min: 3 max: 22 x̄: 12.45 x̃: 12
helped stats (rel) min: 0.13% max: 1.57% x̄: 0.86% x̃: 0.92%
95% mean confidence interval for cycles value: -13.78 -11.13
95% mean confidence interval for cycles %-change: -0.97% -0.76%
Cycles are helped.

No fossil-db changes on any Intel platform.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9122>

nir/algebraic: Remove some redundant b2f logic-op reduction patterns

There are patterns that will re-write the fmin or fmax part into a form
that other patterns will gradually convert to the same ior or iand.  For
example,

    fmax(b2f(a), b2f(b)) != 0
    b2f(a || b) != 0
    a || b

No shader-db or fossil-db changes on any Intel platform.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9122>

nir/algebraic: Fix some min/max of b2f replacements

fmin(-A, -B) is -fmax(A, B), and fmax(-A, -B) is -fmin(A, B).  Therefore
the logic joining A and B should toggle between ior and iand for the
negated versions.

At the very least, a shader from Euro Truck Simulator 2 in shader-db is
affected by this.  The KIL instruction in the (ARB assembly) shader ends
up with the wrong logic.  This is _probably_ the source of
https://gitlab.freedesktop.org/mesa/mesa/-/issues/1346.

That said, the issue mentions that Mesa 18.0.5 works, but commit
68420d8322c ("nir: Simplify min and max of b2f") was added in 17.3.
Moreover, I was not able to reproduce the error in the ETS2 shader from
shader-db from any Mesa commit near the time the original fd.o bugzilla
was submitted (December 2018). :shrug:

In fact, the current error in that shader starts with 9167324a86b
("nir/algebraic: Mark some logic-joined comparison reductions as
exact").  That's a bit of a red herring as 9167324a86b just sets off a
chain of replacements that eventually leads to the incorrect min/max of
b2f patterns fixed by this commit.

The other affected shaders in the shader-db results are from Cargo
Commander.  These are also ARB assembly shaders.

I think any ARB assembly shader that uses the pattern

    SLT    r0, ...;
    ...
    KIL    -r0;

will suffer from issues related to this.

This change fixes the piglit
tests/spec/arb_fragment_program/kil-of-slt.shader_test test added in
https://gitlab.freedesktop.org/mesa/piglit/-/merge_requests/454.

shader-db results:

All Gen6+ platforms had similar result. (Ice Lake shown)
total instructions in shared programs: 20034604 -> 20034486 (<.01%)
instructions in affected programs: 3885 -> 3767 (-3.04%)
helped: 47
HURT: 2
helped stats (abs) min: 2 max: 4 x̄: 2.64 x̃: 2
helped stats (rel) min: 2.33% max: 8.33% x̄: 3.48% x̃: 3.39%
HURT stats (abs)   min: 3 max: 3 x̄: 3.00 x̃: 3
HURT stats (rel)   min: 13.64% max: 16.67% x̄: 15.15% x̃: 15.15%
95% mean confidence interval for instructions value: -2.83 -1.99
95% mean confidence interval for instructions %-change: -3.84% -1.60%
Instructions are helped.

total cycles in shared programs: 979881379 -> 979879406 (<.01%)
cycles in affected programs: 119873 -> 117900 (-1.65%)
helped: 46
HURT: 3
helped stats (abs) min: 10 max: 756 x̄: 45.41 x̃: 26
helped stats (rel) min: 0.53% max: 19.72% x̄: 1.67% x̃: 1.26%
HURT stats (abs)   min: 28 max: 56 x̄: 38.67 x̃: 32
HURT stats (rel)   min: 1.44% max: 3.54% x̄: 2.75% x̃: 3.27%
95% mean confidence interval for cycles value: -70.83 -9.70
95% mean confidence interval for cycles %-change: -2.23% -0.57%
Cycles are helped.

Iron Lake and GM45 had similar results. (Iron Lake shown)
total instructions in shared programs: 8115098 -> 8115076 (<.01%)
instructions in affected programs: 2592 -> 2570 (-0.85%)
helped: 32
HURT: 2
helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1
helped stats (rel) min: 0.88% max: 2.70% x̄: 1.35% x̃: 1.31%
HURT stats (abs)   min: 5 max: 5 x̄: 5.00 x̃: 5
HURT stats (rel)   min: 17.24% max: 18.52% x̄: 17.88% x̃: 17.88%
95% mean confidence interval for instructions value: -1.15 -0.15
95% mean confidence interval for instructions %-change: -1.83% 1.39%
Inconclusive result (%-change mean confidence interval includes 0).

total cycles in shared programs: 238189718 -> 238189802 (<.01%)
cycles in affected programs: 75076 -> 75160 (0.11%)
helped: 3
HURT: 31
helped stats (abs) min: 2 max: 130 x̄: 44.67 x̃: 2
helped stats (rel) min: 0.18% max: 5.70% x̄: 2.02% x̃: 0.19%
HURT stats (abs)   min: 2 max: 70 x̄: 7.03 x̃: 4
HURT stats (rel)   min: 0.07% max: 6.41% x̄: 0.53% x̃: 0.15%
95% mean confidence interval for cycles value: -7.27 12.21
95% mean confidence interval for cycles %-change: -0.33% 0.94%
Inconclusive result (value mean confidence interval includes 0).

No fossil-db changes on any Intel platform.

Fixes: 68420d8322c ("nir: Simplify min and max of b2f")
Closes: #1346
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9122>

freedreno/ir3/print: Improve branch printing

Handle the instruction suffix better, and don't try to print src regs in
a generic way, since that doesn't really work out.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9142>

freedreno/ir3/print: More sane ssa src/dst display

Give src/dst a "ssa_%u" name generated from the instruction's unique
serialno.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9142>

st/mesa: fix PBO download for TEXTURE_1D_ARRAY textures

Fixes 'nir_tex_src_coord' param was provided to NIR 'txf' operation as a
vec3 for TEXTURE_1D_ARRAY target, causing an assert.
Only following targets require vec3: TEXTURE_2D_ARRAY, TEXTURE_3D,
TEXTURE_CUBE, TEXTURE_CUBE_ARRAY. The rest must use vec2.

Packing layer value into Y-coordinate the same way it was done in
'create_fs' in commit 2bf6dfac.

Fixes: a01ad311 ("st/mesa: Add NIR versions of the PBO upload/download
shaders. ")

Signed-off-by: Yevhenii Kharchenko <yevhenii.kharchenko@globallogic.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9014>

iris: Reference the shader variant for last_vue_map as well

We call update_last_vue_map after updating the shaders, which compares
the new and old VUE maps.  Except...updating the shaders may have
dropped the last reference to the variant that ice->shaders.last_vue_map
belonged to, leading to a classic use-after-free.

Fix this by taking a reference to the variant for the last VUE stage,
so it stays around until we're done with it.

Fixes: 1afed51445c ("iris: Store a list of shader variants in the shader itself")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4311
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9143>

turnip,freedreno/a6xx: tell hw the size of shared mem used by CS

Before, we only used 2k of shared memory.

It was found that 5 lower bits of SP_CS_UNKNOWN_A9B1 do control
the available size of shared memory for compute shaders, with
AVAILABLE_SIZE = (SP_CS_UNKNOWN_A9B1_SHARED_SIZE + 1) * 1k
up to 32k. And SP_CS_UNKNOWN_A9B1_SHARED_SIZE being zero enables
all 32k of shared memory.

Fixes tests:
 dEQP-VK.rasterization.line_continuity.line-strip
 dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.buffer.guard_nonlocal.workgroup.comp
 dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_nonlocal.workgroup.guard_local.buffer.comp
 dEQP-VK.memory_model.write_after_read.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.image.guard_nonlocal.workgroup.comp

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9157>

ci: Move specific driver testing to separate files in separate dirs.

The top-level gitlab-ci.yml is big and unwieldy when one wants to work on
CI for a single driver.  Move the drivers to separate include files for
ease of finding all your driver's tests, and also to pave the way for work
on a single driver's CI to not retest all other drivers.

Reviewed-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9139>

microsoft/clc: Add a test with an unused kernel arg

Reviewed-by: Bill Kristiansen <billkris@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9160>

microsoft/clc: Re-order dead variable removal after uniform vars_to_explicit_types

Since vars_to_explicit_types is now where driver_location/offset is filled out,
we need to make sure that we still have all app-provided kernel arg variables
at that point in time so they all get assigned unique offsets. That means
that we can't have removed dead uniforms yet, which also means we can't have
filled out metadata for inline samplers (since usage of them generates tons
of duplicate uniforms).

Reviewed-by: Bill Kristiansen <billkris@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9160>

turnip: consider tile_max_h when calculating tiling config

Otherwise we may get a tile height exceeding the maximum.

Fixes tests:
 dEQP-VK.pipeline.render_to_image.core.2d.huge.height.r8g8b8a8_unorm
 dEQP-VK.pipeline.render_to_image.core.2d.huge.height.r8g8b8a8_unorm_d16_unorm
 dEQP-VK.pipeline.render_to_image.core.2d.huge.height.r8g8b8a8_unorm_s8_uint

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9159>

xlib: Fix build regression since 99e25d183d9

Just named the variable wrong. Can't really fault anyone for breaking a
target we don't build in CI...

Fixes: 99e25d183d9 mesa: optimize set_varying_vp_inputs by precomputing the conditions
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9132>

ac/rgp: set gfxip in elf_hdr.e_flags

This patch will set the gfxip into elf_hdr.e_flags by reading data
from struct rad_info->chip_class instead of hardcoding.

v2: remove unused #define

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp: dump co, col, pso database to rgp profile file

This patch dumps code object, code object loader events, pso correlation
databses to given rgp profile file. The data to dump is taken from
struct rgp_code_object, struct rgp_loader_events and
struct rgp_pso_correlation.

v2: fix sqtt_file structure name to match existing names in file.
    fix checking wrong variable for pso correlation record_count.
    remove freeing rgp database here.

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp,radeonsi,radv: pass struct thread_trace_data to ac_sqtt_dump_data()

struct thread_trace_data holds struct rgp_code_object, struct
rgp_loader_events, struct rgp_pso_correlation data. This data is required
in function ac_sqtt_dump_data(). This patch makes the code changes
required to pass struct thread_trace_data to function ac_sqtt_dump_data().

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp: expose data structure to populate co, col, pso database

This patch exposes struct rgp_code_object, struct rgp_loader_events and
struct rgp_pso_correlation into struct ac_thread_trace_data. Other modules
can fill in this data structure. The code object, code object loader events,
pso correlation database is written to rgp profile file using data from
these data structure exposed by this patch.

v2: move the data structures to struct ac_thread_trace_data

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp: add helper function to write rgp elf oject

This patch adds helper function ac_rgp_write_elf_object() which
creates elf object from struct rgp_code_object_record. The elf
object is written to given output rgp profile file.

v2: for newly added files, change copyright year from 2020 to 2021
v3: fix 32bit compilation failure

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp: add rgp co, col, pso data structures

This patch adds data structures used to collect data for code object,
code object loader event and pso correlation databases present in
a rgp profile.

v2: fix code review comments from Pierre-Eric
v3: Make loader_event_type into enum

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

ac/rgp: add ac_msgpack.h/c

This patch adds functions to create msgpack formatted data.
For msgpack specification refer to
github.com/msgpack/msgpack/blob/master/spec.md

This patch only adds formats from msgpac specification that
are required for rgp profile data.

v2: for newly added files, change copyright year from 2020 to 2021

Signed-off-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8609>

util: fix gcc vsnprintf overflow

Anything higher than INT_MAX results in overflow although the parameter is declared as size_t.

Worse, with (size_t)-1 it is silently ignored and Woverflow is not emitted.

Closes #4226

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Tested-by: Prodea Alexandru-Liviu <liviuprodea@yahoo.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9134>

turnip: consider HW limit on number of views when apply multipos opt

Blob doesn't apply multipos optimization starting from 11 views
even on a650, however in practice, with the limit of 16 views,
tests pass on a640/a650 and fail on a630.

Fixes tests:
 dEQP-VK.multiview.draw_indexed.max_multi_view_view_count
 dEQP-VK.multiview.input_attachments.max_multi_view_view_count
 dEQP-VK.multiview.masks.max_multi_view_view_count
 dEQP-VK.multiview.multisample.max_multi_view_view_count
 dEQP-VK.multiview.queries.max_multi_view_view_count
 dEQP-VK.multiview.renderpass2.index.fragment_shader.max_multi_view_view_count
 dEQP-VK.multiview.secondary_cmd_buffer.max_multi_view_view_count

Fixes: 8d275778 ("tu: Enable multi-position output")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9135>

radv: emit the trap handler registers earlier

Directly into the GFX init IB.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9076>

radv: make the TMA/TBA BOs resident buffers

They are always used if the trap handler is enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9076>

radv: make the trace BO a resident buffer

It's always used if RADV_DEBUG=hang is set.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9076>

radv: make the border color BO a resident buffer

It's always used if the feature is enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9076>

zink: fix win32 build

Fixes: 57575627, c4cc3d91

Tested-by: Prodea Alexandru-Liviu <liviuprodea@yahoo.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9126>

radv: set correct value for OFFCHIP_BUFFERING on GFX10+

Higher values break tessellation. I was only able to reproduce this
by switching back/from AMDVLK which was really weird...

According to Marek (1c6eca23fdd8), it looks like it's related to
register shadowing and PAL enables it, that probably explains a bit.

Copied from PAL and RadeonSI.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4207
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2498
Fixes: 74d69299d16 ("radv/gfx10: double the number of tessellation offchip buffers per SE")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9141>

nv50/ir: Initialize Instruction members.

Fix defects reported by Coverity Scan.

uninit_member: Non-static class member serial is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member sched is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member bb is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9037>

zink: use vkGetFenceStatus when we're obviously checking for status

a timeout of 0 indicates that gallium wants to know whether a fence is done,
so we can use a simpler call here

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9154>

zink: break out even more of zink_blit state saving

for reuse in upcoming clear refactoring

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9150>

freedreno/a6xx: set SP_PERFCTR_ENABLE in computerator

Set this register to have properly working SP perfcntrs in computerator.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

freedreno/a6xx: update some registers

Some sorting, adding unknown fields, documenting some fields, etc.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

freedreno/a6xx: always use reg64 for address registers (no LO/HI)

Reduce noise in a6xx.xml by removing LO/HI versions of address registers.

Also fix type="address" registers in register packing (use bit size instead
of checking for "waddress" to use qword)

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

freedreno/a6xx: update perfcntr registers (declare as arrays)

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

freedreno/registers: use macro instead of inline function for array regs

This is to allow use in places where an inline function isn't allowed,
such as a static initializer.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

freedreno/a6xx: Document threadsize-related fields

We'll need to use if we want to start playing around with thread sizes.
At least now we know what the actual threadsize is.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8423>

zink: just end the current renderpass in zink_batch_no_rp()

since we aren't (currently) parallelizing and now have barriers, we don't need to cycle
the batch here, which lets us avoid submitting too-small command buffers

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9148>

zink: break out buffer copying into util function with batch param

if we want to explicitly specify compute batch, we need to be able
to pass that along

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9147>

zink: add buffer barriers for resource_copy_region

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9147>

zink: just call context destructor on creation fail

this more reliably handles the failure case

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9149>

zink: update relnotes

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9151>

zink: tag some missing ES features

we're only missing advanced blend

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9151>

zink: fix surface creation for cube slices

if first==last layer, this should be a 2D slice of the cube

else if this isn't all the layers, this should be an array of slices

fixes a bunch of spec@arb_shader_image_size@builtin cases

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9080>

lavapipe: use os_time for timing related things

drop the use of the Linux code.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavapipe: make OPT macro MSVC compatible

This macros is ugly, make it uglier but msvc compatible.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavepipe: some misc msvc fixes

drop some headers, get rid of printflike, drop the __builtin_ffs

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavapipe: use msvc compatible 0 init

trivial changes

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavapipe: fix some void ptr arithmetic

msvc disagrees with it, and they are trivial to fix.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavapipe/meson: drop megadrivers build req

This just seems to be cut-n-paste, so lose it.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

lavapipe: implement physical device group enumeration

This was missing when I added physical device groups, and
was causing crashes on win32.

Fixes: 6af87193c ("lavapipe: add basic vulkan device group support.")
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9120>

zink: be more explicit with image barriers for copy operations

we know the access and pipeline stage here, so we can pass those along
to ensure synchronization

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add general zink_resource_barrier() wrapper

this simplifies a bit of code where we may want to be throwing in barriers
regardless of the resource type

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: zink_resource_barrier -> zink_resource_image_barrier

need this namespace

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: improve barrier usage for clear functions

ensure that we're always using the right layout for images that are
being cleared

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add batch references for resources in clear functions

need to make sure we track these writes

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add access info for update_descriptor image barriers

this forces resources to transition where necessary to ensure that
data from previous ops have been made available (e.g., clear -> shader image)

we also switch needs_barrier over to a more accurate signature here since we
can now do it without breaking update_descriptors()

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add access param for image resource barriers

we need more detail on some of these to ensure proper synchronization
and availability/visibility of image data between commands/stages

the signature for needs_barrier() is still funky here to avoid breaking
usage in update_descriptors()

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add barriers for index and draw param buffers

need to ensure synchronization here too

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: assert batch is not in a renderpass when emitting pipeline barrier

this is only valid for subpasses which have self-dependencies set, and we
don't currently do that

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: take struct zink_batch param instead of direct cmdbuf in barrier helpers

this is weird and prevents us from verifying batch states

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: combine resource barriers where possible during update_descriptors

if we have a resource in the same state for multiple shader stages, we can
emit a single barrier for all the stage bits instead of multiple barriers

this also helps with detecting potentially redundant barriers when we go to
emit them later on

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: break out barrier transitioning in update_descriptors

this hopefully makes the code a little more readable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: avoid emitting unnecessary pipeline barriers during update_descriptors

if the current state of the resource matches the desired state, we don't need
to emit anything, and we can potentially avoid ending a renderpass

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add generic wrapper for checking whether a resource needs a barrier

handy to not need to check ahead of time

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: use define for max descriptor array size

this was getting a bit unwieldy

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add helper for image resource barriers and avoid unnecessary barriers

same as buffer barriers now

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add a VkPipelineStageFlags param to zink_resource_barrier()

this matches the buffer variant in its flexibility now

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: remove aspect param from zink_resource_barrier

we have a helper function for this now that we can reuse

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: rework xfb barrier transitions when reusing as vertex inputs

if we have proper barrier usage to start with, then we don't need to do
any kind of weird flushing upon changing vertex inputs and can also remove
a flag from zink_resource

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: rework xfb counter resource barriers

using the new helper functions, we can now more accurately determine
exactly which barriers we need/want and avoid setting unnecessary barriers
or ending a renderpass early

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: set buffer resource barriers for descriptor resources in update_descriptors()

need to make sure these are properly synchronized

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

zink: add helper function for getting pipeline stage from shader stage

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8945>

panfrost/ci: Mark flaky test

I cannot reproduce the issue in local runs.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9105>

docs/features: Mark atomics/images done on Bifrost

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9105>

panfrost: Advertise FRAMEBUFFER_NO_ATTACHMENTS

Passes dEQP-GLES31.functional.fbo.no_attachments.*, it doesn't look like
we need to do anything special.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9105>

panfrost: Comment on state of ARB_shader_clock

Pending on kernel work.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9105>

pan/bi: Use explicit move even for RT#0 of MRT

Otherwise we get a bad RA if RT 0 = RT 3 (for example), fixes
dEQP-GLES31.functional.shaders.opaque_type_indexing.sampler.const_literal.fragment.sampler2d

Fixes: a6f1500bedd ("pan/bi: Workaround BLEND precolour with explicit moves")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9105>