nv50/ir: Initialize ValueDef member origin in constructors.

Fix defects reported by Coverity Scan.

Uninitialized pointer field (UNINIT_CTOR)
uninit_member: Non-static class member origin is not initialized in this
constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9036>

lavapipe: set viewport state dirty on first execute

This makes sure the viewport state gets emitted so that
draw gets viewports setup and the vertex shader doesn't crash.

Fixes bug uncovered with dEQP-VK.api.descriptor_set_layout_lifetime.graphics
once asan fix was done.

Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9093>

lavapipe: avoid pointer to pipeline layout in execution

pipeline layout lifetime is only during command buffer recording,
Don't store pointers to it, just extract them around it.

Fixes asan use-after-free in
dEQP-VK.api.pipeline.pipeline_layout.lifetime.destroy_after_end

Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9093>

lavapipe: add reference counting to descriptor set layout

asan flagged some use after frees, copy the anv solution
of refcounting the descriptor set layout objects.

dEQP-VK.api.descriptor_set.descriptor_set_layout_lifetime.*

Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9093>

ci/freedreno: Drop the "arm64" in front of job names.

All our jobs are arm64, and it made it hard to find jobs in the gitlab UI
which loves to truncate names.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9070>

ci/freedreno: Add a fractional gles31 run with asan enabled.

We have to disable the GLSL unit tests because with asan it runs way too
much code under qemu and times out.  Those unit tests have coverage on
x86, anyway.

I also included a vulkan run, which is disabled by default due to timeouts
that I need to sort out still.  It should be a useful tool for turnip
devs, though.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9070>

ci: Allow better customization of the name of the artifacts for minio.

To have asan testing, we need two different sets of artifacts per arch.
Plus, "UPLOAD_FOR_LAVA" was misnamed at this point anyway.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9070>

mesa/st: Make sure to unbind cb0 on transition away from gs/tess shaders.

This atom tries to unbind cb0 when it's not used any more (the params &&
params->NumParameters check), but if you transitioned to not having a
gs/tess enabled at all, you'd skip unbinding it.  This was mostly
harmless, since if you don't have a GS, why are you looking at GS
constants?  However, if a new program came along that didn't use cb0 at
all, we wouldn't end up in this atom to get the disable, and now you have
a GS enabled but a GS constbuf pointing at potentially freed data.

Dereferencing the freed cb0 data ended up happening in freedreno's
fallback UBO upload path with this combination of tests (which execute in
that order):

dEQP-GLES31.functional.shaders.opaque_type_indexing.sampler.uniform.geometry.sampler2darray
dEQP-GLES31.functional.shaders.opaque_type_indexing.ubo.const_literal_fragment
dEQP-GLES31.functional.shaders.opaque_type_indexing.ubo.dynamically_uniform_geometry

and it seems also affected softpipe as well.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9070>

v3d/qpu: Avoid leaking memory in the QPU disasm test.

Required to run this test under ASan, as we'll be soon doing for building
ARM drivers with asan testing.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9070>

freedreno: driver-thread annotations

Use clangs thread-safety annotations to implement a virtual lock
protecting context fields that should only be accessed from driver-
thread.  This should let the compiler help us detect problems where
ctx is used unsafely from things that could be called by the fe/st
thread.

This does end up sprinkled far and wide, it would be nice if the
compiler could be a bit smarter about understanding call-graphs
(at least with static fxns), but at least it makes it clear where
things are called from which thread.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9061>

freedreno: Split batch_flush_reset_dependencies()

Flushing a batch needs to happen on driver thread, but the reset-but-
dont-actually-flush case does not.  Decouple these two different cases
to prepare for thread-safety annotations.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9061>

freedreno: Quiet fallthrough warnings

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9061>

macros: Add thread-safety annotation macros

Extracted from !7529

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9061>

zink: move command pool to the batch

this simplifies things for the future when batches will have multiple
cmdbufs, letting us skip having to track/apply resets for them

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9064>

zink: handle dual blending override from driconf

when this is enabled, we need to push gl_FragData[1] to location 0 and
index 1 so that it gets blended like the application expects

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9095>

anv/formats: Advertise linear sampling on depth formats

They've all supported it since either forever or Iron Lake which is
equivalent to forever for Vulkan.

From Kenneth Graunke's GitLab review:

    "Linear blending of depth buffer data is usually fairly nonsense
    (something's 2 meters away?  another thing's 6 meters away?  let's
    just report 4 meters?)...but it's definitely a thing we can do, so
    we may as well let apps do it, and trust them not when it doesn't
    make sense."

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9110>

intel/anv: Fix condition for planar yuv surface

Test the sampler->conversion for NULL pointer before dereferencing it.

Fixes: Regressions in VulkanCTS.
Fixes: 226316116cd "intel/anv: Fix condition to set MipModeFilter for YUV surface"
Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

aco: remove special handling of load_helper_invocation

These should now behave the same as is_helper_invocation.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9058>

nir: lower is/load_helper to zero if no helper lanes are needed

If there are no helper invocations required during the
execution of the shader, we can assume that there also
are no helper invocations active.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9058>

nir: lower load_helper to is_helper if the shader uses demote()

load_helper_invocation is an Input Builtin, for which the
value should not change during the execution of a shader.
This new pass inserts an is_helper intrinsic before any
demote() instruction and re-uses its value.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9058>

intel/compiler: Use CMPN for min / max on Gen4 and Gen5

On Intel platforms before Gen6, there is no min or max instruction.
Instead, a comparison instruction (*more on this below) and a SEL
instruction are used.  Per other IEEE rules, the regular comparison
instruction, CMP, will always return false if either source is NaN.  A
sequence like

    cmp.l.f0.0(16)  null<1>F        g30<8,8,1>F     g22<8,8,1>F
    (+f0.0) sel(16) g8<1>F          g30<8,8,1>F     g22<8,8,1>F

will generate the wrong result for min if g22 is NaN.  The CMP will
return false, and the SEL will pick g22.

To account for this, the hardware has a special comparison instruction
CMPN.  This instruction behaves just like CMP, except if the second
source is NaN, it will return true.  The intention is to use it for min
and max.  This sequence will always generate the correct result:

    cmpn.l.f0.0(16) null<1>F        g30<8,8,1>F     g22<8,8,1>F
    (+f0.0) sel(16) g8<1>F          g30<8,8,1>F     g22<8,8,1>F

The problem is... for whatever reason, we don't emit CMPN.  There was
even a comment in lower_minmax that calls out this very issue!  The bug
is actually older than the "Fixes" below even implies.  That's just when
the comment was added.  That we know of, we never observed a failure
until #4254.

If src1 is known to be a number, either because it's not float or it's
an immediate number, use CMP.  This allows cmod propagation to still do
its thing.  Without this slight optimization, about 8,300 shaders from
shader-db are hurt on Iron Lake.

Fixes the following piglit tests (from piglit!475):

    tests/spec/glsl-1.20/execution/fs-nan-builtin-max.shader_test
    tests/spec/glsl-1.20/execution/fs-nan-builtin-min.shader_test
    tests/spec/glsl-1.20/execution/vs-nan-builtin-max.shader_test
    tests/spec/glsl-1.20/execution/vs-nan-builtin-min.shader_test

Closes: #4254
Fixes: 2f2c00c7279 ("i965: Lower min/max after optimization on Gen4/5.")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Iron Lake and GM45 had similar results. (Iron Lake shown)
total instructions in shared programs: 8115134 -> 8115135 (<.01%)
instructions in affected programs: 229 -> 230 (0.44%)
helped: 0

HURT: 1
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9027>

intel/compiler: Make the CMPN builder work like the CMP builder

Since the CMPN builder was never used, there was no reason to make its
interface usable. :)

Fixes: 2f2c00c7279 ("i965: Lower min/max after optimization on Gen4/5.")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9027>

intel/compiler: Enable the ability to emit CMPN instructions

v2: Move checks to the EU validator.  Suggested by Jason.

Fixes: 2f2c00c7279 ("i965: Lower min/max after optimization on Gen4/5.")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9027>

intel/eu/validate: Add some checks for CMP and CMPN

These checks were originally assertions elsewhere either in the existing
code or later in this MR.

Suggested-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9027>

virgl: Drop a context dependency from part of the shader compile path.

We should be exposing PIPE_CAP_SHAREABLE_SHADERS, but we aren't.  Clear up
one place that was using it so the next person has less to look through,
and document what's left.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9088>

gallium: Flip the default value of PIPE_CAP_SHAREABLE_SHADERS.

We should be exposing it in every driver, since it's required eventually
to reduce jank.  Make drivers have to explicitly opt out instead of opt
in.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9088>

ci: build gfxreconstruct v0.9.5

https://github.com/LunarG/gfxreconstruct/issues/328 and
https://github.com/LunarG/gfxreconstruct/issues/402 are already fixed
in the "master" branch.

Updated the piglit version so it supports this version.

Additionally, LunarG's VulkanTools are not built any more since
GFXReconstruct is now able to generate screenshots on its own without
using the VK_LAYER_LUNARG_screenshot layer.

v2:
  - Explain the VulkanTools removal in the commit log (Martin).

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9032>

ci: only install piglit dependencies when installing piglit

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9032>

ci: remove pytest since we don't need it any more

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9032>

microsoft/clc: Only apply float scaling to 32bit fdiv

Reviewed By: Bill Kristiansen <billkris@microsoft.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9091>

intel/anv: Fix condition to set MipModeFilter for YUV surface

Mip Mode Filter must be set to MIPFILTER_NONE for Planar YUV surfaces.
Add the missing condition to check for planar format.

Fixes: b24b93d5843 "anv: enable VK_KHR_sampler_ycbcr_conversion"
Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

ci: piglit runner colors diff output on failures

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Martin Peres <martin.peres@mupuf.org>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9104>

nir: Fix grammar error

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9103>

radv: do not allow to capture SQTT on the compute queue

This currently hangs and I don't know why. It looks better to
print a message instead of hanging the whole system.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

radv: stop emitting pipeline bind markers

RGP actually crashes if pipeline bind markers are emitted without
PSO correlation objects.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

radv: make sure to allocate enough space when emitting SQTT userdata

If we emit a lot of markers, we might reach the limit easily.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

radv: remove an outdated TODO about SQTT cache flushes

It's fully implemented now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

radv: move SQTT parameters initialization to radv_thread_trace_init()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

radv: create the start/stop CS for SQTT dynamically

This seems much cleaner and will help for future work. Also, this
might help if we want to dynamically change some SQTT parameters
in the future.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9097>

ci: tracie dashboard URLs only in the failure after the testcase

When adding the direct link to the diff page in the resulting
JUnit XML file we were not correctly skipping the first line from the
matching range.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4261
Fixes: 27f8c466486 ("ci: recover tracie dashboard URLs for failing traces")
Signed-off-by: Andres Gomez <agomez@igalia.com>
Acked-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8939>

ci: correct artifacts location for piglit's runner messages

We are now using pages.

v2:
  - Define a helper variable for the artifacts base URL (Juan).

Signed-off-by: Andres Gomez <agomez@igalia.com>
Acked-by: Eric Anholt <eric@anholt.net> [v1]
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9092>

ci: make sure piglit's artifacts are not overwritten

The ".test" template defines a generic artifact location to upload and
the ".piglit-test" a custom one we don't want to get overwritten by
the generic one.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9092>

ci: correct piglit's HTML summary location for artifacts upload

Signed-off-by: Andres Gomez <agomez@igalia.com>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9092>

ci: clean paths used in the piglit runner

When running with baremetal, the results path becomes //results. The
unexpected double backslash causes troubles when using sed
later.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9092>

radeonsi: force dcc clear to use compute clear

After the previous commit, when running the following
deqp-gles31 caselist:

  dEQP-GLES31.functional.image_load_store.2d.format_reinterpret.rgba32f_rgba32ui
  dEQP-GLES31.functional.image_load_store.2d.format_reinterpret.rgba32f_rgba32i

The second test always fails on gfx10. I don't know why,
but forcing the dcc clear from si_decompress_dcc to use
compute fixes the problem.

The test caselist wasn't failing before because the dcc
disable step was done in si_resource_copy_region, before
calling si_compute_copy_image.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8958>

radeonsi: enable dcc image stores on gfx10+

This was implemented in 1d3bffaf9cb7ade0676bab969b5d33d6bdabcec8,
but missing the WRITE_COMPRESS_ENABLE bit, then disabled by
4dc6ed2a59040f04648eadbffeb1522587d00f3.

This commits reimplements it to:
- avoid disabling dcc when uploading FP16 textures
  (see si_use_compute_copy_for_float_formats)
- being able to use compute to upload textures in more cases, rather
  than using the blit path

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8958>

radeonsi: replace force_cp_dma arg of si_clear_buffer by enum

The new enum has 3 values:
 - SI_CP_DMA_CLEAR_METHOD: equivalent to force_cp_dma = true
 - SI_COMPUTE_CLEAR_METHOD: to force the clear to use compute
 - SI_AUTO_SELECT_CLEAR_METHOD: equivalent to force_cp_dma = false

No functional change yet, but this will be used later.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8958>

radv,aco: allow unaligned LDS access on GFX9+

fossil-db (GFX10.3):
Totals from 223 (0.16% of 139391) affected shaders:
SGPRs: 10032 -> 10096 (+0.64%)
VGPRs: 7480 -> 7592 (+1.50%)
CodeSize: 853960 -> 821920 (-3.75%); split: -3.76%, +0.01%
MaxWaves: 5916 -> 5908 (-0.14%)
Instrs: 154935 -> 150281 (-3.00%); split: -3.01%, +0.01%
Cycles: 3202496 -> 3080680 (-3.80%); split: -3.81%, +0.00%
VMEM: 48187 -> 46671 (-3.15%); split: +0.29%, -3.44%
SMEM: 13869 -> 13850 (-0.14%); split: +1.52%, -1.66%
VClause: 3110 -> 3085 (-0.80%); split: -1.03%, +0.23%
SClause: 4376 -> 4381 (+0.11%)
Copies: 12132 -> 12065 (-0.55%); split: -2.61%, +2.06%
Branches: 5204 -> 5203 (-0.02%)
PreVGPRs: 6304 -> 6359 (+0.87%); split: -0.10%, +0.97%

See https://reviews.llvm.org/D82788

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8762>

radv: relax shared alignment requirements in mem_vectorize_callback

fossil-db (GFX10.3):
Totals from 13 (0.01% of 139391) affected shaders:
CodeSize: 241316 -> 238684 (-1.09%)
Instrs: 42251 -> 41912 (-0.80%); split: -0.81%, +0.01%
Cycles: 1991192 -> 1974668 (-0.83%); split: -0.83%, +0.00%
VMEM: 4067 -> 4102 (+0.86%)
SMEM: 569 -> 578 (+1.58%)
SClause: 848 -> 858 (+1.18%)
Copies: 3077 -> 2992 (-2.76%); split: -3.12%, +0.36%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8762>

gallium/trace: add a pipe_screen::get_compiler_options method

this fixes crashes on startup

Fixes: a3512ddfdf7 ("st/mesa: don't enable NV_copy_depth_to_color if NIR doesn't support FP64")

fixes mesa/mesa#4312

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9082>

glthread: ignore the return value of glUnmapBuffer, don't sync, and return true

We always return GL_TRUE from the Unmap functions.

gl_marshal.py is modified so as not to use "return" in the unmarshal
function, which always returns void.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9029>

glthread: don't sync when using pixel buffer objects

The pointer is a GPU offset if a PBO is bound.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9029>

glthread: don't declare pointers with const in unmarshal functions

so that GL functions with a non-const pointer don't print a warning when
we call them, such as glGetTexImage with a PBO where the pointer is really
just an offset.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9029>

glthread: assume all parameters are fixed if marshal_sync is present

We want glthread to ignore variable-sized parameters if the only thing
we want is to pass the pointer parameter as-is, e.g. when a PBO is bound.
Making it conditional on marshal_sync is kinda hacky, but it's the easiest
path towards handling PBOs, which will use marshal_sync to check whether
a PBO is bound.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9029>

radeonsi: set MEM_ORDERED optimally

It must be 1 only if both sampler and non-sampler VMEM instructions
that return something are used. BVH counts as a sampler instruction.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: gather shader info about VMEM usage for MEM_ORDERED

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: gather shader info about indirect UBO/SSBO/samplers/images

A future commit will use it.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: gather info about bindless images and memory stores with strstr(intr)

This is only code simplification.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: fix the value of uses_bindless_samplers

We don't have any nir_variables for uniforms, so this code wasn't
doing anything. Also, uniform handles are almost always uniforms.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: do late NIR optimizations after uniform inlining

This was missing.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: allocate filled_size for streamout targets in set_streamout_buffers

so that create_stream_output_target doesn't use the context and can be
called from any thread. This is for u_threaded_context.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: improve comments in si_emit_derived_tess_state

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: for tess, determine the minimum num_patches before optimizing tg size

Doing these MINs at the end could have undone optimizations for the LDS
size and threadgroup size, so move the MINs up.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

ac/surface: use family_id so as not to crash with SI_FORCE_FAMILY in addrlib

We can't invoke gfx6 addrlib (overridden by SI_FORCE_FAMILY) with a gfx9
family ID.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9028>

radeonsi: fix si_check_render_feedback

si_check_render_feedback only relied on si_images::enabled_mask and
si_samplers::enabled_mask to determine if a texture was being used
both as input and output.

Given that some samplers/images can be considered active (so accounted
for by enabled_mask) but not used by the current shader this could
lead to false-positive.

This commit fixes this by and-ing the above mask with the information
from shader_info for each active shader.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4227
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8869>

radeonsi: fix read from compute / write from draw sync

A compute dispatch should see the result of a previous draw command.
radeonsi was missing this implicit sync, causing rendering artifacts:
the compute shader was reading from a texture still being written to
by the previous draw.

Framebuffer BOs are marked with RADEON_USAGE_NEEDS_IMPLICIT_SYNC,
so compute jobs will sync.

v2: use RADEON_USAGE_NEEDS_IMPLICIT_SYNC
v3: unconditionally make CB coherent after a flush

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com> (v3)
Reviewed-by: Marek Olšák <marek.olsak@amd.com> (v3)
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4032
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2878
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/1336
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8869>

radeonsi: store si_context::xxx_shader members in union

This allows to access them individually (sctx->shader.ps) or
using array indexing (sctx->shaders[PIPE_SHADER_FRAGMENT]).

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8869>

radeonsi: fix indentation issue in si_texture.c

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8869>

broadcom/compiler: don't dump shader-db stats for failed shaders

Shaders that fail register allocation were dumped with an instruction
count of 0, so getting them to compile would show up as an instruction
count regression. Also, the LOST/GAINED stats depend on us not dumping
data for failed shaders, which is why we were always seeing 0/0 there.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9077>

broadcom/compiler: use a helper function to decide on TMU spilling

As we add more compiler optimizations that can increase register pressure
we may decide to disallow TMU spilling in more cases so it is probably
better to move this to its own helper function.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9077>

broadcom/compiler: don't emit redundant ldunif

If we emit a new uniform and that uniform has already been emitted
in the same block we can just reuse that.

There is a balancing game here between reducing ldunif instructions
and not increasing register pressure too much though, so we put
a limit to how far back we are willing to look for a previous
definition of the uniform. Based on shader-db results, 20 instructions
produces best results.

total instructions in shared programs: 14928266 -> 14907432 (-0.14%)
instructions in affected programs: 6431841 -> 6411007 (-0.32%)
helped: 15270
HURT: 10772
Instructions are helped.

total uniforms in shared programs: 3944672 -> 3840276 (-2.65%)
uniforms in affected programs: 1827184 -> 1722788 (-5.71%)
helped: 30423
HURT: 845
Uniforms are helped.

total inst-and-stalls in shared programs: 14957813 -> 14936873 (-0.14%)
inst-and-stalls in affected programs: 6475349 -> 6454409 (-0.32%)
helped: 15287
HURT: 10852
Inst-and-stalls are helped.

v2 (Eric):
 - consider ldunifrf too
 - check that no other instruction writes to the register

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9077>

turnip: VK_EXT_memory_budget implementation

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8524>

turnip: keep track of memory heap usage, size and flags

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8524>

util: fix parsing of /proc/meminfo MemAvailable value

It was interpreting the value as hexadecimal when it is unsigned.

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8524>

etnaviv: Fix memory leak in etna_vertex_elements_state_create.

Fix defect reported by Coverity Scan.

Resource leak (RESOURCE_LEAK)
leaked_storage: Variable cs going out of scope leaks the storage it points to.

Fixes: c9e8b49b885 ("etnaviv: gallium driver for Vivante GPUs")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9034>

intel/fs: Shuffle can't handle source modifiers

On Gen7, we have to split shuffles into two MOVs for 64-bit types so we
can't handle source modifiers.  On Gen12.5, we have to use integer types
all the time so we can't use them there either.  Fixing that will be a
different commit but it interacts with this one.

Fixes: 90c9f29518d "i965/fs: Add support for nir_intrinsic_shuffle"
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9068>

nir/opt_large_constants: Handle generic pointers

We already throw out any variables which may have a complex use so we
just need to make sure that our mode checks don't assert if we have a
deref which may_be but not must_be nir_var_function_temp.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9068>

nir: Fix parameter order in the bcsel-of-shuffle optimization

Fixes: 4ff4d4e56966a40 "nir/opt_intrinsic: Optimize bcsel(b, shuffle..."
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9068>

nir: Don't optimize bcsel-of-shuffle across blocks

We can't move the shuffle to a new block so this only works if the
shuffle and the bcsel are in the same block.  Fortunately, in the
motivating case, this is true.

Also, we have to be careful around discard.  We could try really hard to
just avoid moving them past discard but we choose to simply bail if we
see a discard instead.

Fixes: 4ff4d4e56966a40 "nir/opt_intrinsic: Optimize bcsel(b, shuffle..."
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9068>

nouveau: Use format modifiers in buffer allocation

The nvc0 nouveau backend already claimed to support format modifiers, but
in practice it ignored them when allocating buffers outside of a
perfunctory check for the linear modifier in the first element of the
format modifier list.

This change deduces the supported modifiers, if any, for a given miptree
creation request, prioritizes them based on performance and memory waste
properties, compares the requested modifiers against the prioritized list
of supported modifiers, and overrides the internal layout calculations
based on the layout defined by the resulting modifier.

Additionally, if modifiers are provided and none are compatible with the
miptree creation request, the function now fails.  This brings the nouveau
behavior in line with other drivers such as i965 and etnaviv.

Signed-off-by: James Jones <jajones@nvidia.com>
Tested-by: Karol Herbst <kherbst@redhat.com>
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3724>

nouveau: no modifier != the invalid modifier

Other drivers fail resource allocation when a list of modifiers for the
resource is provided but none are supported. This includes cases when the
never-supported DRM_FORMAT_MOD_INVALID modifier is explicitly passed.  To
enable matching that functionality in nouveau, use an empty modifier list
rather than creating a one-entry list containing only
DRM_FORMAT_MOD_INVALID when the non-modifier resource creation function is
used.

This change stops short of failing allocations when no modifier is
specified, because the current code ignores all modifiers except the linear
modifier when creating resources, so there is not yet a framework in place
to determine which modifiers are valid for a given resource creation
request, and hence no way to reject only those which are invalid.

Signed-off-by: James Jones <jajones@nvidia.com>
Tested-by: Karol Herbst <kherbst@redhat.com>
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3724>

nouveau: Use DRM_FORMAT_MOD_NVIDIA_BLOCK_LINEAR_2D

Replace existing usage of the NVIDIA_16BX2_BLOCK format modifiers with
parameterized use of the more general macro.  Nouveau will now report
support for slightly different modifiers depending on whether the
underlying chip is a tegra GPU or not, and will potentially report valid
format modifiers for more resource types, but overall this should be a
functional no-op for existing applications.

Signed-off-by: James Jones <jajones@nvidia.com>
Tested-by: Karol Herbst <kherbst@redhat.com>
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3724>

nouveau: Stash supported sector layout in screen

Older Tegra GPUs use a different sector bit swizzling layout than desktop
and Xavier GPUs. Hence their format modifiers must be differentiated from
those of other GPUs.  As a precursor to supporting more expressive block
linear format modifiers, deduce the sector layout used for a given GPU from
its chipset and stash the layout in the nouveau screen structure.

Signed-off-by: James Jones <jajones@nvidia.com>
Tested-by: Karol Herbst <kherbst@redhat.com>
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3724>

zink: add a disk cache for pipeline objects

this writes the pipeline cache to disk on shutdown

ideally we'd rather write this incrementally any time we make a new pipeline,
but that ends up breaking the disk cache infrastructure since we're always writing
to the same file, so this is the best we can do for now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9094>

zink: create a VkPipelineCache object on the screen and use it

this enables reuse of shaders across pipelines

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9094>

zink: hook up valid_buffer_range for buffer resources using util_range

this lets us avoid stalling during transfer map in some cases

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9062>

zink: add flag for no-oping fence finish

if a fence hasn't been activated then we don't need to wait on it

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9067>

freedreno/ir3: Add missing shader prog cache invalidation

I'm not aware of actually hitting this case, but we need to consider
more than just vs/fs when shader state is deleted.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Async shader compile

Draw-time variants are still synchronous, but I'm not sure there is much
(easy) benefit from generating them asynchronously.  Without patching
the cmdstream later before batch submit, we'd end up waiting for them
immediately.  But we should mostly only hit draw-time variants for
desktop GL (and mostly legacy features).

Note: new xfb xfail on a5xx, but most of the xfb tests are already xfail
so I think we just managed to change the timing a bit, rather than this
being related to async compile.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3857
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Reshuffle compute state creation

There was just a single remaining caller of ir3_shader_create_compute(),
so fold that into ir3_shader_compute_state_create().

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Reshuffle ir3_shader_create()

This had only a single caller, so no need to be exported.  With
that done, fold the ir3_shader creation (ie. the cheap part) into
ir3_shader_state_create(), and rename what is left.

This is prep to moving initial variant creation to a work queue.

It does slightly change the error handling, in that we don't
cleanup the shader hwcso.  We wouldn't be able to do this anyways
with async compile.  But it ends up using the same error handling
paths that we'd hit if we got a compile failure for a draw-time
variant.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Add ir3_screen_fini()

Move ir3_compiler_destroy() into ir3_screen_fini().  This gives us a
good place to cleanup an sync compile queue.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Move ir3_compiler_create()

All the ir3 using backends already call ir3_screen_init(), so lets just
move compiler creation there.

In a subsequent patch, we'll add initialization of the queue for async
compile.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3: Add ir3_shader_state

Initially just a wrapper for ir3_shader, but this is where we'll hook in
the bookkeeping for async compile.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/ir3+a5xx+a6xx: De-duplicate create_compute_state()

These were identical between a5xx and a6xx, so move into shared helper
that can be directly plugged into pctx, similar to the various 3d shader
state creation.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/a6xx: Drop fd6_compute_stateobj

It wasn't really doing anything useful.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno/a5xx: Drop fd5_compute_stateobj

It wasn't really doing anything useful.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

freedreno: Misc cleanup

Some whitespace cleanup + comment addition.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8795>

zink: use nir_shader_instructions_pass for draw params pass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9090>

clover: Add -fgnu89-inline to Clang command line

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9085>