crocus: align staging resource pitch on gen4/5 to allow BLT usage.

Aligning the pitch to 4 bytes allows the BLT engine to be used for
transfers to/from these surfaces.

Fixes: f3630548f1da ("crocus: initial gallium driver for Intel gfx 4-7")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12329>

crocus/blt: add pitch/offset checks to fix blt corruption

I lost these in my conversion from i965 but they are necessary.

This should fix corruption in qt fonts at seen in the minecraft
launcher.

Fixes: f3630548f1da ("crocus: initial gallium driver for Intel gfx 4-7")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12329>

pan/bi: Unit test DISCARD+FCMP fusing

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Fuse DISCARD with conditions

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Add fclamp unit tests

The negative cases here did not pass before this series, showing the bug
in the clamp optimization. By introducing the FCLAMP pseudo op, the bug
is fixed. Let's ensure we don't regress.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Use FCLAMP pseudo op for clamp prop

Map nir_op_fsat/etc to FCLAMP pseudo ops, instead of FADD. There are
significantly fewer knobs on FCLAMP, meaning significantly fewer things
to get wrong.

This fixes two(!) classes of bugs:

* Swizzles (failing to lower/compose swizzles on clamps)
* Numerical bugs (incorrectly treating +0.0 as an additive identity)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Add optimizer unit tests

Writing these tests brought to light the cluster of bugs fixed in the
previous commits. Now that things work, let's ensure they stay working.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Use FABSNEG pseudo ops for modifier prop

Simplifies pattern matching. This commit by itself fixes multiple
numerical issues -- the previous fabsneg check failed to check the round
mode or the sign of the zero. That will break Vulkan/OpenCL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Add shader equality helper for unit tests

Optimizer tests really are global.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Fuse abs/neg more on Valhall

Some of these Bifrost restrictions may be skipped on Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Simplify bi_compose_clamp

Realized this trick when reversing Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Unit test new constant folding patterns

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/bi: Constant fold texturing lowerings

This ensures we can constant fold the ALU ops used to lower:

* explicit LOD calculations
* array textures
* texture offsets
* multisample indices

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

pan/va: Document IEEE 754 conformance of clamps

These rules are not obvious. But they turn out to be exactly what's
required by the spec.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12205>

panfrost: Test src*dst + dst*src blending

Validates the prior commit.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Leverage Bifrost's 2*src blend factor

Bifrost adds a value for the C factor equaling 2*src. This does not
correspond directly to API blend modes so it is not too useful in
general. However, it's required for src*dest + dest*src blending to be
done in hardware instead of a blend shader. GFXbench uses that blend
mode, so it must be important ;-)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Add basic fixed-function blending tests

Add unit tests for the fixed-function blending helpers in pan_blend.c.
Each test consists of a Porter-Duff blend mode and the associated
hardware state. In this commit, we add tests for the most common modes.

For motivation, this code has NOT been properly tested in CI. True,
functional correctness of the blend module as a whole is tested by
dEQP-GLES3.functional.fragment_ops.blend.* among other integration
tests. However, this testing is insufficient to check for regressions.
Crucially, the following broken patch would clear CI:

   bool pan_can_fixed_function(...) {
      return false;
   }

In that case, blend shaders are used 100% of the time, which will
regress performance horribly but still pass dEQP. The only clue
something went wrong would be some traces changing checksum due to the
fixed-function blender producing slightly different output than
equivalent blend shaders. By unit testing the fixed blend path, we
ensure we always use the fixed-function path when we expect it to.

Similarly, using incorrect values for the blend metadata may not affect
functional correctness but will increase power consumption. Let's check
all the data we export to drivers.

Note: due to additive commutativity, there are many pairs of equivalent
Mali blend modes. Unfortunately, the vendor is... inconsistent about how
to resolve ambiguous modes. Our algorithm for computing modes is
correct; the "preferred" values are left in comments since otherwise our
tests fail despite correct code. I want to blame Bifrost for this, but
Midgard was patient zero.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Simplify blend_factor_constant_mask

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Fix is_opaque when blend_enable=false

Needed to pass the "replace" unit test.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Add blend helper packing the equation

This is more convenient for the Gallium driver and easier to test.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Use _PU for non-dithered formats

This is required to disable dithering on a per-draw basis when OPAQUE
output is used (bypassing the blender which normally uses the
round_to_framebuffer_precision flag to do the same).

This functionally reverts:

   ebc07f4b2f3 ("panfrost: Remove padded unorm blendable formats")
   fae90a79404 ("panfrost: Always pick dithered tb formats")

while adding the functionality to make them useful.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12152>

panfrost: Remove unused #defines

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12328>

panfrost: Add LINEAR debug option

Useful to cross off CPU texture tiling as the source of bugs.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12328>

pan/bi: Add a noopt debug option

To rule out buggy optimization passes when debugging.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12328>

pan/bi: Make bi_opt_push_ubo optional

It's an optimization pass -- omitting it should not cause MMU faults
(!). Make sure the UBO push mask is set regardless of whether the pass
is called, and just call the pass when required.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12328>

nir/lower_shader_calls: remove empty phis

This is confusing opt_cse.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 8dfb240b1f0633 ("nir: Add raytracing shader call lowering pass.")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11953>

zink: use nir_shader_instructions_pass in nir_lower_dynamic_bo_access

Changes:
- nir_metadata_preserve(..., nir_metadata_dominance)
  is called only when pass makes progress
- nir_metadata_preserve(..., nir_metadata_all) is called when pass doesn't
  make progress

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

zink: use nir_shader_instructions_pass in lower_discard_if

Changes:
- nir_metadata_preserve(..., nir_metadata_dominance)
  is called only when pass makes progress
- nir_metadata_preserve(..., nir_metadata_all) is called when pass doesn't
  make progress

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

microsoft/compiler: use nir_shader_instructions_pass in dxil_nir_lower_double_math

No functional changes.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

microsoft/compiler: use nir_shader_instructions_pass in dxil_nir_split_clip_cull_distance

No functional changes.

v2: fix build

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

microsoft/compiler: preserve all metadata when upcast_phi doesn't make progress

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

microsoft/clc: use nir_shader_instructions_pass in clc_nir_dedupe_const_samplers

Changes:
- nir_metadata_preserve(..., nir_metadata_all) is called when pass doesn't
  make progress

v2: fix build

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

microsoft/clc: preserve only valid metadata in clc_lower_printf_base

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

d3d12: use nir_metadata_none instead of its value

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

intel/compiler: use nir_shader_instructions_pass in brw_nir_apply_attribute_workarounds

Changes:
- removal of attr_wa_state (it's passed directly)
- nir_metadata_preserve(..., nir_metadata_all) is called when pass doesn't
  make progress

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

nir/builder: invalidate metadata per function

Fixes: a62098fff20 ("nir: Add a helper for general instruction-modifying passes.")
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>

freedreno/decode: print estimated crash location without colored output

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12302>

nir: add a pass to optimize "gl_FragDepth = gl_FragCoord.z" away

gl_FragDepth default value is gl_FragCoord.z so if a shader does:

   gl_FragDepth = gl_FragCoord.z

we can drop this assignment.

v2: use nir_ssa_scalar_resolved and don't do this is gl_FragDepth
    is wrote multiple times (Jason)
v3: - move to its own pass (Jason)
    - handle var = NULL (Rhys)
v4: refactoring (Jason)

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10697>

iris: Drop dead drm_ioctl prototype

We now use intel_ioctl instead.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12206>

iris: Improve the memory layout of iris_bo by fixing pahole issues

We had a 4 byte hole and a 4-byte field breaking up a run of bools.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12206>

iris: Rename bo->gtt_offset to bo->address

This is the virtual memory address of the buffer object.  Calling it the
BO's address is a lot more obvious than calling it an offset in one of
the now many graphics translation tables.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12206>

v3d,v3dv: add options to force 32-bit or 16-bit TMU precision

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12303>

anv/android: fix build error due refactoring

Fixes: e08370dc37e ("anv: disable aux for exportable images without modifiers")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5208
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12300>

docs: add llvmpipe host memory extensions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12316>

lavapipe: add host ptr support.

This actually doesn't need any backend support.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12316>

llvmpipe: add support for user memory pointers

This is useful for clover, but throw it at CI at least

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12316>

pan/bi: Use the computed scale for fexp NaN propagation

This makes pow(NaN, x) return NaN rather than 1.0.

Fixes: 499397700c1 ("pan/bi: Don't lower fpow")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5189
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12269>

Revert "nir/algebraic: Convert some f2u to f2i"

Per https://gitlab.freedesktop.org/mesa/mesa/-/issues/5178#note_1019666,
the assumption fundamental to this optimization is false.  Section
2.4.1 (Float to Integer) of Ivy Bridge PRMs describes the situation.
The wording of the section is somewhat confusing (because it doesn't
clearly delineate between signed and unsigned integers), but the last
two rows of the table make it clear that F->UD conversion clamps
negative float values to 0.

All other hardware mentioned in that thread seems to behave the same
way.

The real problem is that, with hardware that behaves in this ways,
converting f2u(2147483648.0) to f2i(2147483648.0) changes the bit pattern
that would be produced from 0x80000000 to 0x7fffffff.

This reverts commit ad059202583e8c86bbccf0d65c5ce35bc4ab20f1.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12297>

nir/opcodes: Use u_intN_(min|max)

uadd_sat was updated using sed, so I didn't even notice the surrounding
opcodes.  Oops.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12297>

clover: only return CLC version as 1.2 (even for 3.0)

Fixes CTS compiler opencl_c_versions

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12286>

clover/nir: don't convert to NIR on library link

If just creating a library, just link the spir-v and store it.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12286>

clover: fix compilation with clang + llvm 12.

clang in llvm 12 no longer accepts "-cl-denorms-are-zero" as a cc1
options which is how this code uses it.

For now just pick the correct cc1 equivalent.

This fixes a crash with llvm master and CL conversions tests

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12286>

pick-ui: show commit date

With our ff-only merge setup, the commit date ends up being when the
commit actually landed (as opposed to when it was first written).

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12289>

pick-ui: show nomination type in the UI

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12289>

pick-ui: drop assert that optional argument is passed

Let's just make it not-optional instead.

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12289>

freedreno: Use correct key for binning pass shader

We updated the key correctly for whether we wanted to use a
safe_constlen binning pass variant, but then passed the wrong
key to ir3_shader_variant().

Fixes: 1dd24bf27b2 ("freedreno: Share constlen between different stages properly")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12314>

nir/lower_mediump: Fix metadata in all passes

Fixes: fb29cef8dda ("nir: add many passes that lower and optimize 16-bit input/outputs and samplers")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11732>

nir/lower_mediump_io: Don't remap base unless needed

Otherwise drivers that don't use 16-bit slots for varyings will get
confused and have their driver_locations scribbled over. This has caused
multiple problems for both Panfrost and Asahi this week. Given the only
other user of the pass for varyings is radeonsi, which needs both
together, I think this is the least controversial fix.

Fixes: fb29cef8dda ("nir: add many passes that lower and optimize 16-bit input/outputs and samplers")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11732>

tu: add "flushall" and "syncdraw" debug options

They will be useful to check whether some issue is due to the lack
of flushing or waiting.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12283>

nine: init more draw info members

Reviewed-by: <Axel Davy davyaxel0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12284>

nine: init take_index_buffer_ownership for draws

Reviewed-by: <Axel Davy davyaxel0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12284>

u_driconf: Use a macro to avoid repeating option names

Suggested-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12158>

wgl: Add a driver name for driconf

Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12158>

wgl: Parse driconf options

Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12158>

xmlconfig: Use static inline for regex fallback to prevent -O0 issues

A non-static inline function body is only actually emitted by GCC during optimization passes,
so running -O0 ends up never emitting the body, producing linker errors.

Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12158>

gallium/dri: Move driConf -> st option processing to aux/util

Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12158>

tu: Read some input attachments directly

It can happen that the user reads an input attachment as the first use
of that attachment. In that case there are no subpass dependencies
required at all, because there could be a pipeline barrier before the
renderpass instead, and in any case we assume that dependencies with the
first subpass as a destination can be executed only once outside the
renderpass. The result is that we only do a CACHE_INVALIDATE once
before the entire renderpass, but it's actually required after each GMEM
load, because input attachments read GMEM through UCHE and those writes
to GMEM invalidate UCHE.

While we could add the missing CACHE_INVALIDATE "by hand" somehow, it
turns out it's actually just as easy to do an optimization the blob
does, where it simply doesn't patch those input attachments and reads
them directly instead. This means we can skip allocating memory in GMEM
for them entirely in some circumstances.

This fixes e.g.
dEQP-VK.api.copy_and_blit.core.resolve_image.whole_array_image.4_bit
with TU_DEBUG=forcebin.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12213>

intel/eu: Set scope to TILE for TGM flushes

Setting it to GPU can cause an L3$ flush in certain cases.  That's not
what we want as we really only care about coherency within the GPU.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Sagar Ghuge <sagar@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12291>

radv: allow fast clears for concurrent images if comp-to-single is supported

Only GFX10+ is affected because older chips don't support
comp-to-single. For them, we need to implement FCE on compute with DCC
and eventually CMASK.

Fixes the gap between concurrent vs exclusive queue with Scarlet Nexus,
also gives a boost with Doom Eternal.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12088>

glsl: evaluate switch expression once

v2: intialize test_val in constructor

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5185

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Cc: mesa-stable
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12234>

broadcom/compiler: rewrite partial update liveness tracking

The code we had for this was a work in progress and not finished. Also,
it was geared towards partial writes caused by output packing (i.e.
fp16) and was ignoring partial updates caused by conditional writes,
which are far more common in our case.

This change provides an implementation for tracking conditional writes
that works in tandem with the previous spill change to narrow liveness
for their spills.

Fixes register allocation failures in:
dEQP-VK.graphicsfuzz.spv-stable-maze-flatten-copy-composite

We also gain one shader from shader-db:

total instructions in shared programs: 13339969 -> 13338584 (-0.01%)
instructions in affected programs: 185520 -> 184135 (-0.75%)
helped: 375
HURT: 130
Instructions are helped.

total threads in shared programs: 412038 -> 412040 (<.01%)
threads in affected programs: 2 -> 4 (100.00%)
helped: 1
HURT: 0

total uniforms in shared programs: 3746581 -> 3746585 (<.01%)
uniforms in affected programs: 49 -> 53 (8.16%)
helped: 0
HURT: 1

total max-temps in shared programs: 2359960 -> 2359947 (<.01%)
max-temps in affected programs: 289 -> 276 (-4.50%)
helped: 7
HURT: 0
Max-temps are helped.

total sfu-stalls in shared programs: 34351 -> 34359 (0.02%)
sfu-stalls in affected programs: 218 -> 226 (3.67%)
helped: 35
HURT: 37
Inconclusive result (value mean confidence interval includes 0).

total inst-and-stalls in shared programs: 13374320 -> 13372943 (-0.01%)
inst-and-stalls in affected programs: 186653 -> 185276 (-0.74%)
helped: 373
HURT: 132
Inst-and-stalls are helped.

LOST:   0
GAINED: 1

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12278>

broadcom/compiler: make spills of conditional writes also conditional

A spill of a conditional write generates code like this:

mov.ifa t5000, 0
mov tmud, t5000
nop t5001; ldunif (0x00008100 / 0.000000)
add tmua, t11, t5001

Here, we are spilling t5000, which has a conditional write, and we
produce an inconditional spill for it. This implicitly means that
our spill requires a correct value for all channels of t5000.

If we do a conditional spill, then we emit:

mov.ifa t5000, 0
mov tmud.ifa, t5000
nop t5001; ldunif (0x00008100 / 0.000000)
add tmua.ifa, t11, t5001

Which only uses channels of t5000 that have been written by the
instruction being spilled.

By doing the latter, we can then narrow down the liveness for t5000
more effectively, as we can use this to detect that the block only reads
(in the tmud instruction) the values that have been written previously
in the same block (in the mov instruction). This means that values in
other channels are not used, and therefore, we don't need them to be
alive at the start of the block. This means that if this is the only
write of t5000 in this block, we can consider that the block
completely defines t5000.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12278>

broadcom/compiler: Flags are per-thread state in V3D 4.2+

This means they survive a thread switch, so we can remove redundant
flag setups across thread switches.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12278>

broadcom/compiler: add a vir_get_cond helper

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12278>

radv: enable DCC fast-clears with comp-to-single on GFX10+

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: skip FCE for images that are fast-cleared using comp-to-single

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: implement DCC fast clears with comp-to-single

When an image supports comp-to-single, DCC is cleared to 0x10 (single)
and the clear color value is written to the beginning of each 256B
block in the image.

This allows to skip FCE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: determine if an image support fast clears using comp-to-single

Only on GFX10+ with DCC enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: add RADV_DCC_CLEAR_SINGLE

When DCC is cleared with that code, the hardware expects the clear
color value to be stored at the beginning of each 256B block in
the image.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: pass an image view to vi_get_fast_clear_parameters()

image_format was unused.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

radv: use more explicit DCC clear codes

No functional changes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10518>

virgl/ci: Set NIR_VALIDATE=0 on the host

As we aren't testing LLVMPipe in these jobs, and shader compilation is
currrently the bottleneck.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12196>

virgl/ci: Wait a bit before shutting the VM down

Sometimes, the VM powered off before all the output from the guest got
to the console.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12196>

virgl/ci: Rebalance concurrency

Crosvm deals with virtio-gpu commands sequentially, so parallelization
in the host doesn't help much.

Also, too much parallelization in the guest causes some tests to time
out.

So reduce the number of dEQP instances being run concurrently, make sure
we dont limit the number of CPUs being used in the host and schedule
more jobs in CI to keep the times below 10 minutes.

Closes: #5172
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12196>

virgl/ci: Have LLVMPipe use more threads for rendering

dEQP isn't high on rendering, but that is in the critical path as all
dEQP processes are waiting for Crosvm to single-threadedly service their
requests.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12196>

radv: fix reported sample counts for VRS 1x1

The Vulkan spec requires ~0 for 1x1.

Fixes dEQP-VK.fragment_shading_rate.misc.shading_rates.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12245>

radv: bump maxFragmentShadingRateCoverageSamples to 32

Minimum required value is 16 but we support up to 32
(2x2 VRS with MSAA 8x).

Fixes dEQP-VK.fragment_shading_rate.misc.limits.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12245>

radv: disable fragmentShadingRateWithCustomSampleLocations

From the Vulkan spec 1.2.187.

"fragmentShadingRateWithCustomSampleLocations specifies whether
 custom sample locations are supported for multi-pixel fragments.
 It must be VK_FALSE if VK_EXT_sample_locations is not supported."

VK_EXT_sample_locations is disabled on GFX10+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12245>

radv: bump maxFragmentSizeAspectRatio to 2

Minimum required value is 2.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12245>

ci/vc4: update piglit expected results

Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12281>

docs: flag VK_KHR_multiview as implemented for v3dv

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12279>

etnaviv: always try to create KMS side handles for imported resources

By creating the KMS side handles we allow GBM to return the proper KMS
side GEM handles for imported buffers. Always creating the KMS side
handles adds a bit of overhead, as we don't need them on all imported
resources, but seems like the most robust solution for now.

Cc: mesa-stable
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12018>

renderonly: don't complain when GPU import fails

There are a number of drivers which do a trial-and-error import
of buffers into the KMS side via renderonly. Some of those imports
are expected to fail, so we should not print a error message in
this case. All callers do proper error handling themselves.

Cc: mesa-stable
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12018>

clover: add kernel attributes support for SPIR-V

Fixes CTS api kernel_attributes

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12225>

clover: return CL_INVALID_PLATFORM properly.

If the platform isn't clover platform return an error,

Fixes CTS api negative_get_platform_info

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12225>

clover: fix api zero sized enqueue

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12225>

radv: flush caches before performing separate depth/stencil aspect init

It's a RMW operation, also note that DB doesn't use L2 on GFX6-8.

Fixes test_clear_depth_stencil_view() and test_discard_resource() tests
from vkd3d-proton.

Cc: 21.2 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12223>

anv: Make anv_image_aspect_to_plane take an anv_image*

It's called anv_image_* so it really should take an anv_image.  For the
couple of cases where we really want to pass in a set of aspects, we
leave an anv_aspect_to_plane() helper.  anv_image_aspect_to_plane() is
then just a wrapper around it which grabs the aspects from the image.

While we're in the area, sprinkle some const around.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12141>

anv: Rework our aspect/plane helpers

The new versions should have identical output, just a simpler (and
probably faster) implementation and more/better asserts.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12141>

anv/image: Rework YCbCr image aspects

The Vulkan 1.2.184 spec says:

    "When creating a VkImageView, if sampler Y′CBCR conversion is
    enabled in the sampler, the aspectMask of a subresourceRange used by
    the VkImageView must be VK_IMAGE_ASPECT_COLOR_BIT.

    When creating a VkImageView, if sampler Y′CBCR conversion is not
    enabled in the sampler and the image format is multi-planar, the
    image must have been created with
    VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT, and the aspectMask of the
    VkImageView’s subresourceRange must be VK_IMAGE_ASPECT_PLANE_0_BIT,
    VK_IMAGE_ASPECT_PLANE_1_BIT or VK_IMAGE_ASPECT_PLANE_2_BIT."

Previously, for YCbCr images, we were flipping this around.  For single-
plane views where VK_IMAGE_ASPECT_PLANE_N_BIT would be passed in by the
app, we would store VK_IMAGE_ASPECT_COLOR_BIT.  For multi-plane views
where the client says VK_IMAGE_ASPECT_COLOR_BIT, we would store a all of
the planes.  (There was also an extra bit of remapping that would
compact the planes in the non-existent case of a format with a non-
contiguous set of planes.)  The idea behind this was that for things
like rendering or single-plane sampling, storage, or compute, we want it
to look as much like a single-plane image as possible but we wanted the
multi-plane case to be the awkward one.

This commit changes it around so that iview->aspects is always exactly
the subset of image->vk.aspects represented by the view.  This is
identical to how aspects work for depth/stencil so it gains us some
consistency.

This commit also changes anv_image_view::aspect_mask to aspects to force
a full audit of the field.  As can be seen, there are only a few uses of
this field and they're all mostly fine:

 - A bunch of them are used to check for depth/stencil.  That hasn't
   changed.

 - Most of the checks for color already used ANY_COLOR_BIT, only one
   needed fixing.

 - There's a check that both src/depth are color for MSAA resolves.
   However, we don't support MSAA on YCbCr so there's no point in
   checking for ANY_COLOR_BIT.

There is a hidden usage of planes in anv_descriptor_set_write_image_view
that's not as obvious.  However, this function simply looks at
anv_image_view::n_planes and blindly fills out the descriptor
accordingly.  As long as image views with a single plane continue to
claim n_planes == 1, this will be fine.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12141>