ac/nir/ngg,radv: move LDS layout calculation out of nir ngg lowering

Use lds base load intrinsics in nir ngg lowering to get layout, left
its calulation to driver.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>

nir: add two amd ngg lds base load intrinsics

These two values are not known when compile for radeonsi.
They are relocated when link/upload time.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>

ac/nir/ngg: pass primitive_id_location as param for nogs lower

radeonsi need to use packed driver location for all outputs,
while radv need to use VARYING_SLOT_*. To meet both drivers'
needs.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>

ac/nir/ngg: support user edge flags for ngg lower

Pack user edge flag into arg code is ported from radeonsi
gfx10_ngg_build_export_prim.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>

ac/llvm: get back intrinsics used by NGG

Will be used by radeonsi.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>

glx: clean-ups in drisw_glx.c

Replace tabs with spaces.  Fix up function pointer calls (don't use
the old style (*foo)(arg) syntax).

Signed-off-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19329>

glx: clean-ups in create_context.c

Replace tabs w/ spaces, remove trailing whitespace.

Signed-off-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19329>

frontends/dri: clean-ups in dri_util.c

Replace tabs with spaces.  Rename __ATTRIB macro to SIMPLE_CASE to
be a bit more readable.

NFC.

Signed-off-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19329>

frontend/dri: assorted clean-ups in dri-screen.c

Replace tabs with spaces, fix indentation.
Move 'format' var decl and type (it's an integer array index, not
actually a mesa format).

NFC.

Signed-off-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19329>

nv50/ir: nir_op_b2i8 and nir_op_b2i16

Signed-off-by: Yusuf Khan <yusisamerican@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19256>

docs: update to latest venus driver support

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19285>

venus: add VK_EXT_depth_clip_control support

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19285>

venus: add VK_EXT_primitives_generated_query support

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19285>

venus: sync to latest venus protocol headers

This brings in:
- VK_KHR_push_descriptor
- VK_EXT_depth_clip_control
- VK_EXT_primitives_generated_query

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19285>

venus: handle VkAndroidHardwareBufferFormatProperties2ANDROID

Fixes: 4d80ccbf2d0 ("venus: Enable VK_KHR_format_feature_flags2")

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19287>

venus: remove redundant codes

This is some left over from prior 1.3 effort.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19287>

nir/lower_bool_to_int32: add support for lowering functions.

Change the function parameters to 32-bit.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19291>

nir/divergence_analysis: add missing desc_set_address_intel

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19320>

nir/divergence_analysis: add some missing RT intrinsics

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19320>

vulkan/wsi/wl: correctly find whether the compositor uses the same GPU

Using the wl_drm protocol we can check whether the compositor uses the
same GPU as the application.

This allows to run vulkan applications using a DG2 GPU with the
compositor using another card.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Simon Ser <contact@emersion.fr>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19224>

anv: init major/minor before WSI

So that we can provide that information to WSI if it asks for it
immediately.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19224>

anv: disable mesh in memcpy

We can't have streamout and mesh enabled at the same time.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: ef04caea9b8b ("anv: Implement Mesh Shading pipeline")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19323>

Zink: add Zink profiles file

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19192>

zink: add profile documentation

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19192>

zink: handle broken resource mapping deadlocks

some apps (most notably Wolfenstein: The New Order) have broken multi-context
buffer usage in which one context will attempt to write to a buffer while
another context holds unflushed usage, and the unflushed context will never
flush until the buffer write completes

it's impossible to handle this scenario correctly without deadlocking,
so add some handling to try waiting and then yolo the buffer write if
a deadlock would occur

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19141>

nir: Unconditionally call nir_trim_vector in nir_lower_readonly_images_to_tex

It will already short-circuit if the number of components matches.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19301>

nir: Misc. style fixes to nir_lower_readonly_images_to_tex

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19301>

nir: Use nir_shader_instructions_pass in nir_lower_readonly_images_to_tex

nir_shader_lower_instructions is overkill and this makes the pass
generally easier to understand.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19301>

nir: Reformat a comment

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19301>

etnaviv: properly reference flush_resources

The flush_resources recorded in the context need to stay alive until
the context is flushed, at which point additional resolve operations
are done to those resources. While the backing BO is alive due to being
referenced in the cmdstream, the resource might already be destroyed
at this point.

Keep a reference to the resource to make sure it is still available at
context flush time.

Fixes: 7b9d8d1936d7 ("etnaviv: flush used render buffers on context flush when neccessary")
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19280>

winsys/amdgpu: Set RADEON_FLAG_32BIT again

Avoids hang running

 rendercheck -t cacomposite -f a8r8g8b8

via glamor on Navi 14.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7167
Fixes: 7833c5139a54 ("winsys/amdgpu: use cached GTT for command buffers and don't set the 32BIT flag")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19276>

zink: limit gl_Layer clamping to drivers that need it

So far, only IMG drivers cannot handle out of bounds layer values.
Ideally, a vulkan extension will be drafted to detail this behavior.
But for now if KHR-GL46.texture_cube_map_array.color_depth_attachments
fails, then needs_sanitised_layer is probably needed.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: clamp gl_Layer output to 0, if framebuffer is not layered

GL spec forces driver to ignore gl_Layer, if layered rendering
is not enabled.
Since vulkan doesn't have the same bavior, emulate this by forcing
gl_Layer to 0, based on driver internal state.
This was seen as failure in
KHR-GL46.texture_cube_map_array.color_depth_attachments

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: add new framebuffer_is_layered state

This state is needed to make sure gl_Layer values are set to 0,
when the framebuffer is not layered accorfing to GL spec.
Specifically Section 9.8 Layered Framebuffers of GL46 spec:
A layer number written by a geometry shader has no effect if
the framebuffer is not layered.

Vulkan has no carve out for this, so zink must handle this by
sanitising gl_Layer (next commit in the series).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: add pushconst only pipeline layout

Now that all gfx pipelines share the same push constant layout,
create a screen wide push const only layout that is compatible
with all future programs.
This layout will be used to update push constant values, so that
the update can happen at any point before draw call.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: cleanup zink_pipeline_layout_create

move the hashing to the caller, since it's not related to this.
Additionally, remove dependance on zink_program argument.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: use unified pushconst layour for passthorugh tcs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

zink: cleanup pushconst interface between driver/compiler

Extend vs_pushconst structure to all gfx stages and make sure,
the push constant memory layout is defined in one place and
is therefore always correct.
No functional change, but should make adding new members to
zink_*_push_constant easier.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

lavapipe: stop allocating 0 size const buffer

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19163>

anv: enable localized loads for lower_shader_calls

On Q2RTX shaders :

Instructions in all programs: 31039 -> 26150 (-15.8%)
SENDs in all programs: 1587 -> 1148 (-27.7%)
Loops in all programs: 4 -> 4 (+0.0%)
Cycles in all programs: 420218 -> 392179 (-6.7%)
Spills in all programs: 157 -> 132 (-15.9%)
Fills in all programs: 337 -> 262 (-22.3%)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

radv: tweak lower_shader_calls parameters

On Q2RTX shaders :

MaxWaves: 62 -> 69 (+11.29%)
Instrs: 41626 -> 41575 (-0.12%); split: -0.27%, +0.15%
CodeSize: 224960 -> 223740 (-0.54%); split: -0.62%, +0.08%
VGPRs: 800 -> 704 (-12.00%)
Scratch: 75776 -> 70656 (-6.76%)
Latency: 922219 -> 977997 (+6.05%)
InvThroughput: 212154 -> 201746 (-4.91%); split: -5.54%, +0.64%
VClause: 1120 -> 1155 (+3.12%); split: -1.88%, +5.00%
SClause: 1148 -> 1144 (-0.35%); split: -0.70%, +0.35%
Copies: 5840 -> 5788 (-0.89%); split: -0.94%, +0.05%
PreVGPRs: 753 -> 651 (-13.55%)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: run opt_cse after lower stack intrinsics

In particular when using scratch_base_ptr

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: move scratch loads closer to where they're needed

The intel backend compiler is not dealing with the scratch loads
emitted by this pass very well. There are 2 reasons for this :

  - all loads are at the top of the shader

  - the loads are global load intrinsics (cannot be differentiated
    from ssbo loads for example)

This leads the backend to generate ridiculous amount of spills.

To help a bit (actually quite a lot), we can move the scratch loads in
the blocks where they're needed, using the dominance information.
Quite often that also ends up moving loads in a block that might not
be reached by all the lanes, so we're potentially avoiding some loads.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: add a pass to sort/pack values on the stack

The previous pass shrinking values stored on the stack might have left
some gaps on the stack (a vec4 turned into a vec3 for instance).

This pass reorders variables on the stack, by component bit size and
by ssa value number. The component size is useful to pack smaller
values together. The ssa value number is also important because if we
have 2 calls spilling the same values, then we can avoid reemiting the
spillings if the values are stored in the same location.

v2: Remove unused sorting function (Konstantin)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: add a pass to trim scratch values

For example, if we store to scratch a vec4 but only a subset of
components are used after the load operation.

v2: Use nir_intrinsic_write_mask (Konstantin)
    Use u_foreach_bit() instead of u_bit_scan() (Konstantin)
    Fix mask building loop (Konstantin)

v3: Fix reswizzle (Konstantin)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: add an option structure for future optimizations

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: cleanup shaders a bit more post split

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: add NIR_PASS_V internally

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: rematerialize values in more complex cases

Previously when considering whether to rematerialize or spill/fill
ssa_1954, we would go for a spill/fill :

   vec4 32 ssa_388 = (float32)txf ssa_387 (texture_handle), ssa_86 (coord), ssa_23 (lod), 0 (texture), 0 (sampler)
   ...
   vec1 32 ssa_1953 = load_const (0xbd23d70a = -0.040000)
   vec1 32 ssa_1954 = fadd ssa_388.x, ssa_1953
   vec1 32 ssa_1955 = fneg ssa_1954

This is because when looking at ssa_1955 the first time, we would
consider ssa_388 unrematerialiable, and therefore all values built on
top of it would be considered unrematerialiable as well.

The missing piece when considering whether to rematerialize ssa_1954
is that we should look at filled values. Now that ssa_388 has been
spilled/filled, we can rebuild ssa_1955 on top of the filled value and
avoid spilling/filling ssa_1955 at all.

This requires a bit more work though. We can't just look at an
instruction in isolation, we need to go through the ssa chains until
we find values we can rematerialize or not.

In this change we build a list of all ssa values involved in building
a given value, up to the point there we find a filled or a
rematerializable value.

In this particular case, looking at ssa_1955 :

   * We can rematerialize ssa_388 from its filled value

   * We can rematerialize ssa_1953 trivially

   * We can rematerialize ssa_1954 because its 2 inputs are rematerializable

   * We can rematerialize ssa_1955 because ssa_1954 is rematerializable

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: avoid respilling values

Currently we do something like this :

  ssa_0 = ...
  ssa_1 = ...
  * spill ssa_0, ssa_1
  call1()
  * fill ssa_0, ssa_1
  ssa_2 = ...
  ssa_3 = ...
  * spill ssa_0, ssa_1, ssa_2, ssa_3
  call2()
  * fill ssa_0, ssa_1, ssa_2, ssa_3

If we assign the same possition to ssa_0 & ssa_1 in the spilling
stack, then on call2(), we know that those values are already present
in memory at the right location and we can avoid respilling them.

The result would be something like this :

  ssa_0 = ...
  ssa_1 = ...
  * spill ssa_0, ssa_1
  call1()
  * fill ssa_0, ssa_1
  ssa_2 = ...
  ssa_3 = ...
  * spill ssa_2, ssa_3
  call2()
  * fill ssa_0, ssa_1, ssa_2, ssa_3

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: lower scratch access to format internally

For a follow up optimization, we would like to track scratch loads.
This isn't possible with global load/store intrinsics. So use a couple
of special intrinsic in the pass and only lower it to global
intrinsics at the end.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

nir/lower_shader_calls: rematerialize more trivial values

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>

broadcom/compiler: call nir_opt_gcm with a custom strategy

nir_opt_gcm get us worse shader-db stats, but that is expected. But we
want to prevent to get worse values on spill/fills. Analyzing the
outcome with shader-db, this mostly happen with shaders that are
already complex, and are already spilling/filling.

So the best option here is adding a new strategy, that fall backs if
we get spill/fill using nir_opt_gcm.

It is not clear in which order we should disable gcm. For now we
disable it before loop unrolling.

We get a slight performance gain (in average) using nir_opt_gcm.

We don't show the shaderdb stats, as they are worse, but as mentioned,
this is expected.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

broadcom/compiler: pass a strategy struct to vir_compile_init

That allows to reduce the number of parameters of the method. And
after all, they were already filled using an existing strategy struct.

This would make easier adding new fields on a strategy.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

v3dv/pipeline: use v3d_optimize_nir

Instead of using a custom optimize_nir method, with the same purpose.

Running the fossils for the v3dv well know applications (ue4 demos,
Quake3d, etc) we got somewhat inconclusive outcome in general,
although slightly worse values:
  Instrs: 265129 -> 265277 (+0.06%); split: -0.06%, +0.12%
  Thread Count: 5504 -> 5506 (+0.04%)

  Totals from 153 (10.23% of 1495) affected shaders:
  Instrs: 84603 -> 84751 (+0.17%); split: -0.19%, +0.37%
  Thread Count: 316 -> 318 (+0.63%)

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

broadcom/compiler: add more lowerings/optimizations on v3d_optimize_nir

Optimizations that we are already calling on the Vulkan driver. As
preparation to the Vulkan frontend to use v3d_optimize_nir too.

We need to add a new parameter to v3d_optimize_nir in order to know if
we can call nir_opt_find_array_copies. As we don't track if we are
calling nir_var_lower_copies, we explicitly call it when we create the
uncompiled shader create. So instead of tracking, we assume that each
driver (v3d/v3dv) would call it when the shader is created. So when
v3d_optimize_nir is called as part of the process to compile it at the
compiler, we call it with allow_copies as false.

We exclude on purpose nir_opt_gcm as it is a case of a optimization
that could help performance even if it hurts shader db stats.

shaderdb stats:
  total instructions in shared programs: 11705923 -> 11705034 (<.01%)
  instructions in affected programs: 88350 -> 87461 (-1.01%)
  helped: 201
  HURT: 80
  Instructions are helped.

  total threads in shared programs: 375552 -> 375558 (<.01%)
  threads in affected programs: 6 -> 12 (100.00%)
  helped: 3
  HURT: 0

  total uniforms in shared programs: 3486108 -> 3485789 (<.01%)
  uniforms in affected programs: 7473 -> 7154 (-4.27%)
  helped: 90
  HURT: 1
  Uniforms are helped.

  total max-temps in shared programs: 2021860 -> 2021802 (<.01%)
  max-temps in affected programs: 800 -> 742 (-7.25%)
  helped: 21
  HURT: 3
  Max-temps are helped.

  total sfu-stalls in shared programs: 19299 -> 19296 (-0.02%)
  sfu-stalls in affected programs: 18 -> 15 (-16.67%)
  helped: 10
  HURT: 7
  Inconclusive result (value mean confidence interval includes 0).

  total inst-and-stalls in shared programs: 11725222 -> 11724330 (<.01%)
  inst-and-stalls in affected programs: 88402 -> 87510 (-1.01%)
  helped: 201
  HURT: 80
  Inst-and-stalls are helped.

  total nops in shared programs: 269674 -> 269386 (-0.11%)
  nops in affected programs: 3641 -> 3353 (-7.91%)
  helped: 103
  HURT: 29
  Nops are helped.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

broadcom/compiler: update how we compute return_words_of_texture_data on non-ssa

For the non-ssa case, we were trying to use reg->num_components. But
this is not the same that nir_ssa_def_components_read. It is the
number of components of the destination register. And in the 16bit
case, even if nir_lower_tex packs the outcome, it doesn't update the
number of components, as nir_tex_instr_dest_size would still return
4. And nir validate would check that those values are the same.

So this change focuses on the last part of this comment at
nir_lower_tex:

 * Note that we don't change the destination num_components, because
 * nir_tex_instr_dest_size() will still return 4.  The driver is just
 * expected to not store the other channels, given that nothing at the
 * NIR level will read them.

We just limit how many channels we would use for the f16 case.

It is also worth to note, based on the CTS and different applications
we test, that this is a corner case.

This was detected when we experimented to enable nir_opt_gcm for v3d,
that lead to raise an assertion slightly below with some shaderdb
tests, but technically it could happen without it.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

broadcom/compiler: don't call nir_opt_load_store_vectorize on all v3d_optimize_nir calls

For compute shaders, to avoid a crash with that optimization, it requires
doing some optimizations and lowerings before. Example:

  static void
  lower_cs_shared(struct nir_shader *nir)
  {
     NIR_PASS_V(nir, nir_lower_vars_to_explicit_types,
                nir_var_mem_shared, shared_type_info);
     NIR_PASS_V(nir, nir_lower_explicit_io,
                nir_var_mem_shared, nir_address_format_32bit_offset);
  }

In the same way other drivers (like anv) calls
nir_opt_load_store_vectorize as part of their post-process-nir.

So one option would be to move nir_opt_load_store_vectorize outsize
the common v3d_nir_optimize, to a post-process nir method.

To make things simpler, this change calls that optimization only if we
have a v3d_compiler object, that is when each frontend has already
done their lowerings, and call the v3d_compiler to get the final
assembly (so we are already on a kind of post processing nir step).

This avoids dEQP-VK.memory_model.shared.basic_types.3 crashing if we
start to call v3d_optimize_nir on v3dv directly.

Slight shaderdb changes, but not significant.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17185>

nouveau: put nv04 push macros in nouveau_winsys

Remove some of the duplicated code that comes from it

Signed-off-by: Yusuf Khan <yusisamerican@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18857>

rusticl/mem: can only map staging textures directly

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19213>

rusticl: force BIND_LINEAR on staging resources

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19213>

r600/sfn: evaluate LDS location for color and clip-vertex too

This is required to support compatibility contexts

Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6926

Fixes: 3340c7ce359252ad09b3e4d338837944482fb248
   r600/sfn: lower CLIPVERTEX to clip planes

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600: Account for color and clipvertex when evaluating LDS space

Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6926

Fixes: 3340c7ce359252ad09b3e4d338837944482fb248
   r600/sfn: lower CLIPVERTEX to clip planes

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: elimiate dead registers too

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Increase scheduling priority on uniform reads and non-ssa writes

* Non-ssa values are pre-allocated, so assigning values to these is
  unlikely to increase register pressure.

* Uniforms are often used early in the shader to evaluate dependend
  values so, don't penalize their scheduling priority like literal

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: improve scheduling of tex sources

Let tex sources switch the channel to unused channels.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Trigger TEX CF based on max TEX CF size

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Allow copy-prop of group dest into origin

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Add test for channel changes in TEX source from opt

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600: Fix printing t-channel in diss-assambly

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Handle nir_op_seq and nir_op_sne

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

r600/sfn: Unify the handling of resource IDs in instruction

Fetch, GDS, Texture, and RAT instructions all use resources
with a possible offset defined by the index register. Unify
the handling of resource ID and the offset register for these
instruction types.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19300>

v3d: Fix initializer-overrides warning.

../src/gallium/drivers/v3d/v3d_screen.c:733:27: warning: initializer overrides prior initialization of this subobject [-Winitializer-overrides]
        .lower_mul_high = true,
                          ^~~~
/usr/lib/llvm-14/lib/clang/14.0.0/include/stdbool.h:16:14: note: expanded from macro 'true'
             ^
../src/gallium/drivers/v3d/v3d_screen.c:726:27: note: previous initialization is here
        .lower_mul_high = true,
                          ^~~~
/usr/lib/llvm-14/lib/clang/14.0.0/include/stdbool.h:16:14: note: expanded from macro 'true'
             ^

Fixes: 73e8fc3efbf ("broadcom/compiler: don't use imprecise_32bit_lowering for idiv lowering")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19262>

xlib: Remove the usage of global constructor in xlib.c, so the xm_public.h are removed

The usage of global constructor should be limited, only in absolutely needed case.
The call style of xlib_create_screen referenced to osmesa_create_screen

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

xlib: Getting xmesa_strict_invalidate to be function

So that doesn't be called in global constructor

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

ci: Getting debian-clang-release to building with glx=xlib

Because this is just building, didn't not used by CTS runner,
So enable glx=xlib in this build variant

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

ci/macOS: Getting the installed binary to be artifacts

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

xlib: Indent with space and trim trailing spaces of xlib files

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

gallium: Remove mgl* prefix in linker script osmesa.sym and libgl-xlib.sym

Seeing no mgl prefixed function in codebase, so removed it

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

xlib: Remove the linkage hack in lib.c by use meson link_whole option

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

xlib: Fixes compiling on linux

configure command line:
meson ../.. -Dglx=xlib -Dosmesa=true

link error:
virgl_screen.c.o: in function `virgl_is_video_format_supported':
/mnt/c/work/xemu/mesa/build/windows-wsl2/../../src/gallium/drivers/virgl/virgl_screen.c:885: undefined reference to `vl_video_buffer_is_format_supported'

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7545

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19258>

panfrost: Use proper formats for pntc varying

The formats of special attributes are supposed to match their architectural
definitions, and point coordinates are architecturally defined as RGBA32F. In
practice this doesn't seem to fix anything.

Cc: mesa-stable
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19237>

panfrost: Don't use lower_wpos_pntc on Midgard

gl_PointCoord is implemented via a special attribute descriptor on Midgard. This
descriptor has an orientation bit, the orientation is driver-controlled. That
means we can map rast->sprite_coord_mode to this bit, rather than lowering in
the shader.

This is a bug fix for point sprites, which are implemented natively on Midgard
for dubious reasons and need to be flipped this way. It is also an optimization
for apps reading gl_PointCoord, removing the extra arithmetic to flip, although
the value of this is somewhat dubious.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19237>

ci/freedreno: Update known flakes/timeouts.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19304>

ci/freedreno: Update CivV trace expectation.

Rendered scene still looks good, just a few pixels changed a bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19304>

aux/trace: add set_global_binding

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19214>

zink: elide a buffer samplerview update conditional

this is really part of the previous conditional blocks

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19306>

zink: rescope/simplify some push descriptor code

this is only used in the push descriptor block, so move it there and
simplify redundant cases

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19306>

zink: flag push state changed in batch changed conditional

this will matter after future refactoring

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19306>

zink: split out uniform decriptor bindings in shader data

this is both more usable and fixes some descriptor util functionality

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19306>

zink: simplify some program descriptor helpers

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19306>

zink: fix spirv_builder_spec_const_uint

We need to create the type upfront otherwise the spir-v binary gets
corrupted.

Fixes: c305a2c9625 ("zink: move spec constant emission to the types/consts block")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19310>

freedreno: Add 'replay' tool which allows to replay cmdstreams

Replay command stream obtained from:
- /sys/kernel/debug/dri/0/rd
- /sys/kernel/debug/dri/0/hangrd
!!! Command stream capture should be done with ALL buffers:
- echo 1 > /sys/module/msm/parameters/rd_full

Requires kernel with MSM_INFO_SET_IOVA support.

This tool is intended for reproduction of various GPU issues:
- GPU hangs, note that command stream obtained from hangrd
  may not reproduce (rarely) the hang, since the buffers are
  snapshotted at the moment of the hang and not at the start
  of the hanging command stream.
- TODO: Misrendering, would require marking framebuffer images
  at each renderpass in order to fetch and decode them.

Code from Freedreno/Turnip is not re-used here since the relevant
pieces may introduce additional allocations which cannot be allowed
during the replay.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19171>

aco: add support for device clock on GFX11

According to LLVM, s_sendmsg_rtn(GET_REALTIME) should be used instead
of s_memrealtime.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19267>

aco: split the sendmsg enumeration into sendmsg_rtn

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19267>

aco: add support for s_sendmsg_rtn_b{32,64}

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19267>

ac/llvm: add support for device clock on GFX11

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19267>

ac/nir: add legacy streamout and GS copy shader helpers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19302>

radv,nir: add intrinsics for streamout and GS copy shaders

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19302>

turnip: Enable LRZ testing (not writing) in the presence of discards.

We can LRZ test to not rasterize maybe-discarded pixels, as long as we
don't LRZ write a Z that may be discarded.  We can drop this check,
because LRZ writes are already disabled by TU_LRZ_FORCE_DISABLE_WRITE, and
tu_6_build_depth_plane_z_mode() uses has_kill to choose EARLY_LRZ_LATE_Z
(or just LATE_Z if LRZ isn't enabled).

gfxbench aztec ruins:    +7.93104% +/- 0.117845% (n=4)
ANGLE trex_200:          +11.7208% +/- 0.476166% (n=3)
ANGLE aztec_ruins_high:  +11.9138% +/- 0.147586% (n=3)

Fixes: #6327 (now vk-5-normal is +0.712724% +/- 0.0655751% to gl-5-normal)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19286>