st/mesa: fix potential use-after-free in draw_bitmap_quad

This is super unlikely to be freed before use, but let's fix it anyway.

setup_render_state calls set_sampler_views(take_ownership=true), which
means it takes ownership of the sampler view reference and is free to
unreference it, so we can't use sv after setup_render_state.

Fixes: feda6e9c5d101 - st/mesa: set take_ownership = true in set_sampler_views

Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17780>

mesa: create glBitmap textures while creating display lists

This makes glCallList just a textured draw, which is blazingly fast.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17780>

Revert "mesa: implement a display list / glBitmap texture atlas"

This reverts commit b26ddda12fe7dbb6a4e6af3b47c1e837cc7ebb03 and
commit 06d3b0a006f35dc232d512d09f45a6cb4f13cfdf.

Reviewed-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17780>

intel/fs: bump max SIMD size for A64 atomics with LSC

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>.
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

intel/fs: port block a64/surface messages to use LSC

v2: Fixup block load/store on surfaces/shared-memory (Rohan)

v3: drop write specific size_written case (Rohan)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

intel/fs: switch register allocation spilling to use LSC on Gfx12.5+

v2: drop the hardcoded inst->mlen=1 (Rohan)

v3: Move back to LOAD/STORE messages (limited to SIMD16 for LSC)

v4: Also use 4 GRFs transpose loads for fills (Curro)

v5: Reduce amount of needed register to build per lane offsets (Curro)
    Drop some now useless SIMD32 code
    Unify unspill code

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

intel/fs: fixup SEND validation check on overlapping src0/src1

With the following SEND instruction :

   send(1)         nullUD          nullUD          g0UD            0x4200c504                a0.1<0>UD

This instruction although valid but somewhat nonsensical (SEND message
to write at offset contained in NULL register), triggers an error in
the validator.

The restriction is that we cannot have overlapping sources. The
validator not checking the type of register incorrectly thinks that
the null register (offset 0) is the same as g0.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

intel/fs: remove unused opcode

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

intel/fs: switch compute push constant loads to LSC

We're now able to load up to 8 GRFs in one send.

v2: Switch to use transpose + vector of up to 64 (Thanks Curro!)

v3: Increase parallelism by not reusing the same register for push
    constant offset (Curro)

v4: Drop dead ADD() instruction (Curro)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17555>

tu: fix invalid free on alloc failure

this is not an allocated pointer

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18230>

radv: Fold 16bit image sources.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18106>

aco: Combine 16bit undef and constants instead of using s_pack.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18106>

aco: Implement storage image A16.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18106>

nir/fold_16bit_tex_image: Add an option to fold image sources.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18106>

nir_lower_atomics_to_ssbo: Initialize deref struct

This fixes the use of an uninitialzed value:

Conditional jump or move depends on uninitialised value(s)
  bcmp (vg_replace_strmem.c:1203)
  _mesa_add_sized_state_reference (prog_parameter.c:434)
  st_nir_assign_uniform_locations(gl_context*, gl_program*, nir_shader*) (st_glsl_to_nir.cpp:209)
  st_finalize_nir (st_glsl_to_nir.cpp:1041)
  by 0x58271B9: st_glsl_to_nir_post_opts(st_context*, gl_program*, gl_shader_program*) (st_glsl_to_nir.cpp:571)
  ...
Uninitialised value was created by a heap allocation
  malloc (vg_replace_malloc.c:381)
  ralloc_size (ralloc.c:114)
  ralloc_array_size (ralloc.c:218)
  deref_offset_var (nir_lower_atomics_to_ssbo.c:47)
  lower_instr (nir_lower_atomics_to_ssbo.c:111)
  nir_lower_atomics_to_ssbo (nir_lower_atomics_to_ssbo.c:204)

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18227>

nir: Add nir_ssa_scalar_is_undef.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18183>

ci: fix leftover tag in image-tags.yml

Fixes: eb6ce47d4f75 ("ci: Use mold for x86-64 and AArch64 builds")

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18228>

radv: Advertise subgroup ops for rt stages

Closes: #7098
Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18169>

zink: support PIPE_CAP_FBFETCH_COHERENT

that's what VK_EXT_rasterization_order_attachment_access is for

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18133>

vulkan: Update the XML and headers to 1.3.225

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18133>

radv: merge gather_tess_info() with radv_fill_shader_info()

Shouldn't introduce any functional changes. The dependencies between
stages might be improved with a new helper that will link shader_info.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18184>

radv: remove unused num_tess_patches assignment for VS

This is never used.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18184>

radv: remove unused tcs_vertices_out assignment for VS

This is never used.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18184>

radv: remove redundant assignment of tcs.tcs_vertices_out

It's already assigned from radv_nir_shader_info_pass() and it's only
used to configure the VGT_TF_PARAM register. Otherwise, we read it
from NIR shader info during compilation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18184>

etnaviv: mark instanced draw extensions as supported in docs/features.txt

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18046>

etnaviv: expose ARB_draw_instanced

Just set the pipe cap correctly. The InstanceID support is already
hooked up in the NIR compiler. All enabled piglit tests pass.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18046>

zink: Remove duplicate variable zero.

Fix defect reported by Coverity Scan.

Evaluation order violation (EVALUATION_ORDER)
write_write_typo: In zero = zero = nir_imm_zero(b, nir_dest_num_components(intr->dest), nir_dest_bit_size(intr->dest)),
zero is written twice with the same value.

Fixes: 0f97e317e33 ("zink: rewrite all undefined shader reads as 0001 instead of undef")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18164>

glsl: fix location for array subscript

xfb_decl_assign_location() assumes that arrays are going to be packed.
But some conditions might prevent packing (e.g: explicit location or
smooth interpolation mode).

Instead of assuming that packing will happen, this commit adds a check to
determine if it'll happen and use the result to compute the proper location.

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2214

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18175>

glsl: make packed varying helper needs_lowering() external

We will use this helper to correctly calculate xfb offsets in the
following patch.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18175>

radeonsi: fix tcs_out_lds_offsets arg alignment

tcs_out_lds_offsets is not sure to be 16 byte aligned, it's
calculated like this:

  num_patches * patch_vertices * lshs_vertex_stride

num_patches and patch_vertices are not sure to be any value aligned,
lshs_vertex_stride is added one extra dword, so it's only 4 byte
aligned.

This may cause problem even before we switch to nir tess output
lower when write tess factor before read tail of input. But it's
more likely to cause problem after we switch to nir tess output
lower because the main body won't eliminate the low 4bit offset
but epilog will, so they use different offset to read/write tess
factor.

Fixes: 7598bfd768f ("radeonsi: replace llvm tcs output with nir lower pass")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7083
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18174>

intel/compiler: Use fs_reg helpers for GS icp_handle selection

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18221>

intel/compiler: Use fs_reg helpers for TCS icp_handle selection

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18221>

intel/compiler: Rename 8_PATCH to MULTI_PATCH

Make it clearer we are dealing with multiple patches,
works better in constrast with SINGLE_PATCH.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18151>

intel/compiler: Remove INTEL_DEBUG=tcs8

For Gen11 and prior, the dispatch mode for TCS was SINGLE_PATCH, and
this debug setting could be used to change it to 8_PATCH (falling back
to SINGLE_PATCH when shader couldn't be in the multi dispatch mode).
However after talking to Ken, seems this debug setting is not really
worth keeping around, so removing it.

For Gen12+ the only option is 8_PATCH, so it was always using that
dispatch mode as before.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18151>

vulkan/wsi: Take max extent into consideration for modifier selection.

For AMD we kinda have some modifiers with a max size ... (Which is
really a compositor/kms issue, but getting them to try kinda falls
into the unsolved  "how to allocate/what pitch to use" bucket, so
we solve it on the allocating side)

Cc: mesa-stable
Tested-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Joshua Ashton <joshua@froggi.es>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18139>

ci: Use mold for x86-64 and AArch64 builds

mold is a fancy new linker that's really fast.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6877
Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17992>

iris: Drop extra file-descriptor dup in iris_drm_screen_create()

In a99e85db9eb, we added a dup into iris_screen_create(). Apparently
some android code paths must be hitting iris_screen_create() without
calling iris_drm_screen_create(). After a99e85db9eb, the code paths
that do hit iris_drm_screen_create() will now dup the fd twice, but
iris_screen_destroy() will only close 1 of these fds.

Fixes: a99e85db9eb ("iris:Duplicate DRM fd internally instead of reuse.")
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18020>

intel/fs: fixup scratch load/store handling on Gfx12.5+

We did not handle the operation with data size < 4. It works fine on
all other messages (global/shared). The initial commit was just too
restrictive.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 1e242785c315 ("intel/fs: Implement load/store_scratch on XeHP")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16964>

intel/fs: fix load_scratch intrinsic

The selection of the internal opcode to deal with load_scratch is
incorrect.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: c6439792287f ("intel/fs: Choose memory message type based on bit size")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16964>

intel/compiler: Make component() work for FIXED_GRF/ARF

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18157>

intel/fs: Fix horiz_offset() to handle FIXED_GRFs with non-trivial 2D regions.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18157>

ci: update CI to reflect clovers LLVM version bump

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16047>

chore(deps): clover: raise the minimum LLVM version to 11.0.0

LLVM 11 was released in October 2020. If you want to build against
Mesa's Git version, that seems like enough time to upgrade to at least
LLVM 11 (Debian stable has this too).

It reduces the amount of #if gates we need and more will be incoming
again, given the Opaque Pointer transition.

Additionally radeonsi is already requiring LLVM 11. Therefore the
minimum will have been LLVM 11 for many builds anyway.

Note that clc is kept to LLVM 10 for the time being.

Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16047>

radv: Fix mipmap views on GFX10+

As explained in the previous commit, GFX9+ has issues with addressing
mipmaps in block-compressed images. In the case of copy commands, we fix
this by doing an extra copy for the missing blocks.

For GFX10, the mipmap layout in memory allows us to do better than that. We
can change the base level of the descriptor to one level bigger than the
requested level and adjust the extent and address to match. This is done by
ComputeNonBlockCompressedView in addrlib. Thus on GFX10 we can skip the
fixup copy workaround, and this will also fix cases outside of explicit
copy commands.

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17970>

radv: Fix corrupted mipmap copies on GFX9+

GFX9+ hardware has an issue where mipmap degradations are calculated
incorrectly due to using divide-by-two integer math and certain mipmap
sizes lose blocks.

This issue has been documented before, and we ported a workaround from
AMDVLK to increase the extent that is programmed into the descriptor, so
that the hardware arrives at the correct result. However, this is
insufficient as we cannot safely increase the extent beyond the physical
extent of the image in memory. If we can't increase it enough, the image
will still be missing blocks.

But there is still hope. In cases where RADV is responsible for copying to
or from an image (such as vkCmdCopyBufferToImage/vkCmdCopyImageToBuffer),
we can perform a second copy of the blocks that the hardware excluded so
that the resulting image is complete. This is another workaround from
AMDVLK.

This fixes corrupted textures in Halo: The Master Chief Collection.

v2: Add RADV_CMD_FLAG_INV_L2 | RADV_CMD_FLAG_INV_VCACHE to flush_bits
    just in case (Samuel Pitoiset)

Closes: #3347

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17970>

radv: Only apply mipmap view adjustments to block compressed images

This workaround need not apply to subsampled formats.

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17970>

vulkan: Introduce vk_format_is_block_compressed function

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17970>

radv: Add get_addrlib function to radv_radeon_winsys

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17970>

anv: convert assert into unreachable to avoid fallthrough error

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18192>

nvc0: limit max global and alloc size

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50/nir: A group barrier is CTA-level not global-level

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50/ir: Avoid generating splits of splits

Among others, it would result in the spill offsets being wrong due to
being relative to the parent split and not absolute.

For example when computing a 64-bit multiply on Tesla (which only
supports 16-bit mul in hardware), the sources will first be split into
32-bit values and then a second time down to 16-bit ones. Looking at the
first source, the spill offsets ended being computed as follows:

    { .hihi = +2, .hilo = +0, .lohi = +2, .lolo = +0 }

instead of the expected

    { .hihi = +6, .hilo = +4, .lohi = +2, .lolo = +0 }

This is resolved with this patch.

Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50/ra: Fix the offset computation for compounds

compMask is expressed in terms of colours, not bytes, where on Tesla we
have 1 colour per 16-bit (whereas it is 1 per 32-bit for later
architectures). By multiplying by units we will get back to a result in
bytes.

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50/peephole: Disallow combining sub 4-byte ld/st for now

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50/ir: Handle non-32-bit values when cst folding SPLIT

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50: Mark RESOURCE_FROM_USER_MEMORY_COMPUTE_ONLY as unsupported

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50: Disallow allocating more than VRAM size

Allocations larger than the amount of VRAM available might be possible,
but has not been tested.

v4: Change it back to just VRAM size
v3: Increase the minimum from 32 MB to 128 MB, as OpenCL 1.x mandated at
    least 128 MB.
v2: Further lower the allocation size.

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50: Report actual VRAM size

v2: handle vram_size == 0 (Karol)

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50: Rename interps to fixups

This matches the denomination used in nvc0, and finishes the name change
that started in f5fe9030.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50: Rename fixups to relocs

This matches the names used on nvc0.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

nv50,nvc0: Do not resize global residents if unnecessary

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Pierre Moreau <dev@pmoreau.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10711>

radv: Don't assume layout == stencilLayout for stencil-only

This assumption is probably valid with render passes but it won't for
dynamic rendering where we may not have a layout for depth at all.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv: Add a image_transition helper for separate depth/stencil layouts

This just reduces churn a bit when we switch to dynamic rendering.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Drop redundant depth_view_can_fast_clear

It's entirely redundant with radv_can_fast_clear_depth except that it's
missing a few checks around view masks and the layer range in the actual
clear rect and assumes you're always clearing the whole image view.
This isn't necessarily true, even for classic Vulkan render passes.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv: Drop set/restore_subpass

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Use Begin/EndRendering for clears

This is only used in the case of resolve image clears which are going to
go away entirely when we start doing dynamic rendering only.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: s/dest/dst/ in a few functions

It's all horribly inconsistent.  Using "dst" has the advantage that it's
the same number of characters as "src".

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Use Begin/EndRendering for resolves

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Drop subpass_att from radv_fast_clear_color

It's redundant with clear_att->colorAttachment.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Fix a stack use-after-free with sample locations

As soon as the if ends (or possibly before?) the VkSampleLocationsInfoEXT
goes out of scope.  We need it to stay valid until after the
vkCmdPipelineBarrier completes.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv/meta: Respect render area for MSAA resolves

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv: Use VK_IMAGE_LAYOUT_ATTACHMENT_FEEDBACK_LOOP_OPTIMAL_EXT for render loops

This commit fixes
dEQP-VK.rasterization.rasterization_order_attachment_access.format on
GFX9 because changing the layout for Vulkan feedback loops will trigger
a fast-clear eliminate.  Though, the root cause is unrelated to that and
it's because the CMASK/FMASK initialization on GFX9 is currently broken
for TC-compatible images (there is a TODO somewhere).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radv: Use both aspects for depth/stencil blit destinations

Even with dynamic rendering, you have to bind both aspects of the image
if the image contains both depth and stencil.  One day, we may see this
restriction lifted but that will require deeper driver surgery into the
way we handle depth/stencil layouts.

Fixes: 42db5900061c ("radv: convert the meta blit 2d path to dynamic rendering")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

vulkan,anv,dozen: Use VK_IMAGE_LAYOUT_ATTACHMENT_FEEDBACK_LOOP_OPTIMAL_EXT

This has basically identical semantics to the pseudo-ext enum we were
using before.  Also, now that it's in the actual Vulkan enum, we can get
rid of all the #pragma garbage to avoid compiler warnings.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18084>

radeonsi/vcn: add AUD syntax to h264 encoding

Adding AUD syntax to h264 bitstream, for easier locate
frame boundaries.

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

radeonsi/vcn: remove rate control double begin IBs

[why]
when start rate control, two begin IBs have been sent

[how]
merge two begin IBs together

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

radeonsi/vcn: support VBAQ modes

add VBAQ supports to all vcn versions.

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

radeonsi/vcn: support encoding preset modes

- support preset modes for all vcn versions
- SAO HEVC cannot use SPEED mode from vcn2 and up

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

radeonsi/vcn: dpb change for supporting pre-encoding

- dpb buffer created while receive the first begin_frame
  moved out from encoder creation.
- dpb buffer name changed from cpb.
- dpb buffer allocation to support pre-encoding
- different vcn version IB package adjustment
- merge reconstructed_picture_v4_0_t to reconstructed_picture_t

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

radeonsi/vcn: add enc quality bits interface

- define quality_level interface in mesa
- interprete quality_level to internal modes

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

frontends/va: improve enc quality interface change

[why]
to enhance va video encoding quality

[how]
use va encoding quality_level interface, and provide
default value and encoding quality adjustment options,
so that users can finetune encoding quality and performance
from va quality interface. (limited to VCNs)

There are 3 settings added:
- preset modes: speed, balance, quality
  they are using different encoding strategies
- vbaq modes:
  vbaq mode is using variance based strategy
  to improve the subjective image quality
- pre-encoding modes:
  Using scaled down input image for pre-encoding to have
  better rate-control reaction and consume more memory
  in the same time. Only preencoding-4x mode is enabled.

Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17994>

ci: Remove -Wno-error=deprecated-declarations cpp option from debian-clang

As all deprecated-declarations warnings are fixed

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

osmesa: Fixes [-Wdeprecated-declarations] in test-render.cpp

Warning message:
../src/gallium/targets/osmesa/test-render.cpp:150:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

virgl: Fixes [-Wdeprecated-declarations] in virgl_staging_mgr_test.cpp

Warning message:
../src/gallium/drivers/virgl/tests/virgl_staging_mgr_test.cpp:188:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

intel/compiler: Fixes [-Wdeprecated-declarations] in test_eu_validate.cpp

Warning message:
../src/intel/compiler/test_eu_validate.cpp:96:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

intel/compiler: Fixes [-Wdeprecated-declarations] in test_eu_compact.cpp

Warning messages:
../src/intel/compiler/test_eu_compact.cpp:238:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

../src/intel/compiler/test_eu_compact.cpp:256:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

glsl: Fixes [-Wdeprecated-declarations] in list_iterators.cpp

Warning messages:
../src/compiler/glsl/tests/list_iterators.cpp:68:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

../src/compiler/glsl/tests/list_iterators.cpp:187:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

util: Fix [-Wdeprecated-declarations] in mesa-sha1_test.cpp

Warning message:
../src/util/tests/mesa-sha1_test.cpp:42:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

nir: Fixes [-Wdeprecated-declarations] in serialize_tests.cpp

Warning messages:
../src/compiler/nir/tests/serialize_tests.cpp:113:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

../src/compiler/nir/tests/serialize_tests.cpp:119:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>

intel/compiler: Get URB handle in emit_task/mesh_intrinsic functions

This will make convenient later to keep track of the urb
handles directly in a Task thread payload struct (to be part of
fs_visitor).

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18188>

intel/compiler: Call get_mesh_urb_handle one level up in call-stack

Call it now from a fs_visitor member functions instead of the static
ones.  This will make convenient later to keep track of the urb
handles directly in a Task thread payload struct (to be part of
fs_visitor).

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18188>

amd/compiler: Fixes warning [-Wunused-variable] in test_optimizer_postRA.cpp

Warning message:
../src/amd/compiler/tests/test_optimizer_postRA.cpp:137:13: warning: unused variable 'reg_s1' [-Wunused-variable]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18205>

amd/compiler: Fixes warning [-Wunused-variable] in test_to_hw_instr.cpp

Warning message:
../src/amd/compiler/tests/test_to_hw_instr.cpp:793:12: warning: unused variable 'reg_s1' [-Wunused-variable]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18205>

anv: fix assert to build with shader cache disabled

When setting -Dshader-cache=disabled the build fails due
no member named 'disk_cache' in 'struct anv_physical_device'

Signed-off-by: sjfricke <spencerfricke@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Fixes: 7f1e8230 ("anv: Switch to the new common pipeline cache")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18181>

iris: Always initialize shader compilation queue ready fence

We use/delete this fence unconditionally, but it was initialized only
when screen->precompile is set. Move the util_queue_fence_init call
to the iris_create_uncompiled_shader to initialize it always.

Fixes: 42c34e1a ("iris: Enable threaded shader compilation")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7074
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Tested-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18182>

anv: remove the LOCAL_MEM allocation bit

We always want to use local memory if possible, we'll just add the
system memory heap if the buffer needs to be host visible.

v2: Drop some usages of ANV_BO_ALLOC_LOCAL_MEM_CPU_VISIBLE

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17873>

anv: prevent trying to mmap non host visible memory

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17873>

glsl/tests: do not perform eol-conversion on windows

These tests fail on Windows, because we open the expected files in
text-mode, performing EOL conversion. Instead, let's read them as binary
files, and manually UTF-8 decode them to get the expected result.

This fixes the tests on Windows for me.

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18179>

anv: fix assert in memory budget code when extension is not supported

First we should only support the extension if we can support reporting
on all the heaps.

Second we should not run any query code if the extension is not
supported.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: fae88d8791b2 ("anv: make use of the new smallbar uAPI")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18153>

anv: fixup assertions on lowered storage formats

With VK_FORMAT_B10G11R11_UFLOAT_PACK32 in particular, we're seeing
applications create image views with swizzle = R,G,B,0

But since the format has no alpha channel, the swizzle value for it
does not matter for the equivalence we're trying to verify.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: a9edc268b9b1 ("anv: validate image view lowered storage formats for storage")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18081>