nir: Combine if_uses with instruction uses
authorAlyssa Rosenzweig <alyssa@collabora.com>
Thu, 6 Apr 2023 17:19:31 +0000 (13:19 -0400)
committerMarge Bot <emma+marge@anholt.net>
Fri, 7 Apr 2023 23:48:03 +0000 (23:48 +0000)
commit7f6491b76d51f35e76715275124d4a8d2eaf8db1
tree4b121bc59573ed650fd4dd506df969c248d7ed84
parentfd9c69218ae2967f8bbc91cf84c86556881363c1
nir: Combine if_uses with instruction uses

Every nir_ssa_def is part of a chain of uses, implemented with doubly linked
lists.  That means each requires 2 * 64-bit = 16 bytes per def, which is
memory intensive. Together they require 32 bytes per def. Not cool.

To cut that memory use in half, we can combine the two linked lists into a
single use list that contains both regular instruction uses and if-uses. To do
this, we augment the nir_src with a boolean "is_if", and reimplement the
abstract if-uses operations on top of that list. That boolean should fit into
the padding already in nir_src so should not actually affect memory use, and in
the future we sneak it into the bottom bit of a pointer.

However, this creates a new inefficiency: now iterating over regular uses
separate from if-uses is (nominally) more expensive. It turns out virtually
every caller of nir_foreach_if_use(_safe) also calls nir_foreach_use(_safe)
immediately before, so we rewrite most of the callers to instead call a new
single `nir_foreach_use_including_if(_safe)` which predicates the logic based on
`src->is_if`. This should mitigate the performance difference.

There's a bit of churn, but this is largely a mechanical set of changes.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>
38 files changed:
src/compiler/nir/nir.c
src/compiler/nir/nir.h
src/compiler/nir/nir_clone.c
src/compiler/nir/nir_control_flow.c
src/compiler/nir/nir_deref.c
src/compiler/nir/nir_from_ssa.c
src/compiler/nir/nir_loop_analyze.c
src/compiler/nir/nir_lower_io.c
src/compiler/nir/nir_lower_regs_to_ssa.c
src/compiler/nir/nir_lower_to_source_mods.c
src/compiler/nir/nir_lower_vec_to_movs.c
src/compiler/nir/nir_opt_copy_propagate.c
src/compiler/nir/nir_opt_dead_cf.c
src/compiler/nir/nir_opt_if.c
src/compiler/nir/nir_opt_intrinsics.c
src/compiler/nir/nir_opt_peephole_select.c
src/compiler/nir/nir_opt_phi_precision.c
src/compiler/nir/nir_opt_ray_queries.c
src/compiler/nir/nir_opt_rematerialize_compares.c
src/compiler/nir/nir_opt_sink.c
src/compiler/nir/nir_opt_uniform_atomics.c
src/compiler/nir/nir_repair_ssa.c
src/compiler/nir/nir_search_helpers.h
src/compiler/nir/nir_serialize.c
src/compiler/nir/nir_to_lcssa.c
src/compiler/nir/nir_validate.c
src/compiler/nir/tests/ssa_def_bits_used_tests.cpp
src/freedreno/ir3/ir3_nir_opt_preamble.c
src/gallium/auxiliary/nir/nir_to_tgsi.c
src/gallium/drivers/etnaviv/etnaviv_compiler_nir.c
src/gallium/drivers/etnaviv/etnaviv_compiler_nir.h
src/gallium/drivers/vc4/vc4_program.c
src/intel/compiler/brw_fs.cpp
src/intel/compiler/brw_nir_opt_peephole_ffma.c
src/microsoft/compiler/dxil_nir_tess.c
src/panfrost/.clang-format
src/panfrost/midgard/nir_fuse_io_16.c
src/panfrost/util/nir_mod_helpers.c