[libc] Improve memcmp latency and codegen
authorGuillaume Chatelet <gchatelet@google.com>
Mon, 12 Jun 2023 07:55:13 +0000 (07:55 +0000)
committerGuillaume Chatelet <gchatelet@google.com>
Mon, 12 Jun 2023 07:56:23 +0000 (07:56 +0000)
commitbd4f978754758d5ef29d1f10370f45362da3de37
tree778b3d242d9a3118f68483da56be955604142a21
parent15a16ef8e06e5da382d135bc5074068ed14be6d6
[libc] Improve memcmp latency and codegen

This is based on ideas from @nafi to:
 - use a branchless version of 'cmp' for 'uint32_t',
 - completely resolve the lexicographic comparison through vector
   operations when wide types are available. We also get rid of byte
   reloads and serializing '__builtin_ctzll'.

I did not include the suggestion to replace comparisons of 'uint16_t'
with two 'uint8_t' as it did not seem to help the codegen. This can
be revisited in sub-sequent patches.

The code been rewritten to reduce nested function calls, making the
job of the inliner easier and preventing harmful code duplication.

Reviewed By: nafi3000

Differential Revision: https://reviews.llvm.org/D148717
16 files changed:
libc/src/__support/macros/properties/architectures.h
libc/src/string/CMakeLists.txt
libc/src/string/memory_utils/CMakeLists.txt
libc/src/string/memory_utils/aarch64/memcmp_implementations.h
libc/src/string/memory_utils/bcmp_implementations.h
libc/src/string/memory_utils/memcmp_implementations.h
libc/src/string/memory_utils/memmove_implementations.h
libc/src/string/memory_utils/memset_implementations.h
libc/src/string/memory_utils/op_aarch64.h
libc/src/string/memory_utils/op_generic.h
libc/src/string/memory_utils/op_riscv.h [new file with mode: 0644]
libc/src/string/memory_utils/op_x86.h
libc/src/string/memory_utils/utils.h
libc/src/string/memory_utils/x86_64/memcmp_implementations.h
libc/test/src/string/memory_utils/op_tests.cpp
utils/bazel/llvm-project-overlay/libc/BUILD.bazel