[LoopVectorize] Don't tail-fold for scalable VFs when there is no scalar tail
authorDavid Sherwood <david.sherwood@arm.com>
Wed, 15 Mar 2023 14:25:21 +0000 (14:25 +0000)
committerDavid Sherwood <david.sherwood@arm.com>
Mon, 27 Mar 2023 08:34:30 +0000 (08:34 +0000)
commit1c4fedfa35aeb8b456e2d8f4f826c0e026b9d863
tree0a54815f2219acb7e023d3f9f1b059948dc3e07c
parent8d3a09daca7d56e38b205437297cc357f2b4cab0
[LoopVectorize] Don't tail-fold for scalable VFs when there is no scalar tail

Currently in LoopVectorize we avoid tail-folding if we can
prove the trip count is always a multiple of the maximum
fixed-width VF. This works because we know the vectoriser
only ever chooses a VF that is a power of 2. However, if
we are also considering scalable VFs then we conservatively
bail out of the optimisation because we don't know the value
of vscale, which could be an odd or prime number, etc.

This patch tries to enable the same optimisation for scalable
VFs by asking if vscale is known to be a power of 2. If so,
we can then query the maximum value of vscale and use the same
logic as we do for fixed-width VFs. I've also added a new TTI
hook called isVScaleKnownToBeAPowerOfTwo that does the same
thing as the existing TargetLowering hook.

Differential Revision: https://reviews.llvm.org/D146199
llvm/include/llvm/Analysis/TargetTransformInfo.h
llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
llvm/include/llvm/CodeGen/BasicTTIImpl.h
llvm/lib/Analysis/TargetTransformInfo.cpp
llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
llvm/lib/Target/AArch64/AArch64ISelLowering.h
llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
llvm/test/Transforms/LoopVectorize/AArch64/eliminate-tail-predication.ll
llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding.ll