[LoopVectorize] Take vscale into account when deciding to create epilogues
authorDavid Sherwood <david.sherwood@arm.com>
Mon, 3 Apr 2023 16:14:09 +0000 (16:14 +0000)
committerDavid Sherwood <david.sherwood@arm.com>
Mon, 17 Apr 2023 10:49:40 +0000 (10:49 +0000)
commit69ee6533131d030bc724193bfc1b91fcc99a81d2
tree83a8bd1a73eb11c1b1835eaabb137dbdabcc914f
parentf2d03a2533c64068de3e85483aed583ab173b18a
[LoopVectorize] Take vscale into account when deciding to create epilogues

In LoopVectorizationCostModel::isEpilogueVectorizationProfitable we
check to see if the chosen main vector loop VF >= 16. If so, we
decide to create a vector epilogue loop. However, this doesn't
take VScaleForTuning into account because we could be targeting a
CPU where vscale > 1, and hence the runtime VF would be a multiple
of the known minimum value.

This patch multiplies scalable VFs by VScaleForTuning and several
tests have been updated that now produce vector epilogues.

Differential Revision: https://reviews.llvm.org/D147522
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
llvm/test/Transforms/LoopVectorize/AArch64/runtime-check-size-based-threshold.ll
llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-vscale-tune.ll [new file with mode: 0644]
llvm/test/Transforms/LoopVectorize/AArch64/sve-fneg.ll
llvm/test/Transforms/LoopVectorize/AArch64/type-shrinkage-zext-costs.ll