[LoopVectorize] Permit fixed-width epilogue loops for scalable vector bodies
authorDavid Sherwood <david.sherwood@arm.com>
Tue, 7 Sep 2021 10:20:41 +0000 (11:20 +0100)
committerDavid Sherwood <david.sherwood@arm.com>
Mon, 8 Nov 2021 09:41:13 +0000 (09:41 +0000)
commitc42bb30b9e2950efd1d1bd2df9ab8e5b098e65c7
tree9793a5124ee4c873a25559978dc01e8127d60adf
parent9b5e2b5261f6cb1e7913e452e5374e6f6ce68e44
[LoopVectorize] Permit fixed-width epilogue loops for scalable vector bodies

At the moment in LoopVectorizationCostModel::selectEpilogueVectorizationFactor
we bail out if the main vector loop uses a scalable VF. This patch adds
support for generating epilogue vector loops using a fixed-width VF when the
main vector loop uses a scalable VF.

I've changed LoopVectorizationCostModel::selectEpilogueVectorizationFactor
so that we convert the scalable VF into a fixed-width VF and do profitability
checks on that instead. In addition, since the scalable and fixed-width VFs
live in different VPlans that means I had to change the calls to
LVP.hasPlanWithVFs so that we only pass in the fixed-width VF.

New tests added here:

  Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll

Differential Revision: https://reviews.llvm.org/D109432
llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll [new file with mode: 0644]
llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-scalable.ll