[LV] Keep track of cost-based ScalarAfterVec in VPWidenPointerInd.
authorFlorian Hahn <flo@fhahn.com>
Mon, 19 Sep 2022 17:14:34 +0000 (18:14 +0100)
committerFlorian Hahn <flo@fhahn.com>
Mon, 19 Sep 2022 17:14:35 +0000 (18:14 +0100)
commit582f8ef19fe791e66c0bbe090edebfd74ba2b55d
tree6b5c5947a4f5ac663e4543973870707e3396f024
parente6f8a6b56e27b86b0563d3c6d547c2b14f0a1ced
[LV] Keep track of cost-based ScalarAfterVec in VPWidenPointerInd.

Epilogue vectorization uses isScalarAfterVectorization to check if
widened versions for inductions need to be generated and bails out in
those cases.

At the moment, there are scenarios where isScalarAfterVectorization
returns true but VPWidenPointerInduction::onlyScalarsGenerated would
return false, causing widening.

This can lead to widened phis with incorrect start values being created
in the epilogue vector body.

This patch addresses the issue by storing the cost-model decision in
VPWidenPointerInductionRecipe and restoring the behavior before 151c144.
This effectively reverts 151c144, but the long-term fix is to properly
support widened inductions during epilogue vectorization

Fixes #57712.
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
llvm/lib/Transforms/Vectorize/VPlan.cpp
llvm/lib/Transforms/Vectorize/VPlan.h
llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
llvm/test/Transforms/LoopVectorize/AArch64/epilog-vectorization-widen-inductions.ll
llvm/test/Transforms/LoopVectorize/AArch64/sve-live-out-pointer-induction.ll
llvm/test/Transforms/LoopVectorize/AArch64/sve-widen-gep.ll
llvm/test/Transforms/LoopVectorize/AArch64/sve-widen-phi.ll
llvm/test/Transforms/LoopVectorize/X86/interleave-opaque-pointers.ll
llvm/test/Transforms/LoopVectorize/opaque-ptr.ll