From 8571ff0ae0922bee292161c7fd61dd127d26a4ed Mon Sep 17 00:00:00 2001 From: Richard Biener Date: Mon, 23 Aug 2021 14:15:14 +0200 Subject: [PATCH] Adjust inner loop cost scaling This makes use of the estimated number of iterations of the inner loop to limit --param vect-inner-loop-cost-factor scaling. It also reduces the maximum value of vect-inner-loop-cost-factor to 10000 making it less likely to cause overflow of costs. 2021-08-23 Richard Biener * doc/invoke.texi (vect-inner-loop-cost-factor): Adjust. * params.opt (--param vect-inner-loop-cost-factor): Adjust maximum value. * tree-vect-loop.c (vect_analyze_loop_form): Initialize inner_loop_cost_factor to the minimum of the estimated number of iterations of the inner loop and vect-inner-loop-cost-factor. --- gcc/doc/invoke.texi | 7 ++++--- gcc/params.opt | 4 ++-- gcc/tree-vect-loop.c | 7 +++++++ 3 files changed, 13 insertions(+), 5 deletions(-) diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index c057cc1..a9d56fe 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -14386,9 +14386,10 @@ The parameter only has an effect on targets that support partial vector loads and stores. @item vect-inner-loop-cost-factor -The factor which the loop vectorizer applies to the cost of statements -in an inner loop relative to the loop being vectorized. The default -value is 50. +The maximum factor which the loop vectorizer applies to the cost of statements +in an inner loop relative to the loop being vectorized. The factor applied +is the maximum of the estimated number of iterations of the inner loop and +this parameter. The default value of this parameter is 50. @item avoid-fma-max-bits Maximum number of bits for which we avoid creating FMAs. diff --git a/gcc/params.opt b/gcc/params.opt index f926488..f414dc1 100644 --- a/gcc/params.opt +++ b/gcc/params.opt @@ -1114,7 +1114,7 @@ Common Joined UInteger Var(param_vect_partial_vector_usage) Init(2) IntegerRange Controls how loop vectorizer uses partial vectors. 0 means never, 1 means only for loops whose need to iterate can be removed, 2 means for all loops. The default value is 2. -param=vect-inner-loop-cost-factor= -Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 999999) Param Optimization -The factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized. +Common Joined UInteger Var(param_vect_inner_loop_cost_factor) Init(50) IntegerRange(1, 10000) Param Optimization +The maximum factor which the loop vectorizer applies to the cost of statements in an inner loop relative to the loop being vectorized. ; This comment is to ensure we retain the blank line above. diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index c521b43a..0c8d992 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -1519,6 +1519,13 @@ vect_analyze_loop_form (class loop *loop, vec_info_shared *shared) stmt_vec_info inner_loop_cond_info = loop_vinfo->lookup_stmt (inner_loop_cond); STMT_VINFO_TYPE (inner_loop_cond_info) = loop_exit_ctrl_vec_info_type; + /* If we have an estimate on the number of iterations of the inner + loop use that to limit the scale for costing, otherwise use + --param vect-inner-loop-cost-factor literally. */ + widest_int nit; + if (estimated_stmt_executions (loop->inner, &nit)) + LOOP_VINFO_INNER_LOOP_COST_FACTOR (loop_vinfo) + = wi::smin (nit, param_vect_inner_loop_cost_factor).to_uhwi (); } gcc_assert (!loop->aux); -- 2.7.4