analyzer: tweak priority of callstrings in worklist::key_t::cmp
authorDavid Malcolm <dmalcolm@redhat.com>
Fri, 11 Jun 2021 13:30:33 +0000 (09:30 -0400)
committerDavid Malcolm <dmalcolm@redhat.com>
Fri, 11 Jun 2021 13:30:33 +0000 (09:30 -0400)
commit9d20ec97475b1102d6ca005ad165056d34615a3d
treeed7895d8589a59139866da3c17c979c6b463c1bc
parentb9ec5ebb605936684e95b8dcc12e43ba7d8f2cb4
analyzer: tweak priority of callstrings in worklist::key_t::cmp

While debugging another issue I noticed that the analyzer could fail to
merge nodes for control flow in which one path had called a function
and another path hadn't:

        BB
       /  \
      /    \
 fn call   no fn call
      \    /
       \  /
     join BB

The root cause was that the worklist sort function wasn't prioritizing
call strings, and thus it was fully exploring the "no function called"
path to the exit BB, and only then exploring the "within the function call"
parts of the "funcion called" path.

This patch prioritizes call strings when sorting the worklist so that
the nodes with deeper call strings are processed before those with shallower
call strings, thus allowing such nodes to be merged at the joinpoint.

gcc/analyzer/ChangeLog:
* engine.cc (worklist::key_t::cmp): Move sort by call_string to
before SCC.

gcc/testsuite/ChangeLog:
* gcc.dg/analyzer/loop-0-up-to-n-by-1-with-iter-obj.c: Update
expected number of enodes after the loop.
* gcc.dg/analyzer/paths-8.c: New test.

Signed-off-by: David Malcolm <dmalcolm@redhat.com>
gcc/analyzer/engine.cc
gcc/testsuite/gcc.dg/analyzer/loop-0-up-to-n-by-1-with-iter-obj.c
gcc/testsuite/gcc.dg/analyzer/paths-8.c [new file with mode: 0644]