[Syntax] Remove delayed folding from tree building.
authorMarcel Hlopko <hlopko@google.com>
Tue, 31 Mar 2020 07:47:49 +0000 (09:47 +0200)
committerDmitri Gribenko <gribozavr@gmail.com>
Tue, 31 Mar 2020 07:47:50 +0000 (09:47 +0200)
commitcdce2fe561eb6e63d77d1d589e521a8e2afb1eef
treec266033ec1d87d4c10e5c288e8e26346562a2b09
parent159343cd1965c69233597b36ab27adce8bd34a14
[Syntax] Remove delayed folding from tree building.

Summary:
This patch removes delayed folding and replaces it with forward peeking.

Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:

```
int a,b;
```

is expressed in the AST as:

```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```

And in the syntax tree we need:

```
*: TranslationUnit
`-SimpleDeclaration
  |-int
  |-SimpleDeclarator
  | `-a
  |-,
  |-SimpleDeclarator
  | `-b
  |-;
```

So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.

This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).

This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
  method in all places.
* Adds a bunch of tests

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76922
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp