[clang][AST] Support AST files larger than 512M
authorDmitry Polukhin <dmitry.polukhin@gmail.com>
Sat, 21 Mar 2020 20:52:51 +0000 (13:52 -0700)
committerDmitry Polukhin <dmitry.polukhin@gmail.com>
Thu, 16 Apr 2020 14:27:43 +0000 (07:27 -0700)
commit30d5946db95fa465d7ee6caceb2b1ff191e3727c
tree7cbcdca8a8a6c9200a7cb9e705f0c4839f980e51
parentf701d8fa5ff939235dc5cc2a3340bef021c07423
[clang][AST] Support AST files larger than 512M

Summary:
Clang uses 32-bit integers for storing bit offsets from the beginning of
the file that results in 512M limit on AST file. This diff replaces
absolute offsets with relative offsets from the beginning of
corresponding data structure when it is possible. And uses 64-bit
offsets for DeclOffests and TypeOffssts because these coder AST
section may easily exceeds 512M alone.

This diff breaks AST file format compatibility so VERSION_MAJOR bumped.

Test Plan:
Existing clang AST serialization tests
Tested on clangd with ~700M and ~900M preamble files

Reviewers: rsmith, dexonsmith

Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76594
clang/include/clang/Serialization/ASTBitCodes.h
clang/include/clang/Serialization/ASTReader.h
clang/include/clang/Serialization/ASTWriter.h
clang/include/clang/Serialization/ModuleFile.h
clang/lib/Serialization/ASTReader.cpp
clang/lib/Serialization/ASTReaderDecl.cpp
clang/lib/Serialization/ASTWriter.cpp
clang/lib/Serialization/ASTWriterDecl.cpp