[clang][AST] Support AST files larger than 512M
authorDmitry Polukhin <dmitry.polukhin@gmail.com>
Thu, 16 Apr 2020 16:24:46 +0000 (09:24 -0700)
committerDmitry Polukhin <dmitry.polukhin@gmail.com>
Fri, 17 Apr 2020 13:17:33 +0000 (06:17 -0700)
commita7afb211dc460bd4cfb2542ad1f9b05876b57ba1
treeaad50413f5a2adbcc4ffcffc53a7f44540cb982b
parent9a39d5a2ecf7c9290d334f2697c3a4e533aa4bea
[clang][AST] Support AST files larger than 512M

Summary:
Clang uses 32-bit integers for storing bit offsets from the beginning of
the file that results in 512M limit on AST file. This diff replaces
absolute offsets with relative offsets from the beginning of
corresponding data structure when it is possible. And uses 64-bit
offsets for DeclOffests and TypeOffssts because these coder AST
section may easily exceeds 512M alone.

This diff breaks AST file format compatibility so VERSION_MAJOR bumped.

Test Plan:
Existing clang AST serialization tests
Tested on clangd with ~700M and ~900M preamble files
check-clang with ubsan

Reviewers: rsmith, dexonsmith

Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76594
clang/include/clang/Serialization/ASTBitCodes.h
clang/include/clang/Serialization/ASTReader.h
clang/include/clang/Serialization/ASTWriter.h
clang/include/clang/Serialization/ModuleFile.h
clang/lib/Serialization/ASTReader.cpp
clang/lib/Serialization/ASTReaderDecl.cpp
clang/lib/Serialization/ASTWriter.cpp
clang/lib/Serialization/ASTWriterDecl.cpp