--- /dev/null
+===================
+FatLTO
+===================
+.. contents::
+ :local:
+ :depth: 2
+
+.. toctree::
+ :maxdepth: 1
+
+Introduction
+============
+
+FatLTO objects are a special type of `fat object file
+<https://en.wikipedia.org/wiki/Fat_binary>`_ that contain LTO compatible IR in
+addition to generated object code, instead of containing object code for
+multiple target architectures. This allows users to defer the choice of whether
+to use LTO or not to link-time, and has been a feature available in other
+compilers, like `GCC
+<https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>`_, for some time.
+
+Under FatLTO the compiler can emit standard object files which contain both the
+machine code in the ``.text`` section and LLVM bitcode in the ``.llvm.lto``
+section.
+
+Overview
+========
+
+Within LLVM, FatLTO is supported by choosing the ``FatLTODefaultPipeline``.
+This pipeline will:
+
+#) Clone the IR module.
+#) Run the pre-link (Thin)LTO pipeline using the cloned module.
+#) Embed the pre-link bitcode in a special ``.llvm.lto`` section.
+#) Optimize the unmodified copy of the module using the normal compilation pipeline.
+#) Emit the object file, including the new ``.llvm.lto`` section.
+
+.. NOTE
+
+ At the time of writing, we conservatively run independent pipelines to
+ generate the bitcode section and the object code, which happen to be
+ identical to those used outside of FatLTO. This results in compiled
+ artifacts that are identical to those produced by the default and (Thin)LTO
+ pipelines. However, this is not a guarantee, and we reserve the right to
+ change this at any time. Explicitly, users should not rely on the produced
+ bitcode or object code to mach their non-LTO counterparts precisely. They
+ will exhibit similar performance characteristics, but may not be bit-for-bit
+ the same.
+
+Internally, the ``.llvm.lto`` section is created by running the
+``EmbedBitcodePass`` at the start of the ``PerModuleDefaultPipeline``. This
+pass is responsible for cloning and optimizing the module with the appropriate
+LTO pipeline and emitting the ``.llvm.lto`` section. Afterwards, the
+``PerModuleDefaultPipeline`` runs normally and the compiler can emit the fat
+object file.
+
+Limitations
+===========
+
+Linkers
+-------
+
+Currently, using LTO with LLVM fat lto objects is supported by LLD and by the
+GNU linkers via :doc:`GoldPlugin`. This may change in the future, but
+extending support to other linkers isn't planned for now.
+
+.. NOTE
+ For standard linking the fat object files should be usable by any
+ linker capable of using ELF objects, since the ``.llvm.lto`` section is
+ marked ``SHF_EXLUDE``.
+
+Supported File Formats
+----------------------
+
+The current implementation only supports ELF files. At time of writing, it is
+unclear if it will be useful to support other object file formats like ``COFF``
+or ``Mach-O``.
* InstructionSimplify APIs now require instructions be inserted into a
parent function.
+* A new FatLTO pipeline was added to support generating object files that have
+ both machine code and LTO compatible bitcode. See the :doc:`FatLTO`
+ documentation and the original
+ `RFC <https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977>`_
+ for more details.
+
Changes to building LLVM
------------------------
DebuggingJITedCode\r
DirectXUsage\r
Docker\r
+ FatLTO\r
ExtendingLLVM\r
GoldPlugin\r
HowToBuildOnARM\r
ModulePassManager buildPerModuleDefaultPipeline(OptimizationLevel Level,
bool LTOPreLink = false);
+ /// Build a fat object default optimization pipeline.
+ ///
+ /// This builds a pipeline that runs the LTO/ThinLTO pre-link pipeline, and
+ /// emits a section containing the pre-link bitcode along side the object code
+ /// generated by running the PerModuleDefaultPipeline, used when compiling
+ /// without LTO. It clones the module and runs the LTO/non-LTO pipelines
+ /// separately to avoid any inconsistencies with an ad-hoc pipeline that tries
+ /// to approximate the PerModuleDefaultPipeline from the pre-link LTO
+ /// pipelines.
+ ModulePassManager buildFatLTODefaultPipeline(OptimizationLevel Level,
+ bool ThinLTO, bool EmitSummary);
+
/// Build a pre-link, ThinLTO-targeting default optimization pipeline to
/// a pass manager.
///
--- /dev/null
+//===-- EmbedBitcodePass.h - Embeds bitcode into global ---------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file provides a pass which clones the current module and runs the
+/// provided pass pipeline on the clone. The optimized module is stored into a
+/// global variable in the `.llvm.lto` section. Primarily, this pass is used
+/// to support the FatLTO pipeline, but could be used to generate a bitcode
+/// section for any arbitrary pass pipeline without changing the current module.
+///
+//===----------------------------------------------------------------------===//
+//
+#ifndef LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
+#define LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+class Module;
+class ModulePass;
+class Pass;
+
+struct EmbedBitcodeOptions {
+ EmbedBitcodeOptions() : EmbedBitcodeOptions(false, false) {}
+ EmbedBitcodeOptions(bool IsThinLTO, bool EmitLTOSummary)
+ : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary) {}
+ bool IsThinLTO;
+ bool EmitLTOSummary;
+};
+
+/// Pass embeds a copy of the module optimized with the provided pass pipeline
+/// into a global variable.
+class EmbedBitcodePass : public PassInfoMixin<EmbedBitcodePass> {
+ bool IsThinLTO;
+ bool EmitLTOSummary;
+ ModulePassManager MPM;
+
+public:
+ EmbedBitcodePass(EmbedBitcodeOptions Opts)
+ : EmbedBitcodePass(Opts.IsThinLTO, Opts.EmitLTOSummary,
+ ModulePassManager()) {}
+ EmbedBitcodePass(bool IsThinLTO, bool EmitLTOSummary, ModulePassManager &&MPM)
+ : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary),
+ MPM(std::move(MPM)) {}
+
+ PreservedAnalyses run(Module &M, ModuleAnalysisManager &);
+
+ static bool isRequired() { return true; }
+};
+
+} // end namespace llvm.
+
+#endif
bool ObjectFile::isSectionBitcode(DataRefImpl Sec) const {
Expected<StringRef> NameOrErr = getSectionName(Sec);
if (NameOrErr)
- return *NameOrErr == ".llvmbc";
+ return *NameOrErr == ".llvmbc" || *NameOrErr == ".llvm.lto";
consumeError(NameOrErr.takeError());
return false;
}
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionImport.h"
return Result;
}
+Expected<EmbedBitcodeOptions> parseEmbedBitcodePassOptions(StringRef Params) {
+ EmbedBitcodeOptions Result;
+ while (!Params.empty()) {
+ StringRef ParamName;
+ std::tie(ParamName, Params) = Params.split(';');
+
+ if (ParamName == "thinlto") {
+ Result.IsThinLTO = true;
+ } else if (ParamName == "emit-summary") {
+ Result.EmitLTOSummary = true;
+ } else {
+ return make_error<StringError>(
+ formatv("invalid EmbedBitcode pass parameter '{0}' ", ParamName)
+ .str(),
+ inconvertibleErrorCode());
+ }
+ }
+ return Result;
+}
+
Expected<MemorySanitizerOptions> parseMSanPassOptions(StringRef Params) {
MemorySanitizerOptions Result;
while (!Params.empty()) {
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/GlobalDCE.h"
if (LTOPreLink)
addRequiredLTOPreLinkPasses(MPM);
+ return MPM;
+}
+ModulePassManager
+PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level, bool ThinLTO,
+ bool EmitSummary) {
+ ModulePassManager MPM;
+ MPM.addPass(EmbedBitcodePass(ThinLTO, EmitSummary,
+ ThinLTO
+ ? buildThinLTOPreLinkDefaultPipeline(Level)
+ : buildLTOPreLinkDefaultPipeline(Level)));
+ MPM.addPass(buildPerModuleDefaultPipeline(Level));
return MPM;
}
},
parseIPSCCPOptions,
"no-func-spec;func-spec")
+MODULE_PASS_WITH_PARAMS("embed-bitcode",
+ "EmbedBitcodePass",
+ [](EmbedBitcodeOptions Opts) {
+ return EmbedBitcodePass(Opts);
+ },
+ parseEmbedBitcodePassOptions,
+ "thinlto;emit-summary")
#undef MODULE_PASS_WITH_PARAMS
#ifndef CGSCC_ANALYSIS
CrossDSOCFI.cpp
DeadArgumentElimination.cpp
ElimAvailExtern.cpp
+ EmbedBitcodePass.cpp
ExtractGV.cpp
ForceFunctionAttrs.cpp
FunctionAttrs.cpp
--- /dev/null
+//===- EmbedBitcodePass.cpp - Pass that embeds the bitcode into a global---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/Bitcode/BitcodeWriterPass.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+#include "llvm/Passes/PassBuilder.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/TargetParser/Triple.h"
+#include "llvm/Transforms/IPO/ThinLTOBitcodeWriter.h"
+#include "llvm/Transforms/Utils/Cloning.h"
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+
+using namespace llvm;
+
+PreservedAnalyses EmbedBitcodePass::run(Module &M, ModuleAnalysisManager &AM) {
+ if (M.getGlobalVariable("llvm.embedded.module", /*AllowInternal=*/true))
+ report_fatal_error("Can only embed the module once",
+ /*gen_crash_diag=*/false);
+
+ Triple T(M.getTargetTriple());
+ if (T.getObjectFormat() != Triple::ELF)
+ report_fatal_error(
+ "EmbedBitcode pass currently only supports ELF object format",
+ /*gen_crash_diag=*/false);
+
+ std::unique_ptr<Module> NewModule = CloneModule(M);
+ MPM.run(*NewModule, AM);
+
+ std::string Data;
+ raw_string_ostream OS(Data);
+ if (IsThinLTO)
+ ThinLTOBitcodeWriterPass(OS, /*ThinLinkOS=*/nullptr).run(*NewModule, AM);
+ else
+ BitcodeWriterPass(OS, /*ShouldPreserveUseListOrder=*/false, EmitLTOSummary)
+ .run(*NewModule, AM);
+
+ embedBufferInModule(M, MemoryBufferRef(Data, "ModuleData"), ".llvm.lto");
+
+ return PreservedAnalyses::all();
+}
--- /dev/null
+; RUN: not opt --mtriple x86_64-unknown-linux-gnu < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
+
+@a = global i32 1
+@llvm.embedded.module = private constant [4 x i8] c"BC\C0\DE"
+
+; CHECK: LLVM ERROR: Can only embed the module once
--- /dev/null
+; RUN: not opt --mtriple powerpc64-unknown-aix < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
+
+@a = global i32 1
+
+; CHECK: LLVM ERROR: EmbedBitcode pass currently only supports ELF object format
--- /dev/null
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto>" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<emit-summary>" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S | FileCheck %s
+
+@a = global i32 1
+
+; CHECK: @a = global i32 1
+;; Make sure the module is in the correct section.
+; CHECK: @llvm.embedded.object = private constant {{.*}}, section ".llvm.lto", align 1
+
+;; Ensure that the metadata is in llvm.compiler.used.
+; CHECK: @llvm.compiler.used = appending global [1 x ptr] [ptr @llvm.embedded.object], section "llvm.metadata"
+
+;; Make sure the metadata correlates to the .llvm.lto section.
+; CHECK: !llvm.embedded.objects = !{!1}
+; CHECK: !0 = !{}
+; CHECK: !{ptr @llvm.embedded.object, !".llvm.lto"}
+
+
+;; Ensure that the .llvm.lto section has SHT_EXCLUDE set.
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S \
+; RUN: | llc --mtriple x86_64-unknown-linux-gnu -filetype=obj \
+; RUN: | llvm-readobj - --sections --elf-output-style=JSON --pretty-print \
+; RUN: | FileCheck %s --check-prefix=EXCLUDE
+
+; EXCLUDE: "Name": ".llvm.lto",
+; EXCLUDE-NEXT: "Value": 7
+; EXCLUDE-NEXT: },
+; EXCLUDE-NEXT: "Type": {
+; EXCLUDE-NEXT: "Name": "SHT_PROGBITS",
+; EXCLUDE-NEXT: "Value": 1
+; EXCLUDE-NEXT: },
+; EXCLUDE-NEXT: "Flags": {
+; EXCLUDE-NEXT: "Value": 2147483648,
+; EXCLUDE-NEXT: "Flags": [
+; EXCLUDE-NEXT: {
+; EXCLUDE-NEXT: "Name": "SHF_EXCLUDE",
+; EXCLUDE-NEXT: "Value": 2147483648
+; EXCLUDE-NEXT: }
+; EXCLUDE-NEXT: ]
+; EXCLUDE-NEXT: },
+