platform/upstream/llvm.git
15 months ago[RISCV] Move shuffle-reverse test to rvv directory. NFC
Luke Lau [Sun, 2 Apr 2023 14:14:44 +0000 (15:14 +0100)]
[RISCV] Move shuffle-reverse test to rvv directory. NFC

Test is related to vectors so probably belongs in the rvv directory

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D147412

15 months agoRevert "[test] Remove occurences of br undef in CodeGen/AMDGPU tests"
Nuno Lopes [Tue, 4 Apr 2023 12:45:56 +0000 (13:45 +0100)]
Revert "[test] Remove occurences of br undef in CodeGen/AMDGPU tests"

This reverts commit 18c594c176036b7ffcd8439ed9c4b08d2085a244.
Build bots broke

15 months ago[X86][mem-fold] Refine code, NFCI
Shengchen Kan [Tue, 4 Apr 2023 12:39:59 +0000 (20:39 +0800)]
[X86][mem-fold] Refine code, NFCI

1. Remove redundant definition of constructor
2. Move the array in .inc to .def file
3. Add a licence for the .def file

15 months agoMachO use generic code to detect atomic support.
Tim Northover [Tue, 4 Apr 2023 10:55:19 +0000 (11:55 +0100)]
MachO use generic code to detect atomic support.

The default code can detect what width of atomic instructions are supported
based on the targeted architecture profile, version etc so there's no need to
hard-code 64 on Darwin targets (especially as it's wrong in most M-class
cases).

15 months ago[SystemZ] Remove unused function 'decodeU6ImmOperand' in SystemZDisassembler.cpp...
Jie Fu [Tue, 4 Apr 2023 12:20:54 +0000 (20:20 +0800)]
[SystemZ] Remove unused function 'decodeU6ImmOperand' in SystemZDisassembler.cpp (NFC)

/data/llvm-project/llvm/lib/Target/SystemZ/Disassembler/SystemZDisassembler.cpp:209:21: error: unused function 'decodeU6ImmOperand' [-Werror,-Wunused-function]
static DecodeStatus decodeU6ImmOperand(MCInst &Inst, uint64_t Imm,
                    ^
1 error generated.

15 months ago[test] Remove occurences of br undef in CodeGen/AMDGPU tests
Kriti Gupta [Tue, 4 Apr 2023 12:14:30 +0000 (13:14 +0100)]
[test] Remove occurences of br undef in CodeGen/AMDGPU tests

Differential Revision: https://reviews.llvm.org/D145622

15 months ago[X86] Convert some tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 12:07:22 +0000 (14:07 +0200)]
[X86] Convert some tests to opaque pointers (NFC)

15 months ago[compiler-rt] Fix scudo build on ARM
Leandro Lupori [Fri, 27 Jan 2023 18:34:00 +0000 (18:34 +0000)]
[compiler-rt] Fix scudo build on ARM

The build of scudo was failing on armv7l, with undefined references
to unwinder symbols, such as __aeabi_unwind_cpp_pr0. These are
needed by RTGwpAsan and thus, on ARM, scudo must also be linked
against an unwind library.

The cmake command that caused the build failure was:

cmake --fresh -S "$PWD/llvm/" -B "$PWD/build/" -G Ninja \
  -DCMAKE_INSTALL_PREFIX="$PWD/install" \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang;lld;lldb;clang-tools-extra;polly" \
  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \
  -DLLVM_TOOLCHAIN_TOOLS="llvm-ar;llvm-ranlib;llvm-objdump;\
llvm-rc;llvm-cvtres;llvm-nm;llvm-strings;llvm-readobj;\
llvm-dlltool;llvm-pdbutil;llvm-objcopy;llvm-strip;llvm-cov;\
llvm-profdata;llvm-addr2line;llvm-symbolizer;llvm-windres;llvm-ml;\
llvm-readelf;llvm-size" \
  -DLLVM_INSTALL_BINUTILS_SYMLINKS=OFF -DLLVM_PARALLEL_LINK_JOBS=1

Fixes #60115

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D142888

15 months ago[DebugInfo] don't make an MDTuple just to leak it
Jameson Nash [Tue, 4 Apr 2023 11:42:25 +0000 (07:42 -0400)]
[DebugInfo] don't make an MDTuple just to leak it

There does not seem to be any purpose to allocating this object, which
is Metadata, so LLVM will simply leak it until the context is destroyed
(the subprogram metadata it contains are distinct, so there is little
chance of it being reused later). This should not have a functional
change.

This seems to be left over from an earlier design, when it used to call
```
CUNode->replaceSubprograms(SPs.get());
```
here also. (removed in http://reviews.llvm.org/D19034)

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D147469

15 months ago[SystemZ] Allow any I5 in RotateSelect*
Ilya Leoshkevich [Tue, 4 Apr 2023 10:11:50 +0000 (12:11 +0200)]
[SystemZ] Allow any I5 in RotateSelect*

For all RotateSelect* instructions, PoP says:

    Bits 0-1 of the I5 field (bits 32-33 of the instruction) are
    ignored.

LLVM, however, completely prohibits using them, e.g.:

    error: invalid operand for instruction
    asm("rxsbg %[r1],%[r2],177,43,228\n"

Lift this unnecessary restriction.

Reviewed By: uweigand

Differential Revision: https://reviews.llvm.org/D146185

15 months ago[X86] Convert test to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 11:48:49 +0000 (13:48 +0200)]
[X86] Convert test to opaque pointers (NFC)

Retaining bitcasts to keep test intent.

addrsig_sym is no longer emitted for metedata_f1, but this
improvement is ultimately inconsequential, as it will get dropped
at a later stage anyway.

15 months ago[lit] Implement -I option for builtin "diff" command to ignore changes where all...
Bing1 Yu [Tue, 4 Apr 2023 11:47:23 +0000 (19:47 +0800)]
[lit] Implement -I option for builtin "diff" command to ignore changes where all lines match RE

Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D147301

15 months ago[Verifier] Rename matrix-intrinsics-opaque-ptrs.ll->matrix-intrinsics.ll
Florian Hahn [Tue, 4 Apr 2023 11:47:15 +0000 (12:47 +0100)]
[Verifier] Rename matrix-intrinsics-opaque-ptrs.ll->matrix-intrinsics.ll

The original typed pointer version of the tests (matrix-intrinsics.ll)
is now obsolete. Replace matrix-intrinsics.ll with
matrix-intrinsics-opaque-ptrs.ll.

15 months ago[X86][NFC] Remove redundant flags used in x86-auto-memfold.td
Shengchen Kan [Tue, 4 Apr 2023 11:36:05 +0000 (19:36 +0800)]
[X86][NFC] Remove redundant flags used in x86-auto-memfold.td

15 months ago[include-cleaner] Only ignore builtins without a header
Kadir Cetinkaya [Mon, 3 Apr 2023 15:38:54 +0000 (17:38 +0200)]
[include-cleaner] Only ignore builtins without a header

Certain standard library functions (e.g. std::move) are also implemented
as builtins. This patch moves filtering logic to the symbol->header
mapping phase to rather generate these references without any providers
only when we don't have a mapping.
That way we can also map them to header names mentioned in the builtin
mappings.

Differential Revision: https://reviews.llvm.org/D147449

15 months ago[X86] Convert test to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:53:49 +0000 (12:53 +0200)]
[X86] Convert test to opaque pointers (NFC)

15 months ago[X86] Regenerate test checks (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:52:59 +0000 (12:52 +0200)]
[X86] Regenerate test checks (NFC)

15 months ago[NFC][AArch64] Late 2022 A-Profile Extension Release Notes
Archibald Elliott [Tue, 4 Apr 2023 10:39:41 +0000 (11:39 +0100)]
[NFC][AArch64] Late 2022 A-Profile Extension Release Notes

15 months ago[Thumb2] Convert tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:25:30 +0000 (12:25 +0200)]
[Thumb2] Convert tests to opaque pointers (NFC)

15 months ago[Thumb2] Name instructions in tests (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:24:52 +0000 (12:24 +0200)]
[Thumb2] Name instructions in tests (NFC)

15 months ago[CodeGen] ExpandReductions - add reduce_and/or(<X x i1> V) -> icmp(iX bitcast(<X...
Simon Pilgrim [Tue, 4 Apr 2023 10:19:30 +0000 (11:19 +0100)]
[CodeGen] ExpandReductions - add reduce_and/or(<X x i1> V) -> icmp(iX bitcast(<X x i1> V)) canonicalization

This already exists in InstCombine but was missing from the late stage ExpandReductions pass

Fixes #53419
Fixes #61923

Differential Revision: https://reviews.llvm.org/D147452

15 months ago[WebAssembly] Convert tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:16:50 +0000 (12:16 +0200)]
[WebAssembly] Convert tests to opaque pointers (NFC)

15 months ago[WinEH] Convert test to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:13:26 +0000 (12:13 +0200)]
[WinEH] Convert test to opaque pointers (NFC)

15 months ago[PowerPC] Convert tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:04:40 +0000 (12:04 +0200)]
[PowerPC] Convert tests to opaque pointers (NFC)

15 months ago[InstCombine] Pre-land tests for maximum(a,b) +* minimum(a,b) => a +* b
Serguei Katkov [Tue, 4 Apr 2023 09:49:59 +0000 (16:49 +0700)]
[InstCombine] Pre-land tests for maximum(a,b) +* minimum(a,b) => a +* b

15 months ago[PowerPC] Name instructions in tests (NFC)
Nikita Popov [Tue, 4 Apr 2023 10:03:33 +0000 (12:03 +0200)]
[PowerPC] Name instructions in tests (NFC)

15 months ago[Hexagon] Convert some tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 09:34:02 +0000 (11:34 +0200)]
[Hexagon] Convert some tests to opaque pointers (NFC)

15 months ago[InstCombine][NFC] Update generated variable names in signed-truncation-check.ll
John McIver [Tue, 4 Apr 2023 09:47:42 +0000 (10:47 +0100)]
[InstCombine][NFC] Update generated variable names in signed-truncation-check.ll

Update signed-truncation-check.ll using latest version of update_test_checks.py.

Differential Revision: https://reviews.llvm.org/D143030

15 months ago[X86] LowerVectorAllEqual - remove repeated VT.getScalarSizeInBits() call. NFC.
Simon Pilgrim [Tue, 4 Apr 2023 09:45:43 +0000 (10:45 +0100)]
[X86] LowerVectorAllEqual - remove repeated VT.getScalarSizeInBits() call. NFC.

15 months ago[mlir][Linalg] NFC - Refactor hoist padding transform and separate packing creation...
Nicolas Vasilache [Mon, 3 Apr 2023 15:17:24 +0000 (08:17 -0700)]
[mlir][Linalg] NFC - Refactor hoist padding transform and separate packing creation functionality in a separate transform

This revision is in prevision for upcoming improvements.

Differential Revision: https://reviews.llvm.org/D147458

15 months ago[mlir] add readonly/consume annotations to transform named sequences
Alex Zinenko [Thu, 30 Mar 2023 12:31:48 +0000 (12:31 +0000)]
[mlir] add readonly/consume annotations to transform named sequences

Use the argument attribute mechanism for function-like operations to
annotate the arguments of named transform sequences as consuming or only
reading the handles passed as arguments. This makes it possible to
correctly specify handle invalidation for external named sequences by
requiring their declarations to always provide such annotations.
Additionally, these annotations remove the need to analyze the body of
a named sequence to understand its effects on the arguments. Make them
required for named sequences that are called from the same file, in
addition to external sequences.

Provide a convenience pass that infers annotations by analyzing bodies
of named sequences provided they are not called from the same file.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D147223

15 months ago[ARM] Regenerate test checks (NFC)
Nikita Popov [Tue, 4 Apr 2023 09:25:13 +0000 (11:25 +0200)]
[ARM] Regenerate test checks (NFC)

15 months ago[ARM] Convert some tests to opaque pointers (NFC)
Nikita Popov [Tue, 4 Apr 2023 09:15:38 +0000 (11:15 +0200)]
[ARM] Convert some tests to opaque pointers (NFC)

15 months ago[ARM] Regenerate test checks (NFC)
Nikita Popov [Tue, 4 Apr 2023 09:15:22 +0000 (11:15 +0200)]
[ARM] Regenerate test checks (NFC)

15 months ago[ARM] Name instructions in test (NFC)
Nikita Popov [Tue, 4 Apr 2023 09:10:24 +0000 (11:10 +0200)]
[ARM] Name instructions in test (NFC)

15 months ago[clang][Interp][NFC] Remove Integral.h include from PrimType.h
Timm Bäder [Thu, 16 Feb 2023 08:01:52 +0000 (09:01 +0100)]
[clang][Interp][NFC] Remove Integral.h include from PrimType.h

15 months ago[clang][Interp] Ignore more non-VarDecl declarations
Timm Bäder [Sun, 12 Mar 2023 07:32:07 +0000 (08:32 +0100)]
[clang][Interp] Ignore more non-VarDecl declarations

They are harmless and handled by other means, but we used to return
false from visitDeclStmt.

Differential Revision: https://reviews.llvm.org/D145861

15 months ago[Test] Commit test for PR61923
Max Kazantsev [Tue, 4 Apr 2023 08:08:36 +0000 (15:08 +0700)]
[Test] Commit test for PR61923

We see increased number of assembly instructions after patch
24780e13e5be1501e34330148137a10fa9965166 for this test. See details
at https://github.com/llvm/llvm-project/issues/61923.

15 months ago[clang][dataflow] Fix -Wdeprecated-declarations after D147302 (NFC)
Jie Fu [Tue, 4 Apr 2023 08:17:25 +0000 (16:17 +0800)]
[clang][dataflow] Fix -Wdeprecated-declarations after D147302 (NFC)

Replace:
 1. createAtomicBoolValue() --> create<AtomicBoolValue>()
 2. createTopBoolValue()    --> create<TopBoolValue>()

/Users/jiefu/llvm-project/clang/include/clang/Analysis/FlowSensitive/DataflowEnvironment.h:386:19: error: 'createAtomicBoolValue' is deprecated: use create<AtomicBoolValue> instead [-Werror,-Wdeprecated-declarations]
    return DACtx->createAtomicBoolValue();
                  ^~~~~~~~~~~~~~~~~~~~~
                  create<AtomicBoolValue>
/Users/jiefu/llvm-project/clang/include/clang/Analysis/FlowSensitive/DataflowAnalysisContext.h:215:3: note: 'createAtomicBoolValue' has been explicitly marked deprecated here
  LLVM_DEPRECATED("use create<AtomicBoolValue> instead",
  ^
/Users/jiefu/llvm-project/llvm/include/llvm/Support/Compiler.h:143:50: note: expanded from macro 'LLVM_DEPRECATED'
                                                 ^
In file included from /Users/jiefu/llvm-project/clang/lib/Analysis/FlowSensitive/Transfer.cpp:14:
In file included from /Users/jiefu/llvm-project/clang/include/clang/Analysis/FlowSensitive/Transfer.h:19:
/Users/jiefu/llvm-project/clang/include/clang/Analysis/FlowSensitive/DataflowEnvironment.h:391:19: error: 'createTopBoolValue' is deprecated: use create<TopBoolValue> instead [-Werror,-Wdeprecated-declarations]
    return DACtx->createTopBoolValue();
                  ^~~~~~~~~~~~~~~~~~
                  create<TopBoolValue>
/Users/jiefu/llvm-project/clang/include/clang/Analysis/FlowSensitive/DataflowAnalysisContext.h:227:3: note: 'createTopBoolValue' has been explicitly marked deprecated here
  LLVM_DEPRECATED("use create<TopBoolValue> instead", "create<TopBoolValue>")
  ^
/Users/jiefu/llvm-project/llvm/include/llvm/Support/Compiler.h:143:50: note: expanded from macro 'LLVM_DEPRECATED'
                                                 ^
2 errors generated.

15 months ago[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
Nikita Popov [Wed, 22 Mar 2023 10:53:01 +0000 (11:53 +0100)]
[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629

15 months ago[X86] Disable masked UNPCKLPD/UNPCKHPD -> SHUFPS transformation
Phoebe Wang [Tue, 4 Apr 2023 07:51:09 +0000 (15:51 +0800)]
[X86] Disable masked UNPCKLPD/UNPCKHPD -> SHUFPS transformation

UNPCKLPD/UNPCKHPD is a 64-bit element operation. The masked version
doesn't match SHUFPS in lanes.
This reverts part of D144763.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D147507

15 months ago[clang-tidy] Small refactor for ExceptionAnalyzer
AMS21 [Tue, 4 Apr 2023 06:39:15 +0000 (06:39 +0000)]
[clang-tidy] Small refactor for ExceptionAnalyzer

- Use llvm::DenseMap<> with pre-allocation instead of std::map<> for FunctionCache
- Avoid double lookup for FunctionCache
- Use try_emplace instead of insert
- Simplify definition of State enum

Reviewed By: PiotrZSL

Differential Revision: https://reviews.llvm.org/D147376

15 months ago[clang-tidy] Allow bugprone-unchecked-optional-access to handle calls to `std::forward`
AMS21 [Tue, 4 Apr 2023 06:38:40 +0000 (06:38 +0000)]
[clang-tidy] Allow bugprone-unchecked-optional-access to handle calls to `std::forward`

The check now understands that calling `std::forward`
will not modify the underlying optional value.

This fixes llvm#59705

Reviewed By: PiotrZSL

Differential Revision: https://reviews.llvm.org/D147383

15 months ago[clang-tidy] Fix readability-static-accessed-through-instance check for anonymous...
AMS21 [Tue, 4 Apr 2023 06:35:08 +0000 (06:35 +0000)]
[clang-tidy] Fix readability-static-accessed-through-instance check for anonymous structs

Previously we would provide a fixit which looked like
this `unnamed struct at ...::f()` but which is obviously
not valid C/C++.

Since there is no real nice way to accesses a static function
from an anonymous struct anyways we simply ignore all
anonymous structs.

Fixes llvm#61736

Reviewed By: PiotrZSL

Differential Revision: https://reviews.llvm.org/D147411

15 months ago[clang][dataflow] Add `create<T>()` methods to `Environment` and `DataflowAnalysisCon...
Martin Braenne [Mon, 3 Apr 2023 14:45:13 +0000 (14:45 +0000)]
[clang][dataflow] Add `create<T>()` methods to `Environment` and `DataflowAnalysisContext`.

These methods provide a less verbose way of allocating `StorageLocation`s and
`Value`s than the existing `takeOwnership(make_unique(...))` pattern.

In addition, because allocation of `StorageLocation`s and `Value`s now happens
within the `DataflowAnalysisContext`, the `create<T>()` open up the possibility
of using `BumpPtrAllocator` to allocate these objects if it turns out this
helps performance.

Reviewed By: ymandel, xazax.hun, gribozavr2

Differential Revision: https://reviews.llvm.org/D147302

15 months ago[gn build] Port 39fc67b8af70
LLVM GN Syncbot [Tue, 4 Apr 2023 06:25:30 +0000 (06:25 +0000)]
[gn build] Port 39fc67b8af70

15 months ago[X86][NFC] Compress CD8_Scale from 7 bits to 3 bits
Shengchen Kan [Tue, 4 Apr 2023 05:00:14 +0000 (13:00 +0800)]
[X86][NFC] Compress CD8_Scale from 7 bits to 3 bits

Post: https://discourse.llvm.org/t/save-some-bits-in-tsflags-for-x86/69025

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D147500

15 months ago[llvm-exegesis] Factor out DisassemblerHelper from the Analysis class
Pavel Kosov [Tue, 4 Apr 2023 06:17:50 +0000 (09:17 +0300)]
[llvm-exegesis] Factor out DisassemblerHelper from the Analysis class

As part of preparing the reports, the Analysis class needs to print
machine instructions in a disassembled form. For this purpose, the class
has four fields (namely Context_, AsmInfo_, InstPrinter_ and Disasm_).
All the constructor of the Analysis class does is conditionally
initializing these four fields.

This commit factors out the logic for decoding machine code and printing
it in an assembler form into a separate DisassemblerHelper class.

~~

Huawei RRI, OS Lab

Reviewed By: courbet

Differential Revision: https://reviews.llvm.org/D147156

15 months ago[IR] Replace calls to ConstantFP::getNullValue with ConstantFP::getZero. NFC
Craig Topper [Tue, 4 Apr 2023 06:14:02 +0000 (23:14 -0700)]
[IR] Replace calls to ConstantFP::getNullValue with ConstantFP::getZero. NFC

There is no getNullValue in ConstantFP. Due to inheritance, we're calling
Constant::getNullValue which handles any type including FP.
Since we already know we want an FP constant we can use ConstantFP::getZero
which might be faster and is a more readable name for an FP zero.

15 months agoRevert "asan_memory_profile: Fix for deadlock in memory profiler code."
Vitaly Buka [Tue, 4 Apr 2023 03:42:16 +0000 (20:42 -0700)]
Revert "asan_memory_profile: Fix for deadlock in memory profiler code."

Deadlocks with internal symbolizer.
https://lab.llvm.org/buildbot/#/builders/37/builds/21181

This reverts commit 129394ff50ed28a0b85d742c8ae315758bb22582.

15 months ago[gn build] Port 50b2a113db19
LLVM GN Syncbot [Tue, 4 Apr 2023 05:30:21 +0000 (05:30 +0000)]
[gn build] Port 50b2a113db19

15 months agoMove "auto-init" instructions to the dominator of their users
serge-sans-paille [Wed, 2 Nov 2022 07:52:29 +0000 (08:52 +0100)]
Move "auto-init" instructions to the dominator of their users

As a result of -ftrivial-auto-var-init, clang generates instructions to
set alloca'd memory to a given pattern, right after the allocation site.
In some cases, this (somehow costly) operation could be delayed, leading
to conditional execution in some cases.

This is not an uncommon situation: it happens ~500 times on the cPython
code base, and much more on the LLVM codebase. The benefit greatly
varies on the execution path, but it should not regress on performance.

This is a recommit of cca01008cc31a891d0ec70aff2201b25d05d8f1b with
MemorySSA update fixes.

Differential Revision: https://reviews.llvm.org/D137707

15 months ago[X86] Remove stale comment. NFC
Craig Topper [Tue, 4 Apr 2023 05:13:36 +0000 (22:13 -0700)]
[X86] Remove stale comment. NFC

15 months agoPR58819: Correct linkage and mangling of lambdas in inline static member initializers
David Blaikie [Fri, 18 Nov 2022 00:26:40 +0000 (00:26 +0000)]
PR58819: Correct linkage and mangling of lambdas in inline static member initializers

https://llvm.org/pr58819 - clang is giving an externally visible lambda in a static data member internal linkage and the wrong linkage name.

Looks like we should be classifying this case the same as a non-static data member, so far as I can tell from the ABI docs and template examples (seems like the non-template inline-defined case should be the same).

This is a change in ABI, but not sure it qualifies as an ABI break as far as Apple and Sony are concerned - do you folks want this change? (it should fix the example in the bug where a static member in such a lambda ends up bifurcated, and I don't /think/ it'll break existing code since the symbol was previously internal anyway)

Looks like GCC has got this mangling slightly wrong (so we'd still end up with GCC+Clang bifurcation of the local static in the lambda, function address inequality, etc) in that they miss the variable name in the mangling in the non-template case. GCC bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107741

Differential Revision: https://reviews.llvm.org/D138247

15 months ago[AArch64][GlobalISel] Move the localizer to run before the legalizer, and always...
Amara Emerson [Mon, 3 Apr 2023 23:12:58 +0000 (16:12 -0700)]
[AArch64][GlobalISel] Move the localizer to run before the legalizer, and always localize globals.

Our strategy for localizing globals in the entry block breaks down when we have
large functions with high register pressure, using lots of globals. When this
happens, our heuristics say that globals with many uses should not be localized,
leading us to cause excessive spills and stack usage. These situations are also
exacerbated by LTO which tends to generate large functions.

For now, moving to a strategy that's simpler and more akin to SelectionDAG
fixes these issues and makes our codegen more similar. This has an overall
neutral effect on size on CTMark, while showing slight improvements with -Os -flto
on benchmarks. For low level firmware software though we see big improvements.

The reason this is neutral, and not an improvement, is because we give up the
gains from CSE'ing globals in cases where we low register pressure. I think
this can be addressed in future with some better heuristics.

Differential Revision: https://reviews.llvm.org/D147484

15 months ago[hwasan] Record allocation thread id in HeapAllocationRecord
Enna1 [Tue, 4 Apr 2023 03:24:46 +0000 (11:24 +0800)]
[hwasan] Record allocation thread id in HeapAllocationRecord

Extend HeapAllocationRecord to record allocation thread id, print thread id in memory allocation stack trace.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D147215

15 months ago[lsan] Fix Asan/Hwasan testing conditions
Vitaly Buka [Tue, 4 Apr 2023 03:01:24 +0000 (20:01 -0700)]
[lsan] Fix Asan/Hwasan testing conditions

15 months ago[sanitizer] Add new symbolizer symbol
Vitaly Buka [Tue, 4 Apr 2023 03:00:38 +0000 (20:00 -0700)]
[sanitizer] Add new symbolizer symbol

15 months ago[Clang][Sema] Fix comparison of constraint expressions
Alexander Shaposhnikov [Tue, 4 Apr 2023 02:22:08 +0000 (02:22 +0000)]
[Clang][Sema] Fix comparison of constraint expressions

This diff switches the approach to comparison of constraint expressions
to the new one based on template args substitution.
It continues the effort to fix our handling of out-of-line definitions
of constrained templates.

The associated GitHub issue: https://github.com/llvm/llvm-project/issues/61414

Test plan:
1/ ninja check-all
2/ bootstrapped Clang passes tests

Differential revision: https://reviews.llvm.org/D146178

15 months ago[X86][NFC] Combine HasVEX_W with hasREX_W to save 1 bit of TSFlags
Shengchen Kan [Mon, 3 Apr 2023 14:15:47 +0000 (22:15 +0800)]
[X86][NFC] Combine HasVEX_W with hasREX_W to save 1 bit of TSFlags

Post: https://discourse.llvm.org/t/save-some-bits-in-tsflags-for-x86/69025

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D147443

15 months ago[gn build] Port 038b7e6b761c
LLVM GN Syncbot [Tue, 4 Apr 2023 01:55:14 +0000 (01:55 +0000)]
[gn build] Port 038b7e6b761c

15 months ago[X86] Support AMX Complex instructions
Xiang1 Zhang [Fri, 31 Mar 2023 08:56:06 +0000 (16:56 +0800)]
[X86] Support AMX Complex instructions

Reviewed By: Wang Pengfei

Differential Revision: https://reviews.llvm.org/D147420

15 months ago[scudo] Temporarily disable the build of Timer.
Chia-hung Duan [Tue, 4 Apr 2023 00:58:40 +0000 (00:58 +0000)]
[scudo] Temporarily disable the build of Timer.

The placeholder was %llu while merged but it may cause problem on some
platforms that enable -Wformat. It was incorrect changed to %lu which
cause the test failure because string_utils doesn't support %lu. Will
bring this back after we fix the placeholder.

Differential Revision: https://reviews.llvm.org/D147490

15 months ago[SmallVector] Fix unit test to eliminate failure with -Werror
Nemanja Ivanovic [Tue, 4 Apr 2023 00:59:30 +0000 (19:59 -0500)]
[SmallVector] Fix unit test to eliminate failure with -Werror

Commit 475dd6f626ee2679578ed570e9fb78f7e957a36d added
a unit test that compares SmallVector::size() (a size_t
value) against 42 (an int value). That causes failures with
-Werror. This patch just adds the cast to silence the warning.

15 months agoUpdate __sanitizer_get_allocated_begin to return const void*
Thurston Dang [Tue, 4 Apr 2023 00:42:37 +0000 (00:42 +0000)]
Update __sanitizer_get_allocated_begin to return const void*

D147005 introduced __sanitizer_get_allocated_begin, with a return
value of void*. This involved a few naughty casts that dropped the
const. This patch adds back the const qualifier.

Differential Revision: https://reviews.llvm.org/D147489

15 months ago[NFC] Update CodeGen/*/nomerge.ll tests with utils/update_llc_test_checks.py.
Zequan Wu [Mon, 3 Apr 2023 17:01:14 +0000 (13:01 -0400)]
[NFC] Update CodeGen/*/nomerge.ll tests with utils/update_llc_test_checks.py.

Precommit this patch for better diff view on D146749.

Reviewed By: nemanjai

Differential Revision: https://reviews.llvm.org/D147454

15 months ago[compiler-rt] Fix -Wformat in timing.h (NFC)
Jie Fu [Mon, 3 Apr 2023 23:48:33 +0000 (07:48 +0800)]
[compiler-rt] Fix -Wformat in timing.h (NFC)

/home/jiefu/llvm-project/compiler-rt/lib/scudo/standalone/timing.h:181:41: error: format specifies type 'unsigned long long' but the argument has type 'u64' (aka 'unsigned long') [-Werror,-Wformat]
    Str.append("%14llu.%llu(ns) %-11s", Integral, Fraction, " ");
                ~~~~~~                  ^~~~~~~~
                %14lu
/home/jiefu/llvm-project/compiler-rt/lib/scudo/standalone/timing.h:181:51: error: format specifies type 'unsigned long long' but the argument has type 'u64' (aka 'unsigned long') [-Werror,-Wformat]
    Str.append("%14llu.%llu(ns) %-11s", Integral, Fraction, " ");
                       ~~~~                       ^~~~~~~~
                       %lu
/home/jiefu/llvm-project/compiler-rt/lib/scudo/standalone/timing.h:185:54: error: format specifies type 'unsigned long long' but the argument has type 'u64' (aka 'unsigned long') [-Werror,-Wformat]
    Str.append("%s (%llu)\n", Timers[HandleId].Name, Occurrence);
                    ~~~~                             ^~~~~~~~~~
                    %lu
3 errors generated.

15 months ago[PowerPC][NFC] Forgot to add requires asserts to ppc-TOC-stats.ll
Stefan Pintilie [Mon, 3 Apr 2023 23:45:27 +0000 (19:45 -0400)]
[PowerPC][NFC] Forgot to add requires asserts to ppc-TOC-stats.ll

When I sumbitted the original patch I forgot to add that:
GodeGen/PowerPC/ppc-TOC-stats.ll
requires asserts.
Added that now.

15 months ago[compiler-rt] Fix -Wcast-qual after D147005 (NFC)
Jie Fu [Mon, 3 Apr 2023 23:40:34 +0000 (07:40 +0800)]
[compiler-rt] Fix -Wcast-qual after D147005 (NFC)

/home/jiefu/llvm-project/compiler-rt/lib/lsan/lsan_allocator.cpp:161:18: error: cast from 'const void *' to 'void *' drops const qualifier [-Werror,-Wcast-qual]
  return (void *)beg;
                 ^
1 error generated.

15 months ago[mlir][sparse] Using SparseTensorType in SparsePackOpConverter
wren romano [Mon, 3 Apr 2023 19:55:59 +0000 (12:55 -0700)]
[mlir][sparse] Using SparseTensorType in SparsePackOpConverter

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D147465

15 months ago[RISCV] Add cost and lowering tests for "transpose" shuffle kinds
Philip Reames [Mon, 3 Apr 2023 22:48:44 +0000 (15:48 -0700)]
[RISCV] Add cost and lowering tests for "transpose" shuffle kinds

These just hit the generic two argument shuffle lowering right now, but that's not how their costed.

15 months agoReland "[scudo] Add a Timer class to assist performance measurement"
Chia-hung Duan [Fri, 24 Mar 2023 22:54:41 +0000 (22:54 +0000)]
Reland "[scudo] Add a Timer class to assist performance measurement"

This reverts commit 2e9bcadb7c8acaa8f6ec7d807e5666246923e468.

Differential Revision: https://reviews.llvm.org/D146772

15 months ago[SelectionDAG] Use MemVT for FoldingSetNodeID in SelectionDAG::getLoadVP.
Craig Topper [Mon, 3 Apr 2023 22:15:48 +0000 (15:15 -0700)]
[SelectionDAG] Use MemVT for FoldingSetNodeID in SelectionDAG::getLoadVP.

Return types and operands are put in the ID by AddNodeIDNode. I'm
pretty sure this was supposed to be the memory VT.

15 months ago[mlir][Vector] Enable masking for ops with index semantics
Diego Caballero [Mon, 3 Apr 2023 21:36:13 +0000 (21:36 +0000)]
[mlir][Vector] Enable masking for ops with index semantics

Masking was already supported for linalg.index and n-D extract but
disabled while waiting for some n-D extract vectorization patches to
land. This patch is just enabling masking for them and adding a couple
of tests.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D147359

15 months ago[mlir][Vector][NFC] Small vector masking clean-up
Diego Caballero [Mon, 3 Apr 2023 21:34:38 +0000 (21:34 +0000)]
[mlir][Vector][NFC] Small vector masking clean-up

We stored static (int) and dynamic (Value) iteration space dims separately
and then merged them by creating constant ops for the static ones. This
merge happened multiple times during vectorization. This PR changes that
to perform the merge once and store in the state instead of the dynamic
values in isolation.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D147351

15 months ago[Clang] Fix failing test on windows and add TODO for gpu headers
Joseph Huber [Mon, 3 Apr 2023 21:53:41 +0000 (16:53 -0500)]
[Clang] Fix failing test on windows and add TODO for gpu headers

Summary:
This test failed because of the path separators on Windows. Also this
was a good excuse to add an extra TODO that @tra wanted.

15 months agoAdd __sanitizer_get_allocated_begin API and implementations
Thurston Dang [Mon, 3 Apr 2023 21:14:40 +0000 (21:14 +0000)]
Add __sanitizer_get_allocated_begin API and implementations

This function will return the start of the allocation, if given a pointer that lies within an allocation. Otherwise, it returns NULL.

It will be useful for detecting dynamic TLS allocations in glibc >=2.25, which
uses malloc (see https://github.com/google/sanitizers/issues/1409#issuecomment-1214244142).

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D147005

15 months ago[SelectionDAG] Move variable declaration to its first assignment. NFC
Craig Topper [Mon, 3 Apr 2023 21:02:26 +0000 (14:02 -0700)]
[SelectionDAG] Move variable declaration to its first assignment. NFC

We declared this variable and assigned it to true, but then overwrote
it before its first use.

15 months ago[SelectionDAGBuilder] Use SmallVectorImpl& for function arguments. NFC
Craig Topper [Mon, 3 Apr 2023 20:59:55 +0000 (13:59 -0700)]
[SelectionDAGBuilder] Use SmallVectorImpl& for function arguments. NFC

Make the reference const since we aren't modifying the vectors.

15 months ago[RISCV][docs] Document which revision of the specification we implement
Philip Reames [Mon, 3 Apr 2023 20:53:22 +0000 (13:53 -0700)]
[RISCV][docs] Document which revision of the specification we implement

This is intended to document the decision made in recent discussion, and ratified at the last risc-v sync up call two weeks ago.

The wording here turned out to be a bit tricky. I ended up using the word "revision" as the specification internally defines several versioning schemes, and RVI assigns particular meaning to the words "specification version" that I really didn't want to get into. If anyone has suggestions on how to improve this, please don't hesitate to chime in.

Differential Revision: https://reviews.llvm.org/D147183

15 months agoUsing global variable in xnu kernel, set # of addressable bits
Jason Molenda [Mon, 3 Apr 2023 20:49:51 +0000 (13:49 -0700)]
Using global variable in xnu kernel, set # of addressable bits

The kernel has a global variable with the TCR_EL1.T1SZ value,
from which was can calculate the number of addressable bits.
Find that symbol in DynamicLoaderDarwinKernel and set the bits
to that value for this Process.

Differential Revision: https://reviews.llvm.org/D147462
rdar://107445318

15 months ago[LLDB] Ensure LLDB symbols are exported in LLDB_EXPORT_ALL_SYMBOLS is provided.
walter erquinigo [Mon, 3 Apr 2023 16:32:53 +0000 (11:32 -0500)]
[LLDB] Ensure LLDB symbols are exported in LLDB_EXPORT_ALL_SYMBOLS is provided.

If we want to export all lldb symbols (i.e LLDB_EXPORT_ALL_SYMBOLS=ON), we need to use default visibility for all LLDB libraries even if a global CMAKE_CXX_VISIBILITY_PRESET=hidden is present. In fact, there are cases in which a global llvm configuration wants CMAKE_CXX_VISIBILITY_PRESET as hidden but simultaneously LLDB_EXPORT_ALL_SYMBOLS=ON is also needed to be able to develop out-of-tree lldb plugins.

Differential Revision: https://reviews.llvm.org/D147453

15 months ago[OpenMP][5.1] Fix parallel masked is ignored #59939
Rafael A. Herrera Guaitero [Mon, 3 Apr 2023 20:29:16 +0000 (20:29 +0000)]
[OpenMP][5.1] Fix parallel masked is ignored #59939

Code generation support for 'parallel masked' directive.

The `EmitOMPParallelMaskedDirective` was implemented.
In addition, the appropiate device functions were added.

Fix #59939.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D143527

15 months ago[SmallVector] Add an explicit SmallVector(size_t Size) constructor.
Craig Topper [Mon, 3 Apr 2023 20:22:36 +0000 (13:22 -0700)]
[SmallVector] Add an explicit SmallVector(size_t Size) constructor.

Previously we used the SmallVector(size_t Size, const T& Value) constructor
with a default constructed Value. That will copy construct every element
in the vector, but not all types can be copy constructed.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D147426

15 months ago[lldb][NFC] Remove outdated TODO message
Alex Langford [Mon, 3 Apr 2023 20:08:52 +0000 (13:08 -0700)]
[lldb][NFC] Remove outdated TODO message

This should have been removed in a52054cfa29d

15 months agoUpdate Clang 16 statuses
Aaron Ballman [Mon, 3 Apr 2023 19:31:20 +0000 (15:31 -0400)]
Update Clang 16 statuses

Updates the C status tracking page for the recent release.

15 months ago[C2x] Implement support for empty brace initialization (WG14 N2900 and WG14 N3011)
Aaron Ballman [Mon, 3 Apr 2023 19:19:15 +0000 (15:19 -0400)]
[C2x] Implement support for empty brace initialization (WG14 N2900 and WG14 N3011)

This implements support for allowing {} to consistently zero initialize
objects. We already supported most of this work as a GNU extension, but
the C2x feature goes beyond what the GNU extension allowed.

The changes in this patch are:

* Removed the -Wgnu-empty-initializer warning group. The extension is
  now a C2x extension warning instead. Note that use of
  `-Wno-gnu-empty-initializer seems` to be quite low in the wild
(https://sourcegraph.com/search?q=context%3Aglobal+-file%3A.*test.*+%22-Wno-gnu-empty-initializer%22&patternType=standard&sm=1&groupBy=repo
  which currently only gives 8 hits total), so this is not expected to
  be an overly disruptive change. But I'm adding the clang vendors
  review group just in case this expectation is wrong.
* Reworded the diagnostic wording to be about a C2x extension, added a
  pre-C2x compat warning.
* Allow {} to zero initialize a VLA

This functionality is exposed as an extension in all older C modes
(same as the GNU extension was), but does *not* allow the extension for
VLA initialization in C++ due to concern about handling non-trivially
constructible types.

Differential Revision: https://reviews.llvm.org/D147349

15 months ago[mlir] Improve FieldParser list container detection
rkayaith [Tue, 28 Mar 2023 23:00:40 +0000 (19:00 -0400)]
[mlir] Improve FieldParser list container detection

The current detection logic will fail for containers with an overloaded
`push_back` member. This causes issues with types like `std::vector` and
`SmallVector<SomeNonTriviallyCopyableT>`, which have both
`push_back(const T&)` and `push_back(T&&)`.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D147101

15 months ago[RuntimeDyld] RuntimeDyldELF: Clear GOTOffsetMap when resetting GOT section.
Graham Markall [Mon, 3 Apr 2023 18:15:36 +0000 (11:15 -0700)]
[RuntimeDyld] RuntimeDyldELF: Clear GOTOffsetMap when resetting GOT section.

When the GOT section ID is reset, the GOTOffsetMap must also be cleared,
otherwise spurious matches can be located when handling GOT relocations
in subsequently-linked objects.

Fixes Issue #61402 - see https://github.com/llvm/llvm-project/issues/61402.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D146938

15 months ago[PowerPC] Add statistics to show the number of entries in the TOC.
Stefan Pintilie [Fri, 31 Mar 2023 18:42:58 +0000 (14:42 -0400)]
[PowerPC] Add statistics to show the number of entries in the TOC.

On Power PC some data is stored in the TOC. This pass adds statistics
to show how many entries are emitted to the TOC and what types of
entries those are.

Reviewed By: amyk

Differential Revision: https://reviews.llvm.org/D146325

15 months ago[JITLink][ELF] Only make sections with SHF_WRITE writable
Job Noorman [Mon, 3 Apr 2023 18:18:19 +0000 (20:18 +0200)]
[JITLink][ELF] Only make sections with SHF_WRITE writable

All non-executable sections used to be mapped as RW- causing read-only
sections such as .rodata to be writable.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D147442

15 months ago[libc] Add strchrnul implementation
Caslyn Tonelli [Mon, 3 Apr 2023 18:04:48 +0000 (11:04 -0700)]
[libc] Add strchrnul implementation

Introduce strchrnul implementation and unit tests.

Submitting on behalf of Caslyn@

Differential Revision: https://reviews.llvm.org/D147346

15 months agoRevert "[mlir] Introduce IRDL dialect"
Mehdi Amini [Mon, 3 Apr 2023 18:05:37 +0000 (11:05 -0700)]
Revert "[mlir] Introduce IRDL dialect"

This reverts commit 31229d48bbfd394b64179d9be94f74ab70c84630.

The build is broken with shared library mode.

15 months ago[LV] Remmove unused default argument to isLegalGatherOrScatter [nfc]
Philip Reames [Mon, 3 Apr 2023 18:03:14 +0000 (11:03 -0700)]
[LV] Remmove unused default argument to isLegalGatherOrScatter [nfc]

15 months ago[Clang] Implicitly include LLVM libc headers for the GPU
Joseph Huber [Mon, 27 Mar 2023 15:01:12 +0000 (10:01 -0500)]
[Clang] Implicitly include LLVM libc headers for the GPU

There is currently work to support basic `libc` functionality on the
GPU. Some basic information about the projects can be found at
https://libc.llvm.org/gpu_mode.html. Typically, including the system
headers on the GPU will result in an error. For this reason the LLVM
`libc` project will generate its own headers that can be used with the
GPU.

The problem is that these headers will use the same name as the system headers.
For that reason, D146970 places it in the `llvm-libc` subfolder. In order to
still pick these files up, this patch adds changes in clang to default to
searching this directory when targeting the GPU. This lets offloading languages
such as OpenMP use the system `string.h` when compiling for the host and then
the LLVM libc `string.h` when targeting the GPU.

Depends on D146970

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D146973

15 months ago[mlir][MemRef] Add pattern to resolve strided metadata of `memref.get_global` operation.
Mahesh Ravishankar [Mon, 3 Apr 2023 17:31:36 +0000 (17:31 +0000)]
[mlir][MemRef] Add pattern to resolve strided metadata of `memref.get_global` operation.

This changes adds patterns to resolve the base pointer, offset, sizes
and strides of the result of a `memref.get_global` operation. Since
the operation can only result in static shaped memrefs, current
resolution kicks in only for non-zero offsets, and identity strides.

Also

- Add a separate `populateResolveExtractStridedMetadata` method that
  adds just the pattern to resolve `<memref op>` ->
  `memref.extract_strided_metadata` operations.
- Refactor the `SubviewFolder` pattern to allow resolving
  `memref.subview` -> `memref.extract_strided_metadata`.

This allows using these patterns for cases where there are already
existing `memref.extract_strided_metadata` operations.

Reviewed By: qcolombet

Differential Revision: https://reviews.llvm.org/D147393

15 months ago[mlir] Introduce IRDL dialect
Mathieu Fehr [Mon, 3 Apr 2023 16:42:53 +0000 (09:42 -0700)]
[mlir] Introduce IRDL dialect

This patch introduces the IRDL dialect, which allow users to represent
dynamic dialect definitions as an MLIR program.

The IRDL dialect defines operations, attributes, and types, using
attribute constraints. For example:

```
module {
  irdl.dialect @cmath {
    irdl.type @complex {
      %0 = irdl.is f32
      %1 = irdl.is f64
      %2 = irdl.any_of(%0, %1)
      irdl.parameters(%2)
    }

    irdl.operation @norm {
      %0 = irdl.any
      %1 = irdl.parametric @complex<%0>
      irdl.operands(%1)
      irdl.results(%0)
    }
}
```

This program will define a new `cmath.complex` type, which expects a single
parameter, which is either an `f32` or an `f64`. It also defines an
`cmath.norm` operation, which expects a single `cmath.complex` type as operand,
and returns a value of the underlying type. Note that like PDL (which IRDL is
heavily inspired from), both uses of `%0` are expected to be of the same attribute.

IRDL handles attributes and types with the same operations, and does this by always
wrapping types in a `TypeAttr`. This is to simplify the language.

Depends on D144690

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D144692

15 months ago[clang-tidy] ignore NRVO const variables in performance-no-automatic-move.
Logan Gnanapragasam [Mon, 3 Apr 2023 16:37:56 +0000 (16:37 +0000)]
[clang-tidy] ignore NRVO const variables in performance-no-automatic-move.

In the following code:

```cc
struct Obj {
  Obj();
  Obj(const Obj &);
  Obj(Obj &&);
  virtual ~Obj();
};

Obj ConstNrvo() {
  const Obj obj;
  return obj;
}
```

performance-no-automatic-move warns about the constness of `obj`. However, NRVO
is applied to `obj`, so the `const` should have no effect on performance.

This change modifies the matcher to exclude NRVO variables.

#clang-tidy

Reviewed By: courbet, PiotrZSL

Differential Revision: https://reviews.llvm.org/D147419

15 months agoRevert "[X86] Simplify some `pternlog` intrinsics"
Noah Goldstein [Mon, 3 Apr 2023 16:59:32 +0000 (11:59 -0500)]
Revert "[X86] Simplify some `pternlog` intrinsics"

This reverts commit cacc0033b12b2e6400eedd6e449644e87ec3099b.

15 months ago[libc] Free the GPU memory allocated in the device loaders
Joseph Huber [Mon, 3 Apr 2023 16:54:48 +0000 (11:54 -0500)]
[libc] Free the GPU memory allocated in the device loaders

Summary:
This part was ignored and we just hoped that shutting down the runtime
freed these correctly. But it's best to be specific and free the memory
we've allocated.