platform/upstream/llvm.git
17 months ago[GlobalIsel][X86] Regenerate G_MUL scalar legalization tests
Simon Pilgrim [Fri, 2 Jun 2023 19:12:52 +0000 (20:12 +0100)]
[GlobalIsel][X86] Regenerate G_MUL scalar legalization tests

Add i8 test coverage

17 months agoRevert "[CMake] Ensure `CLANG_RESOURCE_DIR` is respected."
Martin Storsjö [Sat, 3 Jun 2023 08:44:06 +0000 (11:44 +0300)]
Revert "[CMake] Ensure `CLANG_RESOURCE_DIR` is respected."

This reverts commit 39aa0f5c434b463520ac39a8dbe933ee8c4c5ea7.

This is missing the new GetClangResourceDir.cmake that is being included,
so all clang builds are broken.

17 months ago[llvm] Strip stabs symbols in Mach-O when stripping debug info
Mike Hommey [Sat, 3 Jun 2023 07:31:42 +0000 (09:31 +0200)]
[llvm] Strip stabs symbols in Mach-O when stripping debug info

17 months ago[InstSimplify] Simplify select i1 ConstExpr, i1 true, i1 false to ConstExpr
luxufan [Thu, 1 Jun 2023 05:54:16 +0000 (13:54 +0800)]
[InstSimplify] Simplify select i1 ConstExpr, i1 true, i1 false to ConstExpr

`select i1 non-const, i1 true, i1 false` has been optimized to
`non-const`. There is no reason that we can not optimize `select i1
ConstExpr, i1 true, i1 false` to `ConstExpr`.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D151631

17 months ago[CMake] Ensure `CLANG_RESOURCE_DIR` is respected.
paperchalice [Sat, 3 Jun 2023 06:29:44 +0000 (23:29 -0700)]
[CMake] Ensure `CLANG_RESOURCE_DIR` is respected.

17 months ago[TableGen][GlobalISel] Account for HwMode in RegisterBank register sizes
Nitin John Raj [Sat, 3 Jun 2023 05:08:05 +0000 (22:08 -0700)]
[TableGen][GlobalISel] Account for HwMode in RegisterBank register sizes

This patch adds logic for determining RegisterBank size to RegisterBankInfo, which allows accounting for the HwMode of the target. Individual RegisterBanks cannot be constructed with HwMode information as construction is generated by TableGen, but a RegisterBankInfo subclass can provide the HwMode as a constructor argument. The HwMode is used to select the appropriate RegisterBank size from an array relating sizes to RegisterBanks.

Targets simply need to provide the HwMode argument to the <target>GenRegisterBankInfo constructor. The RISC-V RegisterBankInfo constructor has been updated accordingly (plus an unused argument removed).

Reviewed By: simoncook, craig.topper

Differential Revision: https://reviews.llvm.org/D76007

17 months ago[AMDGPU] Don't flush vmcnt for loops with use/def pairs
Austin Kerbow [Mon, 22 May 2023 17:32:09 +0000 (10:32 -0700)]
[AMDGPU] Don't flush vmcnt for loops with use/def pairs

Conditions for hoisting vmcnt with flat instructions should be similar to VMEM.
If there are use/def pairs in a loop body we cannot guarantee that hosting the
waitcnt will be profitable. Better heuristics are needed to analyse whether
gains from avoiding waitcnt in loop bodys outweighs waiting for loads in the
preheader.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D151126

17 months ago[CodeGen] Use DenseMapBase::lookup (NFC)
Kazu Hirata [Sat, 3 Jun 2023 04:05:13 +0000 (21:05 -0700)]
[CodeGen] Use DenseMapBase::lookup (NFC)

Note that DenseMapBase::lookup and Reg2MIMap::get do exactly the same
thing.

17 months ago[RISCV] Avoid illegal DAG combination for strict-fp nodes.
Yeting Kuo [Fri, 2 Jun 2023 12:38:03 +0000 (20:38 +0800)]
[RISCV] Avoid illegal DAG combination for strict-fp nodes.

Those combines may change the exception behavior and rounding behavior.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D151992

17 months ago[mlir][sparse] fix crash when using sparse_tensor::UnaryOp and ReduceOp.
Peiming Liu [Sat, 3 Jun 2023 00:49:37 +0000 (00:49 +0000)]
[mlir][sparse] fix crash when using sparse_tensor::UnaryOp and ReduceOp.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D152048

17 months ago[libc][NFC] Add LIBC_INLINE and attribute.h header includes to targets' FMA.h.
Tue Ly [Fri, 2 Jun 2023 20:13:19 +0000 (16:13 -0400)]
[libc][NFC] Add LIBC_INLINE and attribute.h header includes to targets' FMA.h.

Targets' FMA.h headers are missing LIBC_INLINE and attributes.h header.

Reviewed By: brooksmoses

Differential Revision: https://reviews.llvm.org/D152024

17 months ago[mlir][sparse] fixed bug with unary op, dense output
Aart Bik [Fri, 2 Jun 2023 23:41:49 +0000 (16:41 -0700)]
[mlir][sparse] fixed bug with unary op, dense output

Note that by sparse compiler convention, dense output
is zerod out when not set, so complement results in
zeros where elements were present.

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D152046

17 months ago[scudo] Add 'inline' to static secondary.h function
Caslyn Tonelli [Fri, 2 Jun 2023 22:13:50 +0000 (22:13 +0000)]
[scudo] Add 'inline' to static secondary.h function

This patches an error flaged by Fuchsia builds e.g.
https://ci.chromium.org/ui/p/turquoise/builders/global.try/core.x64-asan/b8779376650819379137/overview)

```
build failed:

[87176/332302](525) CXX user.libc_x64-asan-ubsan/obj/zircon/system/ulib/c/scudo/gwp-asan-info.gwp_asan_info.cc.o
FAILED: user.libc_x64-asan-ubsan/obj/zircon/system/ulib/c/scudo/gwp-asan-info.gwp_asan_info.cc.o
../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/rbe/cxx_remote_wrapper.py --exec_strategy=remote_local_fallback --  ../../prebuilt/third_party/clang/linux-x64/bin/clang++ -MD -MF user.libc_x64-asan-ubsan/obj/zircon/system/ulib/c/scudo/gwp-asan-info.gwp_asan_info.cc.o.d -o user.libc_x64-asan-ubsan/obj/zircon/system/ulib/c/scudo/gwp-asan-info.gwp_asan_info.cc.o -D_LIBCPP...
In file included from ../../zircon/system/ulib/c/scudo/gwp_asan_info.cc:7:
In file included from ../../third_party/scudo/src/allocator_config.h:12:
In file included from ../../third_party/scudo/src/combined.h:22:
../../third_party/scudo/src/secondary.h:67:13: error: 'static' function 'unmap' declared in header file should be declared 'static inline' [-Werror,-Wunneeded-internal-declaration]
static void unmap(LargeBlock::Header *H) {
            ^
1 error generated.
```

Differential Revision: https://reviews.llvm.org/D152038

17 months ago[gn] port 6b3ae49d3243
Nico Weber [Fri, 2 Jun 2023 23:37:10 +0000 (19:37 -0400)]
[gn] port 6b3ae49d3243

17 months ago[InlineCost] Check for conflicting target attributes early
Kazu Hirata [Fri, 2 Jun 2023 23:00:47 +0000 (16:00 -0700)]
[InlineCost] Check for conflicting target attributes early

When we inline a callee into a caller, the compiler needs to make sure
that the caller supports a superset of instruction sets that the
callee is allowed to use.  Normally, we check for the compatibility of
target features via functionsHaveCompatibleAttributes, but that
happens after we decide to honor call site attribute
Attribute::AlwaysInline.  If the caller contains a call marked with
Attribute::AlwaysInline, which can happen with
__attribute__((flatten)) placed on the caller, the caller could end up
with code that cannot be lowered to assembly code.

This patch fixes the problem by checking the target feature
compatibility before we honor Attribute::AlwaysInline.

Fixes https://github.com/llvm/llvm-project/issues/62664

Differential Revision: https://reviews.llvm.org/D150396

17 months agoclang: Update tests after InstSimplify change
Matt Arsenault [Fri, 2 Jun 2023 22:44:18 +0000 (18:44 -0400)]
clang: Update tests after InstSimplify change

Update tests after 1536e299e63d7788f38117b0212ca50eb76d7a3b

17 months ago[ThinLTO] Fix internalization decisions for weak/linkonce ODR
Teresa Johnson [Thu, 1 Jun 2023 18:07:05 +0000 (11:07 -0700)]
[ThinLTO] Fix internalization decisions for weak/linkonce ODR

This fixes a runtime error that occurred due to incorrect
internalization of linkonce_odr functions where function pointer
equality was broken. This was hit because the prevailing copy was in a
native object, so the IR copies were not exported, and the existing code
internalized all of the IR copies. It could be fixed by guarding this
internalization on whether the defs are (local_)unnamed_addr, meaning
that their address is not significant (which we have in the summary
currently for linkonce_odr via the CanAutoHide flag). Or we can
propagate reference attributes as we do when determining whether a
global variable is read or write-only (reference edges are annotated
with whether they are read-only, write-only, or neither, and taking the
address of a function would result in a reference edge to the function
that is not read or write-only).

However, this exposed a larger issue with the internalization handling.
Looking at test cases, it appears the intent is to internalize when
there is a single definition of a linkonce/weak ODR symbol (that isn't
exported). This makes sense in the case of functions, because the
inliner can apply its last call to static heuristic when appropriate. In
the case where there is no prevailing copy in IR, internalizing all of
the IR copies of a linkonce_odr, even if legal, just increases binary
size. In that case it is better to fall back to the normal handling of
converting all non-prevailing copies to available_externally so that
they are eliminated after inlining.

In the case of variables, the existing code was attempting to
internalize the non-exported linkonce/weak ODR variables if they were
read or write-only. While this is legal (we propagate reference
attributes to determine this information), we don't even need to
internalize these here as there is later separate handling that
internalizes read and write-only variables when we process the module at
the start of the ThinLTO backend (processGlobalForThinLTO). Instead, we
can also internalize any non-exported variable when there is only one
(IR) definition, which is prevailing. And in that case, we don't need to
require that it is read or write-only, since we are guaranteed that all
uses must use that single definition.

In the new LTO API, if there are multiple defs of a linkonce or weak ODR
it will be marked exported, but it isn't clear that this will always be
true for the legacy LTO API. Therefore, require that there is only a
single (non-local) def, and that it is prevailing.

The test cases changes are both to reflect the change in the handling of
linkonce_odr IR copies where the prevailing def is not in IR (the main
correctness bug fix here), and to reflect the more aggressive
internalization of variables when there is only a single def, it is in
IR, and not exported.

I've also added some additional testing via the new LTO API.

Differential Revision: https://reviews.llvm.org/D151965

17 months agoProperly restore SP tag on exceptions
Florian Mayer [Fri, 2 Jun 2023 21:11:38 +0000 (14:11 -0700)]
Properly restore SP tag on exceptions

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D152036

17 months ago[Damangle] convert dlangDemangle to use std::string_view
Nick Desaulniers [Fri, 2 Jun 2023 22:16:10 +0000 (15:16 -0700)]
[Damangle] convert dlangDemangle to use std::string_view

I was doing this API conversion to use std::string_view top-down in
D149104, but this exposed issues in individual demanglers that needed to
get fixed first. There's no issue with the conversion for the D language
demangler, so convert it.

I have a more aggressive refactoring of the entire D language demangler
to use std::string_view more extensively, but the interface with
llvm::nonMicrosoftDemangle is the more interesting one.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D151003

17 months agoInstSimplify: Remove null parent checks
Matt Arsenault [Wed, 24 May 2023 10:42:28 +0000 (11:42 +0100)]
InstSimplify: Remove null parent checks

The feature to operate on incomplete IR was barely used
an poorly tested.

17 months agoInstSimplify: Require instruction be parented
Matt Arsenault [Tue, 23 May 2023 09:28:57 +0000 (10:28 +0100)]
InstSimplify: Require instruction be parented

Unlike every other analysis and transform, simplifyInstruction
permitted operating on instructions which are not inserted
into a function. This created an edge case no other code needs
to really worry about, and limited transforms in cases that
can make use of the context function. Only the inliner and a handful
of other utilities were making use of this, so just fix up these
edge cases. Results in some IR ordering differences since
cloned blocks are inserted eagerly now. Plus some additional
simplifications trigger (e.g. some add 0s now folded out that
previously didn't).

17 months ago[Damangle] convert rustDemangle to use std::string_view
Nick Desaulniers [Fri, 2 Jun 2023 22:08:14 +0000 (15:08 -0700)]
[Damangle] convert rustDemangle to use std::string_view

I was doing this API conversion to use std::string_view top-down in
D149104, but this exposed issues in individual demanglers that needed to
get fixed first. There's no issue with the conversion for the Rust
demangler, so convert it first.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D149784

17 months ago[Demangle] convert itaniumDemangle and nonMicrosoftDemangle to use std::string_view
Nick Desaulniers [Fri, 2 Jun 2023 21:53:46 +0000 (14:53 -0700)]
[Demangle] convert itaniumDemangle and nonMicrosoftDemangle to use std::string_view

D149104 converted llvm::demangle to use std::string_view. Enabling
"expensive checks" (via -DLLVM_ENABLE_EXPENSIVE_CHECKS=ON) causes
lld/test/wasm/why-extract.s to fail. The reason for this is obscure:

Reason #10007 why std::string_view is dangerous:
Consider the following pattern:

  std::string_view s = ...;
  const char *c = s.data();
  std::strlen(c);

Is c a NUL-terminated C style string? It depends; but if it's not then
it's not safe to call std::strlen on the std::string_view::data().
std::string_view::length() should be used instead.

Fixing this fixes the one lone test that caught this.

microsoftDemangle, rustDemangle, and dlangDemangle should get this same
treatment, too. I will do that next.

Reviewed By: MaskRay, efriedma

Differential Revision: https://reviews.llvm.org/D149675

17 months ago[mlir][sparse][gpu] make computeType mandatory
Kun Wu [Fri, 2 Jun 2023 18:29:49 +0000 (18:29 +0000)]
[mlir][sparse][gpu] make computeType mandatory

Differential Revision: https://reviews.llvm.org/D152018

17 months ago[flang][hlfir] Added allocatable/pointer support in hlfir::convertToValue.
Slava Zakharin [Fri, 2 Jun 2023 21:20:21 +0000 (14:20 -0700)]
[flang][hlfir] Added allocatable/pointer support in hlfir::convertToValue.

The code is used, for example, when passing arguments to IO or intrinsic
calls as value. The allocatable/pointer boxes must be dereferenced,
and trivial values have to be loaded. Character and derived values
have to stay boxed.
I am not sure what to do for the array cases, and I have not seen
any test triggering it, so I leave it as a TODO.

Reviewed By: tblah, clementval

Differential Revision: https://reviews.llvm.org/D151925

17 months ago[NFC][lsan] Use hash table to track root regions
Vitaly Buka [Wed, 31 May 2023 07:08:02 +0000 (00:08 -0700)]
[NFC][lsan] Use hash table to track root regions

This avoid O(N) in __lsan_unregister_root_region.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D151784

17 months ago[NFC][sanitizer] Avoid O(N^2) algorithm
Vitaly Buka [Wed, 31 May 2023 05:58:23 +0000 (22:58 -0700)]
[NFC][sanitizer] Avoid O(N^2) algorithm

Usually root_regions size is small so unlikey
this change will provide a noticable difference.

However it's easy to make sure that even with
large number of root_regions it works reasonably
fast.

Differential Revision: https://reviews.llvm.org/D151781

17 months ago[NFC][sanitizer] Delegate ArrayRef constructors
Vitaly Buka [Fri, 2 Jun 2023 21:25:11 +0000 (14:25 -0700)]
[NFC][sanitizer] Delegate ArrayRef constructors

17 months ago[CUDA] Update Kepler(sm_3*) support info.
Artem Belevich [Fri, 2 Jun 2023 20:30:47 +0000 (13:30 -0700)]
[CUDA] Update Kepler(sm_3*) support info.

sm_30 and sm_32 were removed in cuda-11.0
sm_35 and sm_37 were removed in cuda-12.0

Differential Revision: https://reviews.llvm.org/D152027

17 months agoFully qualify MLIR type in TableGen emitter
Mehdi Amini [Fri, 2 Jun 2023 21:07:56 +0000 (14:07 -0700)]
Fully qualify MLIR type in TableGen emitter

This is important for users that don't enclose their dialect within the MLIR
namespace.

17 months ago[libc] Disable the string_to_float test on NVPTX
Joseph Huber [Fri, 2 Jun 2023 20:56:00 +0000 (15:56 -0500)]
[libc] Disable the string_to_float test on NVPTX

This test began failing after recent changes. Disable it for now.

Differential Revision: https://reviews.llvm.org/D152032

17 months ago[sanitizer] Calculate Range sets intersection
Vitaly Buka [Wed, 31 May 2023 05:47:33 +0000 (22:47 -0700)]
[sanitizer] Calculate Range sets intersection

Will be used to handle Root Regions in LSAN D151781.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D151779

17 months ago[TableGen] Allow bit fields in SearchableTables.
Stanislav Mekhanoshin [Tue, 30 May 2023 21:41:29 +0000 (14:41 -0700)]
[TableGen] Allow bit fields in SearchableTables.

Differential Revision: https://reviews.llvm.org/D151756

17 months ago[NFC][CLANG] Fix Static Code Analyzer Concerns with dereference null return value...
Manna, Soumi [Fri, 2 Jun 2023 20:34:32 +0000 (13:34 -0700)]
[NFC][CLANG] Fix Static Code Analyzer Concerns with dereference null return value in applyObjCTypeArgs()

This patch uses castAs instead of getAs to resolve dereference issue with nullptr boundObjC when calling
canAssignObjCInterfaces() or isObjCIdType() in applyObjCTypeArgs() since getAs returns nullptr.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D151964

17 months ago[NFC][CLANG] Fix bug with dereference null return value in GetFunctionTypeForVTable()
Manna, Soumi [Fri, 2 Jun 2023 20:23:51 +0000 (13:23 -0700)]
[NFC][CLANG] Fix bug with dereference null return value in GetFunctionTypeForVTable()

This patch uses castAs instead of getAs which will assert if the type doesn't match in clang::CodeGen::CodeGenTypes::GetFunctionTypeForVTable(clang::GlobalDecl).

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D151957

17 months ago[RISCV] Remove optype parameter from most V instruction multiclasses. NFC
Craig Topper [Fri, 2 Jun 2023 20:24:36 +0000 (13:24 -0700)]
[RISCV] Remove optype parameter from most V instruction multiclasses. NFC

For the most part we already had the classes split and instantiated
in a way that the type is always the same for all instantiations of
the class.

17 months ago[UBSan] Consider zero input to __builtin_clz/ctz to be undefined independent of the...
Craig Topper [Fri, 2 Jun 2023 19:51:42 +0000 (12:51 -0700)]
[UBSan] Consider zero input to __builtin_clz/ctz to be undefined independent of the target.

Previously we checked isCLZForZeroUndef and only added UBSan checks
if it returned true.

The builtin should be considered undefined for 0 regardless of
the target so that code using it is portable. The isCLZForZeroUndef
was only intended to disable optimizations in the middle end and
backend.

See https://discourse.llvm.org/t/should-ubsan-detect-0-input-to-builtin-clz-ctz-regardless-of-target/71060

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D152023

17 months ago[KCFI] Emit debugtrap to make indirect call checks recoverable
Sami Tolvanen [Thu, 1 Jun 2023 23:06:12 +0000 (16:06 -0700)]
[KCFI] Emit debugtrap to make indirect call checks recoverable

KCFI traps should always be recoverable, but as Intrinsic::trap
is marked noreturn, it's not possible to continue execution after
handling the trap as the compiler is free to assume we never
return. Switch to debugtrap instead to ensure we have the option
to resume execution after the trap.

17 months ago[Flang][OpenMP][Parser] Resolve Declare Target Directive Symbols
Andrew Gozillon [Fri, 2 Jun 2023 19:18:50 +0000 (14:18 -0500)]
[Flang][OpenMP][Parser] Resolve Declare Target Directive Symbols

Currently symbols are not resolved for declare target
after they've been modified by prior passes. This can
lead to missing or incorrect symbols in subsequent
compiler phases when declare target is used with
more complex types e.g. common block.

This patch should allow these symbols to be
resolved appropriately.

Reviewers: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D151993

17 months ago[libc] Implement 'errno' on the GPU as a global integer internally
Joseph Huber [Fri, 2 Jun 2023 17:48:05 +0000 (12:48 -0500)]
[libc] Implement 'errno' on the GPU as a global integer internally

The C standard asserts that the `errno` value is an l-value thread local
integer. We cannot provide a generic thread local integer on the GPU
currently without some workarounds. Previously, we worked around this by
implementing the `errno` value as a special consumer class that made all
the writes disappear. However, this is problematic for internal tests.
Currently there are build failures because of this handling and it's
only likely to cause more problems the more we do this.

This patch instead makes the internal target used for testing export the
`errno` value as a simple global integer. This allows us to use and test
the `errno` interface correctly assuming we run with a single thread.
Because this is only used for the non-exported target we still do not
provide this feature in the version that users will use so we do not
need to worrk about it being incorrect in general.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D152015

17 months agoConstexprify all eligible functions in MCRegister and Register
Krzysztof Parzyszek [Thu, 1 Jun 2023 18:57:55 +0000 (11:57 -0700)]
Constexprify all eligible functions in MCRegister and Register

17 months ago[test] Add -Wno-msvc-not-found to fix linker-opts.c on *-windows-msvc
Fangrui Song [Fri, 2 Jun 2023 18:59:22 +0000 (11:59 -0700)]
[test] Add -Wno-msvc-not-found to fix linker-opts.c on *-windows-msvc

17 months ago[RISCV] Simplify vector instruction multiclasses. NFC
Craig Topper [Fri, 2 Jun 2023 18:27:31 +0000 (11:27 -0700)]
[RISCV] Simplify vector instruction multiclasses. NFC

Remove 'string vw' template parameter from classes where it always
has a one value.

For the 2 classes that need it, make it required instead of having a
default.

17 months ago[Clang] Fix missing libraries for the include cleaner check
Joseph Huber [Fri, 2 Jun 2023 17:42:15 +0000 (12:42 -0500)]
[Clang] Fix missing libraries for the include cleaner check

Summary:
Recently, the changes in https://reviews.llvm.org/D148793 introduced
some extra dependencies that caused link failured on my machine. This
patch adds the necessary libraries to resolve the link failures and
allow me to build again.

17 months agoRevert rG2f9a4d308d4517eee85e373bc09920f85f91fdcd "[GlobalISel][X86] Add G_CTLZ_ZERO_...
Simon Pilgrim [Fri, 2 Jun 2023 17:43:02 +0000 (18:43 +0100)]
Revert rG2f9a4d308d4517eee85e373bc09920f85f91fdcd "[GlobalISel][X86] Add G_CTLZ_ZERO_UNDEF legalization handling"

Unintentional commit - G_CTLZ_ZERO_UNDEF will have to be custom handled as BSR needs the bits flipping (and we don't have a pattern for that yet).

17 months ago[bazel] Add include-cleaner targets, fix clang-tidy build for c28506ba4b6961950849f8f...
Haojian Wu [Fri, 2 Jun 2023 17:36:13 +0000 (19:36 +0200)]
[bazel] Add include-cleaner targets, fix clang-tidy build for c28506ba4b6961950849f8fdecd0cf7e503a14f9

17 months ago[mlir][sparse] fix various bug to support sparse pooling
Peiming Liu [Wed, 31 May 2023 04:53:46 +0000 (04:53 +0000)]
[mlir][sparse] fix various bug to support sparse pooling

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D151776

17 months ago[GlobalISel][X86] Add G_CTLZ_ZERO_UNDEF legalization handling
Simon Pilgrim [Fri, 2 Jun 2023 16:31:19 +0000 (17:31 +0100)]
[GlobalISel][X86] Add G_CTLZ_ZERO_UNDEF legalization handling

17 months ago[gn build] Port c28506ba4b69 more
Nico Weber [Fri, 2 Jun 2023 17:24:33 +0000 (13:24 -0400)]
[gn build] Port c28506ba4b69 more

17 months ago[gn build] Port c28506ba4b69
Nico Weber [Fri, 2 Jun 2023 17:16:05 +0000 (13:16 -0400)]
[gn build] Port c28506ba4b69

17 months ago[libc++] Remove min_max_macros test
Louis Dionne [Wed, 31 May 2023 14:30:32 +0000 (07:30 -0700)]
[libc++] Remove min_max_macros test

This test is redundant since we already test the same thing in our
nasty_macros test.

Differential Revision: https://reviews.llvm.org/D152007

17 months ago[libc++] Transition several remaining generated tests to the new .gen format
Louis Dionne [Wed, 31 May 2023 20:35:12 +0000 (13:35 -0700)]
[libc++] Transition several remaining generated tests to the new .gen format

In addition to reducing the amount of boilerplate we need to generate
whenever a new header is added, this also improves the existing tests
by running them in separate Lit tests (so they can be parallelized).
This also creates separate translation units for most header tests,
which is what we really should have done from the start since it
isolates each header we're testing.

Differential Revision: https://reviews.llvm.org/D151654

17 months ago[PowerPC] Add DFP compare instructions.
Stefan Pintilie [Fri, 2 Jun 2023 16:09:14 +0000 (12:09 -0400)]
[PowerPC] Add DFP compare instructions.

This patch adds the DFP compare instructions:
dcmpu, dcmpuq, dcmpo, dcmpoq

Reviewed By: amyk

Differential Revision: https://reviews.llvm.org/D150899

17 months ago[Driver] Move -nostdinc like options into IncludePath_Group
Fangrui Song [Fri, 2 Jun 2023 16:33:08 +0000 (09:33 -0700)]
[Driver] Move -nostdinc like options into IncludePath_Group

With only a link action, we claim all CompileOnly_Group options (including -f*,
-m*, -i*, etc). It makes sense to claim -nostdinc family options as well.
We can achieve this by placing these options into IncludePath_Group, a derivative of
CompileOnly_Group.

Reviewed By: theuni

Differential Revision: https://reviews.llvm.org/D151944

17 months ago[scudo] Support importing custom configuration
Chia-hung Duan [Fri, 2 Jun 2023 16:27:49 +0000 (16:27 +0000)]
[scudo] Support importing custom configuration

To define custom allocation, you only need to put the configuration in
custom_scudo_config.h and define two required aliases, then you will be
switched to the customized config and the tests will also run with your
configuration.

In this CL, we also have a minor refactor the structure of
configuration. Now the essential fields are put under the associated
hierarchy and which will make the defining new configuration easier.

Reviewed By: cferris

Differential Revision: https://reviews.llvm.org/D150481

17 months ago[CodeGen] Use llvm::LLVMContext::MD_invariant_load (NFC)
Kazu Hirata [Fri, 2 Jun 2023 16:25:00 +0000 (09:25 -0700)]
[CodeGen] Use llvm::LLVMContext::MD_invariant_load (NFC)

17 months ago[OpenMP] Fix transformed loop's var privacy
Joel E. Denny [Fri, 2 Jun 2023 16:18:13 +0000 (12:18 -0400)]
[OpenMP] Fix transformed loop's var privacy

Without this patch, the following example crashes Clang:

```
 #pragma omp target map(i)
 #pragma omp tile sizes(2)
 for (i = 0; i < N; ++i)
   ;
```

This patch fixes the crash by changing `Sema::isOpenMPPrivateDecl` not
to identify `i` as private just because it's the loop variable of a
`tile` construct.

While OpenMP TR11 and earlier do specify privacy for loop variables of
loops *generated* from a `tile` construct, I haven't found text
stating that the original loop variable must be private in the above
example, so this patch leaves it shared.  Even so, it is a bit
unexpected that value of `i` after the loop is `N - 1` instead of `N`.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D151356

17 months ago[mlir] Update how scalable indices are printed
Andrzej Warzynski [Thu, 1 Jun 2023 08:30:46 +0000 (09:30 +0100)]
[mlir] Update how scalable indices are printed

This patch makes sure that scalable indices (that would normally
represent scalable tile or vector sizes) are printed correctly, i.e.
with additional square brackets:
```
%1, %loop = transform.structured.tile %0 [2, 8, [4]]
```

This change complements https://reviews.llvm.org/D150944 and is a part
of a larger effort to enable scalable vectorisation in Linalg. See this
RFC for more context:
  * https://discourse.llvm.org/t/rfc-scalable-vectorisation-in-linalg/

Differential Revision: https://reviews.llvm.org/D151978

17 months ago[clang-tidy] Fix docs.
Viktoriia Bakalova [Fri, 2 Jun 2023 15:31:55 +0000 (15:31 +0000)]
[clang-tidy] Fix docs.

17 months ago[GlobalIsel][X86] Merge LZCNT 32-bit and 64-bit test coverage
Simon Pilgrim [Fri, 2 Jun 2023 15:26:25 +0000 (16:26 +0100)]
[GlobalIsel][X86] Merge LZCNT 32-bit and 64-bit test coverage

17 months ago[clang-tidy] Implement an include-cleaner check.
Viktoriia Bakalova [Wed, 19 Apr 2023 07:16:51 +0000 (07:16 +0000)]
[clang-tidy] Implement an include-cleaner check.

Differential Revision: https://reviews.llvm.org/D148793

17 months ago[libc][Obvious] Fix a typo in setting FMA control option for RISCV64.
Tue Ly [Fri, 2 Jun 2023 15:15:29 +0000 (11:15 -0400)]
[libc][Obvious] Fix a typo in setting FMA control option for RISCV64.

17 months ago[Clang][Parser] Accept GNU attributes preceding C++ attributes on templates
Elizabeth Andrews [Thu, 1 Jun 2023 21:18:40 +0000 (14:18 -0700)]
[Clang][Parser] Accept GNU attributes preceding C++ attributes on templates

Clang was rejecting valid code where GNU style attributes preceded C++ style
attributes in template declarations as follows:

template<int a>
__attribute__((deprecated("oh no!"))) [[deprecated("oh no!")]] void foo();

This PR fixes the bug.

Differential Revision: https://reviews.llvm.org/D151837

17 months ago[GlobalIsel][X86] Move G_SHL/G_LSHR/G_ASHR legalization before legacy handling and...
Simon Pilgrim [Fri, 2 Jun 2023 14:55:47 +0000 (15:55 +0100)]
[GlobalIsel][X86] Move G_SHL/G_LSHR/G_ASHR legalization before legacy handling and merge 32-bit/64-bit handling

17 months ago[GlobalIsel][X86] Move G_SDIV/G_SREM/G_UDIV/G_UREM legalization before legacy handlin...
Simon Pilgrim [Fri, 2 Jun 2023 14:46:24 +0000 (15:46 +0100)]
[GlobalIsel][X86] Move G_SDIV/G_SREM/G_UDIV/G_UREM legalization before legacy handling and merge 32-bit/64-bit handling

17 months ago[flang] Relax ALLOCATABLE/POINTER actual argument checks under INTENT(IN)
Peter Klausler [Wed, 31 May 2023 21:06:22 +0000 (14:06 -0700)]
[flang] Relax ALLOCATABLE/POINTER actual argument checks under INTENT(IN)

Per 15.5.2.5 p2, when both a dummy data object and its associated
actual argument are ALLOCATABLE or POINTER, there are rules requiring
that both be unlimited polymorphic if either is, and that both be
polymorphic if either is.  The justifications for the first restriction
is that the called procedure might change the type of an unlimited
polymorphic dummy argument, but as this cannot occur for a dummy
argument with INTENT(IN), we can relax the check to an optional
portability warning.  The justification for the second restriction
is that some implementations would have to create a type descriptor
to associate a monomorphic allocatable/pointer actual argument with
a polymorphic dummy argument, and that doesn't apply to f18 since we
use descriptors for them anyways.

Relaxing these needless checks allows more library procedures to
use "class(*), dimension(..), pointer, intent(in)" dummy arguments
in explicit interfaces.

Differential Revision: https://reviews.llvm.org/D151941

17 months ago[clang-format] Parse the Verilog language option in configuration
sstwcw [Fri, 2 Jun 2023 14:52:34 +0000 (14:52 +0000)]
[clang-format] Parse the Verilog language option in configuration

Reviewed By: HazardyKnusperkeks, MyDeveloperDay

Differential Revision: https://reviews.llvm.org/D151632

17 months agoRevert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"
Marco Elver [Fri, 2 Jun 2023 14:36:45 +0000 (16:36 +0200)]
Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit fc011a72881cdddc95bfa61f3f38916c29b7e362.
This reverts commit 4ad6a0c9a409b19b950a6a2a90d5405cea2e9b89.
This reverts commit 4b1eb4cf0e8eff5f68410720167b4986da597010.

Still causes Windows build bots to fail.

17 months agoFix "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"
Marco Elver [Fri, 2 Jun 2023 14:15:31 +0000 (16:15 +0200)]
Fix "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

The tests already depend on libc through various dependencies. In
addition, including C++STL inline functions may lead to ODR violations
where one version uses sanitizer_common's internal_mem*() functions, and
the other the normal memintrinsics.

17 months ago[libc++] Fix broken transitive includes test
Louis Dionne [Fri, 2 Jun 2023 14:23:36 +0000 (07:23 -0700)]
[libc++] Fix broken transitive includes test

17 months ago[MLIR][Docs] Unwrap video link in Rationale
J. Ryan Stinnett [Fri, 2 Jun 2023 14:22:26 +0000 (15:22 +0100)]
[MLIR][Docs] Unwrap video link in Rationale

17 months ago[AArch64] Add SVE predicated operations tests. NFC
David Green [Fri, 2 Jun 2023 14:08:04 +0000 (15:08 +0100)]
[AArch64] Add SVE predicated operations tests. NFC

Originally from the MVE tests, this adds tests for various operations which can
often be converted to predicated instructions under SVE. Additionally some tests
for commutativity and extra uses of the existing smin/smax operations. See the
patches D149969/ D151084 / D151080 / D149967 / etc.

17 months ago[mlir] Fix unqualified call to 'std::move' in TransformOps.cpp
Jie Fu [Fri, 2 Jun 2023 14:06:46 +0000 (22:06 +0800)]
[mlir] Fix unqualified call to 'std::move' in TransformOps.cpp

/data/llvm-project/mlir/lib/Dialect/Transform/IR/TransformOps.cpp:230:35: error: unqualified call to 'std::move' [-Werror,-Wunqualified-std-cast-call]
  patterns.try_emplace(attr, [f = move(fn)](RewritePatternSet &patternSet) {
                                  ^
                                  std::
1 error generated.

17 months ago[mlir][vector][transform] Register vector dialect patterns
Matthias Springer [Fri, 2 Jun 2023 13:39:04 +0000 (15:39 +0200)]
[mlir][vector][transform] Register vector dialect patterns

Differential Revision: https://reviews.llvm.org/D151983

17 months agoFix "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"
Marco Elver [Fri, 2 Jun 2023 13:47:21 +0000 (15:47 +0200)]
Fix "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

Add interface attribute to __sanitizer_internal_mem* declarations as
well, as otherwise some compilers (MSVC) will complain.

17 months ago[GlobalIsel][X86] Add G_ADD/G_SUB i8/i16 and promotion legalization tests
Simon Pilgrim [Fri, 2 Jun 2023 13:41:15 +0000 (14:41 +0100)]
[GlobalIsel][X86] Add G_ADD/G_SUB i8/i16 and promotion legalization tests

Add i8/i16/i24/i42 ADD/SUB test coverage

17 months ago[GlobalIsel][X86] Regenerate G_ADD/G_SUB vector legalization tests
Simon Pilgrim [Fri, 2 Jun 2023 13:40:03 +0000 (14:40 +0100)]
[GlobalIsel][X86] Regenerate G_ADD/G_SUB vector legalization tests

17 months ago[mlir][tensor] Implement getBufferType for ReshapeOp.
Ingo Müller [Fri, 26 May 2023 09:43:16 +0000 (09:43 +0000)]
[mlir][tensor] Implement getBufferType for ReshapeOp.

This function should be implemented for ops that work in one-shot
bufferization.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D151548

17 months ago[compiler-rt] Avoid memintrinsic calls inserted by the compiler
Marco Elver [Tue, 30 May 2023 09:59:22 +0000 (11:59 +0200)]
[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).

In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.

To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).

In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.

Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:

1. Linker --wrap:  this does not work because --wrap only
   applies to the final link, and would not apply when building
   sanitizer static libraries.

2. Changing references to memset() via objcopy:  this may work,
   but due to the complexities of the build system, introducing
   such a post-processing step for the right object files (in
   particular object files defining memset cannot be touched)
   seems infeasible.

The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").

v3:
- Don't use ALIAS() to alias internal_mem*() functions to
  __sanitizer_internal_mem*() functions, but just define them as
  ALWAYS_INLINE functions instead. This will work on darwin and windows.

v2:
- Fix ubsan_minimal build where compiler decides to insert
  memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
  therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
  aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
  more pedantic about attribute placement around extern "C".

Reviewed By: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D151152

17 months ago[mlir][tensor][transform] Register tensor dialect patterns
Matthias Springer [Fri, 2 Jun 2023 13:27:55 +0000 (15:27 +0200)]
[mlir][tensor][transform] Register tensor dialect patterns

Differential Revision: https://reviews.llvm.org/D151984

17 months ago[GlobalIsel][X86] Move G_BSWAP/G_CTPOP/G_CTLZ legalization before legacy handling
Simon Pilgrim [Fri, 2 Jun 2023 13:05:06 +0000 (14:05 +0100)]
[GlobalIsel][X86] Move G_BSWAP/G_CTPOP/G_CTLZ legalization before legacy handling

Also merge 32-bit/64-bit handling of G_CTPOP/G_CTLZ using legalIf()

17 months ago[mlir][transform] Add ApplyPatternsOp and PatternRegistry
Matthias Springer [Fri, 2 Jun 2023 12:57:22 +0000 (14:57 +0200)]
[mlir][transform] Add ApplyPatternsOp and PatternRegistry

Add a new transform op that applies patterns to a targeted payload op. Patterns can be registered by transform dialect extensions in a pattern registry.

Differential Revision: https://reviews.llvm.org/D151983

17 months ago[mlir][transform] Utilize op interface instead of tensor::TrackingListener
Matthias Springer [Fri, 2 Jun 2023 12:42:56 +0000 (14:42 +0200)]
[mlir][transform] Utilize op interface instead of tensor::TrackingListener

Add a new interface `FindPayloadReplacementOpInterface` to specify ops that should be skipped when looking for payload replacement ops. Such ops are typically metadata-only ops.

With this change, we no longer need to maintain a custom TrackingListener in the tensor dialect.

Note: `CastOpInterface` by itself is not sufficient. Some metadata-only ops such as "tensor.reshape" are not casts, and it would be incorrect for them to implement the `CastOpInterface`.

Differential Revision: https://reviews.llvm.org/D151888

17 months ago[Clang][RISCV] Add test coverage for `_mu` overloaded variant for the load intrinsics...
eopXD [Fri, 2 Jun 2023 12:35:56 +0000 (05:35 -0700)]
[Clang][RISCV] Add test coverage for `_mu` overloaded variant for the load intrinsics. NFC

Maps to amendment in under specification:
riscv-non-isa/rvv-intrinsic-doc#233

Signed-off-by: eop Chen <eop.chen@sifive.com>
17 months ago[ValueTracking] Use correct struct kind for forward declaration (NFC)
Nikita Popov [Fri, 2 Jun 2023 12:33:54 +0000 (14:33 +0200)]
[ValueTracking] Use correct struct kind for forward declaration (NFC)

17 months agoSimpleLoopUnswitch: Add missing test coverage for divergent target check
Matt Arsenault [Fri, 2 Jun 2023 10:18:21 +0000 (06:18 -0400)]
SimpleLoopUnswitch: Add missing test coverage for divergent target check

No tests failed when I removed the hasBranchDivergence check, so
add one.

17 months ago[InstCombine] Handle assumes in multi-use demanded bits simplification
Nikita Popov [Fri, 2 Jun 2023 10:52:27 +0000 (12:52 +0200)]
[InstCombine] Handle assumes in multi-use demanded bits simplification

This fixes the largest remaining discrepancy between results of
computeKnownBits() and SimplifyDemandedBits(). We only care about
the multi-use case here, because the assume necessarily introduces
an extra use.

17 months ago[SVE ACLE] Extend IR combines for fmul, fsub, fadd to cover _u variants
Jolanta Jensen [Wed, 17 May 2023 09:21:40 +0000 (09:21 +0000)]
[SVE ACLE] Extend IR combines for fmul, fsub, fadd to cover _u variants

This patch extends existing IR combines for: fmul, fsub and fadd,
relying on all active predicate to also apply to their equivalent
undef (_u) intrinsics.

Differential Revision: https://reviews.llvm.org/D150768

17 months ago[clangd] Use FileManager for getCanonicalPath, NFC
Haojian Wu [Fri, 2 Jun 2023 10:12:54 +0000 (12:12 +0200)]
[clangd] Use FileManager for getCanonicalPath, NFC

get rid of the SourceManager dependency -- getCanonicalPath doesn't use
other SourceManager fields.

17 months ago[AArch64] Don't use tbl lowering if ZExt can be folded into user.
Florian Hahn [Fri, 2 Jun 2023 10:53:03 +0000 (11:53 +0100)]
[AArch64] Don't use tbl lowering if ZExt can be folded into user.

If the ZExt can be lowered to a single ZExt to the next power-of-2 and
the remaining ZExt folded into the user, don't use tbl lowering.

Fixes #62620.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D150482

17 months ago[GlobalIsel][X86] Update legalization of G_ADD/G_SUB
Simon Pilgrim [Fri, 2 Jun 2023 10:33:51 +0000 (11:33 +0100)]
[GlobalIsel][X86] Update legalization of G_ADD/G_SUB

Replace the legacy legalizers for G_ADD/G_SUB with a pattern that should work for the full mixture of scalar/vector types we need to support for most opcodes.

If we can decide on this as a suitable pattern we can reuse it for the other op types, but G_ADD/G_SUB are the only ones that have decent legalizer test coverage at this stage.

Differential Revision: https://reviews.llvm.org/D151878

17 months ago[MLIR][Bufferization] Improve documentation of introduction and `summary`s
rikhuijzer [Fri, 2 Jun 2023 09:46:46 +0000 (11:46 +0200)]
[MLIR][Bufferization] Improve documentation of introduction and `summary`s

This patch suggests to add a sentence in the introduction which clarifies
the aim of the dialect in one sentence. I hope that I understood it correctly.

For the `summary` fields, I have scrolled through the Bufferization page
(<https://mlir.llvm.org/docs/Bufferization>) and added missing summaries
or standardized existing ones to make the summaries easier to read.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D151829

17 months ago[GlobalIsel][X86] Legalize G_BSWAP III
Thorsten Schütt [Fri, 2 Jun 2023 09:18:38 +0000 (11:18 +0200)]
[GlobalIsel][X86] Legalize G_BSWAP III

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D151915

17 months ago[ValueTracking] Replace Query with SimplifyQuery (NFC)
Nikita Popov [Fri, 2 Jun 2023 09:05:55 +0000 (11:05 +0200)]
[ValueTracking] Replace Query with SimplifyQuery (NFC)

These implement essentially the same thing, so normalize
ValueTracking to use SimplifyQuery. In the future we can directly
expose the SimplifyQuery-based APIs.

17 months ago[LoongArch] Use Subtarget consistently during LoongArchISelLowering init. NFC
Weining Lu [Fri, 2 Jun 2023 08:20:32 +0000 (16:20 +0800)]
[LoongArch] Use Subtarget consistently during LoongArchISelLowering init. NFC

17 months ago[AArch64] Add additional zext/mull tests.
Florian Hahn [Fri, 2 Jun 2023 08:16:30 +0000 (09:16 +0100)]
[AArch64] Add additional zext/mull tests.

Extra test coverage for D150482.

17 months ago[MI] Don't use std::function for all_uses/all_defs iterators (NFC)
Nikita Popov [Fri, 2 Jun 2023 07:17:48 +0000 (09:17 +0200)]
[MI] Don't use std::function for all_uses/all_defs iterators (NFC)

This mitigates the compile-time regression from D151424. The use
of std::function is not necessary here, as we're passing in a
static function.

17 months agoFix isKnownNeverInfOrNaN() call in AMDGPU after ORE removal 97b5cc214aee48e30391bfcd2...
Haojian Wu [Fri, 2 Jun 2023 07:32:46 +0000 (09:32 +0200)]
Fix isKnownNeverInfOrNaN() call in AMDGPU after ORE removal 97b5cc214aee48e30391bfcd2cde4252163d7406

17 months ago[X86] Add schedule module for SapphireRapids
Haohai Wen [Fri, 2 Jun 2023 07:10:24 +0000 (15:10 +0800)]
[X86] Add schedule module for SapphireRapids

The X86SchedSapphireRapids.td file is automatically generated by
schedtool (D130897). Most of instruction's scheduling information
is from SapphireRapids tpt/lat data provided by intel doc. Some data
is from measured ADL-P data in uops.info. The rest instruction's
scheduling information is from skylake server schedule model in order
to get a relative complete model.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D141485

17 months ago[SimplifyLibCalls] Fix isKnownNeverInfinity() call after ORE removal
Nikita Popov [Fri, 2 Jun 2023 07:20:57 +0000 (09:20 +0200)]
[SimplifyLibCalls] Fix isKnownNeverInfinity() call after ORE removal

Missed this in 97b5cc214aee48e30391bfcd2cde4252163d7406.