Noah Goldstein [Thu, 16 Feb 2023 17:57:12 +0000 (11:57 -0600)]
Add new pass `X86FixupInstTuning` for fixing up machine-instruction selection.
There are a variety of cases where we want more control over the exact
instruction emitted. This commit creates a new pass to fixup
instructions after the DAG has been lowered. The pass is only meant to
replace instructions that are guranteed to be interchangable, not to
do analysis for special cases.
Handling these instruction changes in in X86ISelLowering of
X86ISelDAGToDAG isn't ideal, as its liable to either break existing
patterns that expected a certain instruction or generate infinite
loops.
As well, operating as the MachineInstruction level allows us to access
scheduling/code size information for making the decisions.
Currently only implements `{v}permilps` -> `{v}shufps/{v}shufd` but
more transforms can be added.
Differential Revision: https://reviews.llvm.org/D143787
Noah Goldstein [Sat, 25 Feb 2023 03:05:10 +0000 (21:05 -0600)]
Add tests for replacing `{v}permilps` -> `{v}shufps/{v}pshufd`; NFC
Differential Revision: https://reviews.llvm.org/D144779
Noah Goldstein [Thu, 16 Feb 2023 17:56:48 +0000 (11:56 -0600)]
Adding tuning flags for int <-> fp domain switching penalties; NFC
Atom
- No domain switching penalties
Nehalem+
- No penalty on moves
Haswell+
- No penalty on moves / shuffles
Skylake+
- No penality on moves / shuffles / blends
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D143859
Fangrui Song [Tue, 28 Feb 2023 00:19:13 +0000 (16:19 -0800)]
[ELF][PPC64] Actually implement --no-power10-stubs
When a caller that does not use TOC calls a function, a call stub is needed if
the function may use TOC. --no-power10-stubs avoids PC-relative instructions in
the code sequence.
The --no-power10-stubs=no implementation added in D94627 is wrong.
First, the first instruction incorrectly uses `mflr 0` (instead of `mflr 12`).
Second, for the PLT case, it uses addis+addi with getVA instead of addis+ld with
getGotPltVA.
Amir Ayupov [Mon, 27 Feb 2023 23:40:45 +0000 (15:40 -0800)]
[BOLT][NFC] Simplify BinaryFunction::setTrapOnEntry
Reviewed By: #bolt, maksfb
Differential Revision: https://reviews.llvm.org/D144758
Arthur Eubanks [Mon, 27 Feb 2023 23:38:39 +0000 (15:38 -0800)]
[polly] Remove unnecessary -enable-new-pm flags
Amir Ayupov [Mon, 27 Feb 2023 23:26:14 +0000 (15:26 -0800)]
[BOLT][NFC] Log reversing splitting decision
Expose log for testing purposes.
Reviewed By: #bolt, maksfb
Differential Revision: https://reviews.llvm.org/D144674
Amir Ayupov [Mon, 27 Feb 2023 23:21:23 +0000 (15:21 -0800)]
[BOLT] Prevent unsetting unknown control flow for split jump table
In case of a function with unknown control flow but with a single jump
table and a single jump table site, we attempt to match the jump table
and a site and update block successors using jump table targets.
Restrict this behavior for split jump tables which have targets in a
fragment function.
Fixes https://github.com/llvm/llvm-project/issues/60795.
Reviewed By: #bolt, rafauler
Differential Revision: https://reviews.llvm.org/D144602
Amir Ayupov [Wed, 22 Feb 2023 06:09:17 +0000 (22:09 -0800)]
[BOLT][NFC] Const-ify analyzeJumpTable
Avoid modifying `BF`, instead set extra output parameter and modify BF in caller
scope.
Reviewed By: #bolt, rafauler
Differential Revision: https://reviews.llvm.org/D144598
Maksim Panchenko [Fri, 24 Feb 2023 23:45:21 +0000 (15:45 -0800)]
[BOLT] Change call count output for ICF
ICF optimization runs multiple passes and the order in which functions
are folded could be dependent on the order they are being processed.
This order is indeterministic as functions are intermediately stored in
std::unordered_map<>. Note that this order is mostly stable, but is not
guaranteed to be and can change e.g. after switching to a different C++
library implementation.
Because the processing (and folding) order is indeterministic, the
previous way of calculating merged function call count could produce
different results.
Change the way we calculate the ICF call count to make it independent of
the function folding/processing order.
Mostly NFC as the output binary should remain the same, the change
affects only the console output.
Reviewed By: yota9
Differential Revision: https://reviews.llvm.org/D144807
wpmed92 [Mon, 27 Feb 2023 20:51:28 +0000 (12:51 -0800)]
[mlir][core] Fix ValueRange printing in AsmPrinter
The ValueRange printing behaviour of `OpAsmPrinter` and `AsmPrinter` is different, as reported [[ https://github.com/llvm/llvm-project/issues/59334 | here ]]
```
static void testPrint(AsmPrinter &p, Operation *op, ValueRange operands) {
p << '(' << operands << ')';
}
```
Although the base `AsmPrinter` is passed as the first parameter (and not `OpAsmPrinter`), the code compiles fine. However, instead of the SSA values, the types for the operands will be printed. This is a violation of the Liskov Substitution Principle.
The desired behaviour would be that the above code does not compile. The reason it compiles, is that for the above code, the `TypeRange` version will be selected for the `<<` operator, since `ValueRange` is implicitly converted to `TypeRange`:
```
template <typename AsmPrinterT>
inline std::enable_if_t<std::is_base_of<AsmPrinter, AsmPrinterT>::value,
AsmPrinterT &>
operator<<(AsmPrinterT &p, const TypeRange &types) {
llvm::interleaveComma(types, p);
return p;
}
```
Vladislav Dzhidzhoev [Tue, 7 Feb 2023 20:32:50 +0000 (21:32 +0100)]
[AArch64][GlobalISel] Legalize G_SHUFFLE_VECTOR with smaller dest size
Legalize G_SHUFFLE_VECTOR having destination vector length smaller than
source vector length by reshaping destination vector.
Differential Revision: https://reviews.llvm.org/D144670
Fangrui Song [Mon, 27 Feb 2023 22:33:18 +0000 (14:33 -0800)]
[ELF][PPC64] Merge PPC64R12SetupStub and PPC64PCRelPLTStub. NFC
PPC64PCRelPLTStub (from D83669) duplicates lot of code from
PPC64R12SetupStub. Just merge them.
Note: PPC64R12SetupStub does not correctly handle long branch to a
non-preemptible non-TOC code.
Sam Clegg [Fri, 24 Feb 2023 18:09:07 +0000 (10:09 -0800)]
[lld][WebAssembly] Fix handling of mixed strong and weak references
When adding a undefined symbols to the symbol table, if the existing
reference is weak replace the symbol flags with (potentially) non-weak
binding.
Fixes: https://github.com/llvm/llvm-project/issues/60829
Differential Revision: https://reviews.llvm.org/D144747
Arthur Eubanks [Mon, 27 Feb 2023 22:17:24 +0000 (14:17 -0800)]
[test] Remove unnecessary -enable-new-pm=0
Arthur Eubanks [Mon, 27 Feb 2023 19:50:30 +0000 (11:50 -0800)]
[LLVMContextImpl] Separate out opaque pointers
To make the map lookups simpler for opaque pointers and to simplify future typed pointer code removal. No significant compile time wins though.
While we're here, remove the address space 0 optimization for typed pointers.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144910
Maksim Panchenko [Sun, 26 Feb 2023 02:23:53 +0000 (18:23 -0800)]
[BOLT] Fix intermittent crash with instrumentation
When createInstrumentedIndirectCall() was invoked for tail calls, we
attached annotation instruction twice to the new call instruction.
First in createDirectCall(), and then again while copying over the
metadata operands.
As a result, the annotations were not properly stripped for such calls
before the call to freeAnnotations() in LowerAnnotations pass. That lead
to use-after-free while restoring the offsets with setOffset() call.
Reviewed By: yota9
Differential Revision: https://reviews.llvm.org/D144806
Matthew Voss [Mon, 27 Feb 2023 22:04:27 +0000 (14:04 -0800)]
[NFC][PGO] Prefix duplicate profile MemOp entry diagnostic with 'warning:'
Adding this prefix will indicate clearly that the compiler doesn't exit
when it hits this diagnostic. Searches for other non-fatal diagnostics
will also be able to find this diagnostic easily.
Arthur O'Dwyer [Thu, 19 Jan 2023 22:10:09 +0000 (17:10 -0500)]
[libc++] Fix "size_t" constants that should be "bool" or "int", and add tests
`is_placeholder`, despite having an "is_" name, actually returns an int:
1 for `_1`, 2 for `_2`, 3 for `_3`, and so on. But it should still be int,
not size_t.
Simon Pilgrim [Mon, 27 Feb 2023 21:49:18 +0000 (21:49 +0000)]
[X86] Split off x86-64-v* tuning flags. NFC
Noticed when reviewing D143786, we are currently inheriting the x86-64-v* tuning flags from specific CPUs when really we need these to be a mixture of common traits and tuning to avoid specific severe regressions.
Differential Revision: https://reviews.llvm.org/D144832
Michael Jones [Thu, 16 Feb 2023 19:23:44 +0000 (11:23 -0800)]
[libc] use vars in string to num fuzz targets
The string to integer and string to float standalone fuzz targets just
ran the functions and didn't do anything with the output. This was
intentional, since they are intended to be used with sanitizers to
detect buffer overflow bugs. Not using the variables was causing compile
warnings, so this patch adds trivial checks to use the variables.
Reviewed By: sivachandra, lntue
Differential Revision: https://reviews.llvm.org/D144208
Chia-hung Duan [Mon, 27 Feb 2023 21:12:10 +0000 (21:12 +0000)]
Revert "[scudo] Only prepare PageMap entry for partial region"
This reverts commit
0a0b6fa4fbdf3bdeb300ddd58859f66b714b8bdf.
Arthur Eubanks [Mon, 27 Feb 2023 19:29:23 +0000 (11:29 -0800)]
[Bitcode] Remove typed pointer abbreviation
Since typed pointers are deprecated.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144901
Kazu Hirata [Mon, 27 Feb 2023 20:56:19 +0000 (12:56 -0800)]
[AArch64] Fix a warning
This patch fixes:
llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp:582:17: error:
unused variable 'INSvilaneMI' [-Werror,-Wunused-variable]
Michal Paszkowski [Mon, 27 Feb 2023 20:26:09 +0000 (21:26 +0100)]
[SPIR-V] Support TargetExtType for SPIR-V builtin types
This patch adds support for TargetExtType/target(...) representing
SPIR-V builtin types. After D135202, target(...) is the preferred way
for representing SPIR-V builtin types in LLVM IR and the only working
in the opaque pointer mode.
In order to maintain compatibility with LLVM IR generated by older
versions of Clang and LLVM/SPIR-V Translator, pointers-to-opaque-structs
denoting SPIR-V/OpenCL builtin types will be translated to equivalent
SPIR-V target extension types. This translation is only available in the
typed pointer mode (-opaque-pointers=0).
The relevant LIT tests with SPIR-V builtins were converted to use the
new target(...) notation.
Differential Revision: https://reviews.llvm.org/D144494
Vasileios Porpodas [Mon, 27 Feb 2023 18:44:14 +0000 (10:44 -0800)]
[SLP] Fixes crash in BoUpSLP::isGatherShuffledEntry()
Crash caused by:
708eb1b96d9a36f9c0182b7d53c492059778fa35
Differential Revision: https://reviews.llvm.org/D144895
Nilanjana Basu [Thu, 26 Jan 2023 01:35:31 +0000 (17:35 -0800)]
[AArch64] Avoid using intermediate integer registers for copying between source and destination floating point registers
In post-isel code, there are cases where there were redundant copies from a source FPR to an intermediate GPR in order to copy to a destination FPR. In this patch, we identify these patterns in post-isel peephole optimization and replace them with a direct FPR-to-FPR copy.
One example for this will be the insertion of the scalar result of 'uaddlv' neon intrinsic function into a destination vector. During instruction selection phase, 'uaddlv' result is copied to a GPR, & a vector insert instruction is matched separately to copy the previous result to a destination SIMD&FP register.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D142594
Daniel Thornburgh [Sun, 6 Feb 2022 13:20:54 +0000 (08:20 -0500)]
[Clang] [AVR] Fix USHRT_MAX for 16-bit int.
For AVR, the definition of USHRT_MAX overflows.
Reviewed By: aaron.ballman, #clang-language-wg
Differential Revision: https://reviews.llvm.org/D144218
Tamir Duberstein [Mon, 27 Feb 2023 20:02:51 +0000 (20:02 +0000)]
[clang-format-diff] Correctly parse start-of-file diffs
Handle the case where the diff is a pure removal of lines. Before this
change start_line would end up as 0 which is rejected by clang-format.
Submitting on behalf of @tamird.
Differential Revision: https://reviews.llvm.org/D144291
Rong Xu [Mon, 27 Feb 2023 17:51:28 +0000 (09:51 -0800)]
[Pass][CHR] Move ControlHeightReduction to module optimization pipeline
This is a modified version of commit
b374423304a8 by
Arthur (https://reviews.llvm.org/D143424).
Here we invoke to the pass independent of PGOOPT. We now check if the
profile is available through the program summary. This ensures CHR is
called in distributed ThinLTO BE compilation (where PGOOPT might not
be created).
Differential Revision: https://reviews.llvm.org/D144769
Florian Hahn [Mon, 27 Feb 2023 19:38:39 +0000 (20:38 +0100)]
[SCEV] Hoist common cleanup code to function. (NFC)
This allows for easier updating of common code in follow-on patches.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144847
Amara Emerson [Mon, 27 Feb 2023 19:02:37 +0000 (11:02 -0800)]
[AArch64][GlobalISel] Reorder stack up-adjustment and register copies
This change reorders the stack up-adjustment and return value copying phases of
machine-ir generation on Aarch64. Doing so prevents a bug observed for fastcc
calls with >8 arguments, where the up-adjustment required from making that call
is placed in the wrong place relative to spill and reloading code.
See: https://github.com/llvm/llvm-project/issues/60972 for full issue
reproduction and context.
Patch contributed by Bruce Collie
Differential Revision: https://reviews.llvm.org/D144791
David Green [Mon, 27 Feb 2023 19:20:10 +0000 (19:20 +0000)]
[AArch64] Don't remove free sext_inreg(vector_extract(x)) if it leads to multiple extracts
If we have sext_inreg(vector_extract(x)) but the top bits are not used, DAG
will try to remove the sext_inreg, using vector_extract(x) directly. This can
lead to multiple uses of both sext_inreg(vector_extract(x)) and
vector_extract(x), leading to the generation of both umov and smov extracts.
This adds a target hook to prevent that under AArch64 where the sext_inreg can
be considered free if there are multiple uses of the sext and no uses of the
vector_extract. This helps fix a small regression from D144550.
Differential Revision: https://reviews.llvm.org/D144850
Frederik Gossen [Mon, 27 Feb 2023 18:52:15 +0000 (13:52 -0500)]
[MLIR] Add primitive builders for scf.if
Differential Revision: https://reviews.llvm.org/D144886
Chia-hung Duan [Fri, 24 Feb 2023 04:20:30 +0000 (04:20 +0000)]
[scudo] Only prepare PageMap entry for partial region
This reduces the size of PageMap and we are more likely to use the
static local buffer. Note that now this is only supported for single
region case, i.e. on SizeClassAllocator64. For SizeClassAllocator32,
it needs a different way to save the PageMap.
Differential Revision: https://reviews.llvm.org/D142659
Nikolas Klauser [Tue, 24 Jan 2023 08:32:31 +0000 (09:32 +0100)]
[libc++][NFC] Format __split_buffer and move constructors that are marked inline into the class body
Reviewed By: ldionne, #libc
Spies: libcxx-commits
Differential Revision: https://reviews.llvm.org/D142433
Nikolas Klauser [Sun, 26 Feb 2023 14:57:25 +0000 (15:57 +0100)]
[libc++] Simplify the modules_include.sh.cpp script a bit
Reviewed By: #libc, ldionne
Spies: vvereschaka, libcxx-commits
Differential Revision: https://reviews.llvm.org/D144825
Mark de Wever [Fri, 24 Feb 2023 20:35:41 +0000 (21:35 +0100)]
[libc++] Improves clang-format settings.
Add a new test based .clang-format file which inherits from the generic
one. This moves some test specific formatting rules to the test
directory.
The main benefit is that headers are sorted, which makes it more likely
to catch these errors before creating a review instead of spotting the
error in the CI clang-tidy step.
Reviewed By: ldionne, philnik, #libc
Differential Revision: https://reviews.llvm.org/D144755
Mark de Wever [Sat, 25 Feb 2023 14:24:57 +0000 (15:24 +0100)]
[libc++] Fixes operator& hijacking atomic types.
This uses std::addressof everywherein atomic. This is not strictly
needed for the integral and floating point specializations. They should
not be used by user defined types. But it's easier to fix everything.
Note these changes are made using a WIP clang-tidy plugin.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D144786
Arthur Eubanks [Fri, 24 Feb 2023 05:47:03 +0000 (21:47 -0800)]
[LLVMContextImpl] Separate out integer constant ones
Very small compile time improvement:
https://llvm-compile-time-tracker.com/compare.php?from=
6a7a8907e8334eaf551742148079c628f78e6ed7&to=
454d1181fbdb9121f0c7a3ecf526520db32ab420&stat=instructions:u
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144746
Arthur Eubanks [Fri, 24 Feb 2023 05:16:27 +0000 (21:16 -0800)]
[LLVMContextImpl] Separate out integer constant zeroes
Very small compile time improvement:
https://llvm-compile-time-tracker.com/compare.php?from=
a628ca4925f7249b4fbd3e932c9627b12e2770dd&to=
6a7a8907e8334eaf551742148079c628f78e6ed7&stat=instructions:u
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144745
Alexey Bataev [Mon, 27 Feb 2023 17:20:49 +0000 (09:20 -0800)]
[SLP]Fix PR61018: Assertion `Mask[I] == UndefMaskElem && "Multiple uses
of scalars."' failed.
Need to check for the reused indices when checking if 2 insertelement
instruction are from the same buildvector. If the inidices are reused,
better not to match buildvectors and consider them as differenet,
otherwise need to track the order of insertelement operations.
zhongyunde [Mon, 27 Feb 2023 18:00:43 +0000 (02:00 +0800)]
[AMDGPU] Update the CHECK autogenerated as it's expired
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144771
Craig Topper [Mon, 27 Feb 2023 17:19:27 +0000 (09:19 -0800)]
[Sema] Use isSVESizelessBuiltinType instead of isSizelessBuiltinType to prevent crashing on RISC-V.
These 2 spots are protecting calls to SVE specific functions. If RISC-V
sizeless types end up in there we trigger assertions.
Use the more specific isSVESizelessBuiltinType() to avoid letting
RISC-V vectors through.
Reviewed By: asb, c-rhodes
Differential Revision: https://reviews.llvm.org/D144772
Kiran Chandramohan [Mon, 27 Feb 2023 16:42:42 +0000 (16:42 +0000)]
[Flang][OpenMP][OpenACC] Error for loop with no control
Issue error if a DO construct associated with a loop does not have
loop control. Currently, it is issued only for the loop immediately
following the loop construct. This patch extends it to cases like
collapse where there is more than one loop associated. It also fixes
a crash since the existing code always expects loop control.
This is covered in OpenMP 4.5 standard, Section 2.7.1.
"The do-loop cannot be a DO WHILE or a DO loop without loop control."
OpenACC 3.3 covers this indirectly in Section 2.9.1.
The trip count for all loops associated with the collapse clause must
be computable and invariant in all the loops".
Reviewed By: clementval
Differential Revision: https://reviews.llvm.org/D144290
Joseph Huber [Mon, 27 Feb 2023 14:46:32 +0000 (08:46 -0600)]
[OpenMP] Ignore implicit casts on assertion for `use_device_ptr`
There was an assertion triggering when invoking a captured member whose
initializer was in a blase class. This patch fixes it by allowing the
assertion on implicit casts to the base class rather than only the base
class itself.
Fixes https://github.com/llvm/llvm-project/issues/61027
Reviewed By: tianshilei1992
Differential Revision: https://reviews.llvm.org/D144873
Kiran Chandramohan [Mon, 27 Feb 2023 16:31:16 +0000 (16:31 +0000)]
[Flang][OpenMP] NFC: Change a few message/comments to fit 80chars
Changes are all in the OpenMP semantic checks file.
Reviewed By: SBallantyne
Differential Revision: https://reviews.llvm.org/D144874
Nicolas Vasilache [Wed, 22 Feb 2023 13:24:25 +0000 (05:24 -0800)]
[mlir][Linalg] Reimplement hoisting on tensors as a subset-based transformation
This revision significantly rewrites hoisting on tensors.
Previously, `vector.transfer_read/write` and `tensor.extract/insert_slice` would
be clumped together when looking for candidate pairs.
This would significantly increase the complexity of the logic and would not apply
independently to `tensor.extract/insert_slice`.
The new implementation decouples the cases and starts to cast the problem
as a generic matching subset extract/insert, which will be future proof when
other such operation pairs are introduced.
Lastly, the implementation makes the distinction clear between `vector.transfer_read/write` for
which we allow bypasses of the disjoint subsets from `tensor.extract/insert_slice` for which we
do not yet allow it.
This can be extended in the future and unified once we have subset disjunction implemented more generally.
The algorithm can be rewritten to be less of a fixed point with interspersed canonicalizations.
As a consequence, the test explicitly adds a canonicalization to clean up the IR and verify we end up in the same state.
That extra canonicalization exhibited that one of the uses in one of the tests was dead, so we fix the appropriate test.
Differential Revision: https://reviews.llvm.org/D144656
Haojian Wu [Mon, 27 Feb 2023 16:10:33 +0000 (17:10 +0100)]
[mlir] Fix a -Wunused-variable warning, NFC
Nikita Popov [Mon, 27 Feb 2023 15:56:45 +0000 (16:56 +0100)]
[ConstExpr] Avoid creation of select constant expressions
These expressions will now only be created if explicitly requested
in IR/bitcode (and by LowerTypeTests, which has a tricky to remove
use).
This is in preparation for removing these expressions entirely,
but also fixes #60983 in the meantime.
Frederik Gossen [Mon, 27 Feb 2023 15:58:56 +0000 (10:58 -0500)]
[MLIR] Add pass to deduplicate functions
Deduplicate functions that are equivalent in all aspects but their symbol name.
The pass chooses one representative per equivalence class, erases the remainder, and updates function calls accordingly.
Differential Revision: https://reviews.llvm.org/D144738
Haojian Wu [Mon, 27 Feb 2023 15:50:39 +0000 (16:50 +0100)]
Frederik Gossen [Mon, 27 Feb 2023 15:49:22 +0000 (10:49 -0500)]
[MLIR] Expose region equivalence check through OperationEquivalence
Differential Revision: https://reviews.llvm.org/D144735
Nikita Popov [Mon, 27 Feb 2023 15:36:07 +0000 (16:36 +0100)]
[InlineCost] Avoid ConstantExpr::getSelect()
Instead use ConstantFoldSelectInstruction(), which will return
nullptr if it cannot be folded and a constant expression would
be produced instead.
In preparation for removing select constant expressions.
Kohei Yamaguchi [Mon, 27 Feb 2023 15:36:16 +0000 (16:36 +0100)]
[mlir][sparse] Add checking parent op of SortOp
Fix crash with segmentation fault caused by setting a parent operator
that is not func::FuncOp with sparse_tensor SortOp.
fixes https://github.com/llvm/llvm-project/issues/59988
Reviewed By: aartbik, wrengr
Differential Revision: https://reviews.llvm.org/D143874
Kohei Yamaguchi [Mon, 27 Feb 2023 15:34:59 +0000 (16:34 +0100)]
[mlir][NFC] Cleanup Passes documentation
- Fix a place of NVGPU dialect's pass
- Move a summary of `-finalize-memref-to-llvm` into description
- Fix broken links
- Replace back-quote dialect headers with single-quote headers for
improved readability.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D142868
Amir Mohammad Tavakkoli [Mon, 27 Feb 2023 15:28:54 +0000 (16:28 +0100)]
[mlir][LinAlg][Transform][GPU] Add GPU memory hierarchy to the transform.promote op
In this patch we are adding the support of copying a a `memref.subview` to the shared or private memory in GPU. The global to shared memory copy is adopted from codes implemented in IREE (https://github.com/iree-org/iree), but the private memory copy part has not been implemented in IREE. This patch enables transferring a subview from `global->shared`, `global->private`, and `shared->private`.
Our final aim is to provide a copy layout as an affine map to the `transform.promote` op to support transpose memory copy. This map is a permutation of the original affine index map. Although this has been implemented and user can copy data to arbitrary layout , this attempt is not included in this patch since we have still problem with `linalg.generic` operations to change their index map to the transformed index map. You can find more in following links ([[ https://github.com/tavakkoliamirmohammad/iree-llvm-fork/commit/
4fd5f93355951ad0fb338858393ff409bd9c62f8 | Initial attempt to support layout map in promote op in transform dialect ]]) ([[ https://github.com/tavakkoliamirmohammad/iree-llvm-fork/commit/
9062b5849f91d4defb84996392b71087dadf7a8c | Fix data transpose in shared memory ]])
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D144666
Alexey Bataev [Mon, 27 Feb 2023 14:33:08 +0000 (06:33 -0800)]
[SLP]Fix a crash when trying to find reduced ops for the reduced value.
Need to use original reduced value, not the one the compiler gets after
reduction, it may be replaced by the extractelement instruction already.
Nikita Popov [Mon, 27 Feb 2023 15:31:21 +0000 (16:31 +0100)]
[InstCombine] Avoid ConstantExpr::getSelect() use (NFCI)
Instead let IRBuilder take care of constant folding.
In preparation for removing select constantexprs.
David Green [Mon, 27 Feb 2023 14:34:52 +0000 (14:34 +0000)]
[AArch64] Add some tests for multiple uses of extended vector extracts. NFC
Alexander Belyaev [Mon, 27 Feb 2023 14:16:27 +0000 (15:16 +0100)]
[mlir] Insert tensor.cast only when needed when folding tensor.cast into extract_slice.
Differential Revision: https://reviews.llvm.org/D144868
Pavel Kosov [Mon, 27 Feb 2023 14:15:45 +0000 (17:15 +0300)]
[OHOS] Add support for OpenHarmony
Add support for OpenHarmony OS
General OpenHarmony OS discussion on discourse thread "[RFC] Add support for OpenHarmony OS"
https://discourse.llvm.org/t/rfc-add-support-for-openharmony-os/66656
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D138202
Kerry McLaughlin [Mon, 27 Feb 2023 13:38:37 +0000 (13:38 +0000)]
[SME2][AArch64] Add multi-indexed multiply-add long long intrinsics
Adds intrinsics for the following SME2 instructions (1, 2 & 4 vector):
- smlall
- umlall
- smlsll
- umlsll
- sumlall
- usmlall
NOTE: These intrinsics are still in development and are subject to future changes.
Reviewed By: david-arm
Differential Revision: https://reviews.llvm.org/D143278
Nikita Popov [Wed, 22 Feb 2023 15:35:51 +0000 (16:35 +0100)]
[GlobalOpt] Ignore only loaded / only stored global parts in global SRA heuristic
When limiting the number of parts we split a global into, ignore
any parts that are either only loaded or only stored, because we
expect these to be optimized away after SRA.
Differential Revision: https://reviews.llvm.org/D129857
Igor Zhukov [Sun, 19 Feb 2023 00:40:41 +0000 (07:40 +0700)]
[libc++][ranges] Implement LWG-3860 range_common_reference_t is missing
Adrian Kuegel [Mon, 27 Feb 2023 13:01:20 +0000 (14:01 +0100)]
[mlir] Use the same name as the generated parameter name (NFC).
When commenting for which parameter a value is passed, the same name
should be used as is used for the real parameter. In this case, the
parameter name is generated from the TransformOps.td file.
Nimish Mishra [Mon, 27 Feb 2023 11:54:02 +0000 (17:24 +0530)]
[flang][OpenMP] Handle lastprivate on sections construct
This patch adds support for lastprivate on sections construct.
One omp.sections operation can have several omp.section operation. As such, the privatization happens in the lexically last omp.section operation.
Reviewed By: kiranchandramohan, peixin
Differential Revision: https://reviews.llvm.org/D133686
Sacha Ballantyne [Mon, 27 Feb 2023 11:00:45 +0000 (11:00 +0000)]
[Flang] Add Minloc to simplify intrinsics pass
This patch adds minloc to the simplify intrinsics pass, supporting calls with KIND or MASK arguments while calls which have BACK, DIM or have a CHARACTER input array are rejected. This patch is targeting exchange2, and in benchmarks provides a ~11% improvement in performance.
Also included are some minor style changes / cleanup in simplifyIntrinsics.cpp.
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D144103
Max Kazantsev [Mon, 27 Feb 2023 11:22:52 +0000 (18:22 +0700)]
[LoopPredication] Account for critical edges when inserting assumes. PR26496
Loop predication can insert assumes to preserve knowledge about some facts that
may otherwise be lost, because loop predication is a lossy transform. When a guard
is represented as branch by widenable condition, it should insert it in the guarded
block. However, if the guarded block has other predecessors than the guard block,
then the condition might not dominate it. Currently we generate invalid code here.
One possible fix here is to split critical edge and insert the assume there, but in
this case we should modify CFG, which Loop Predication is not currently doing, and we
want to keep it that way.
The fix is to handle this case by inserting a Phi which takes `Cond` as input from the
guard block and `true` from any other blocks. This is valid in terms of IR and does
not introduce any new knowledge if we came from another block.
Differential Revision: https://reviews.llvm.org/D144859
Reviewed By: nikic, skatkov
Nikita Popov [Mon, 20 Feb 2023 08:46:54 +0000 (09:46 +0100)]
Reapply [InstCombine] Remove early constant fold
The reported compile-time regression has been address in
47f9109dff80a1abbe2705ee71dc0882b1d62274.
Additionally, this contains a change to immediately fold zext
with constant operand, even if it's used in a trunc. I'm not sure
if this is relevant for anything, but I noticed it as a behavioral
discrepancy when investigating this issue.
-----
InstCombine currently performs a constant folding attempt as part
of the main InstCombine loop, before visiting the instruction.
However, each visit method will also attempt to simplify the
instruction, which will in turn constant fold it. (Additionally,
we also constant fold instructions before the main InstCombine loop
and use a constant folding IR builder, so this is doubly redundant.)
There is one place where InstCombine visit methods currently don't
call into simplification, and that's casts. To be conservative,
I've added an explicit constant folding call there (though it has
no impact on tests).
This makes for a mild compile-time improvement and in particular
mitigates the compile-time regression from enabling load
simplification in
be88b5814d9efce131dbc0c8e288907e2e6c89be.
Differential Revision: https://reviews.llvm.org/D144369
Marco Elver [Mon, 27 Feb 2023 11:05:02 +0000 (12:05 +0100)]
[SelectionDAG] Transitively copy NodeExtraInfo on RAUW
During legalization of the SelectionDAG, some nodes are replaced with
arch-specific nodes. These may be complex nodes, where the root node no
longer corresponds to the node that should carry the extra info.
Fix the issue by copying extra info to the new node and all its new
transitive operands during RAUW. See code comments for more details.
This fixes the remaining pcsections-atomics.ll tests on X86.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D144677
Marco Elver [Mon, 27 Feb 2023 11:04:52 +0000 (12:04 +0100)]
[X86][FixupBWInsts] Fix propagation of !pcsections metadata
Use MIMetadata() to propagate both DebugLoc and !pcsections metadata.
This fixes several of the non-native sized !pcsections tests in
pcsections-atomics.ll.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D144676
Marco Elver [Mon, 27 Feb 2023 11:04:42 +0000 (12:04 +0100)]
[X86] Improve atomics test for !pcsections
Extend pcsections-atomics.ll to exhaustively test all atomic ops up to
64 bits. This currently shows that some atomic operations do not end up
in PC sections. This will be addressed in a subsequent change.
Differential Revision: https://reviews.llvm.org/D144710
Marco Elver [Mon, 27 Feb 2023 11:04:21 +0000 (12:04 +0100)]
[X86] Move atomics test for !pcsections into separate file
The pcsections.ll test primarily tests that the AsmPrinter produces the
right output in sections. This output is not easily covered by
update_llc_test_checks.py, and as such is hand written. This makes
maintenance rather burdensome. Instead, let's keep pcsections.ll as
simple as possible.
Move the more complex tests that primarily test that some atomic
operations end up in the PC section to pcsections-atomics.ll.
NFC.
Reviewed By: dvyukov, vitalybuka
Differential Revision: https://reviews.llvm.org/D144675
Nikita Popov [Mon, 27 Feb 2023 10:56:36 +0000 (11:56 +0100)]
[InstCombine] Guard against many users when swapping icmp operands
This addresses the compile-time regression reported on D144369.
If we don't fold constant operands early, then we might end up
walking very large use lists of constants here. Explicitly exclude
constants, and also limit the number of inspected users to avoid
degenerate cases like this.
This entire transform shouldn't be part of InstCombine in the
first place though.
Manuel Klimek [Mon, 27 Feb 2023 10:50:05 +0000 (10:50 +0000)]
[clang-format] Fix assertion that doesn't hold under fuzzing.
chendewen [Mon, 27 Feb 2023 09:24:11 +0000 (17:24 +0800)]
[SVE] Add intrinsics for uniform dsp operations that explicitly undefine the result for inactive lanes.
This patch adds new intrinsics for uniform dsp operations and changes the lowering for the following builtins to emit calls to the new aarch64.sve.###.u intrinsics.
svsqsub_x
svsqsub_n_x
svuqsub_x
svuqsub_n_x
svsqsubr_x
svsqsubr_n_x
svuqsubr_x
svuqsubr_n_x
Reviewed By: Paul Walker
Differential Revision: https://reviews.llvm.org/D144704
Manuel Klimek [Mon, 27 Feb 2023 09:39:40 +0000 (09:39 +0000)]
[clang-format] Add macro replacement to fuzzing.
Haojian Wu [Mon, 27 Feb 2023 10:33:45 +0000 (11:33 +0100)]
Michael Platings [Fri, 24 Feb 2023 21:07:27 +0000 (21:07 +0000)]
Allow building with CMAKE_SYSTEM_NAME=Generic
This is important for building runtimes for bare metal targets.
Differential Revision: https://reviews.llvm.org/D144757
Max Kazantsev [Mon, 27 Feb 2023 10:19:17 +0000 (17:19 +0700)]
[Test] Add failing test for PR61022
Details: https://github.com/llvm/llvm-project/issues/61022
Sergey Kachkov [Mon, 27 Feb 2023 09:56:13 +0000 (12:56 +0300)]
Revert "[GVN] Support address translation through select instructions"
This reverts commit
b5bf6f6392a3408be1b7b7e036eb69358c5a2c29.
Nikita Popov [Wed, 22 Feb 2023 16:05:33 +0000 (17:05 +0100)]
[SCEV] Make scalable size representation more explicit
Represent scalable type sizes using C * vscale, where vscale is
the vscale constant expression. This exposes a bit more information
to SCEV, because the vscale multiplier is explicitly modeled in SCEV
(rather than part of the sizeof expression).
This is mainly intended as an alternative to D143642.
Differential Revision: https://reviews.llvm.org/D144624
Augustin Fabre [Mon, 27 Feb 2023 09:51:56 +0000 (01:51 -0800)]
[clang-format] clang-format.el: fix warnings
Differential Revision: https://reviews.llvm.org/D143560
LLVM GN Syncbot [Mon, 27 Feb 2023 09:25:56 +0000 (09:25 +0000)]
[gn build] Port
e7950fceb1e7
Carlos Alberto Enciso [Mon, 27 Feb 2023 08:12:19 +0000 (08:12 +0000)]
[llvm-debuginfo-analyzer] (09/09) - CodeView Reader
llvm-debuginfo-analyzer is a command line tool that processes debug
info contained in a binary file and produces a debug information
format agnostic “Logical View”, which is a high-level semantic
representation of the debug info, independent of the low-level
format.
The code has been divided into the following patches:
1) Interval tree
2) Driver and documentation
3) Logical elements
4) Locations and ranges
5) Select elements
6) Warning and internal options
7) Compare elements
8) ELF Reader
9) CodeView Reader
Full details:
https://discourse.llvm.org/t/llvm-dev-rfc-llvm-dva-debug-information-visual-analyzer/62570
This patch:
This is a high level summary of the changes in this patch.
CodeView Reader
- Support for CodeView/PDB.
LVCodeViewReader, LVTypeVisitor, LVSymbolVisitor, LVLogicalVisitor
Reviewed By: psamolysov, probinson, djtodoro, zequanwu
Differential Revision: https://reviews.llvm.org/D125784
Haojian Wu [Mon, 27 Feb 2023 09:07:16 +0000 (10:07 +0100)]
[include-cleaner] Fix an unintended early return when checking the
incompatible flags in the CLI tool.
Sergey Kachkov [Wed, 25 Jan 2023 13:10:29 +0000 (16:10 +0300)]
[GVN] Support address translation through select instructions
Process cases when phi incoming in predecessor block has select
instruction, and this select address is unavailable, but there
are addresses translated from both sides of select instruction.
Differential Revision: https://reviews.llvm.org/D142705
Christian Ulmann [Mon, 27 Feb 2023 08:49:06 +0000 (09:49 +0100)]
[mlir][llvm] Stop exporting empty debug MD strings
This commit ensures that no empty debug metadata strings are exported as
these are not legal names. Additionally, this commit ensures that
non-existing strings are not accidentially imported as empty strings.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D144263
Diana Picus [Thu, 23 Feb 2023 12:01:02 +0000 (13:01 +0100)]
[AMDGPU] Run update scripts on existing tests. NFC
Update a few tests where the checks aren't exactly kosher.
Differential Revision: https://reviews.llvm.org/D144639
Christian Ulmann [Mon, 27 Feb 2023 08:31:48 +0000 (09:31 +0100)]
[mlir][llvm] Builders dont access null attr (NFC)
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D144267
Matthias Springer [Mon, 27 Feb 2023 08:35:37 +0000 (09:35 +0100)]
[mlir][SCF] Fix incorrect API usage in RewritePatterns
Incorrect API usage was detected by D144552.
Differential Revision: https://reviews.llvm.org/D144636
Jean Perier [Mon, 27 Feb 2023 08:05:11 +0000 (09:05 +0100)]
[flang][hlfir] Lower associate construct to HLFIR
- always use genExprAddr when lowering to HLFIR: it does not create
temporary for array sections without vector subscripts, so there is
no need to have custom logic.
- update mangling to deal with AssocDetailsEntity. Their name is
required in HLFIR so that it can be added to the hlfir.declare
that is created for the selector once it is lowered. This should
allow getting debug info for selector when debug info are generated
from hlfir.declare.
The rest of associate construct lowering is unchanged and shared with
the current lowering.
This patch also enables select type lowering to work properly, but some
other todos (mainly about parent component references) prevents porting
the tests for now, so this will be done later.
Differential Revision: https://reviews.llvm.org/D144740
Jean Perier [Mon, 27 Feb 2023 08:04:20 +0000 (09:04 +0100)]
[flang][hlfir] Lower allocatable assignment to HLFIR
Nothing much to do except set the right attributes on hlfir.assign.
Differential Revision: https://reviews.llvm.org/D144727
Jean Perier [Mon, 27 Feb 2023 08:03:09 +0000 (09:03 +0100)]
[flang][hlfir] add allocatable assignment semantic to hlfir.assign
Differential Revision: https://reviews.llvm.org/D144723
Nilanjana Basu [Sat, 11 Feb 2023 05:34:43 +0000 (21:34 -0800)]
[AArch64] Added tests for inserting scalar result of uaddlv neon instrinsic function into a vector
Inserting scalar result of 'uaddlv' neon intrinsic function to a destination vector currently makes use of the integer unit. Subsequent patches will eliminate the redundant use of the integer registers in a more generic way that will include this special case. This is an initial set of tests for this functionality.
Differential Revision: https://reviews.llvm.org/D143038
Lei Zhang [Mon, 27 Feb 2023 06:22:56 +0000 (06:22 +0000)]
[mlir][spirv] Fix Physical32/Physical64 support for OpenCL
We use `use64bitIndex` in the option to decide the target device
address bitwidth. This makes it consistent with index type
conversion too.
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D144827
Lei Zhang [Mon, 27 Feb 2023 06:15:18 +0000 (06:15 +0000)]
[mlir][spirv] Respect client API requirements for 64-bit index
Vulkan requires GPU processor ID/count builtin variables to be
32-bit scalar or vector for all the cases. Similarly there
are special requirements for OpenCL. We need to make sure those
rules are respected when converting using 64bit for index.
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D144819
Rahul Kayaith [Sat, 25 Feb 2023 08:51:31 +0000 (03:51 -0500)]
[mlir][python] Don't emit diagnostics when printing invalid ops
The asm printer grew the ability to automatically fall back to the
generic format for invalid ops, so this logic doesn't need to be in the
bindings anymore. The printer already handles supressing diagnostics
that get emitted while checking if the op is valid.
Reviewed By: mehdi_amini, stellaraccident
Differential Revision: https://reviews.llvm.org/D144805
Jun Zhang [Mon, 27 Feb 2023 04:22:18 +0000 (12:22 +0800)]
Precommit test for D144777, NFC
Signed-off-by: Jun Zhang <jun@junz.org>