[AArch64][SVE] Add intrinsics for non-temporal scatters/gathers
authorAndrzej Warzynski <andrzej.warzynski@arm.com>
Wed, 4 Mar 2020 11:21:20 +0000 (11:21 +0000)
committerAndrzej Warzynski <andrzej.warzynski@arm.com>
Thu, 12 Mar 2020 13:55:56 +0000 (13:55 +0000)
commit46b9f14d712d47fa0bebd72edd8cfe58cae11f53
tree5ea96471d038b128a2fb86f8ec2385966c106600
parenta66dc755db4cd0af678b0dd7a84ca64fd66518f6
[AArch64][SVE] Add intrinsics for non-temporal scatters/gathers

Summary:
This patch adds the following intrinsics for non-temporal gather loads
and scatter stores:
  * aarch64_sve_ldnt1_gather_index
  * aarch64_sve_stnt1_scatter_index
These intrinsics implement the "scalar + vector of indices" addressing
mode.

As opposed to regular and first-faulting gathers/scatters, there's no
instruction that would take indices and then scale them. Instead, the
indices for non-temporal gathers/scatters are scaled before the
intrinsics are lowered to `ldnt1` instructions.

The new ISD nodes, GLDNT1_INDEX and SSTNT1_INDEX, are only used as
placeholders so that we can easily identify the cases implemented in
this patch in performGatherLoadCombine and performScatterStoreCombined.
Once encountered, they are replaced with:
  * GLDNT1_INDEX -> SPLAT_VECTOR + SHL + GLDNT1
  * SSTNT1_INDEX -> SPLAT_VECTOR + SHL + SSTNT1

The patterns for lowering ISD::SHL for scalable vectors (required by
this patch) were missing, so these are added too.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D75601
llvm/include/llvm/IR/IntrinsicsAArch64.td
llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
llvm/lib/Target/AArch64/AArch64ISelDAGToDAG.cpp
llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
llvm/lib/Target/AArch64/AArch64ISelLowering.h
llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
llvm/lib/Target/AArch64/SVEInstrFormats.td
llvm/test/CodeGen/AArch64/sve2-intrinsics-nt-gather-loads-64bit-scaled-offset.ll [new file with mode: 0644]
llvm/test/CodeGen/AArch64/sve2-intrinsics-nt-scatter-stores-64bit-scaled-offset.ll [new file with mode: 0644]