Peter Zijlstra [Wed, 23 Mar 2022 23:05:55 +0000 (00:05 +0100)]
crypto: x86/poly1305 - Fixup SLS
Due to being a perl generated asm file, it got missed by the mass
convertion script.
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_init_x86_64()+0x3a: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_x86_64()+0xf2: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_x86_64()+0x37: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_block()+0x6d: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_init_avx()+0x1e8: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0x18a: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0xaf8: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_avx()+0x99: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x18a: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x776: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x18a: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x796: missing int3 after ret
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x10bd: missing int3 after ret
Fixes:
f94909ceb1ed ("x86: Prepare asm files for straight-line-speculation")
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Peter Zijlstra [Tue, 22 Mar 2022 11:48:10 +0000 (12:48 +0100)]
crypto: x86/chacha20 - Avoid spurious jumps to other functions
The chacha_Nblock_xor_avx512vl() functions all have their own,
identical, .LdoneN label, however in one particular spot {2,4} jump to
the 8 version instead of their own. Resulting in:
arch/x86/crypto/chacha-x86_64.o: warning: objtool: chacha_2block_xor_avx512vl() falls through to next function chacha_8block_xor_avx512vl()
arch/x86/crypto/chacha-x86_64.o: warning: objtool: chacha_4block_xor_avx512vl() falls through to next function chacha_8block_xor_avx512vl()
Make each function consistently use its own done label.
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Zheng Yongjun [Thu, 17 Mar 2022 13:16:13 +0000 (13:16 +0000)]
crypto: stm32 - fix reference leak in stm32_crc_remove
pm_runtime_get_sync() will increment pm usage counter even it
failed. Forgetting to call pm_runtime_put_noidle will result
in reference leak in stm32_crc_remove, so we should fix it.
Signed-off-by: Zheng Yongjun <zhengyongjun3@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Wed, 16 Mar 2022 22:55:13 +0000 (10:55 +1200)]
crypto: arm/aes-neonbs-cbc - Select generic cbc and aes
The algorithm __cbc-aes-neonbs requires a fallback so we need
to select the config options for them or otherwise it will fail
to register on boot-up.
Fixes:
00b99ad2bac2 ("crypto: arm/aes-neonbs - Use generic cbc...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Wed, 9 Mar 2022 03:20:01 +0000 (15:20 +1200)]
crypto: xilinx - Turn SHA into a tristate and allow COMPILE_TEST
This patch turns the new SHA driver into a tristate and also allows
compile testing.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Longfang Liu [Wed, 9 Mar 2022 02:52:44 +0000 (10:52 +0800)]
MAINTAINERS: update HPRE/SEC2/TRNG driver maintainers list
Zaibo moved projects and is not looking into crypto stuff.
I am responsible for checking the patches of these modules.
so the maintainers list needs to be updated.
I take care of HPRE, Qian Weili take care of TRNG,
Ye Kai and me take care of SEC2.
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Kai Ye <yekai12@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Jiapeng Chong [Tue, 8 Mar 2022 07:55:37 +0000 (15:55 +0800)]
crypto: dh - Remove the unused function dh_safe_prime_dh_alg()
Fix the following W=1 kernel warnings:
crypto/dh.c:311:31: warning: unused function 'dh_safe_prime_dh_alg'
[-Wunused-function]
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Miaoqian Lin [Mon, 7 Mar 2022 11:07:24 +0000 (11:07 +0000)]
hwrng: nomadik - Change clk_disable to clk_disable_unprepare
The corresponding API for clk_prepare_enable is clk_disable_unprepare,
other than clk_disable_unprepare.
Fix this by changing clk_disable to clk_disable_unprepare.
Fixes:
beca35d05cc2 ("hwrng: nomadik - use clk_prepare_enable()")
Signed-off-by: Miaoqian Lin <linmq006@gmail.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tom Rix [Sun, 6 Mar 2022 13:34:20 +0000 (05:34 -0800)]
crypto: arm64 - cleanup comments
For spdx, use // for *.c files
Replacements
significanty to significantly
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Fri, 4 Mar 2022 18:03:56 +0000 (18:03 +0000)]
crypto: qat - fix initialization of pfvf rts_map_msg structures
Initialize fully the structures rts_map_msg containing the ring to
service map from the host.
This is to fix the following warning when compiling the QAT driver
using the clang compiler with CC=clang W=2:
drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c:144:51: warning: missing field 'map' initializer [-Wmissing-field-initializers]
struct ring_to_svc_map_v1 rts_map_msg = { { 0 }, };
^
Fixes:
e1b176af3d7e ("crypto: qat - exchange ring-to-service mappings over PFVF")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Fri, 4 Mar 2022 18:03:55 +0000 (18:03 +0000)]
crypto: qat - fix initialization of pfvf cap_msg structures
Initialize fully the structures cap_msg containing the device
capabilities from the host.
This is to fix the following warning when compiling the QAT driver
using the clang compiler with CC=clang W=2:
drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c:99:44: warning: missing field 'ext_dc_caps' initializer [-Wmissing-field-initializers]
struct capabilities_v3 cap_msg = { { 0 }, };
^
Fixes:
851ed498dba1 ("crypto: qat - exchange device capabilities over PFVF")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Fri, 4 Mar 2022 18:03:54 +0000 (18:03 +0000)]
crypto: qat - remove unneeded assignment
The function adf_gen4_get_vf2pf_sources() computes a mask which is
stored in a variable which is returned and not used.
Remove superfluous assignment of variable.
This is to fix the following warning when compiling the QAT driver
with clang scan-build:
drivers/crypto/qat/qat_common/adf_gen4_pfvf.c:46:9: warning: Although the value stored to 'sou' is used in the enclosing expression, the value is never actually read from 'sou' [deadcode.DeadStores]
return sou &= ~mask;
^ ~~~~~
Fixes:
5901b4af6e07 ("crypto: qat - fix access to PFVF interrupt registers for GEN4")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Fri, 4 Mar 2022 17:54:47 +0000 (17:54 +0000)]
crypto: qat - disable registration of algorithms
The implementations of aead and skcipher in the QAT driver do not
support properly requests with the CRYPTO_TFM_REQ_MAY_BACKLOG flag set.
If the HW queue is full, the driver returns -EBUSY but does not enqueue
the request.
This can result in applications like dm-crypt waiting indefinitely for a
completion of a request that was never submitted to the hardware.
To avoid this problem, disable the registration of all crypto algorithms
in the QAT driver by setting the number of crypto instances to 0 at
configuration time.
Cc: stable@vger.kernel.org
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kai Ye [Fri, 4 Mar 2022 09:46:47 +0000 (17:46 +0800)]
crypto: hisilicon/qm - fix memset during queues clearing
Due to that extra page addr is used as a qp error flag when the device
resetting. So it not should to clear this qp flag in userspace.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Fri, 4 Mar 2022 07:36:48 +0000 (07:36 +0000)]
crypto: xilinx: prevent probing on non-xilinx hardware
The zynqmp-sha driver is always loaded and register its algorithm even on
platform which do not have the proper hardware.
This lead to a stacktrace due to zynqmp-sha3-384 failing its crypto
self tests.
So check if hardware is present via the firmware API call get_version.
While at it, simplify the platform_driver by using module_platform_driver()
Furthermore the driver should depend on ZYNQMP_FIRMWARE since it cannot
work without it.
Fixes:
7ecc3e34474b ("crypto: xilinx - Add Xilinx SHA3 driver")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Jiapeng Chong [Fri, 4 Mar 2022 05:57:42 +0000 (13:57 +0800)]
crypto: marvell/octeontx - Use swap() instead of open coding it
Clean the following coccicheck warning:
./drivers/crypto/marvell/octeontx/otx_cptvf_algs.c:1645:16-17: WARNING
opportunity for swap().
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Jianglei Nie [Thu, 3 Mar 2022 02:23:37 +0000 (10:23 +0800)]
crypto: ccree - Fix use after free in cc_cipher_exit()
kfree_sensitive(ctx_p->user.key) will free the ctx_p->user.key. But
ctx_p->user.key is still used in the next line, which will lead to a
use after free.
We can call kfree_sensitive() after dev_dbg() to avoid the uaf.
Fixes:
63ee04c8b491 ("crypto: ccree - add skcipher support")
Signed-off-by: Jianglei Nie <niejianglei2021@163.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dāvis Mosāns [Mon, 28 Feb 2022 03:15:45 +0000 (05:15 +0200)]
crypto: ccp - ccp_dmaengine_unregister release dma channels
ccp_dmaengine_register adds dma_chan->device_node to dma_dev->channels list
but ccp_dmaengine_unregister didn't remove them.
That can cause crashes in various dmaengine methods that tries to use dma_dev->channels
Fixes:
58ea8abf4904 ("crypto: ccp - Register the CCP as a DMA...")
Signed-off-by: Dāvis Mosāns <davispuh@gmail.com>
Acked-by: John Allen <john.allen@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Yang Yingliang [Fri, 25 Feb 2022 07:27:24 +0000 (15:27 +0800)]
crypto: octeontx2 - fix missing unlock
Add the missing unlock before return from error path.
Fixes:
4363f3d3ce8f ("crypto: octeontx2 - add synchronization between mailbox accesses")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wan Jiabing [Fri, 25 Feb 2022 06:38:59 +0000 (14:38 +0800)]
hwrng: cavium - fix NULL but dereferenced coccicheck error
Fix following coccicheck warning:
./drivers/char/hw_random/cavium-rng-vf.c:182:17-20: ERROR:
pdev is NULL but dereferenced.
Signed-off-by: Wan Jiabing <wanjiabing@vivo.com>
Reviewed-by: Sunil Goutham <sgoutham@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andy Shevchenko [Wed, 23 Feb 2022 16:26:20 +0000 (18:26 +0200)]
crypto: cavium/nitrox - don't cast parameter in bit operations
While in this particular case it would not be a (critical) issue,
the pattern itself is bad and error prone in case the location
of the parameter is changed.
Don't cast parameter to unsigned long pointer in the bit operations.
Instead copy to a local variable on stack of a proper type and use.
Fixes:
cf718eaa8f9b ("crypto: cavium/nitrox - Enabled Mailbox support")
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Petr Vorel [Wed, 23 Feb 2022 15:11:15 +0000 (16:11 +0100)]
crypto: vmx - add missing dependencies
vmx-crypto module depends on CRYPTO_AES, CRYPTO_CBC, CRYPTO_CTR or
CRYPTO_XTS, thus add them.
These dependencies are likely to be enabled, but if
CRYPTO_DEV_VMX=y && !CRYPTO_MANAGER_DISABLE_TESTS
and either of CRYPTO_AES, CRYPTO_CBC, CRYPTO_CTR or CRYPTO_XTS is built
as module or disabled, alg_test() from crypto/testmgr.c complains during
boot about failing to allocate the generic fallback implementations
(2 == ENOENT):
[ 0.540953] Failed to allocate xts(aes) fallback: -2
[ 0.541014] alg: skcipher: failed to allocate transform for p8_aes_xts: -2
[ 0.541120] alg: self-tests for p8_aes_xts (xts(aes)) failed (rc=-2)
[ 0.544440] Failed to allocate ctr(aes) fallback: -2
[ 0.544497] alg: skcipher: failed to allocate transform for p8_aes_ctr: -2
[ 0.544603] alg: self-tests for p8_aes_ctr (ctr(aes)) failed (rc=-2)
[ 0.547992] Failed to allocate cbc(aes) fallback: -2
[ 0.548052] alg: skcipher: failed to allocate transform for p8_aes_cbc: -2
[ 0.548156] alg: self-tests for p8_aes_cbc (cbc(aes)) failed (rc=-2)
[ 0.550745] Failed to allocate transformation for 'aes': -2
[ 0.550801] alg: cipher: Failed to load transform for p8_aes: -2
[ 0.550892] alg: self-tests for p8_aes (aes) failed (rc=-2)
Fixes:
c07f5d3da643 ("crypto: vmx - Adding support for XTS")
Fixes:
d2e3ae6f3aba ("crypto: vmx - Enabling VMX module for PPC64")
Suggested-by: Nicolai Stange <nstange@suse.de>
Signed-off-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Harsha [Wed, 23 Feb 2022 10:35:04 +0000 (16:05 +0530)]
MAINTAINERS: Add maintainer for Xilinx ZynqMP SHA3 driver
This patch adds an entry for ZynqMP SHA3 driver in the list of
Maintainers.
Signed-off-by: Harsha <harsha.harsha@xilinx.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Harsha [Wed, 23 Feb 2022 10:35:03 +0000 (16:05 +0530)]
crypto: xilinx - Add Xilinx SHA3 driver
This patch adds SHA3 driver support for the Xilinx ZynqMP SoC.
Xilinx ZynqMP SoC has SHA3 engine used for secure hash calculation.
The flow is
SHA3 request from Userspace -> SHA3 driver-> ZynqMp driver-> Firmware ->
SHA3 HW Engine
SHA3 HW engine in Xilinx ZynqMP SoC, does not support parallel processing
of 2 hash requests.
Therefore, software fallback is being used for init, update, final,
export and import in the ZynqMP SHA driver
For digest, the calculation of SHA3 hash is done by the hardened
SHA3 accelerator in Xilinx ZynqMP SoC.
Signed-off-by: Harsha <harsha.harsha@xilinx.com>
Signed-off-by: Kalyani Akula <kalyani.akula@xilinx.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Harsha [Wed, 23 Feb 2022 10:35:02 +0000 (16:05 +0530)]
firmware: xilinx: Add ZynqMP SHA API for SHA3 functionality
This patch adds zynqmp_pm_sha_hash API in the ZynqMP firmware to compute
SHA3 hash of given data.
Signed-off-by: Harsha <harsha.harsha@xilinx.com>
Signed-off-by: Kalyani Akula <kalyani.akula@xilinx.com>
Acked-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Harsha [Wed, 23 Feb 2022 10:35:01 +0000 (16:05 +0530)]
crypto: xilinx - Updated Makefile for xilinx subdirectory
This patch updates the Makefile for xilinx subdirectory.
CONFIG_CRYPTO_DEV_ZYNQMP_AES protects zynqmp-aes-gcm.o and it is used
twice (in drivers/crypto/Makefile and drivers/crypto/xilinx/Makefile)
and it is enough to use it once.
Signed-off-by: Harsha <harsha.harsha@xilinx.com>
Reviewed-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Wed, 23 Feb 2022 07:07:01 +0000 (08:07 +0100)]
crypto: crypto_xor - use helpers for unaligned accesses
Dereferencing a misaligned pointer is undefined behavior in C, and may
result in codegen on architectures such as ARM that trigger alignments
traps and expensive fixups in software.
Instead, use the get_aligned()/put_aligned() accessors, which are cheap
or even completely free when CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y.
In the converse case, the prior alignment checks ensure that the casts
are safe, and so no unaligned accessors are necessary.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tom Rix [Mon, 21 Feb 2022 22:31:18 +0000 (14:31 -0800)]
crypto: cleanup comments
For spdx
/* */ for *.h, // for *.c
Space before spdx tag
Replacements
paramenters to parameters
aymmetric to asymmetric
sigature to signature
boudary to boundary
compliled to compiled
eninges to engines
explicity to explicitly
Signed-off-by: Tom Rix <trix@redhat.com>
Acked-by: Jarkko Sakkinen <jarkko@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:11:01 +0000 (13:11 +0100)]
crypto: dh - calculate Q from P for the full public key verification
As the ->q in struct dh_ctx gets never set anywhere, the code in
dh_is_pubkey_valid() for doing the full public key validation in accordance
to SP800-56Arev3 is effectively dead.
However, for safe-prime groups Q = (P - 1)/2 by definition and
as the safe-prime groups are the only possible groups in FIPS mode (via
those ffdheXYZ() templates), this enables dh_is_pubkey_valid() to calculate
Q on the fly for these.
Implement this.
With this change, the last code accessing struct dh_ctx's ->q is now gone.
Remove this member from struct dh_ctx.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:11:00 +0000 (13:11 +0100)]
lib/mpi: export mpi_rshift
A subsequent patch will make the crypto/dh's dh_is_pubkey_valid() to
calculate a safe-prime groups Q parameter from P: Q = (P - 1) / 2. For
implementing this, mpi_rshift() will be needed. Export it so that it's
accessible from crypto/dh.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:59 +0000 (13:10 +0100)]
crypto: dh - disallow plain "dh" usage in FIPS mode
SP800-56Arev3, sec. 5.5.2 ("Assurance of Domain-Parameter Validity")
asserts that an implementation needs to verify domain paramtere validity,
which boils down to either
- the domain parameters corresponding to some known safe-prime group
explicitly listed to be approved in the document or
- for parameters conforming to a "FIPS 186-type parameter-size set",
that the implementation needs to perform an explicit domain parameter
verification, which would require access to the "seed" and "counter"
values used in their generation.
The latter is not easily feasible and moreover, SP800-56Arev3 states that
safe-prime groups are preferred and that FIPS 186-type parameter sets
should only be supported for backward compatibility, if it all.
Mark "dh" as not fips_allowed in testmgr. Note that the safe-prime
ffdheXYZ(dh) wrappers are not affected by this change: as these enforce
some approved safe-prime group each, their usage is still allowed in FIPS
mode.
This change will effectively render the keyctl(KEYCTL_DH_COMPUTE) syscall
unusable in FIPS mode, but it has been brought up that this might even be
a good thing ([1]).
[1] https://lore.kernel.org/r/
20211217055227.GA20698@gondor.apana.org.au
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:58 +0000 (13:10 +0100)]
crypto: api - allow algs only in specific constructions in FIPS mode
Currently we do not distinguish between algorithms that fail on
the self-test vs. those which are disabled in FIPS mode (not allowed).
Both are marked as having failed the self-test.
Recently the need arose to allow the usage of certain algorithms only
as arguments to specific template instantiations in FIPS mode. For
example, standalone "dh" must be blocked, but e.g. "ffdhe2048(dh)" is
allowed. Other potential use cases include "cbcmac(aes)", which must
only be used with ccm(), or "ghash", which must be used only for
gcm().
This patch allows this scenario by adding a new flag FIPS_INTERNAL to
indicate those algorithms that are not FIPS-allowed. They can then be
used as template arguments only, i.e. when looked up via
crypto_grab_spawn() to be more specific. The FIPS_INTERNAL bit gets
propagated upwards recursively into the surrounding template
instances, until the construction eventually matches an explicit
testmgr entry with ->fips_allowed being set, if any.
The behaviour to skip !->fips_allowed self-test executions in FIPS
mode will be retained. Note that this effectively means that
FIPS_INTERNAL algorithms are handled very similarly to the INTERNAL
ones in this regard. It is expected that the FIPS_INTERNAL algorithms
will receive sufficient testing when the larger constructions they're
a part of, if any, get exercised by testmgr.
Note that as a side-effect of this patch algorithms which are not
FIPS-allowed will now return ENOENT instead of ELIBBAD. Hopefully
this is not an issue as some people were relying on this already.
Link: https://lore.kernel.org/r/YeEVSaMEVJb3cQkq@gondor.apana.org.au
Originally-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:57 +0000 (13:10 +0100)]
crypto: dh - allow for passing NULL to the ffdheXYZ(dh)s' ->set_secret()
Ephemeral key generation can be requested from any of the ffdheXYZ(dh)
variants' common ->set_secret() by passing it an (encoded) struct dh
with the key parameter being unset, i.e. with ->key_size == 0. As the
whole purpose of the ffdheXYZ(dh) templates is to fill in the group
parameters as appropriate, they expect ->p and ->g to be unset in any
input struct dh as well. This means that a user would have to encode an
all-zeroes struct dh instance via crypto_dh_encode_key() when requesting
ephemeral key generation from a ffdheXYZ(dh) instance, which is kind of
pointless.
Make dh_safe_prime_set_secret() to decode a struct dh from the supplied
buffer only if the latter is non-NULL and initialize it with all zeroes
otherwise.
That is, it is now possible to call
crypto_kpp_set_secret(tfm, NULL, 0);
on any ffdheXYZ(dh) tfm for requesting ephemeral key generation.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:56 +0000 (13:10 +0100)]
crypto: testmgr - add keygen tests for ffdheXYZ(dh) templates
Now that the ffdheXYZ(dh) templates support ephemeral key generation, add
->keygen = 1 TVs for each of them to the testmgr.c.
In order to facilitate string merging by the compiler, set party B's secret
and public keys to the ones specified for party A in the respective
existing known answer test. With GCC 7.5 on x86_64, this leads to an
increase of testmgr.o size by less than half a kB.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:55 +0000 (13:10 +0100)]
crypto: dh - implement private key generation primitive for ffdheXYZ(dh)
The support for NVME in-band authentication currently in the works ([1])
needs to generate ephemeral DH keys for use with the RFC 7919 safe-prime
FFDHE groups.
In analogy to ECDH and its ecc_gen_privkey(), implement a
dh_safe_prime_gen_privkey() and invoke it from the ffdheXYZ(dh) templates'
common ->set_secret(), i.e. dh_safe_prime_set_secret(), in case the input
->key_size is zero.
As the RFC 7919 FFDHE groups are classified as approved safe-prime groups
by SP800-56Arev3, it's worthwhile to make the new
dh_safe_prime_gen_privkey() to follow the approach specified in
SP800-56Arev3, sec. 5.6.1.1.3 ("Key-Pair Generation Using Extra Random
Bits") in order to achieve conformance.
SP800-56Arev3 specifies a lower as well as an upper bound on the generated
key's length:
- it must be >= two times the maximum supported security strength of
the group in question and
- it must be <= the length of the domain parameter Q.
For any safe-prime group Q = (P - 1)/2 by definition and the individual
maximum supported security strengths as specified by SP800-56Arev3 have
been made available as part of the FFDHE dh_safe_prime definitions
introduced with a previous patch. Make dh_safe_prime_gen_privkey() pick
twice the maximum supported strength rounded up to the next power of two
for the output key size. This choice respects both, the lower and upper
bounds given by SP800-90Arev3 for any of the approved safe-prime groups and
is also in line with the NVME base spec 2.0, which requires the key size to
be >= 256bits.
[1] https://lore.kernel.org/r/
20211202152358.60116-1-hare@suse.de
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:54 +0000 (13:10 +0100)]
crypto: testmgr - add known answer tests for ffdheXYZ(dh) templates
Add known answer tests for the ffdhe2048(dh), ffdhe3072(dh), ffdhe4096(dh),
ffdhe6144(dh) and ffdhe8192(dh) templates introduced with the previous
patch to the testmgr. All TVs have been generated with OpenSSL.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:53 +0000 (13:10 +0100)]
crypto: dh - implement ffdheXYZ(dh) templates
Current work on NVME in-band authentication support ([1]) needs to invoke
DH with the FFDHE safe-prime group parameters specified in RFC 7919.
Introduce a new CRYPTO_DH_RFC7919_GROUPS Kconfig option. If enabled, make
dh_generic register a couple of ffdheXYZ(dh) templates, one for each group:
ffdhe2048(dh), ffdhe3072(dh), ffdhe4096(dh), ffdhe6144(dh) and
ffdhe8192(dh). Their respective ->set_secret() expects a (serialized)
struct dh, just like the underlying "dh" implementation does, but with the
P and G values unset so that the safe-prime constants for the given group
can be filled in by the wrapping template.
Internally, a struct dh_safe_prime instance is being defined for each of
the ffdheXYZ(dh) templates as appropriate. In order to prepare for future
key generation, fill in the maximum security strength values as specified
by SP800-56Arev3 on the go, even though they're not needed at this point
yet.
Implement the respective ffdheXYZ(dh) crypto_template's ->create() by
simply forwarding any calls to the __dh_safe_prime_create() helper
introduced with the previous commit, passing the associated dh_safe_prime
in addition to the received ->create() arguments.
[1] https://lore.kernel.org/r/
20211202152358.60116-1-hare@suse.de
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:52 +0000 (13:10 +0100)]
crypto: dh - introduce common code for built-in safe-prime group support
Recent work on NVME in-band authentication support ([1]) needs to invoke
the "dh" KPP with the FFDHE safe-prime group parameters as specified in
RFC 7919 and generate ephemeral keys suitable for the respective group. By
coincidence, the requirements from NIST SP800-56Arev3,
sec. 5.5.2 ("Assurance of Domain-Parameter Validity") basically boil down
to disallowing any group parameters not among the approved safe-prime
groups specified in either RFC 7919 or RFC 3526 in FIPS mode. Furthermore,
SP800-56Arev3 specifies the respective security strength for each of the
approved safe-prime groups, which has a direct impact on the minimum key
lengths.
In this light, it's desirable to introduce built-in support for the
RFC 7919 safe-prime groups to the kernel's DH implementation, provide a
SP800-56Arev3 conforming key generation primitive for those and render
non-approved group parameters unusable in FIPS mode on the way.
As suggested ([2]) in the course of discussion to previous iterations of
this patchset, the built-in support for ffdhe groups would be best made
available in the form of templates wrapping the existing "dh"
implementation, one for each group specified by RFC 7919: ffdhe2048(dh),
ffdhe3072(dh), ffdhe4096(dh), ffdhe6144(dh) and ffdhe8192(dh). As these
templates differ only in the safe-prime constants they'd configure the
inner "dh" transforms with, they can share almost all of their
"dh"-wrapping template implementation code.
Introduce this common code to dh_generic. The actual dump of the RFC 7919
safe-prime constants will be deferred to the next patch in order to
facilitate review. The ephemeral key generation primitive mentioned above
likewise deserves a patch on its own, as does the mechanism by which
unapproved groups are rendered unusable in FIPS mode.
Define a struct dh_safe_prime container for specifying the individual
templates' associated safe-prime group constants. All ffdheXYZ(dh) template
instances will store a pointer to such a dh_safe_prime in their context
areas each. Implement the common __dh_safe_prime_create() template
instantiation helper. The intention is that the individual ffdheXYZ(dh)
crypto_templates' ->create() implementations will simply forward any calls
to __dh_safe_prime_create(), passing a suitable dh_safe_prime in addition
to the received ->create() arguments. __dh_safe_prime_create() would then
create and register a kpp_instance as appropriate, storing the given
dh_safe_prime pointer alongside a crypto_kpp_spawn for the inner "dh"
kpp_alg in the context area.
As the ffdheXYZ(dh) kpp_instances are supposed to act as proxies to the
inner "dh" kpp_alg, make each of their associated crypto_kpp transforms to
in turn own an inner "dh" transform, a pointer to which gets stored in the
context area. Setup and teardown are getting handled from the outer
->init_tfm() and ->exit_tfm() respectively.
In order to achieve the overall goal and let the ffdheXYZ(dh) kpp_instances
configure the inner "dh" transforms with the respective group parameters,
make their common ->set_secret(), the new dh_safe_prime_set_secret(), fill
in the P and G values before forwarding the call to the inner "dh"'s
->set_secret(). Note that the outer ->set_secret() can obtain the P value
associated with the given ffdheXYZ(dh) kpp_instance by means of the
dh_safe_prime referenced from the latter's context. The value of G OTOH
always equals constant 2 for the safe-prime groups.
Finally, make the remaining two kpp_alg primitives both operating on
kpp_requests, i.e. ->generate_public_key() and ->compute_shared_secret(),
to merely forward any request to the inner "dh" implementation. However, a
kpp_request instance received from the outside cannot get simply passed
on as-is, because its associated transform (crypto_kpp_reqtfm()) will have
been set to the outer ffdheXYZ(dh) one. In order to handle this, reserve
some space in the outer ffdheXYZ(dh) kpp_requests' context areas for in
turn storing an inner kpp_request suitable for "dh" each. Make the outer
->generate_public_key() and ->compute_shared_secret() respectively to setup
this inner kpp_request by means of the new dh_safe_prime_prepare_dh_req()
helper before handing it over to the "dh" implementation for further
processing. dh_safe_prime_prepare_dh_req() basically copies the outer
kpp_request received from the outside over to the inner one, but installs
the inner transform and its own ->complete() proxy callback therein. This
completion callback, the new dh_safe_prime_complete_req(), doesn't do
anything beyond completing the outer request. Note that there exist some
examples in crypto/, which would simply install the completion handler
from the outer request at the inner one in similar setups, e.g. seqiv.
However, this would mean that the user-provided completion handler won't
get called with the address of the outer kpp_request initially submitted
and the handler might not be prepared for this. Users could certainly work
around this by setting the callback ->data properly, but IMO it's cleaner
this way. Furthermore, it might make sense to extend
dh_safe_prime_complete_req() in the future and move e.g. those
post-computation FIPS checks from the generic "dh" implementation to the
ffdheXYZ(dh) templates.
[1] https://lore.kernel.org/r/
20211202152358.60116-1-hare@suse.de
[2] https://lore.kernel.org/r/
20211217055227.GA20698@gondor.apana.org.au
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:51 +0000 (13:10 +0100)]
crypto: dh - split out deserialization code from crypto_dh_decode()
A subsequent commit will introduce "dh" wrapping templates of the form
"ffdhe2048(dh)", "ffdhe3072(dh)" and so on in order to provide built-in
support for the well-known safe-prime ffdhe group parameters specified in
RFC 7919.
Those templates' ->set_secret() will wrap the inner "dh" implementation's
->set_secret() and set the ->p and ->g group parameters as appropriate on
the way inwards. More specifically,
- A ffdheXYZ(dh) user would call crypto_dh_encode() on a struct dh instance
having ->p == ->g == NULL as well as ->p_size == ->g_size == 0 and pass
the resulting buffer to the outer ->set_secret().
- This outer ->set_secret() would then decode the struct dh via
crypto_dh_decode_key(), set ->p, ->g, ->p_size as well as ->g_size as
appropriate for the group in question and encode the struct dh again
before passing it further down to the inner "dh"'s ->set_secret().
The problem is that crypto_dh_decode_key() implements some basic checks
which would reject parameter sets with ->p_size == 0 and thus, the ffdheXYZ
templates' ->set_secret() cannot use it as-is for decoding the passed
buffer. As the inner "dh"'s ->set_secret() will eventually conduct said
checks on the final parameter set anyway, the outer ->set_secret() really
only needs the decoding functionality.
Split out the pure struct dh decoding part from crypto_dh_decode_key() into
the new __crypto_dh_decode_key().
__crypto_dh_decode_key() gets defined in crypto/dh_helper.c, but will have
to get called from crypto/dh.c and thus, its declaration must be somehow
made available to the latter. Strictly speaking, __crypto_dh_decode_key()
is internal to the dh_generic module, yet it would be a bit over the top
to introduce a new header like e.g. include/crypto/internal/dh.h
containing just a single prototype. Add the __crypto_dh_decode_key()
declaration to include/crypto/dh.h instead.
Provide a proper kernel-doc annotation, even though
__crypto_dh_decode_key() is purposedly not on the function list specified
in Documentation/crypto/api-kpp.rst.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:50 +0000 (13:10 +0100)]
crypto: dh - constify struct dh's pointer members
struct dh contains several pointer members corresponding to DH parameters:
->key, ->p and ->g. A subsequent commit will introduce "dh" wrapping
templates of the form "ffdhe2048(dh)", "ffdhe3072(dh)" and so on in order
to provide built-in support for the well-known safe-prime ffdhe group
parameters specified in RFC 7919. These templates will need to set the
group parameter related members of the (serialized) struct dh instance
passed to the inner "dh" kpp_alg instance, i.e. ->p and ->g, to some
constant, static storage arrays.
Turn the struct dh pointer members' types into "pointer to const" in
preparation for this.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:49 +0000 (13:10 +0100)]
crypto: dh - remove struct dh's ->q member
The only current user of the DH KPP algorithm, the
keyctl(KEYCTL_DH_COMPUTE) syscall, doesn't set the domain parameter ->q
in struct dh. Remove it and any associated (de)serialization code in
crypto_dh_encode_key() and crypto_dh_decode_key. Adjust the encoded
->secret values in testmgr's DH test vectors accordingly.
Note that the dh-generic implementation would have initialized its
struct dh_ctx's ->q from the decoded struct dh's ->q, if present. If this
struct dh_ctx's ->q would ever have been non-NULL, it would have enabled a
full key validation as specified in NIST SP800-56A in dh_is_pubkey_valid().
However, as outlined above, ->q is always NULL in practice and the full key
validation code is effectively dead. A later patch will make
dh_is_pubkey_valid() to calculate Q from P on the fly, if possible, so
don't remove struct dh_ctx's ->q now, but leave it there until that has
happened.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:48 +0000 (13:10 +0100)]
crypto: kpp - provide support for KPP spawns
The upcoming support for the RFC 7919 ffdhe group parameters will be
made available in the form of templates like "ffdhe2048(dh)",
"ffdhe3072(dh)" and so on. Template instantiations thereof would wrap the
inner "dh" kpp_alg and also provide kpp_alg services to the outside again.
The primitves needed for providing kpp_alg services from template instances
have been introduced with the previous patch. Continue this work now and
implement everything needed for enabling template instances to make use
of inner KPP algorithms like "dh".
More specifically, define a struct crypto_kpp_spawn in close analogy to
crypto_skcipher_spawn, crypto_shash_spawn and alike. Implement a
crypto_grab_kpp() and crypto_drop_kpp() pair for binding such a spawn to
some inner kpp_alg and for releasing it respectively. Template
implementations can instantiate transforms from the underlying kpp_alg by
means of the new crypto_spawn_kpp(). Finally, provide the
crypto_spawn_kpp_alg() helper for accessing a spawn's underlying kpp_alg
during template instantiation.
Annotate everything with proper kernel-doc comments, even though
include/crypto/internal/kpp.h is not considered for the generated docs.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Nicolai Stange [Mon, 21 Feb 2022 12:10:47 +0000 (13:10 +0100)]
crypto: kpp - provide support for KPP template instances
The upcoming support for the RFC 7919 ffdhe group parameters will be
made available in the form of templates like "ffdhe2048(dh)",
"ffdhe3072(dh)" and so on. Template instantiations thereof would wrap the
inner "dh" kpp_alg and also provide kpp_alg services to the outside again.
Furthermore, it might be perhaps be desirable to provide KDF templates in
the future, which would similarly wrap an inner kpp_alg and present
themselves to the outside as another kpp_alg, transforming the shared
secret on its way out.
Introduce the bits needed for supporting KPP template instances. Everything
related to inner kpp_alg spawns potentially being held by such template
instances will be deferred to a subsequent patch in order to facilitate
review.
Define struct struct kpp_instance in close analogy to the already existing
skcipher_instance, shash_instance and alike, but wrapping a struct kpp_alg.
Implement the new kpp_register_instance() template instance registration
primitive. Provide some helper functions for
- going back and forth between a generic struct crypto_instance and the new
struct kpp_instance,
- obtaining the instantiating kpp_instance from a crypto_kpp transform and
- for accessing a given kpp_instance's implementation specific context
data.
Annotate everything with proper kernel-doc comments, even though
include/crypto/internal/kpp.h is not considered for the generated docs.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 12:08:33 +0000 (12:08 +0000)]
crypto: engine - check if BH is disabled during completion
When doing iperf over ipsec with crypto hardware sun8i-ce, I hit some
spinlock recursion bug.
This is due to completion function called with enabled BH.
Add check a to detect this.
Fixes:
735d37b5424b ("crypto: engine - Introduce the block request crypto engine framework")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 12:08:32 +0000 (12:08 +0000)]
crypto: gemini - call finalize with bh disabled
Doing ipsec produces a spinlock recursion warning.
This is due to not disabling BH during crypto completion function.
Fixes:
46c5338db7bd45b2 ("crypto: sl3516 - Add sl3516 crypto engine")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 12:08:31 +0000 (12:08 +0000)]
crypto: amlogic - call finalize with bh disabled
Doing ipsec produces a spinlock recursion warning.
This is due to not disabling BH during crypto completion function.
Fixes:
48fe583fe541 ("crypto: amlogic - Add crypto accelerator for amlogic GXL")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 12:08:30 +0000 (12:08 +0000)]
crypto: sun8i-ce - call finalize with bh disabled
Doing ipsec produces a spinlock recursion warning.
This is due to not disabling BH during crypto completion function.
Fixes:
06f751b61329 ("crypto: allwinner - Add sun8i-ce Crypto Engine")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 12:08:29 +0000 (12:08 +0000)]
crypto: sun8i-ss - call finalize with bh disabled
Doing ipsec produces a spinlock recursion warning.
This is due to not disabling BH during crypto completion function.
Fixes:
f08fcced6d00 ("crypto: allwinner - Add sun8i-ss cryptographic offloader")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Mon, 21 Feb 2022 11:52:34 +0000 (11:52 +0000)]
crypto: cavium/zip - register algorithm only if hardware is present
On my renesas salavator-X, I saw some cavium driver failing crypto
self-tests.
But salvator does not have such hardware.
This is due to cavium/zip driver registering algorithms even if hardware
is not present.
The solution is to move algorithm registration in the probe function.
This permits also to simplify module init/exit by using
module_pci_driver.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:28 +0000 (09:59 +0200)]
hwrng: atmel - remove extra line
Remove extra line.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:27 +0000 (09:59 +0200)]
hwrng: atmel - add runtime pm support
Add runtime PM support.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:26 +0000 (09:59 +0200)]
hwrng: atmel - use __maybe_unused and pm_ptr() for pm ops
Use __maybe_unused and pm_ptr() for pm ops.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:25 +0000 (09:59 +0200)]
hwrng: atmel - move set of TRNG_HALFR in atmel_trng_init()
Move set of TRNG_HALFR in atmel_trng_init() as this function is
also called on resume path. In case of SAMA7G5 where backup and
self-refresh PM mode is available most of the SoC parts are
powered of (including TRNG) when entering suspend. In that case
on resuming path TRNG_HALFR should also be re-configured.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:24 +0000 (09:59 +0200)]
hwrng: atmel - rename enable/disable functions to init/cleanup
s/atmel_trng_disable/atmel_trng_cleanup/g and
s/atmel_trng_enable/atmel_trng_init/g to cope with
struct hwrng::{init, cleanup} members.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:23 +0000 (09:59 +0200)]
hwrng: atmel - disable trng on failure path
Call atmel_trng_disable() on failure path of probe.
Fixes:
a1fa98d8116f ("hwrng: atmel - disable TRNG during suspend")
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Claudiu Beznea [Mon, 21 Feb 2022 07:59:22 +0000 (09:59 +0200)]
hwrng: atmel - add wait for ready support on read
Add wait for ready support on read.
Signed-off-by: Claudiu Beznea <claudiu.beznea@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kai Ye [Sat, 19 Feb 2022 08:08:08 +0000 (16:08 +0800)]
crypto: hisilicon/sec - fix the aead software fallback for engine
Due to the subreq pointer misuse the private context memory. The aead
soft crypto occasionally casues the OS panic as setting the 64K page.
Here is fix it.
Fixes:
6c46a3297bea ("crypto: hisilicon/sec - add fallback tfm...")
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Gilad Ben-Yossef [Thu, 17 Feb 2022 19:27:26 +0000 (21:27 +0200)]
crypto: ccree - don't attempt 0 len DMA mappings
Refuse to try mapping zero bytes as this may cause a fault
on some configurations / platforms and it seems the prev.
attempt is not enough and we need to be more explicit.
Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com>
Reported-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Fixes:
ce0fc6db38de ("crypto: ccree - protect against empty or NULL
scatterlists")
Tested-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
YueHaibing [Wed, 16 Feb 2022 11:45:21 +0000 (19:45 +0800)]
crypto: x86/des3 - Remove unused inline function des3_ede_enc_blk_3way()
This is unused after commit
768db5fee3bb ("crypto: x86/des - drop CTR mode implementation")
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
YueHaibing [Wed, 16 Feb 2022 11:44:06 +0000 (19:44 +0800)]
crypto: x86/blowfish - Remove unused inline functions
This is unused after commit
c0a64926c53e ("crypto: x86/blowfish - drop CTR mode implementation")
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Harman Kalra [Fri, 4 Feb 2022 12:46:01 +0000 (18:16 +0530)]
crypto: octeontx2 - add synchronization between mailbox accesses
Since there are two workqueues implemented in CPTPF driver - one
for handling mailbox requests from VFs and another for handling FLR.
In both cases PF driver will forward the request to AF driver by
writing to mailbox memory. A race condition may arise if two
simultaneous requests are written to mailbox memory. Introducing
locking mechanism to maintain synchronization between multiple
mailbox accesses.
Signed-off-by: Harman Kalra <hkalra@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Sun, 13 Feb 2022 20:46:31 +0000 (21:46 +0100)]
hwrng: core - introduce rng_quality sysfs attribute
The rng_quality sysfs attribute returns the quality setting for the
currently active hw_random device, in entropy bits per 1024 bits of
input. Storing a value between 0 and 1024 to this file updates this
estimate accordingly.
Based on the updates to the quality setting, the rngd kernel thread
may be stopped (if no hw_random device is trusted to return entropy),
may be started (if the quality setting is increased from zero), or
may use a different hw_random source (if that has higher quality
output).
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Sun, 13 Feb 2022 20:46:30 +0000 (21:46 +0100)]
hwrng: core - use per-rng quality value instead of global setting
The current_quality variable exposed as a module parameter is
fundamentally broken: If it is set at boot time, it is overwritten once
the first hw rng device is loaded; if it is set at runtime, it is
without effect if the hw rng device had its quality value set to 0 (and
no default_quality was set); and if a new rng is selected, it gets
overwritten. Therefore, mark it as obsolete, and replace it by the
per-rng quality setting.
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Sun, 13 Feb 2022 20:46:29 +0000 (21:46 +0100)]
hwrng: core - start and stop in-kernel rngd in separate function
Extract the start/stop logic for the in-kernel rngd thread to
a separate function.
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Sun, 13 Feb 2022 20:46:28 +0000 (21:46 +0100)]
hwrng: core - do not bother to order list of devices by quality
There is no real reason why this list needs to be kept ordered by
the driver-provided quality value -- a value which is set only by
a handful of hw_random devices anyway.
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Fri, 11 Feb 2022 12:16:17 +0000 (12:16 +0000)]
crypto: rockchip - ECB does not need IV
When loading rockchip crypto module, testmgr complains that ivsize of ecb-des3-ede-rk
is not the same than generic implementation.
In fact ECB does not use an IV.
Fixes:
ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kai Ye [Fri, 11 Feb 2022 09:08:18 +0000 (17:08 +0800)]
crypto: hisilicon/sec - not need to enable sm4 extra mode at HW V3
It is not need to enable sm4 extra mode in at HW V3. Here is fix it.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kai Ye [Fri, 11 Feb 2022 09:07:58 +0000 (17:07 +0800)]
crypto: hisilicon/sec - add the register configuration for HW V3
Added the register configuration of the SVA mode for HW V3.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Julia Lawall [Thu, 10 Feb 2022 20:42:18 +0000 (21:42 +0100)]
crypto: ux500 - use GFP_KERNEL
Platform_driver probe functions aren't called with locks held
and thus don't need GFP_ATOMIC. Use GFP_KERNEL instead.
Problem found with Coccinelle.
Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rikard Falkeborn [Thu, 10 Feb 2022 20:28:05 +0000 (21:28 +0100)]
crypto: nx - Constify static attribute_group structs
The only usage of these is to pass their address to
sysfs_{create,remove}_group(), which takes pointers to const struct
attribute_group. Make them const to allow the compiler to put them in
read-only memory.
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rikard Falkeborn [Thu, 10 Feb 2022 20:28:04 +0000 (21:28 +0100)]
crypto: omap-sham - Constify static attribute_group
The only usage of omap_sham_attr_group is to pass its address to
sysfs_{create,remove}_group(), which takes pointers to const struct
attribute_group. Make it const to allow the compiler to put it in
read-only memory.
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rikard Falkeborn [Thu, 10 Feb 2022 20:28:03 +0000 (21:28 +0100)]
crypto: omap-aes - Constify static attribute_group
The only usage of omap_aes_attr_group is to pass its address to
sysfs_{create,remove}_group(), which takes pointers to const struct
attribute_group. Make it const to allow the compiler to put it in
read-only memory.
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 10 Feb 2022 13:38:27 +0000 (13:38 +0000)]
crypto: qat - enable power management for QAT GEN4
Add support for HW QAT Power Management (PM) feature.
This feature is enabled at init time (1) by sending an admin message to
the firmware, targeting the admin AE, that sets the idle time before
the device changes state and (2) by unmasking the PM source of interrupt
in ERRMSK2.
The interrupt handler is extended to handle a PM interrupt which
is triggered by HW when a PM transition occurs. In this case, the
driver responds acknowledging the transaction using the HOST_MSG
mailbox.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Co-developed-by: Marcinx Malinowski <marcinx.malinowski@intel.com>
Signed-off-by: Marcinx Malinowski <marcinx.malinowski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Marco Chiappero <marco.chiappero@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 10 Feb 2022 13:38:26 +0000 (13:38 +0000)]
crypto: qat - move and rename GEN4 error register definitions
Move error source related CSRs from 4xxx to the wider GEN4 header file.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Marco Chiappero <marco.chiappero@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wojciech Ziemba [Thu, 10 Feb 2022 13:38:25 +0000 (13:38 +0000)]
crypto: qat - add misc workqueue
In an effort to reduce the amount of workqueues, scattered across
the QAT driver, introduce the misc workqueue. This queue will be used
to handle bottom halves, Power Management and more in the future.
The function adf_misc_wq_queue_work() has been added to simplify
the enqueuing of jobs.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Marco Chiappero <marco.chiappero@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 10 Feb 2022 02:31:13 +0000 (13:31 +1100)]
crypto: lrw - Add dependency on ecb
The lrw template relies on ecb to work. So we need to declare
a Kconfig dependency as well as a module softdep on it.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 10 Feb 2022 02:09:40 +0000 (13:09 +1100)]
crypto: xts - Add softdep on ecb
The xts module needs ecb to be present as it's meant to work
on top of ecb. This patch adds a softdep so ecb can be included
automatically into the initramfs.
Reported-by: rftc <rftc@gmx.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andy Shevchenko [Wed, 9 Feb 2022 18:00:47 +0000 (20:00 +0200)]
crypto: qat - don't cast parameter in bit operations
While in this particular case it would not be a (critical) issue,
the pattern itself is bad and error prone in case the location
of the parameter is changed.
Don't cast parameter to unsigned long pointer in the bit operations.
Instead copy to a local variable on stack of a proper type and use.
Fixes:
b4b7e67c917f ("crypto: qat - Intel(R) QAT ucode part of fw loader")
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Corentin Labbe [Wed, 9 Feb 2022 10:21:58 +0000 (10:21 +0000)]
crypto: cavium/nitrox - fix typo on crypto
crypto had a typo, fix it.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Sat, 5 Feb 2022 15:23:46 +0000 (16:23 +0100)]
crypto: arm/xor - make vectorized C code Clang-friendly
The ARM version of the accelerated XOR routines are simply the 8-way C
routines passed through the auto-vectorizer with SIMD codegen enabled.
This used to require GCC version 4.6 at least, but given that 5.1 is now
the baseline, this check is no longer necessary, and actually
misidentifies Clang as GCC < 4.6 as Clang defines the GCC major/minor as
well, but makes no attempt at doing this in a way that conveys feature
parity with a certain version of GCC (which would not be a great idea in
the first place).
So let's drop the version check, and make the auto-vectorize pragma
(which is based on a GCC-specific command line option) GCC-only. Since
Clang performs SIMD auto-vectorization by default at -O2, no pragma is
necessary here.
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/496
Link: https://github.com/ClangBuiltLinux/linux/issues/503
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Sat, 5 Feb 2022 15:23:45 +0000 (16:23 +0100)]
lib/xor: make xor prototypes more friendly to compiler vectorization
Modern compilers are perfectly capable of extracting parallelism from
the XOR routines, provided that the prototypes reflect the nature of the
input accurately, in particular, the fact that the input vectors are
expected not to overlap. This is not documented explicitly, but is
implied by the interchangeability of the various C routines, some of
which use temporary variables while others don't: this means that these
routines only behave identically for non-overlapping inputs.
So let's decorate these input vectors with the __restrict modifier,
which informs the compiler that there is no overlap. While at it, make
the input-only vectors pointer-to-const as well.
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/563
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tudor Ambarus [Fri, 4 Feb 2022 13:35:22 +0000 (15:35 +0200)]
crypto: atmel-tdes - Add support for the TDES IP available on sama7g5 SoC
Add support for the TDES IP found on sama7g5.
Signed-off-by: Tudor Ambarus <tudor.ambarus@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephan Müller [Tue, 1 Feb 2022 08:41:32 +0000 (09:41 +0100)]
crypto: hmac - disallow keys < 112 bits in FIPS mode
FIPS 140 requires a minimum security strength of 112 bits. This implies
that the HMAC key must not be smaller than 112 in FIPS mode.
This restriction implies that the test vectors for HMAC that have a key
that is smaller than 112 bits must be disabled when FIPS support is
compiled.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephan Müller [Tue, 1 Feb 2022 08:40:58 +0000 (09:40 +0100)]
crypto: hmac - add fips_skip support
By adding the support for the flag fips_skip, hash / HMAC test vectors
may be marked to be not applicable in FIPS mode. Such vectors are
silently skipped in FIPS mode.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Colin Ian King [Sun, 30 Jan 2022 23:05:15 +0000 (23:05 +0000)]
crypto: sl3516 - remove redundant initializations of pointers in_sg and out_sg
Pointers in_sg and out_sg are being initialized with values that are
never read, they are being re-assigned the same values later on. The
initializations are redundant, remove them in preference to the later
assignments that are closer to when the pointers are being used.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Colin Ian King [Sun, 30 Jan 2022 22:53:25 +0000 (22:53 +0000)]
crypto: marvell/octeontx - remove redundant initialization of variable c_size
Variable c_size is being initialized with a value that is never read, it
is being re-assigned with a different value later on. The initialization
is redundant and can be removed.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Shijith Thotton [Fri, 28 Jan 2022 13:57:42 +0000 (19:27 +0530)]
crypto: octeontx2 - remove CONFIG_DM_CRYPT check
No issues were found while using the driver with dm-crypt enabled. So
CONFIG_DM_CRYPT check in the driver can be removed.
This also fixes the NULL pointer dereference in driver release if
CONFIG_DM_CRYPT is enabled.
...
Unable to handle kernel NULL pointer dereference at virtual address
0000000000000008
...
Call trace:
crypto_unregister_alg+0x68/0xfc
crypto_unregister_skciphers+0x44/0x60
otx2_cpt_crypto_exit+0x100/0x1a0
otx2_cptvf_remove+0xf8/0x200
pci_device_remove+0x3c/0xd4
__device_release_driver+0x188/0x234
device_release_driver+0x2c/0x4c
...
Fixes:
6f03f0e8b6c8 ("crypto: octeontx2 - register with linux crypto framework")
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tianjia Zhang [Fri, 28 Jan 2022 08:38:35 +0000 (16:38 +0800)]
crypto: tcrypt - remove all multibuffer ahash tests
The multibuffer algorithms was removed already in 2018, so it is
necessary to clear the test code left by tcrypt.
Suggested-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kavyasree Kotagiri [Fri, 28 Jan 2022 07:17:55 +0000 (12:47 +0530)]
crypto: atmel - add support for AES and SHA IPs available on lan966x SoC
This patch adds support for hardware version of AES and SHA IPs
available on lan966x SoC.
Signed-off-by: Kavyasree Kotagiri <kavyasree.kotagiri@microchip.com>
Reviewed-by: Tudor Ambarus <tudor.ambarus@microchip.com>
Tested-by: Tudor Ambarus <tudor.ambarus@microchip.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Fri, 28 Jan 2022 07:02:54 +0000 (08:02 +0100)]
hwrng: core - credit entropy for low quality sources of randomness
In case the entropy quality is low, there may be less than one bit to
credit in the call to add_hwgenerator_randomness(): The number of bytes
returned by rng_get_data() multiplied by the current quality (in entropy
bits per 1024 bits of input) must be larger than 128 to credit at least
one bit. However, imx-rngc.c sets the quality to 19, but may return less
than 32 bytes; hid_u2fzero.c sets the quality to 1; and users may override
the quality setting manually.
In case there is less than one bit to credit, keep track of it and add
that credit to the next iteration.
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Thu, 27 Jan 2022 11:35:45 +0000 (12:35 +0100)]
crypto: arm64/aes-neonbs-xts - use plain NEON for non-power-of-2 input sizes
Even though the kernel's implementations of AES-XTS were updated to
implement ciphertext stealing and can operate on inputs of any size
larger than or equal to the AES block size, this feature is rarely used
in practice.
In fact, in the kernel, AES-XTS is only used to operate on 4096 or 512
byte blocks, which means that not only the ciphertext stealing is
effectively dead code, the logic in the bit sliced NEON implementation
to deal with fewer than 8 blocks at a time is also never used.
Since the bit-sliced NEON driver already depends on the plain NEON
version, which is slower but can operate on smaller data quantities more
straightforwardly, let's fallback to the plain NEON implementation of
XTS for any residual inputs that are not multiples of 128 bytes. This
allows us to remove a lot of complicated logic that rarely gets
exercised in practice.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Thu, 27 Jan 2022 11:35:44 +0000 (12:35 +0100)]
crypto: arm64/aes-neonbs-ctr - fallback to plain NEON for final chunk
Instead of processing the entire input with the 8-way bit sliced
algorithm, which is sub-optimal for inputs that are not a multiple of
128 bytes in size, invoke the plain NEON version of CTR for the
remainder of the input after processing the bulk using 128 byte strides.
This allows us to greatly simplify the asm code that implements CTR, and
get rid of all the branches and special code paths. It also gains us a
couple of percent of performance.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Thu, 27 Jan 2022 11:35:43 +0000 (12:35 +0100)]
crypto: arm/aes-neonbs-ctr - deal with non-multiples of AES block size
Instead of falling back to C code to deal with the final bit of input
that is not a round multiple of the block size, handle this in the asm
code, permitting us to use overlapping loads and stores for performance,
and implement the 16-byte wide XOR using a single NEON instruction.
Since NEON loads and stores have a natural width of 16 bytes, we need to
handle inputs of less than 16 bytes in a special way, but this rarely
occurs in practice so it does not impact performance. All other input
sizes can be consumed directly by the NEON asm code, although it should
be noted that the core AES transform can still only process 128 bytes (8
AES blocks) at a time.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Thu, 27 Jan 2022 09:52:11 +0000 (10:52 +0100)]
crypto: arm64/aes-neon-ctr - improve handling of single tail block
Instead of falling back to C code to do a memcpy of the output of the
last block, handle this in the asm code directly if possible, which is
the case if the entire input is longer than 16 bytes.
Cc: Nathan Huckleberry <nhuck@google.com>
Cc: Eric Biggers <ebiggers@google.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Srujana Challa [Tue, 25 Jan 2022 18:26:24 +0000 (23:56 +0530)]
crypto: octeontx2 - increase CPT HW instruction queue length
LDWB is getting incorrectly used in HW when
CPT_AF_LF()_PTR_CTL[IQB_LDWB]=1 and CPT instruction queue has less than
320 free entries. So, increase HW instruction queue size by 320 and give
320 entries less for SW/NIX RX as a SW workaround.
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Srujana Challa [Tue, 25 Jan 2022 18:26:23 +0000 (23:56 +0530)]
crypto: octeontx2 - disable DMA black hole on an DMA fault
When CPT_AF_DIAG[FLT_DIS] = 0 and a CPT engine access to
LLC/DRAM encounters a fault/poison, a rare case may result
in unpredictable data being delivered to a CPT engine.
So, this patch adds code to set FLT_DIS as a workaround.
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Srujana Challa [Tue, 25 Jan 2022 18:26:22 +0000 (23:56 +0530)]
crypto: octeontx2 - CN10K CPT to RNM workaround
When software sets CPT_AF_CTL[RNM_REQ_EN]=1 and RNM in not producing
entropy(i.e., RNM_ENTROPY_STATUS[NORMAL_CNT] < 0x40), the first cycle of
the response may be lost due to a conditional clocking issue. Due to
this, the subsequent random number stream will be corrupted. So, this
patch adds support to ensure RNM_ENTROPY_STATUS[NORMAL_CNT] = 0x40
before writing CPT_AF_CTL[RNM_REQ_EN] = 1, as a workaround.
Signed-off-by: Srujana Challa <schalla@marvell.com>
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Mon, 24 Jan 2022 20:29:50 +0000 (21:29 +0100)]
hwrng: core - break out of hwrng_fillfn if current rng is not trusted
For two reasons, current_quality may become zero within the rngd
kernel thread: (1) The user lowers current_quality to 0 by writing
to the sysfs module parameter file (note that increasing the quality
from zero is without effect at the moment), or (2) there are two or
more hwrng devices registered, and those which provide quality>0 are
unregistered, but one with quality==0 remains.
If current_quality is 0, the randomness is not trusted and cannot help
to increase the entropy count. That will lead to continuous calls to
the hwrngd thread and continuous stirring of the input pool with
untrusted bits.
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Mon, 24 Jan 2022 20:29:49 +0000 (21:29 +0100)]
hwrng: core - only set cur_rng_set_by_user if it is working
In case the user-specified rng device is not working, it is not used;
therefore cur_rng_set_by_user must not be set to 1.
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dominik Brodowski [Mon, 24 Jan 2022 20:29:48 +0000 (21:29 +0100)]
hwrng: core - use rng_fillbuf in add_early_randomness()
Using rng_buffer in add_early_randomness() may race with rng_dev_read().
Use rng_fillbuf instead, as it is otherwise only used within the kernel
by hwrng_fillfn() and therefore never exposed to userspace.
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>