crypto: af_alg: Support MSG_SPLICE_PAGES

Make AF_ALG sendmsg() support MSG_SPLICE_PAGES.  This causes pages to be
spliced from the source iterator.

This allows ->sendpage() to be replaced by something that can handle
multiple multipage folios in a single transaction.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: "David S. Miller" <davem@davemloft.net>
cc: Eric Dumazet <edumazet@google.com>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Jens Axboe <axboe@kernel.dk>
cc: Matthew Wilcox <willy@infradead.org>
cc: linux-crypto@vger.kernel.org
cc: netdev@vger.kernel.org
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>

diff --git a/crypto/af_alg.c b/crypto/af_alg.c
index fd56ccf..9c12530 100644 (file)
--- a/crypto/af_alg.c
+++ b/crypto/af_alg.c
@@ -985,7 +985,7 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size,
        while (size) {
                struct scatterlist *sg;
                size_t len = size;
-               size_t plen;
+               ssize_t plen;
 
                /* use the existing memory in an allocated page */
                if (ctx->merge) {
@@ -1030,7 +1030,27 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size,
                if (sgl->cur)
                        sg_unmark_end(sg + sgl->cur - 1);
 
-               if (1 /* TODO check MSG_SPLICE_PAGES */) {
+               if (msg->msg_flags & MSG_SPLICE_PAGES) {
+                       struct sg_table sgtable = {
+                               .sgl            = sg,
+                               .nents          = sgl->cur,
+                               .orig_nents     = sgl->cur,
+                       };
+
+                       plen = extract_iter_to_sg(&msg->msg_iter, len, &sgtable,
+                                                 MAX_SGL_ENTS, 0);
+                       if (plen < 0) {
+                               err = plen;
+                               goto unlock;
+                       }
+
+                       for (; sgl->cur < sgtable.nents; sgl->cur++)
+                               get_page(sg_page(&sg[sgl->cur]));
+                       len -= plen;
+                       ctx->used += plen;
+                       copied += plen;
+                       size -= plen;
+               } else {
                        do {
                                struct page *pg;
                                unsigned int i = sgl->cur;

diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c
index 8298780..35bfa28 100644 (file)
--- a/crypto/algif_aead.c
+++ b/crypto/algif_aead.c
@@ -9,8 +9,8 @@
  * The following concept of the memory management is used:
  *
  * The kernel maintains two SGLs, the TX SGL and the RX SGL. The TX SGL is
- * filled by user space with the data submitted via sendpage/sendmsg. Filling
- * up the TX SGL does not cause a crypto operation -- the data will only be
+ * filled by user space with the data submitted via sendpage. Filling up
+ * the TX SGL does not cause a crypto operation -- the data will only be
  * tracked by the kernel. Upon receipt of one recvmsg call, the caller must
  * provide a buffer which is tracked with the RX SGL.
  *
@@ -113,19 +113,19 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg,
        }
 
        /*
-        * Data length provided by caller via sendmsg/sendpage that has not
-        * yet been processed.
+        * Data length provided by caller via sendmsg that has not yet been
+        * processed.
         */
        used = ctx->used;
 
        /*
-        * Make sure sufficient data is present -- note, the same check is
-        * also present in sendmsg/sendpage. The checks in sendpage/sendmsg
-        * shall provide an information to the data sender that something is
-        * wrong, but they are irrelevant to maintain the kernel integrity.
-        * We need this check here too in case user space decides to not honor
-        * the error message in sendmsg/sendpage and still call recvmsg. This
-        * check here protects the kernel integrity.
+        * Make sure sufficient data is present -- note, the same check is also
+        * present in sendmsg. The checks in sendmsg shall provide an
+        * information to the data sender that something is wrong, but they are
+        * irrelevant to maintain the kernel integrity.  We need this check
+        * here too in case user space decides to not honor the error message
+        * in sendmsg and still call recvmsg. This check here protects the
+        * kernel integrity.
         */
        if (!aead_sufficient_data(sk))
                return -EINVAL;

diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c
index a251cd6..b1f321b 100644 (file)
--- a/crypto/algif_skcipher.c
+++ b/crypto/algif_skcipher.c
@@ -9,10 +9,10 @@
  * The following concept of the memory management is used:
  *
  * The kernel maintains two SGLs, the TX SGL and the RX SGL. The TX SGL is
- * filled by user space with the data submitted via sendpage/sendmsg. Filling
- * up the TX SGL does not cause a crypto operation -- the data will only be
- * tracked by the kernel. Upon receipt of one recvmsg call, the caller must
- * provide a buffer which is tracked with the RX SGL.
+ * filled by user space with the data submitted via sendmsg. Filling up the TX
+ * SGL does not cause a crypto operation -- the data will only be tracked by
+ * the kernel. Upon receipt of one recvmsg call, the caller must provide a
+ * buffer which is tracked with the RX SGL.
  *
  * During the processing of the recvmsg operation, the cipher request is
  * allocated and prepared. As part of the recvmsg operation, the processed