crypto: api - Use data directly in completion function

This patch does the final flag day conversion of all completion
functions which are now all contained in the Crypto API.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Handle EBUSY correctly

As it is seqiv only handles the special return value of EINPROGERSS,
which means that in all other cases it will free data related to the
request.

However, as the caller of seqiv may specify MAY_BACKLOG, we also need
to expect EBUSY and treat it in the same way. Otherwise backlogged
requests will trigger a use-after-free.

Fixes: 0a270321dbf9 ("[CRYPTO] seqiv: Add Sequence Number IV Generator")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

mm, treewide: rename kzfree() to kfree_sensitive()

As said by Linus:

A symmetric naming is only helpful if it implies symmetries in use.
Otherwise it's actively misleading.

In "kzalloc()", the z is meaningful and an important part of what the
caller wants.

In "kzfree()", the z is actively detrimental, because maybe in the
future we really _might_ want to use that "memfill(0xdeadbeef)" or
something. The "zero" part of the interface isn't even _relevant_.

The main reason that kzfree() exists is to clear sensitive information
that should not be leaked to other future users of the same memory
objects.

Rename kzfree() to kfree_sensitive() to follow the example of the recently
added kvfree_sensitive() and make the intention of the API more explicit.
In addition, memzero_explicit() is used to clear the memory to make sure
that it won't get optimized away by the compiler.

The renaming is done by using the command sequence:

git grep -w --name-only kzfree |\
xargs sed -i 's/kzfree/kfree_sensitive/'

followed by some editing of the kfree_sensitive() kerneldoc and adding
a kzfree backward compatibility macro in slab.h.

[akpm@linux-foundation.org: fs/crypto/inline_crypt.c needs linux/slab.h]
[akpm@linux-foundation.org: fix fs/crypto/inline_crypt.c some more]

Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Howells <dhowells@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: Joe Perches <joe@perches.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Dan Carpenter <dan.carpenter@oracle.com>
Cc: "Jason A . Donenfeld" <Jason@zx2c4.com>
Link: http://lkml.kernel.org/r/20200616154311.12314-3-longman@redhat.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

crypto: seqiv - remove seqiv_create()

seqiv_create() is pointless because it just checks that the template is
being instantiated as an AEAD, then calls seqiv_aead_create(). But
seqiv_aead_create() does the exact same check, via aead_geniv_alloc().

Just remove seqiv_create() and use seqiv_aead_create() directly.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: geniv - remove unneeded arguments from aead_geniv_alloc()

The type and mask arguments to aead_geniv_alloc() are always 0, so
remove them.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: geniv - convert to new way of freeing instances

Convert the "seqiv" template to the new way of freeing instances where a
->free() method is installed to the instance struct itself. Also remove
the unused implementation of the old way of freeing instances from the
"echainiv" template, since it's already using the new way too.

In doing this, also simplify the code by making the helper function
aead_geniv_alloc() install the ->free() method, instead of making seqiv
and echainiv do this themselves. This is analogous to how
skcipher_alloc_instance_simple() works.

This will allow removing support for the old way of freeing instances.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-or-later

has been chosen to replace the boilerplate/reference in 3029 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

crypto: run initcalls for generic implementations earlier

Use subsys_initcall for registration of all templates and generic
algorithm implementations, rather than module_init. Then change
cryptomgr to use arch_initcall, to place it before the subsys_initcalls.

This is needed so that when both a generic and optimized implementation
of an algorithm are built into the kernel (not loadable modules), the
generic implementation is registered before the optimized one.
Otherwise, the self-tests for the optimized implementation are unable to
allocate the generic implementation for the new comparison fuzz tests.

Note that on arm, a side effect of this change is that self-tests for
generic implementations may run before the unaligned access handler has
been installed. So, unaligned accesses will crash the kernel. This is
arguably a good thing as it makes it easier to detect that type of bug.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use kmemdup in seqiv_aead_encrypt()

Use kmemdup rather than duplicating its implementation

Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: null - Remove VLA usage of skcipher

In the quest to remove all stack VLA usage from the kernel[1], this
replaces struct crypto_skcipher and SKCIPHER_REQUEST_ON_STACK() usage
with struct crypto_sync_skcipher and SYNC_SKCIPHER_REQUEST_ON_STACK(),
which uses a fixed stack size.

[1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.com

Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Remove unused alg/spawn variable

This patch remove two unused variable and some dead "code" using it.

Fixes: 92932d03c2b3 ("crypto: seqiv - Remove AEAD compatibility code")

Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic

Instead of unconditionally forcing 4 byte alignment for all generic
chaining modes that rely on crypto_xor() or crypto_inc() (which may
result in unnecessary copying of data when the underlying hardware
can perform unaligned accesses efficiently), make those functions
deal with unaligned input explicitly, but only if the Kconfig symbol
HAVE_EFFICIENT_UNALIGNED_ACCESS is set. This will allow us to drop
the alignmasks from the CBC, CMAC, CTR, CTS, PCBC and SEQIV drivers.

For crypto_inc(), this simply involves making the 4-byte stride
conditional on HAVE_EFFICIENT_UNALIGNED_ACCESS being set, given that
it typically operates on 16 byte buffers.

For crypto_xor(), an algorithm is implemented that simply runs through
the input using the largest strides possible if unaligned accesses are
allowed. If they are not, an optimal sequence of memory accesses is
emitted that takes the relative alignment of the input buffers into
account, e.g., if the relative misalignment of dst and src is 4 bytes,
the entire xor operation will be completed using 4 byte loads and stores
(modulo unaligned bits at the start and end). Note that all expressions
involving misalign are simply eliminated by the compiler when
HAVE_EFFICIENT_UNALIGNED_ACCESS is defined.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: skcipher - Remove top-level givcipher interface

This patch removes the old crypto_grab_skcipher helper and replaces
it with crypto_grab_skcipher2.

As this is the final entry point into givcipher this patch also
removes all traces of the top-level givcipher interface, including
all implicit IV generators such as chainiv.

The bottom-level givcipher interface remains until the drivers
using it are converted.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use skcipher

This patch replaces use of the obsolete blkcipher with skcipher.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use generic geniv init/exit helpers

This patch replaces the seqiv init/exit handlers with the generic
geniv helpers.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Remove AEAD compatibility code

Now that we no longer have any legacy AEAD implementations the
compatibility code path can no longer be triggered. This patch
removes it.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Remove seqniv

Now that IPsec no longer uses seqniv we can remove it.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Replace seqniv with seqiv

This patch replaces the seqniv generator with seqiv when the
underlying algorithm understands the new calling convention.

This not only makes more sense as now seqiv is solely responsible
for IV generation rather than also determining how the IV is going
to be used, it also allows for optimisations in the underlying
implementation. For example, the space for the IV could be used
to add padding for authentication.

This patch also removes the unnecessary copying of IV to dst
during seqiv decryption as the IV is part of the AD and not cipher
text.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Add compatibility support without RNG

When seqiv is used in compatibility mode, this patch allows it
to function even when an RNG Is not available. It also changes
the RNG allocation for the new explicit seqiv interface so that
we only hold a reference to the RNG during initialisation.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Move IV seeding into init function

We currently do the IV seeding on the first givencrypt call in
order to conserve entropy. However, this does not work with
DRBG which cannot be called from interrupt context. In fact,
with DRBG we don't need to conserve entropy anyway. So this
patch moves the seeding into the init function.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Fix module unload/reload crash

On module unload we weren't unregistering the seqniv template,
thus leading to a crash the next time someone walks the template
list.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Fix IV size in context size calculation

This patch fixes a bug in the context size calculation where we
were still referring to the old cra_aead.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use common IV generation code

This patch makes use of the new common IV generation code.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Copy AD along with plain/cipher text

As the AD does not necessarily exist in the destination buffer
it must be copied along with the plain/cipher text.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: aead - Remove unused cryptoff parameter

This patch removes the cryptoff parameter now that all users
set it to zero.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Stop using cryptoff

The cryptoff parameter was added to facilitate the skipping of
IVs that sit between the AD and the plain/cipher text. However,
it was never implemented correctly as and we do not handle users
such as IPsec setting cryptoff. It is simply ignored.

Implementing correctly is in fact more trouble than what it's
worth.

This patch removes the uses of cryptoff by moving the AD forward
to fill the gap left by the IV. The AD is moved back after the
underlying AEAD processing is finished.

This is in fact beter than the cryptoff solution because it allows
algorithms that use seqniv (i.e., GCM and CCM) to hash the whole
packet as a single piece, while cryptoff meant that there was
guaranteed to be a gap.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use aead_register_instance

New style AEAD instances must use aead_register_instance. This
worked by chance because aead_geniv_alloc is still setting things
the old way.

This patch converts the template over to the create model where
we are responsible for instance registration so that we can call
the correct function.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Add seqniv

This patch adds a new IV generator seqniv which is identical to
seqiv except that it skips the IV when authenticating. This is
intended to be used by algorithms such as rfc4106 that does the
IV authentication implicitly.

Note that the code used for seqniv is in fact identical to the
compatibility case for seqiv.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Add support for new AEAD interface

This patch converts the seqiv IV generator to work with the new
AEAD interface where IV generators are just normal AEAD algorithms.

Full backwards compatibility is paramount at this point since
no users have yet switched over to the new interface. Nor can
they switch to the new interface until IV generation is fully
supported by it.

So this means we are adding two versions of seqiv alongside the
existing one. The first one is the one that will be used when
the underlying AEAD algorithm has switched over to the new AEAD
interface. The second one handles the current case where the
underlying AEAD algorithm still uses the old interface.

Both versions export themselves through the new AEAD interface.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Use crypto_aead_set_reqsize helper

This patch uses the crypto_aead_set_reqsize helper to avoid directly
touching the internals of aead.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: seqiv - Ensure that IV size is at least 8 bytes

Since seqiv is designed for IPsec we need to be able to accomodate
the whole IPsec sequence number in order to ensure the uniqueness
of the IV.

This patch forbids any algorithm with an IV size of less than 8
from using it. This should have no impact on existing users since
they all have an IV size of 8.

Reported-by: Maciej ?enczykowski <zenczykowski@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Maciej ?enczykowski <zenczykowski@gmail.com>

crypto: include crypto- module prefix in template

This adds the module loading prefix "crypto-" to the template lookup
as well.

For example, attempting to load 'vfat(blowfish)' via AF_ALG now correctly
includes the "crypto-" prefix at every level, correctly rejecting "vfat":

net-pf-38
algif-hash
crypto-vfat(blowfish)
crypto-vfat(blowfish)-all
crypto-vfat

Reported-by: Mathias Krause <minipli@googlemail.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
Acked-by: Mathias Krause <minipli@googlemail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: Resolve shadow warnings

Change formal parameters to not clash with global names to
eliminate many W=2 warnings.

Signed-off-by: Mark Rustad <mark.d.rustad@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: use ERR_CAST

Replace PTR_ERR followed by ERR_PTR by ERR_CAST, to be more concise.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression err,x;
@@
- err = PTR_ERR(x);
if (IS_ERR(x))
- return ERR_PTR(err);
+ return ERR_CAST(x);
// </smpl>

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.

2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).

* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

crypto: skcipher - Use RNG interface instead of get_random_bytes

This patch makes the IV generators use the new RNG interface so
that the user can pick an RNG other than the default get_random_bytes.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

[CRYPTO] seqiv: Add AEAD support

This patch adds support for using seqiv with AEAD algorithms. This is
useful for those AEAD algorithms that performs authentication before
encryption because the IV generated by the underlying encryption algorithm
won't be available for authentication.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

[CRYPTO] seqiv: Add Sequence Number IV Generator

This generator generates an IV based on a sequence number by xoring it
with a salt. This algorithm is mainly useful for CTR and similar modes.

This patch also sets it as the default IV generator for ctr.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>