lib: checksum: hide unused expected_csum_ipv6_magic[]

When CONFIG_NET is disabled, an extra warning shows up for this
unused variable:

lib/checksum_kunit.c:218:18: error: 'expected_csum_ipv6_magic' defined but not used [-Werror=unused-const-variable=]

Replace the #ifdef with an IS_ENABLED() check that makes the compiler's
dead-code-elimination take care of the link failure.

Fixes: f24a70106dc1 ("lib: checksum: Fix build with CONFIG_NET=n")
Suggested-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Simon Horman <horms@kernel.org>
Tested-by: Simon Horman <horms@kernel.org> # build-tested
Signed-off-by: David S. Miller <davem@davemloft.net>

kunit: Fix again checksum tests on big endian CPUs

Commit b38460bc463c ("kunit: Fix checksum tests on big endian CPUs")
fixed endianness issues with kunit checksum tests, but then
commit 6f4c45cbcb00 ("kunit: Add tests for csum_ipv6_magic and
ip_fast_csum") introduced new issues on big endian CPUs. Those issues
are once again reflected by the warnings reported by sparse.

So, fix them with the same approach, perform proper conversion in
order to support both little and big endian CPUs. Once the conversions
are properly done and the right types used, the sparse warnings are
cleared as well.

Reported-by: Erhard Furtner <erhard_f@mailbox.org>
Fixes: 6f4c45cbcb00 ("kunit: Add tests for csum_ipv6_magic and ip_fast_csum")
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Tested-by: Charlie Jenkins <charlie@rivosinc.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
Link: https://lore.kernel.org/r/73df3a9e95c2179119398ad1b4c84cdacbd8dfb6.1708684443.git.christophe.leroy@csgroup.eu
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

lib: checksum: Fix build with CONFIG_NET=n

The generic ipv6 checksums are only defined with CONFIG_NET=y, so gate
the test as well.

Fixes: 6f4c45cbcb00 ("kunit: Add tests for csum_ipv6_magic and ip_fast_csum")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401192143.jLdjbIy3-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202401192357.WU4nPRdN-lkp@intel.com/
Reviewed-By: Charlie Jenkins <charlie@rivosinc.com>
Link: https://lore.kernel.org/r/20240119145600.3093-2-palmer@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

kunit: Add tests for csum_ipv6_magic and ip_fast_csum

Supplement existing checksum tests with tests for csum_ipv6_magic and
ip_fast_csum.

Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
Link: https://lore.kernel.org/r/20240108-optimize_checksum-v15-5-1c50de5f2167@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

kunit: Fix checksum tests on big endian CPUs

On powerpc64le checksum kunit tests work:

[ 2.011457][ T1] KTAP version 1
[ 2.011662][ T1] # Subtest: checksum
[ 2.011848][ T1] 1..3
[ 2.034710][ T1] ok 1 test_csum_fixed_random_inputs
[ 2.079325][ T1] ok 2 test_csum_all_carry_inputs
[ 2.127102][ T1] ok 3 test_csum_no_carry_inputs
[ 2.127202][ T1] # checksum: pass:3 fail:0 skip:0 total:3
[ 2.127533][ T1] # Totals: pass:3 fail:0 skip:0 total:3
[ 2.127956][ T1] ok 1 checksum

But on powerpc64 and powerpc32 they fail:

[ 1.859890][ T1] KTAP version 1
[ 1.860041][ T1] # Subtest: checksum
[ 1.860201][ T1] 1..3
[ 1.861927][ T58] # test_csum_fixed_random_inputs: ASSERTION FAILED at lib/checksum_kunit.c:243
[ 1.861927][ T58] Expected result == expec, but
[ 1.861927][ T58] result == 54991 (0xd6cf)
[ 1.861927][ T58] expec == 33316 (0x8224)
[ 1.863742][ T1] not ok 1 test_csum_fixed_random_inputs
[ 1.864520][ T60] # test_csum_all_carry_inputs: ASSERTION FAILED at lib/checksum_kunit.c:267
[ 1.864520][ T60] Expected result == expec, but
[ 1.864520][ T60] result == 255 (0xff)
[ 1.864520][ T60] expec == 65280 (0xff00)
[ 1.868820][ T1] not ok 2 test_csum_all_carry_inputs
[ 1.869977][ T62] # test_csum_no_carry_inputs: ASSERTION FAILED at lib/checksum_kunit.c:306
[ 1.869977][ T62] Expected result == expec, but
[ 1.869977][ T62] result == 64515 (0xfc03)
[ 1.869977][ T62] expec == 0 (0x0)
[ 1.872060][ T1] not ok 3 test_csum_no_carry_inputs
[ 1.872102][ T1] # checksum: pass:0 fail:3 skip:0 total:3
[ 1.872458][ T1] # Totals: pass:0 fail:3 skip:0 total:3
[ 1.872791][ T1] not ok 3 checksum

This is because all expected values were calculated for X86 which
is little endian. On big endian systems all precalculated 16 bits
halves must be byte swapped.

And this is confirmed by a huge amount of sparse errors when building
with C=2

So fix all sparse errors and it will naturally work on all endianness.

Fixes: 688eb8191b47 ("x86/csum: Improve performance of `csum_partial`")
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: David S. Miller <davem@davemloft.net>

x86/csum: Improve performance of `csum_partial`

1) Add special case for len == 40 as that is the hottest value. The
nets a ~8-9% latency improvement and a ~30% throughput improvement
in the len == 40 case.

2) Use multiple accumulators in the 64-byte loop. This dramatically
improves ILP and results in up to a 40% latency/throughput
improvement (better for more iterations).

Results from benchmarking on Icelake. Times measured with rdtsc()
len lat_new lat_old r tput_new tput_old r
8 3.58 3.47 1.032 3.58 3.51 1.021
16 4.14 4.02 1.028 3.96 3.78 1.046
24 4.99 5.03 0.992 4.23 4.03 1.050
32 5.09 5.08 1.001 4.68 4.47 1.048
40 5.57 6.08 0.916 3.05 4.43 0.690
48 6.65 6.63 1.003 4.97 4.69 1.059
56 7.74 7.72 1.003 5.22 4.95 1.055
64 6.65 7.22 0.921 6.38 6.42 0.994
96 9.43 9.96 0.946 7.46 7.54 0.990
128 9.39 12.15 0.773 8.90 8.79 1.012
200 12.65 18.08 0.699 11.63 11.60 1.002
272 15.82 23.37 0.677 14.43 14.35 1.005
440 24.12 36.43 0.662 21.57 22.69 0.951
952 46.20 74.01 0.624 42.98 53.12 0.809
1024 47.12 78.24 0.602 46.36 58.83 0.788
1552 72.01 117.30 0.614 71.92 96.78 0.743
2048 93.07 153.25 0.607 93.28 137.20 0.680
2600 114.73 194.30 0.590 114.28 179.32 0.637
3608 156.34 268.41 0.582 154.97 254.02 0.610
4096 175.01 304.03 0.576 175.89 292.08 0.602

There is no such thing as a free lunch, however, and the special case
for len == 40 does add overhead to the len != 40 cases. This seems to
amount to be ~5% throughput and slightly less in terms of latency.

Testing:
Part of this change is a new kunit test. The tests check all
alignment X length pairs in [0, 64) X [0, 512).
There are three cases.
1) Precomputed random inputs/seed. The expected results where
generated use the generic implementation (which is assumed to be
non-buggy).
2) An input of all 1s. The goal of this test is to catch any case
a carry is missing.
3) An input that never carries. The goal of this test si to catch
any case of incorrectly carrying.

More exhaustive tests that test all alignment X length pairs in
[0, 8192) X [0, 8192] on random data are also available here:
https://github.com/goldsteinn/csum-reproduction

The reposity also has the code for reproducing the above benchmark
numbers.

Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/all/20230511011002.935690-1-goldstein.w.n%40gmail.com