Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.

Register a new directory in common/lib/libc/misc

Registe misc/ with ubsan.c.

branches: 1.17.2; 1.17.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.

Register a new directory in common/lib/libc/misc

Registe misc/ with ubsan.c.

branches: 1.17.2; 1.17.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.

Register a new directory in common/lib/libc/misc

Registe misc/ with ubsan.c.

branches: 1.17.2;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

Register a new directory in common/lib/libc/misc

Registe misc/ with ubsan.c.

branches: 1.17.2;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU

Include compiler-rt in libc, libm and libkern.

Include bsd.own.mk in case it hasn't been yet.

Use the quad support from compiler-rt for MKLIBGCC=no.

branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@

branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.

branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.

Hook into sys

branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.

Hook the atomic ops in. Right now they are only built into libkern, and
only if available.

branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.

lack of COMMON_ARCHDIR is not fatal.

print a meaningful error rather than "Need an operator".

x86_64 uses i386 assembly files in standalone mode.

Include string in the cpp flags.

Merge libkern + libc common files. As requested by core.

sync with HEAD

sync with HEAD

Add build glue for i386 atomic ops.

this is shared with the kernel now.

file Makefile.inc was added on branch newlock2 on 2006-10-27 21:23:16 +0000

Sync w/ -current. 34 merge conflicts to follow.

sync with head.

Catchup with rmind-uvmplock merge.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")

sync with head

Rebase to HEAD as of a few days ago.