MFC r312443:
mppc - Finish pluging NETGRAPH_MPPC_COMPRESSION.

There were several places where reference to compression were left
unfinished. Furthermore, KASSERTs contained references to MPPC_INVALID
which is not defined in the tree and therefore were sure to break with
INVARIANTS: comment them out.

Reported by: Eugene Grosbein
PR: 216265

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Pull in r267961 and r267973 again. Fix for issues reported will follow.

Revert r267961, r267973:

These changes prevent sysctl(8) from returning proper output,
such as:

1) no output from sysctl(8)
2) erroneously returning ENOMEM with tools like truss(1)
or uname(1)
truss: can not get etype: Cannot allocate memory

Extend the meaning of the CTLFLAG_TUN flag to automatically check if
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.

Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.

MFC after: 2 weeks
Sponsored by: Mellanox Technologies

Make ng_mppc to not disable the node in case of multiple packet loss.
Quite often it can be just packet reorder, and killing link in such case
is inconvenient. Add few sysctl's to control that behavior.

PR: kern/182212
Submitted by: Eugene Grosbein <egrosbein@rdtc.ru>
MFC after: 2 weeks

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags within sys.

Exceptions:

- sys/contrib not touched
- sys/mbuf.h edited manually

Revert my local not yet properly tested changes, that leaked in
with r235923.

Revert r220768 for ng_ksocket. This node is special and
when it is cloning, its constructor method may be called
in a context that isn't allowed to sleep.

Noticed by: Vadim Goncharov

Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

remove RESTARTABLE_PANICS option

This is done per request/suggestion from John Baldwin
who introduced the option. Trying to resume normal
system operation after a panic is very unpredictable
and dangerous. It will become even more dangerous
when we allow a thread in panic(9) to penetrate all
lock contexts.
I understand that the only purpose of this option was
for testing scenarios potentially resulting in panic.

Suggested by: jhb
Reviewed by: attilio, jhb
X-MFC-After: never
Approved by: re (kib)

Node constructor methods are supposed to be called in syscall
context always. Convert nodes to consistently use M_WAITOK flag
for memory allocation.

Reviewed by: julian

Remove some more alignment constraints.

If source mbuf chain consists of only one mbuf, use it directly as source
buffer to avoid extra copying.

Use m_unshare()+m_copyback() instead of m_freem()+m_devget() to keep
original mbuf chain headers. It can be less efficient in some cases, but it
looks better then mess of copying headers into the nonempty chain.

Unroll two loops of SHA1Update(). 60 bytes of static memory is not a price.

Carefully handle memory errors to keep peers compression/encryption state
consistent. There are some cases reported where peers fatally getting out
of sync without any visible reason. I hope this solve the problem.

Fix a number of style issues in the MALLOC / FREE commit. I've tried to
be careful not to fix anything that was already broken; the NFSv4 code is
particularly bad in this respect.

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Fix build with NETGRAPH_MPPC_COMPRESSION but without NETGRAPH_MPPC_ENCRYPTION.

Approved by: glebius (mentor)

Performance optimization of the "encryption without compression" case by
avoiding memory allocation and data copying.
Encrypting directly at the original mbuf chain.

Approved by: glebius (mentor)

Avoid extra rc4_init() when ng_mppc_updatekey() going to do it anyway.

Approved by: glebius (mentor)

Compact code a bit

Approved by: glebius (mentor)

Make coherency counter 12bit as it should

Approved by: glebius (mentor)

Fix small mistake (sizeof(pad2) instead of sizeof(pad1))

Approved by: glebius (mentor)

Remove unneded bzero().
SHA1Final() does not require clean buffer.

Approved by: glebius (mentor)

In preparation for making the modules actually use opt_*.h files
provided in the kernel build directory, fix modules that were
failing to build this way due to not quite correct kernel option
usage. In particular:

ng_mppc.c uses two complementary options, both of which are listed
in sys/conf/files. Ideally, there should be a separate option for
including ng_mppc.c in kernel build, but now only
NETGRAPH_MPPC_ENCRYPTION is usable anyway, the other one requires
proprietary files.

nwfs and smbfs were trying to ensure they were built with proper
network components, but the check was rather questionable.

Discussed with: ru

MPPC node is not thread safe.

PR: kern/79990
Reported by: Arcadiy Ivanov
Reported by: atckoe.zlo @ gmail.com
MFC after: 1 week

/* -> /*- for license, minor formatting changes

Switch to using C99 sparse initialisers for the type methods array.
Should make no binary difference.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>
Reviewed by: Harti Brandt <harti@freebsd.org>
MFC after: 1 week

Take the rc4 code out of ng_mppc module so we don't fail to load when
we have the rc4 code already in the kernel (via wlan stuff or awi).
Add a dependency on the rc4 module so if it doesn't exist then load it.

Reviewed by: archie

Fix two bugs:

(a) Save control message return address only if NGM_MPPC_CONFIG_DECOMP
(b) Properly count the number of required re-key operations
when we loose synchronization and have to resync

MFC after: 3 days

Don't use "NULL" when "0" is really meant.

o Move NTOHL() and associated macros into <sys/param.h>. These are
deprecated in favor of the POSIX-defined lowercase variants.
o Change all occurrences of NTOHL() and associated marcros in the
source tree to use the lowercase function variants.
o Add missing license bits to sparc64's <machine/endian.h>.
Approved by: jake
o Clean up <machine/endian.h> files.
o Remove unused __uint16_swap_uint32() from i386's <machine/endian.h>.
o Remove prototypes for non-existent bswapXX() functions.
o Include <machine/endian.h> in <arpa/inet.h> to define the
POSIX-required ntohl() family of functions.
o Do similar things to expose the ntohl() family in libstand, <netinet/in.h>,
and <sys/param.h>.
o Prepend underscores to the ntohl() family to help deal with
complexities associated with having MD (asm and inline) versions, and
having to prevent exposure of these functions in other headers that
happen to make use of endian-specific defines.
o Create weak aliases to the canonical function name to help deal with
third-party software forgetting to include an appropriate header.
o Remove some now unneeded pollution from <sys/types.h>.
o Add missing <arpa/inet.h> includes in userland.

Tested on: alpha, i386
Reviewed by: bde, jake, tmm

Add support for 56 bit MPPE encryption.

MFC after: 3 days

Update to C99, s/__FUNCTION__/__func__/,
also don't use ANSI string concatenation.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Fix some memory leaks
Add memory leak detection assitance.

Part 2 of the netgraph rewrite.
This is mostly cosmetic changes, (though I caught a bug or two while
makeing them)
Reviewed by: archie@freebsd.org

Rewrite of netgraph to start getting ready for SMP.
This version is functional and is aproaching solid..
notice I said APROACHING. There are many node types I cannot test
I have tested: echo hole ppp socket vjc iface tee bpf async tty
The rest compile and "Look" right. More changes to follow.
DEBUGGING is enabled in this code to help if people have problems.

Divorce the kernel binary ABI version number from the message
format version number. (userland programs should not need to be
recompiled when the netgraph kernel internal ABI is changed.

Also fix modules that don;t handle the fact that a caller may not supply
a return message pointer. (benign at the moment because the calling code
checks, but that will change)

Reviewed by: Archie@freebsd.org
This clears out my outstanding netgraph changes.
There is a netgraph change of design in the offing and this is to some
extent a superset of soem of the new functionality and some of the old
functionality that may be removed.

This code works as before, but allows some new features that I want to
work with and evaluate. It is the basis for a version of netgraph
with integral locking for SMP use.

This is running on my test machine with no new problems :-)

Add the use of M_ZERO to netgraph.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>
Submitted by: archie
Approved by: archie

Since neither archie nor I work at Whistle any more, change our email
addresses to be the more usefu @freebsd.org ones
so we can keep getting bug-reports.
- man pages to follow..

Remove unnecessary #include's as reported by phk's script.

Allocate all memory (including within node constructors) with M_NOWAIT
instead of M_WAITOK, to allow for maximum flexibility.

Allocate memory with M_NOWAIT instead of M_WAITOK because we could
be called in an interrupt context.

Two simple changes to the kernel internal API for netgraph modules,
to support future work in flow-control and 'packet reject/replace'
processing modes.

reviewed by: phk, archie

Remove ~25 unneeded #include <sys/conf.h>
Remove ~60 unneeded #include <sys/malloc.h>

A netgraph node that implements Microsoft Point-to-Point compression
(MPPC) and Microsoft Point-to-Point encryption (MPPE) protocols.

Note: the MPPC part is disabled as it requires proprietary files.

Obtained from: Whistle source tree