Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Simplify kvm symbol resolution and error handling. The symbol table
nl_symbols will eventually be organized into several modules depending
on MK_* variables.

Convert netstat to use libxo.

Obtained from: Phil Shafer <phil@juniper.net>
Ported to -current by: alfred@ (mostly), Kim Shrier
Formatting: marcel@
Sponsored by: Juniper Networks, Inc.

Merge from projects/sendfile:

o Introduce a notion of "not ready" mbufs in socket buffers. These
mbufs are now being populated by some I/O in background and are
referenced outside. This forces following implications:
- An mbuf which is "not ready" can't be taken out of the buffer.
- An mbuf that is behind a "not ready" in the queue neither.
- If sockbet buffer is flushed, then "not ready" mbufs shouln't be
freed.

o In struct sockbuf the sb_cc field is split into sb_ccc and sb_acc.
The sb_ccc stands for ""claimed character count", or "committed
character count". And the sb_acc is "available character count".
Consumers of socket buffer API shouldn't already access them directly,
but use sbused() and sbavail() respectively.
o Not ready mbufs are marked with M_NOTREADY, and ready but blocked ones
with M_BLOCKED.
o New field sb_fnrdy points to the first not ready mbuf, to avoid linear
search.
o New function sbready() is provided to activate certain amount of mbufs
in a socket buffer.

A special note on SCTP:
SCTP has its own sockbufs. Unfortunately, FreeBSD stack doesn't yet
allow protocol specific sockbufs. Thus, SCTP does some hacks to make
itself compatible with FreeBSD: it manages sockbufs on its own, but keeps
sb_cc updated to inform the stack of amount of data in them. The new
notion of "not ready" data isn't supported by SCTP. Instead, only a
mechanical substitute is done: s/sb_cc/sb_ccc/.
A proper solution would be to take away struct sockbuf from struct
socket and allow protocols to implement their own socket buffers, like
SCTP already does. This was discussed with rrs@.

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Use more fine-grained kvm(3) symbol lookup: routing code retrieves only
necessary symbols needed per subsystem. Main kvm(3) init is now delayed
as much as possbile. This finally fixes performance issues reported in
kern/167204.
Some non-working code (ng_socket.ko symbol addresses calculation) removed.
Some global variables eliminated.

PR: kern/167204
MFC after: 4 weeks

In ng_socket(4) expose less kernel internals to userland. This commit
breaks ABI, but makes probability of ABI breakage in future less.

Now fix functionality of 'netstat -f netgraph' that hasn't worked
starting from netgraph import in 1999.

netstat(8) used pointer to node as node address, oops. That didn't
work, we need the node ID in brackets to successfully address a node.
We can't look into ng_node, due to inability to include netgraph/netgraph.h
in userland code. So let the node make a hint for a userland, storing
the node ID in its private data.

MFC after: 2 weeks

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

style(9)
+ kread is not a boolean, so check it as such
+ fix $FreeBSD$ Ids
+ denote copyrights with /*-
+ misc whitespace changes.

Restore netstat -M functionality for most statistics on core dumps. In
general, when support was added to netstat for fetching data using sysctl,
no provision was left for fetching equivalent data from a core dump, and
in fact, netstat would _always_ fetch data from the live kernel using
sysctl even when -M was specified resulting in the user believing they
were getting data from coredumps when they actually weren't. Some specific
changes:
- Add a global 'live' variable that is true if netstat is running against
the live kernel and false if -M has been specified.
- Stop abusing the sysctl flag in the protocol tables to hold the protocol
number. Instead, the protocol is now its own field in the tables, and
it is passed as a separate parameter to the PCB and stat routines rather
than overloading the KVM offset parameter.
- Don't run PCB or stats functions who don't have a namelist offset if we
are being run against a crash dump (!live).
- For the inet and unix PCB routines, we generate the same buffer from KVM
that the sysctl usually generates complete with the header and trailer.
- Don't run bpf stats for !live (before it would just silently always run
live).
- kread() no longer trashes memory when opening the buffer if there is an
error on open and the passed in buffer is smaller than _POSIX2_LINE_MAX.
- The multicast routing code doesn't fallback to kvm on live kernels if
the sysctl fails. Keeping this made the code rather hairy, and netstat
is already tied to the kernel ABI anyway (even when using sysctl's since
things like xinpcb contain an inpcb) so any kernels this is run against
that have the multicast routing stuff should have the sysctls.
- Don't try to dig around in the kernel linker in the netgraph PCB routine
for core dumps.

Other notes:
- sctp's PCB routine only works on live kernels, it looked rather
complicated to generate all the same stuff via KVM. Someone can always
add it later if desired though.
- Fix the ipsec removal bug where N_xxx for IPSEC stats weren't renumbered.
- Use sysctlbyname() everywhere rather than hardcoded mib values.

MFC after: 1 week
Approved by: re (rwatson)

- Achieve WARNS=3 by using sparse initializers or avoiding initializers at all.
- Fix a nlist initialization: it should be terminated by a NULL entry.
- Constify.
- Catch an unused parameter.

Tested on: i386 amd64 ia64

Achieve WARNS=2 by using uintmax_t to pass around 64-bit quantities,
including to printf(). Using uintmax_t is also robust to further
extensions in both the C language and the bitwidth of kernel counters.

Tested on: i386 amd64 ia64

Add __FBSDID. Replace local variable sin by sockin to not conflict with sin(3).
Use warnx() instead of warn() when error message is not of any interest. Add
prototypes.

Warns cleanups for netstat:
1) Include arpa/inet.h for ntohs.
2) Constness fixes.
3) Fix shadowing except for "sin" which shouldn't be in scope.
4) Remove register keyword.
5) Add missing initialsers to user defined structs.
5) Make prototype of netname6 globally visable.
6) Use right macros for printing syncache stats (even though entrie isn't
a word).

Widen struct sockbuf's sb_timeo member to int from short. With
non-default but reasonable values of hz this member overflowed,
breaking NFS over UDP.

Also, as long as I'm plowing up struct sockbuf ... Change certain
members from u_long/long to u_int/int in order to reduce wasted
space on 64-bit machines. This change was requested by Andrew
Gallatin.

Netstat and systat need to be rebuilt. I am incrementing
__FreeBSD_version in case any ports need to change.

remove warnings
remove superfluous declarations
make things more consistent

revert removal of warning and K&R support

Requested by: bde

remove most of the warnings

Use macro API to <sys/queue.h>

Submitted by: "Jason" <jsmethers@pdq.net>
Reviewed by: phk

Put include <netgraph.h> back in - problem was due to a cvsup mess on
freefall. There may also be a real problem w/ buildworld but the fix
would go somewhere else, not here.

Cleanup missing includes, stale includes, and a few printf format
inconsistancies.

Whistle's Netgraph link-layer (sometimes more) networking infrastructure.
Been in production for 3 years now. Gives Instant Frame relay to if_sr
and if_ar drivers, and PPPOE support soon. See:
ftp://ftp.whistle.com/pub/archie/netgraph/index.html
for on-line manual pages.

Reviewed by: Doug Rabson (dfr@freebsd.org)
Obtained from: Whistle CVS tree