Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

sys/netgraph: spelling fixes in comments.

No functional change.

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten
years for head. However, it is continuously misused as the mpsafe argument
for callout_init(9). Deprecate the flag and clean up callout_init() calls
to make them more consistent.

Differential Revision: https://reviews.freebsd.org/D2613
Reviewed by: jhb
MFC after: 2 weeks

Refactor the name hash and the ID hash, that are used to address nodes:

- Make hash sizes growable, to satisfy users running large mpd
installations, having thousands of nodes.
- NG_NAMEHASH() proved to give a very bad distribution in real life
name sets, while generic hash32_str(name, HASHINIT) proved to give
an even one, so you the latter for name hash.
- Do not store unnamed nodes in slot 0 of name hash, no reason for that.
- Use the ID hash in cases when we need to run through all nodes: the
NGM_LISTNODES command and in the vnet_netgraph_uninit().
- Implement NGM_LISTNODES and NGM_LISTNAMES as separate code, the former
iterates through the ID hash, and the latter through the name hash.
- Keep count of all nodes and of named nodes, so that we don't need
to count nodes in NGM_LISTNODES and NGM_LISTNAMES. The counters are
also used to estimate whether we need to grow hashes.
- Close a race between two threads running ng_name_node() assigning same
name to different nodes.

Constify "address" argument of ng_address_path().

- If KDB & NETGRAPH_DEBUG are on, print traces on discovered failed
invariants.
- Reduce tautology in NETGRAPH_DEBUG output.

Fix build with NETGRAPH_DEBUG.

Fix build with NETGRAPH_DEBUG.

- Use refcount(9) API to manage node and hook refcounting.
- Make ng_unref_node() void, since caller shouldn't be
interested in whether node is valid after call or not,
since it can't be guaranteed to be valid. [1]

Ok from: julian [1]

Introduce and use a sysinit-based initialization scheme for virtual
network stacks, VNET_SYSINIT:

- Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will
occur each time a network stack is instantiated and destroyed. In the
!VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT.
For the VIMAGE case, we instead use SYSINIT's to track their order and
properties on registration, using them for each vnet when created/
destroyed, or immediately on module load for already-started vnets.
- Remove vnet_modinfo mechanism that existed to serve this purpose
previously, as well as its dependency scheme: we now just use the
SYSINIT ordering scheme.
- Implement VNET_DOMAIN_SET() to allow protocol domains to declare that
they want init functions to be called for each virtual network stack
rather than just once at boot, compiling down to DOMAIN_SET() in the
non-VIMAGE case.
- Walk all virtualized kernel subsystems and make use of these instead
of modinfo or DOMAIN_SET() for init/uninit events. In some cases,
convert modular components from using modevent to using sysinit (where
appropriate). In some cases, do minor rejuggling of SYSINIT ordering
to make room for or better manage events.

Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup)
Discussed with: jhb, bz, julian, zec
Reviewed by: bz
Approved by: re (VIMAGE blanket)

Build on Jeff Roberson's linker-set based dynamic per-CPU allocator
(DPCPU), as suggested by Peter Wemm, and implement a new per-virtual
network stack memory allocator. Modify vnet to use the allocator
instead of monolithic global container structures (vinet, ...). This
change solves many binary compatibility problems associated with
VIMAGE, and restores ELF symbols for virtualized global variables.

Each virtualized global variable exists as a "reference copy", and also
once per virtual network stack. Virtualized global variables are
tagged at compile-time, placing the in a special linker set, which is
loaded into a contiguous region of kernel memory. Virtualized global
variables in the base kernel are linked as normal, but those in modules
are copied and relocated to a reserved portion of the kernel's vnet
region with the help of a the kernel linker.

Virtualized global variables exist in per-vnet memory set up when the
network stack instance is created, and are initialized statically from
the reference copy. Run-time access occurs via an accessor macro, which
converts from the current vnet and requested symbol to a per-vnet
address. When "options VIMAGE" is not compiled into the kernel, normal
global ELF symbols will be used instead and indirection is avoided.

This change restores static initialization for network stack global
variables, restores support for non-global symbols and types, eliminates
the need for many subsystem constructors, eliminates large per-subsystem
structures that caused many binary compatibility issues both for
monitoring applications (netstat) and kernel modules, removes the
per-function INIT_VNET_*() macros throughout the stack, eliminates the
need for vnet_symmap ksym(2) munging, and eliminates duplicate
definitions of virtualized globals under VIMAGE_GLOBALS.

Bump __FreeBSD_version and update UPDATING.

Portions submitted by: bz
Reviewed by: bz, zec
Discussed with: gnn, jamie, jeff, jhb, julian, sam
Suggested by: peter
Approved by: re (kensmith)

Introduce a mechanism for detecting calls from outbound path of the
network stack when reentering the inbound path from netgraph, and
force queueing of mbufs at the outbound netgraph node.

The mechanism relies on two components. First, in netgraph nodes
where outbound path of the network stack calls into netgraph, the
current thread has to be appropriately marked using the new
NG_OUTBOUND_THREAD_REF() macro before proceeding to call further
into the netgraph topology, and unmarked using the
NG_OUTBOUND_THREAD_UNREF() macro before returning to the caller.
Second, netgraph nodes which can potentially reenter the network
stack in the inbound path have to mark their inbound hooks using
NG_HOOK_SET_TO_INBOUND() macro. The netgraph framework will then
detect when there is a danger of a call graph looping back from
outbound to inbound path via netgraph, and defer handing off the
mbufs to the "inbound" node to a worker thread with a clean stack.

In this first pass only the most obvious netgraph nodes have been
updated to ensure no outbound to inbound calls can occur. Nodes
such as ng_ipfw, ng_gif etc. should be further examined whether a
potential for outbound to inbound call looping exists.

This commit changes the layout of struct thread, but due to
__FreeBSD_version number shortage a version bump has been omitted
at this time, nevertheless kernel and modules have to be rebuilt.

Reviewed by: julian, rwatson, bz
Approved by: julian (mentor)

Change the curvnet variable from a global const struct vnet *,
previously always pointing to the default vnet context, to a
dynamically changing thread-local one. The currvnet context
should be set on entry to networking code via CURVNET_SET() macros,
and reverted to previous state via CURVNET_RESTORE(). Recursions
on curvnet are permitted, though strongly discuouraged.

This change should have no functional impact on nooptions VIMAGE
kernel builds, where CURVNET_* macros expand to whitespace.

The curthread->td_vnet (aka curvnet) variable's purpose is to be an
indicator of the vnet context in which the current network-related
operation takes place, in case we cannot deduce the current vnet
context from any other source, such as by looking at mbuf's
m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so
far curvnet has turned out to be an invaluable consistency checking
aid: it helps to catch cases when sockets, ifnets or any other
vnet-aware structures may have leaked from one vnet to another.

The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros
was a result of an empirical iterative process, whith an aim to
reduce recursions on CURVNET_SET() to a minimum, while still reducing
the scope of CURVNET_SET() to networking only operations - the
alternative would be calling CURVNET_SET() on each system call entry.
In general, curvnet has to be set in three typicall cases: when
processing socket-related requests from userspace or from within the
kernel; when processing inbound traffic flowing from device drivers
to upper layers of the networking stack, and when executing
timer-driven networking functions.

This change also introduces a DDB subcommand to show the list of all
vnet instances.

Approved by: julian (mentor)

In preparation to make options VIMAGE operational, where needed,
initialize / release netgraph related state in iattach() / idetach()
functions called via the vnet module registration / initialization
framework, instead of initialization / cleanups being done in
mod_event handlers.

While here, introduce a crude hack aimed at preventing ng_ether to
autoattach to ng_eiface ifnets, which are also netgraph nodes already.

Reviewed by: bz
Approved by: julian (mentor)

Revert rev. 183277:

Remove ng_rmnode_flags() function.
ng_rmnode_self() was made to be called only while having node locked.
When node is properly locked, any function call sent to it will always be
queued. So turning ng_rmnode_self() into the ng_rmnode_flags() is not just
meaningless, but incorrent, as it violates node locking when called outside.

No objections: julian, thompsa

Conditionally compile out V_ globals while instantiating the appropriate
container structures, depending on VIMAGE_GLOBALS compile time option.

Make VIMAGE_GLOBALS a new compile-time option, which by default will not
be defined, resulting in instatiations of global variables selected for
V_irtualization (enclosed in #ifdef VIMAGE_GLOBALS blocks) to be
effectively compiled out. Instantiate new global container structures
to hold V_irtualized variables: vnet_net_0, vnet_inet_0, vnet_inet6_0,
vnet_ipsec_0, vnet_netgraph_0, and vnet_gif_0.

Update the VSYM() macro so that depending on VIMAGE_GLOBALS the V_
macros resolve either to the original globals, or to fields inside
container structures, i.e. effectively

#ifdef VIMAGE_GLOBALS
#define V_rt_tables rt_tables
#else
#define V_rt_tables vnet_net_0._rt_tables
#endif

Update SYSCTL_V_*() macros to operate either on globals or on fields
inside container structs.

Extend the internal kldsym() lookups with the ability to resolve
selected fields inside the virtualization container structs. This
applies only to the fields which are explicitly registered for kldsym()
visibility via VNET_MOD_DECLARE() and vnet_mod_register(), currently
this is done only in sys/net/if.c.

Fix a few broken instances of MODULE_GLOBAL() macro use in SCTP code,
and modify the MODULE_GLOBAL() macro to resolve to V_ macros, which in
turn result in proper code being generated depending on VIMAGE_GLOBALS.

De-virtualize local static variables in sys/contrib/pf/net/pf_subr.c
which were prematurely V_irtualized by automated V_ prepending scripts
during earlier merging steps. PF virtualization will be done
separately, most probably after next PF import.

Convert a few variable initializations at instantiation to
initialization in init functions, most notably in ipfw. Also convert
TUNABLE_INT() initializers for V_ variables to TUNABLE_FETCH_INT() in
initializer functions.

Discussed at: devsummit Strassburg
Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Unhide declarations of network stack virtualization structs from
underneath #ifdef VIMAGE blocks.

This change introduces some churn in #include ordering and nesting
throughout the network stack and drivers but is not expected to cause
any additional issues.

In the next step this will allow us to instantiate the virtualization
container structures and switch from using global variables to their
"containerized" counterparts.

Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit

Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.

Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().

Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).

All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).

(*) netipsec/keysock.c did not validate depending on compile time options.

Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Add ng_rmnode_flags() so the caller can pass NG_QUEUE and have the node
destroyed asynchronously due to locking or other constraints.

Reviewed by: julian

Make it simpler to build netgraph modules outside of the kernel source
tree. This change follows similar ones in the device tree.

MFC after: 2 weeks

Several changes breaking netgraph module ABI collected together:
- reorder structures fields (XX_refs) a bit to group fields modified
same time together. According to my tests it gives up to 10%
SMP performance benefit on real workload due to reduced inter-CPU
cache trashing.
- change q_flags from long to int as long is not really needed there and
it's usage with atomics is argued by some people.
- move NGF_WORKQ flag into the separate field q_flags2 as it protected by
queue mutex instead of node writer protection used by the rest of flags.
- move nd_work queue entry to ng_queue structure to which it is more
related and make it STAILQ instead of TAILQ as now it is a classic FIFO.
- remove q_node pointer from ng_queue structure as it is not really needed.
- reimplement item queue using STAILQ instead of own equal implementation.
As soon as BT subsystem has own item queues using ng_item.el_next update
it also.
- change depth field in ng_item from uintptr_t to u_int. It was made
uintptr_t to keep ABI compatibility.

Reviewed by: julian, emax
Tested with: Netperf cluster

Improve apply callback error reporting:
Before this patch callback returned result of the last finished call chain.
Now it returns last nonzero result from all call chain results in this request.

As soon as this improvement gives reliable error reporting, it is now possible
to remove dirty workaround in ng_socket, made to return ENOBUFS error statuses
of request-response operations. That workaround was responsible for returning
ENOBUFS errors to completely unrelated requests working at the same time
on socket.

Implement stack protection based on GET_STACK_USAGE() macro.
This fixes system panics possible with complicated netgraph setups
and allows to avoid unneded extra queueing for stack unwrapping.

- Merge all the ng_send_fn2* functions into one - ng_send_fn2(),
removing some copy&pasted code.
- Reduce copy and paste in ng_apply_item().
- Resurrect ng_send_fn() as a valid symbol, not a define.

Reviewed by: mav, julian

Implement new apply callback mechanism to handle item forwarding.
When item forwarded refence counter is incremented, when item
processed, counter decremented. When counter reaches zero,
apply handler is getting called.
Now it allows to report right connect() call status from user-level
at the right time.

Replace references to NET_CALLOUT_MPSAFE with CALLOUT_MPSAFE, and remove
definition of NET_CALLOUT_MPSAFE, which is no longer required now that
debug.mpsafenet has been removed.

The once over: bz
Approved by: re (kensmith)

- ng_address_ID() has already freed the message, don't do double free.
- Get error from ng_address_ID().

Reported by: Coverity via pjd

Use NET_CALLOUT_MPSAFE for netgraph callout initializer.

Provide additional macros for sending netgraph items, which allow
to use non-default flags for netgraph functions. Implement current
macros via new ones.

Remove old debugging leftover.

Reviewed by: julian

Some whitespace and style cleanup.

Rework the ng_item queueing on nodes:
- Introduce a new flags NGQF_QREADER and NGQF_QWRITER,
which tell how the item should be actually applied,
overriding NGQF_READER/NGQF_WRITER flags.
- Do not differ between pending reader or writer. Use only
one flag that is raised, when there are pending items.
- Schedule netgraph ISR in ng_queue_rw(), so that callers
do not need to do this job.
- Fix several comments.

Submitted by: julian

Remove ng_callout_init_mtx() macro. Mutexed callouts are incompatible
with netgraph(4).

Add ng_callout_ini_mtx() macro.

reorder a couple of cases of void __inline to __inline void
to silence warnings from compilig with -W

Submitted by: sandvine inc
MFC after: 1 week

Fix up the comment.

Bump NG_ABI_VERSION. A number of ABI breakeges had been done since
RELENG_5 fork.

Enhance struct ng_hook - add hk_type field. This field will describe
data link type of the hook. It will be used to ease autoconfiguration
of netgraph and also to print warning messages, when incompatoble nodes
are connected together.

In the splnet times, netgraph was functional and synchronous. Nowadays,
an item may be queued and processed later. While this is OK for mbufs,
this is a problem for control messages.

In the framework:
- Add optional callback function pointer to an item. When item gets
applied the callback is executed from ng_apply_item().
- Add new flag NG_PROGRESS. If this flag is supplied, then return
EINPROGRESS instead of 0 in case if item failed to deliver
synchronously and was queued.
- Honor NG_PROGRESS in ng_snd_item().

In ng_socket:
- When userland sends control message add callback to the item.
- If ng_snd_item() returns EINPROGRESS, then sleep.

This change fixes possible races in ngctl(8) scripts.

Reviewed by: julian
Approved by: re (scottl)

Use NG_QUEUE instead of constant.

- Extend interface of ng_getqblk(), so that malloc wait flags are
specified by caller.
- Change ng_send_item() interface - use 'flags' argument instead of
boolean 'queue'.
- Extend ng_send_fn(), ng_package_data() and ng_package_msg()
interface - add possibility to pass flags. Rename ng_send_fn() to
ng_send_fn1(). Create macro for ng_send_fn().
- Update all macros, that use ng_package_data() and ng_package_msg().

Reviewed by: julian

Make NETGRAPH_DEBUG a kernel option, so that it can't be turned off
without hacking source.

In collaboration with: ru, julian

Use uma(9) for allocating netgraph items:
- ng_getqblk() simply runs uma_zalloc().
- ng_free_item() simply frees.
- ngq_mtx is pushed down under NETGRAPH_DEBUG.
- NGQF_FREE is removed.

Increase default maxalloc to 512.

Reviewed by: julian

Since there is no way to queue a function call to node, create
ng_queue_fn() - a queue version of ng_send_fn().

Plug item leak in macro NG_RESPOND_MSG. Item was leaked when destination
node couldn't be addressed.

Submitted by: Roselyn Lee

Make netgraph ISR and callout MPSAFE.

Reviewed by: rwatson, ru

/* -> /*- for license, minor formatting changes

Mechanically rename s/ng_timeout/ng_callout/g, s/ng_untimeout/ng_uncallout/g.
This is done to keep both versions in RELENG_5 and support both APIs.

Reviewed by: scottl
Approved by: julian (mentor), implicitly

- Make ng_timeout() to use callout() interface instead of timeout().
- Remove callout-hacking from ng_untimeout().

Approved by: julian (mentor)
MFC after: 1 month

Align netgraph message fields ready for 64-bit (and 128 bit :-) machines.
requires a recompile of netgraph users.
Also change the size of a field in the bluetooth code
that was waiting for the next change that needed recompiles so
it could piggyback its way in.

Submitted by: jdp, maksim
MFC after: 2 days

When making a peer of unknown type framework tries to load module
using linker_load_module(). This works OK if NGM_MKPEER message came
from userland and we have process associated with thread. But when
NGM_MKPEER was queued because target node was busy, linker_load_module()
is called from netisr thread leading to panic.
To workaround that we do not load modules by framework, instead ng_socket
loads module (if this is required) before sending NGM_MKPEER.
However, the race condition between return from NgSendMsg() and actual
creation of node still exist and needs to be solved.

PR: kern/62789
Approved by: julian

Slight cosmetic changes.
Also introduce a macro to be called by persistent nodes to signal their
persistence during shutdown to hide this mechanism from the node author.

Make node flags have a consistent style in naming.

Document the change.

Remove the home-grown metadata facility in favour of the now generic
mbuf tags facility. Netgraph modules will all need a recompile.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>

Convert Netgraph to use mbuf tags to pass its meta information around.
Thanks to Sam for importing tags in a way that allowed this to be done.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>
Also allow the sr and ar drivers to create netgraph versions of their modules.
Document the change to the ksocket node.

Add a new netgraph method to allow restoration of some
behaviour lost in the change from 4.x style netgraph tee nodes.
Alter the tee node to use the new method. Document the behaviour.

Step the ABI version number... old netgraph klds will refuse to load.
Better than just crashing.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>

Get rid of the deprecated *LEN constants and use the new *SIZ
(that include the trailing \0) constants instead.

Double length of node names, hook names, command strings and types. Add
defines for these constants that include the trailing NUL byte. These
new constants have SIZ in their name instead of LEN. As soon as all
consumers in the tree are converted to use the new defines the old
defines will be put under BURN_BRIDGES.

Reviewed by: archie, julian, ru
Approved by: re (in principle)

Use a single style of multiple inclusion protection for Netgraph headers.

Reviewed by: archie, harti, emax

Don't use "NULL" when "0" is really meant.

When declaring local variables in macros, always use "_name" instead
of "name" to avoid ugly problems when the containing code already has
a variable named "name".

Official timeout routines for netgraph nodes that know how to
use (and abuse) the node locking system.

MFC after: 1 week

Add an external function to unlink a netgraph type from the types list.

Catch up to header include changes:
- <sys/mutex.h> now requires <sys/systm.h>
- <sys/mutex.h> and <sys/sx.h> now require <sys/lock.h>

netgraph.h:
Change a prototype.
Add a function version of ng_ref_node() when debugging so
a breakpoint can be set on it.
ng_base.c:
add 'node' as an argument to ng_apply_item so that it is up
to the caller to take over and release the item's reference on
the node. If the release reports back that the node went away
due to the reference going to 0, the caller should cease referencing
the now defunct node. (e.g. the item was a 'kill node' message).
Alter ng_unref_node to report back the residual references as a result.
ng_pptpgre.c:
Don't reference a node after we dropped a reference to it.
(What if it was the last?)
Fixes a node leak reported by Harti Brandt <brandt@fokus.gmd.de>
which was due to an incorrect earlier attempt to fix the
"accessing node after dropping the last reference" problem.

Cleanups to Macros for sending data between netgraph nodes.

Add parenthesis to a macro.
This took me 2 whole days to track down. (bleah)

Shuffle sysctls a bit (thankyou whoever made them dynamic for modules)
and add a sysctl to pppoe to activate non standard ethertypes
so that idiot ISPs (apparently in France) who use
equipment from idiot suppliers (rumour says 3com)
who use nonstandard ethertypes can still connect.

"yep, sure we do pppoe, we use a different identifier to that dictated in
the standard, but sure it's pppoe!"

sysctl -w net.graph.stupid_isp=1 enables the changeover.

Make netgraph modules refuse to link with modules of a different ABI version.
also try implement teh documented behaviour in socket nodes
so that when there is only one hook, an unaddressed write/send
will DTRT and send the data to that hook.

Change the kernel internal ABI number as the HOOK structure has changed.
Forgotten by: me

Add the ability to declare ore-ride methods on a per-hook basis
for the rcvdata() and rcvmsg() methods.

Also bring the man page up to sync with my last commit. (and this one)

Implement direct support for semipersistant nodes.
(e.g. ethernet nodes are persistent until you rip out the hardware)
Use this support in the ethernet and sample nodes.
Add some more abstraction on the 'item's so that node and
hook reference counting can be checked easier.
Slight man page correction.
Make pppoe type dependent on ethernet type.
Clean up node shutdown a little.
Move a mutex from MTX_SPIN to MTX_DEF (oops)
Fix small ref-counting bug.
remove warning on one2many type.

Add a generic "queued function callin" mechanism
Use it to implement (hopefully) SMP safe node/hook addition
and removal.
Fix some debug stuff.

Add an exported function ng_rmhook_self() that removes a hook
from a node, but does it via the locking queue, thus ensuring that the
node is locked when it's hook is removed.

Add 'deadnode' and 'deadhook' structures for when a node or hook is
invalidated but not yet freed. (not yet freed)

Unbreak compilation.

Fix some memory leaks
Add memory leak detection assitance.

Part 2 of the netgraph rewrite.
This is mostly cosmetic changes, (though I caught a bug or two while
makeing them)
Reviewed by: archie@freebsd.org

Rewrite of netgraph to start getting ready for SMP.
This version is functional and is aproaching solid..
notice I said APROACHING. There are many node types I cannot test
I have tested: echo hole ppp socket vjc iface tee bpf async tty
The rest compile and "Look" right. More changes to follow.
DEBUGGING is enabled in this code to help if people have problems.

Divorce the kernel binary ABI version number from the message
format version number. (userland programs should not need to be
recompiled when the netgraph kernel internal ABI is changed.

Also fix modules that don;t handle the fact that a caller may not supply
a return message pointer. (benign at the moment because the calling code
checks, but that will change)

Reviewed by: Archie@freebsd.org
This clears out my outstanding netgraph changes.
There is a netgraph change of design in the offing and this is to some
extent a superset of soem of the new functionality and some of the old
functionality that may be removed.

This code works as before, but allows some new features that I want to
work with and evaluate. It is the basis for a version of netgraph
with integral locking for SMP use.

This is running on my test machine with no new problems :-)

Since neither archie nor I work at Whistle any more, change our email
addresses to be the more usefu @freebsd.org ones
so we can keep getting bug-reports.
- man pages to follow..

Back out the previous change to the queue(3) interface.
It was not discussed and should probably not happen.

Requested by: msmith and others

Change the way that the queue(3) structures are declared; don't assume that
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by: phk
Reviewed by: phk
Approved by: mdodd

Macro call to ng_send_dataq() should have been to ng_send_data() instead.

Use 'type_name' structure field instead of 'typename', which is
a C++ reserved work.

Add a ng_copy_meta() function.

Minimal tweak to make the ng_XXX modules depend on netgraph so that they
see its symbols and link ok.

Two simple changes to the kernel internal API for netgraph modules,
to support future work in flow-control and 'packet reject/replace'
processing modes.

reviewed by: phk, archie

Some minor prototype tweaks.

Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL"
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.

Add a new function ng_findhook() for finding a node's hook;
if the node type provides a more efficient implementation than
the normal linear scan, use it.

Reviewed by: julian

Add two new generic control messages, NGM_ASCII2BINARY and
NGM_BINARY2ASCII, which convert control messages to ASCII and back.
This allows control messages to be sent and received in ASCII form
using ngctl(8), which makes ngctl a lot more useful.

This also allows all the type-specific debugging code in libnetgraph
to go away -- instead, we just ask the node itself to do the ASCII
translation for us.

Currently, all generic control messages are supported, as well as
messages associated with the following node types: async, cisco,
ksocket, and ppp.

See /usr/share/examples/netgraph/ngctl for an example of using this.

Also give ngctl(8) the ability to print out incoming data and
control messages at any time. Eventually nghook(8) may be subsumed.

Several other misc. bug fixes.

Reviewed by: julian

Add typedefs for node methods
Suggested by phk.

Start making the contents of the generic framework opaque to the nodes.
This step: IDs are no-longer the address of the node.
Reviewd by: Archie@freebsd.org

Whistle's Netgraph link-layer (sometimes more) networking infrastructure.
Been in production for 3 years now. Gives Instant Frame relay to if_sr
and if_ar drivers, and PPPOE support soon. See:
ftp://ftp.whistle.com/pub/archie/netgraph/index.html
for on-line manual pages.

Reviewed by: Doug Rabson (dfr@freebsd.org)
Obtained from: Whistle CVS tree