- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove most of the remaining sysctl name list macros. They were only
ever intended for use in sysctl(8) and it has not used them for many
years.

Reviewed by: bde
Tested by: exp-run by bdrewery

Remove the large part of struct ipsecstat. Only few fields of this
structure is used, but they already have equal fields in the struct
newipsecstat, that was introduced with FAST_IPSEC and then was merged
together with old ipsecstat structure.

This fixes kernel stack overflow on some architectures after migration
ipsecstat to PCPU counters.

Reported by: Taku YAMAMOTO, Maciej Milewski

Migrate structs ahstat, espstat, ipcompstat, ipipstat, pfkeystat,
ipsec4stat, ipsec6stat to PCPU counters.

Prepare network statistics structures for migration to PCPU counters.
Use uint64_t as type for all fields of structures.

Changed structures: ahstat, arpstat, espstat, icmp6_ifstat, icmp6stat,
in6_ifstat, ip6stat, ipcompstat, ipipstat, ipsecstat, mrt6stat, mrtstat,
pfkeystat, pim6stat, pimstat, rip6stat, udpstat.

Discussed with: arch@

Use corresponding macros to update statistics for AH, ESP, IPIP, IPCOMP,
PFKEY.

MFC after: 2 weeks

Use IPSECSTAT_INC() and IPSEC6STAT_INC() macros for ipsec statistics
accounting.

MFC after: 2 weeks

Use IP6STAT_INC/IP6STAT_DEC macros to update ip6 stats.

MFC after: 1 week

Use m_get2() + m_align() instead of hand made key_alloc_mbuf(). Code
examination shows, that although key_alloc_mbuf() could return chains,
the callers never use chains, so m_get2() should suffice.

Sponsored by: Nginx, Inc.

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags within sys.

Exceptions:

- sys/contrib not touched
- sys/mbuf.h edited manually

Do not reduce ip_len by size of IP header in the ip_input()
before passing a packet to protocol input routines.
For several protocols this mean that now protocol needs to
do subtraction itself, and for another half this means that
we do not need to add header length back to the packet.

Make ip_stripoptions() to adjust ip_len, since now we enter
this function with a packet header whose ip_len does represent
length of entire packet, not payload only.

- Fix one more miss from r241913.
- Add XXX comment about necessity of the entire block,
that "fixes up" the IP header.

Couple of changes missed from r241913, which converted
IPv4 stack to network byte order.

Switch the entire IPv4 stack to keep the IP packet header
in network byte order. Any host byte order processing is
done in local variables and host byte order values are
never[1] written to a packet.

After this change a packet processed by the stack isn't
modified at all[2] except for TTL.

After this change a network stack hacker doesn't need to
scratch his head trying to figure out what is the byte order
at the given place in the stack.

[1] One exception still remains. The raw sockets convert host
byte order before pass a packet to an application. Probably
this would remain for ages for compatibility.

[2] The ip_input() still subtructs header len from ip->ip_len,
but this is planned to be fixed soon.

Reviewed by: luigi, Maxim Dounin <mdounin mdounin.ru>
Tested by: ray, Olivier Cochard-Labbe <olivier cochard.me>

Mechanically remove the last stray remains of spl* calls from net*/*.
They have been Noop's for a long time now.

Add missing break

In NAT-T transport mode, allow a client to open a new connection just after
closing another.
It worked only in tunnel mode before.

Submitted by: Andreas Longwitz <longwitz@incore.de>
MFC after: 1M

Merge the projects/pf/head branch, that was worked on for last six months,
into head. The most significant achievements in the new code:

o Fine grained locking, thus much better performance.
o Fixes to many problems in pf, that were specific to FreeBSD port.

New code doesn't have that many ifdefs and much less OpenBSDisms, thus
is more attractive to our developers.

Those interested in details, can browse through SVN log of the
projects/pf/head branch. And for reference, here is exact list of
revisions merged:

r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330,
r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656,
r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782,
r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868,
r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223,
r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456,
r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505,
r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168,
r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230,
r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398,
r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548,
r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672,
r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169,
r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442,
r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522,
r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661,
r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212.

I'd like to thank people who participated in early testing:

Tested by: Florian Smeets <flo freebsd.org>
Tested by: Chekaluk Vitaly <artemrts ukr.net>
Tested by: Ben Wilber <ben desync.com>
Tested by: Ian FREISLICH <ianf cloudseed.co.za>

Unexpand a couple of TAILQ_FOREACH()s.

Fix a bug introduced in r221129 that leads to a panic wen using bundled
SAs. For now allow same address family bundles. While discovered with
ESP and AH, which does not make a lot of sense, IPcomp could be a possible
problematic candidate.

PR: kern/164400
MFC after: 3 days

Merge multi-FIB IPv6 support from projects/multi-fibv6/head/:

Extend the so far IPv4-only support for multiple routing tables (FIBs)
introduced in r178888 to IPv6 providing feature parity.

This includes an extended rtalloc(9) KPI for IPv6, the necessary
adjustments to the network stack, and user land support as in netstat.

Sponsored by: Cisco Systems, Inc.
Reviewed by: melifaro (basically)
MFC after: 10 days

Clean up some #endif comments removing from short sections. Add #endif
comments to longer, also refining strange ones.

Properly use #ifdef rather than #if defined() where possible. Four
#if defined(PCBGROUP) occurances (netinet and netinet6) were ignored to
avoid conflicts with eventually upcoming changes for RSS.

Reported by: bde (most)
Reviewed by: bde
MFC after: 3 days

Remove unused 'plen' variable.

The esp_max_ivlen global variable is not needed, we can just use
EALG_MAX_BLOCK_LEN.

malloc(M_WAITOK) never fails, so there is no need to check for NULL.

Eliminate 'err' variable and just use existing 'error'.

Simplify code a bit.

There is no need to virtualize esp_max_ivlen.

Add missing va_end() in an error case to clean up after va_start()
(already done in the non-error case).

CID: 4726
Found with: Coverity Prevent(tm)
MFC after: 1 week

Update packet filter (pf) code to OpenBSD 4.5.

You need to update userland (world and ports) tools
to be in sync with the kernel.

Submitted by: mlaier
Submitted by: eri

Release SP's refcount in key_get_spdbyid().

PR: 156676
Submitted by: Tobias Brunner (tobias@strongswan.org)
MFC after: 1 week

Make IPsec compile without INET adding appropriate #ifdef checks.

Unfold the IPSEC_COMMON_INPUT_CB() macro in xform_{ah,esp,ipcomp}.c
to not need three different versions depending on INET, INET6 or both.

Mark two places preparing for not yet supported functionality with IPv6.

Reviewed by: gnn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems
MFC after: 4 days

Do not allow recursive RFC3173 IPComp payload.

Reviewed by: Tavis Ormandy (taviso cmpxchg8b.com)
MFC after: 5 days
Security: CVE-2011-1547

Optimisation in IPSEC(4):
- Remove contention on ISR during the crypto operation by using rwlock(9).
- Remove a second lookup of the SA in the callback.

Gain on 6 cores CPU with SHA1/AES128 can be up to 30%.

Reviewed by: vanhu
MFC after: 1 month

Fix two SA refcount:
- AH does not release the SA like in ESP/IPCOMP when handling EAGAIN
- ipsec_process_done incorrectly release the SA.

Reviewed by: vanhu
MFC after: 1 week

Fixed IPsec's HMAC_SHA256-512 support to be RFC4868 compliant.
This will break interoperability with all older versions of
FreeBSD for those algorithms.

Reviewed by: bz, gnn
Obtained from: NETASQ
MFC after: 1w

After some off-list discussion, revert a number of changes to the
DPCPU_DEFINE and VNET_DEFINE macros, as these cause problems for various
people working on the affected files. A better long-term solution is
still being considered. This reversal may give some modules empty
set_pcpu or set_vnet sections, but these are harmless.

Changes reverted:

------------------------------------------------------------------------
r215318 | dim | 2010-11-14 21:40:55 +0100 (Sun, 14 Nov 2010) | 4 lines

Instead of unconditionally emitting .globl's for the __start_set_xxx and
__stop_set_xxx symbols, only emit them when the set_vnet or set_pcpu
sections are actually defined.

------------------------------------------------------------------------
r215317 | dim | 2010-11-14 21:38:11 +0100 (Sun, 14 Nov 2010) | 3 lines

Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.

------------------------------------------------------------------------
r215316 | dim | 2010-11-14 21:23:02 +0100 (Sun, 14 Nov 2010) | 2 lines

Add macros to define static instances of VNET_DEFINE and DPCPU_DEFINE.

Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.

Announce both IPsec and UDP Encap (NAT-T) if available for
feature_present(3) checks.

This will help to run-time detect and conditionally handle specific
optionas of either feature in user space (i.e. in libipsec).

Descriptions read by: rwatson
MFC after: 2 weeks

Fix typo in comment.

Make the IPsec SADB embedded route cache a union to be able to hold both the
legacy and IPv6 route destination address.
Previously in case of IPv6, there was a memory overwrite due to not enough
space for the IPv6 address.

PR: kern/122565
MFC After: 2 weeks

Remove dead code:
assignment to a local variable not used anywhere after that.

MFC after: 3 days

Style: make the asterisk go with the variable name, not the type.

MFC after: 3 days

MFp4 @178283:

Improve IPsec flow distribution for better netisr parallelism.
Instead of using the pointer that would have the last bits masked in a %
statement in netisr_select_cpuid() to select the queue, use the SPI.

Reviewed by: rwatson
MFC after: 4 weeks

Set SA's natt_type before calling key_mature() in key_add(),
as the SA may be used as soon as key_mature() has been done.

Obtained from: NETASQ
MFC after: 1 week

Update SA's NAT-T stuff before calling key_mature() in key_update(),
as SA may be used as soon as key_mature() has been called.

Obtained from: NETASQ
MFC after: 1 week

MFP4: @176978-176982, 176984, 176990-176994, 177441

"Whitspace" churn after the VIMAGE/VNET whirls.

Remove the need for some "init" functions within the network
stack, like pim6_init(), icmp_init() or significantly shorten
others like ip6_init() and nd6_init(), using static initialization
again where possible and formerly missed.

Move (most) variables back to the place they used to be before the
container structs and VIMAGE_GLOABLS (before r185088) and try to
reduce the diff to stable/7 and earlier as good as possible,
to help out-of-tree consumers to update from 6.x or 7.x to 8 or 9.

This also removes some header file pollution for putatively
static global variables.

Revert VIMAGE specific changes in ipfilter::ip_auth.c, that are
no longer needed.

Reviewed by: jhb
Discussed with: rwatson
Sponsored by: The FreeBSD Foundation
Sponsored by: CK Software GmbH
MFC after: 6 days

Locks SPTREE when setting some SP entries to state DEAD.
This can prevent kernel panics when updating SPs while
there is some traffic for them.

Obtained from: NETASQ
MFC after: 1m

Fix a logic error in ipsec code that extracts
information from the packets.

Reviewed by: bz, mlaier
Approved by: mlaier(mentor)
MFC after: 1 month

When tearing down IPsec as part of a (virtual) network stack,
do not try to free the same list twice but free both the
acquiring list and the security policy acquiring list.

Reviewed by: anchie
MFC after: 3 days

Correct typo in comment.

Enable IPcomp by default.

PR: kern/123587
MFC after: 5 days

Add more statistics variables for IPcomp.

Try to version the struct in a backward compatible way.
People asked for the versioning of the stats structs in general before.

MFC after: 5 days

Assimilate very similar input and output code paths
(no real functional change).

MFC after: 5 days

Only add the IPcomp header if crypto reported success and we have a lower
payload size. Before we had always added the header, no matter if we
actually send out compressed data or not.

With this, after the opencrypto/deflate changes, IPcomp starts to work
apart from edge cases. Leave it disabled by default until those are
fixed as well.

PR: kern/123587
MFC after: 5 days

Remove whitespace.

MFC after: 6 days

Directly send data uncompressed if the packet payload size is lower than
the compression algorithm threshold.

MFC after: 6 days

Correct a typo.

MFC after: 6 days

fixed two race conditions when inserting/removing SAs via PFKey,
which can both lead to a kernel panic when adding/removing quickly
a lot of SAs.

Obtained from: NETASQ
MFC after: 2w (MFC on 8 before 8.0 release ???)

Changed an IPSEC_ASSERT to a simple test, as such invalid packets
may come from outside without being discarded before.

Submitted by: aurelien.ansel@netasq.com
Reviewed by: bz (secteam)
Obtained from: NETASQ
MFC after: 1m

When checking traffic endpoint's adresses families in key_spdadd(),
compare them together instead of comparing each one with respective
tunnel endpoint.

PR: kern/138439
Submitted by: aurelien.ansel@netasq.com
Obtained from: NETASQ
MFC after: 1 m

Silent gcc? Yeah, you wish. What I ment was to silence gcc.

Spotted by: julian

Initialize state_valid and arraysize variable so gcc won't complain.

Reported by: bz

Improve code a bit by eliminating goto and having one unlock per lock.

Correct typo in comment.

Rework global locks for interface list and index management, correcting
several critical bugs, including race conditions and lock order issues:

Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an
sxlock. Either can be held to stablize the lists and indexes, but both
are required to write. This allows the list to be held stable in both
network interrupt contexts and sleepable user threads across sleeping
memory allocations or device driver interactions. As before, writes to
the interface list must occur from sleepable contexts.

Reviewed by: bz, julian
MFC after: 3 days

Merge the remainder of kern_vimage.c and vimage.h into vnet.c and
vnet.h, we now use jails (rather than vimages) as the abstraction
for virtualization management, and what remained was specific to
virtual network stacks. Minor cleanups are done in the process,
and comments updated to reflect these changes.

Reviewed by: bz
Approved by: re (vimage blanket)

Introduce and use a sysinit-based initialization scheme for virtual
network stacks, VNET_SYSINIT:

- Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will
occur each time a network stack is instantiated and destroyed. In the
!VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT.
For the VIMAGE case, we instead use SYSINIT's to track their order and
properties on registration, using them for each vnet when created/
destroyed, or immediately on module load for already-started vnets.
- Remove vnet_modinfo mechanism that existed to serve this purpose
previously, as well as its dependency scheme: we now just use the
SYSINIT ordering scheme.
- Implement VNET_DOMAIN_SET() to allow protocol domains to declare that
they want init functions to be called for each virtual network stack
rather than just once at boot, compiling down to DOMAIN_SET() in the
non-VIMAGE case.
- Walk all virtualized kernel subsystems and make use of these instead
of modinfo or DOMAIN_SET() for init/uninit events. In some cases,
convert modular components from using modevent to using sysinit (where
appropriate). In some cases, do minor rejuggling of SYSINIT ordering
to make room for or better manage events.

Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup)
Discussed with: jhb, bz, julian, zec
Reviewed by: bz
Approved by: re (VIMAGE blanket)

Garbage collect vnet module registrations that have neither constructors
nor destructors, as there's no actual work to do.

In most cases, the constructors weren't needed because of the existing
protocol initialization functions run by net_init_domain() as part of
VNET_MOD_NET, or they were eliminated when support for static
initialization of virtualized globals was added.

Garbage collect dependency references to modules without constructors or
destructors, notably VNET_MOD_INET and VNET_MOD_INET6.

Reviewed by: bz
Approved by: re (vimage blanket)

Reimplement and/or implement vnet list locking by replacing a mostly
unused custom mutex/condvar-based sleep locks with two locks: an
rwlock (for non-sleeping use) and sxlock (for sleeping use). Either
acquired for read is sufficient to stabilize the vnet list, but both
must be acquired for write to modify the list.

Replace previous no-op read locking macros, used in various places
in the stack, with actual locking to prevent race conditions. Callers
must declare when they may perform unbounded sleeps or not when
selecting how to lock.

Refactor vnet sysinits so that the vnet list and locks are initialized
before kernel modules are linked, as the kernel linker will use them
for modules loaded by the boot loader.

Update various consumers of these KPIs based on whether they may sleep
or not.

Reviewed by: bz
Approved by: re (kib)

Remove unused VNET_SET() and related macros; only VNET_GET() is
ever actually used. Rename VNET_GET() to VNET() to shorten
variable references.

Discussed with: bz, julian
Reviewed by: bz
Approved by: re (kensmith, kib)

Build on Jeff Roberson's linker-set based dynamic per-CPU allocator
(DPCPU), as suggested by Peter Wemm, and implement a new per-virtual
network stack memory allocator. Modify vnet to use the allocator
instead of monolithic global container structures (vinet, ...). This
change solves many binary compatibility problems associated with
VIMAGE, and restores ELF symbols for virtualized global variables.

Each virtualized global variable exists as a "reference copy", and also
once per virtual network stack. Virtualized global variables are
tagged at compile-time, placing the in a special linker set, which is
loaded into a contiguous region of kernel memory. Virtualized global
variables in the base kernel are linked as normal, but those in modules
are copied and relocated to a reserved portion of the kernel's vnet
region with the help of a the kernel linker.

Virtualized global variables exist in per-vnet memory set up when the
network stack instance is created, and are initialized statically from
the reference copy. Run-time access occurs via an accessor macro, which
converts from the current vnet and requested symbol to a per-vnet
address. When "options VIMAGE" is not compiled into the kernel, normal
global ELF symbols will be used instead and indirection is avoided.

This change restores static initialization for network stack global
variables, restores support for non-global symbols and types, eliminates
the need for many subsystem constructors, eliminates large per-subsystem
structures that caused many binary compatibility issues both for
monitoring applications (netstat) and kernel modules, removes the
per-function INIT_VNET_*() macros throughout the stack, eliminates the
need for vnet_symmap ksym(2) munging, and eliminates duplicate
definitions of virtualized globals under VIMAGE_GLOBALS.

Bump __FreeBSD_version and update UPDATING.

Portions submitted by: bz
Reviewed by: bz, zec
Discussed with: gnn, jamie, jeff, jhb, julian, sam
Suggested by: peter
Approved by: re (kensmith)

Add address list locking for in6_ifaddrhead/ia_link: as with locking
for in_ifaddrhead, we stick with an rwlock for the time being, which
we will revisit in the future with a possible move to rmlocks.

Some pieces of code require significant further reworking to be
safe from all classes of writer-writer races.

Reviewed by: bz
MFC after: 6 weeks

Add a new global rwlock, in_ifaddr_lock, which will synchronize use of the
in_ifaddrhead and INADDR_HASH address lists.

Previously, these lists were used unsynchronized as they were effectively
never changed in steady state, but we've seen increasing reports of
writer-writer races on very busy VPN servers as core count has gone up
(and similar configurations where address lists change frequently and
concurrently).

For the time being, use rwlocks rather than rmlocks in order to take
advantage of their better lock debugging support. As a result, we don't
enable ip_input()'s read-locking of INADDR_HASH until an rmlock conversion
is complete and a performance analysis has been done. This means that one
class of reader-writer races still exists.

MFC after: 6 weeks
Reviewed by: bz

Convert netinet6 to using queue(9) rather than hand-crafted linked lists
for the global IPv6 address list (in6_ifaddr -> in6_ifaddrhead). Adopt
the code styles and conventions present in netinet where possible.

Reviewed by: gnn, bz
MFC after: 6 weeks (possibly not MFCable?)

Move setting of ports from NAT-T below key_getsah() and actually
below key_setsaval().
Without that, the lookup for the SA had failed as we were looking for
a SA with the new, updated port numbers instead of the old ones and
were comparing the ports in key_cmpsaidx().
This makes updating the remote -> local SA on the initiator work again.

Problem introduced with: p4 changeset 152114

Add the explicit include of vimage.h to another five .c files still
missing it.

Remove the "hidden" kernel only include of vimage.h from ip_var.h added
with the very first Vimage commit r181803 to avoid further kernel poisoning.

Added support for NAT-Traversal (RFC 3948) in IPsec stack.

Thanks to (no special order) Emmanuel Dreyfus (manu@netbsd.org), Larry
Baird (lab@gta.com), gnn, bz, and other FreeBSD devs, Julien Vanherzeele
(julien.vanherzeele@netasq.com, for years of bug reporting), the PFSense
team, and all people who used / tried the NAT-T patch for years and
reported bugs, patches, etc...

X-MFC: never

Reviewed by: bz
Approved by: gnn(mentor)
Obtained from: NETASQ

Properly hide IPv4 only variables and functions under #ifdef INET.

After r193232 rt_tables in vnet.h are no longer indirectly dependent on
the ROUTETABLES kernel option thus there is no need to include opt_route.h
anymore in all consumers of vnet.h and no longer depend on it for module
builds.

Remove the hidden include in flowtable.h as well and leave the two
explicit #includes in ip_input.c and ip_output.c.

Introduce an infrastructure for dismantling vnet instances.

Vnet modules and protocol domains may now register destructor
functions to clean up and release per-module state. The destructor
mechanisms can be triggered by invoking "vimage -d", or a future
equivalent command which will be provided via the new jail framework.

While this patch introduces numerous placeholder destructor functions,
many of those are currently incomplete, thus leaking memory or (even
worse) failing to stop all running timers. Many of such issues are
already known and will be incrementaly fixed over the next weeks in
smaller incremental commits.

Apart from introducing new fields in structs ifnet, domain, protosw
and vnet_net, which requires the kernel and modules to be rebuilt, this
change should have no impact on nooptions VIMAGE builds, since vnet
destructors can only be called in VIMAGE kernels. Moreover,
destructor functions should be in general compiled in only in
options VIMAGE builds, except for kernel modules which can be safely
kldunloaded at run time.

Bump __FreeBSD_version to 800097.
Reviewed by: bz, julian
Approved by: rwatson, kib (re), julian (mentor)

Reimplement the netisr framework in order to support parallel netisr
threads:

- Support up to one netisr thread per CPU, each processings its own
workstream, or set of per-protocol queues. Threads may be bound
to specific CPUs, or allowed to migrate, based on a global policy.

In the future it would be desirable to support topology-centric
policies, such as "one netisr per package".

- Allow each protocol to advertise an ordering policy, which can
currently be one of:

NETISR_POLICY_SOURCE: packets must maintain ordering with respect to
an implicit or explicit source (such as an interface or socket).

NETISR_POLICY_FLOW: make use of mbuf flow identifiers to place work,
as well as allowing protocols to provide a flow generation function
for mbufs without flow identifers (m2flow). Falls back on
NETISR_POLICY_SOURCE if now flow ID is available.

NETISR_POLICY_CPU: allow protocols to inspect and assign a CPU for
each packet handled by netisr (m2cpuid).

- Provide utility functions for querying the number of workstreams
being used, as well as a mapping function from workstream to CPU ID,
which protocols may use in work placement decisions.

- Add explicit interfaces to get and set per-protocol queue limits, and
get and clear drop counters, which query data or apply changes across
all workstreams.

- Add a more extensible netisr registration interface, in which
protocols declare 'struct netisr_handler' structures for each
registered NETISR_ type. These include name, handler function,
optional mbuf to flow ID function, optional mbuf to CPU ID function,
queue limit, and ordering policy. Padding is present to allow these
to be expanded in the future. If no queue limit is declared, then
a default is used.

- Queue limits are now per-workstream, and raised from the previous
IFQ_MAXLEN default of 50 to 256.

- All protocols are updated to use the new registration interface, and
with the exception of netnatm, default queue limits. Most protocols
register as NETISR_POLICY_SOURCE, except IPv4 and IPv6, which use
NETISR_POLICY_FLOW, and will therefore take advantage of driver-
generated flow IDs if present.

- Formalize a non-packet based interface between interface polling and
the netisr, rather than having polling pretend to be two protocols.
Provide two explicit hooks in the netisr worker for start and end
events for runs: netisr_poll() and netisr_pollmore(), as well as a
function, netisr_sched_poll(), to allow the polling code to schedule
netisr execution. DEVICE_POLLING still embeds single-netisr
assumptions in its implementation, so for now if it is compiled into
the kernel, a single and un-bound netisr thread is enforced
regardless of tunable configuration.

In the default configuration, the new netisr implementation maintains
the same basic assumptions as the previous implementation: a single,
un-bound worker thread processes all deferred work, and direct dispatch
is enabled by default wherever possible.

Performance measurement shows a marginal performance improvement over
the old implementation due to the use of batched dequeue.

An rmlock is used to synchronize use and registration/unregistration
using the framework; currently, synchronized use is disabled
(replicating current netisr policy) due to a measurable 3%-6% hit in
ping-pong micro-benchmarking. It will be enabled once further rmlock
optimization has taken place. However, in practice, netisrs are
rarely registered or unregistered at runtime.

A new man page for netisr will follow, but since one doesn't currently
exist, it hasn't been updated.

This change is not appropriate for MFC, although the polling shutdown
handler should be merged to 7-STABLE.

Bump __FreeBSD_version.

Reviewed by: bz

Lock SPTREE before parsing it in key_spddump()

Approved by: gnn(mentor)
Obtained from: NETASQ
MFC after: 2 weeks

Only decrease refcnt once when flushing SPD entries, to
avoid flushing entries which are still used.

Approved by: gnn(mentor)
Obtained from: NETASQ
MFC after: 1 month

Add sysctls to toggle the behaviour of the (former) IPSEC_FILTERTUNNEL
kernel option.
This also permits tuning of the option per virtual network stack, as
well as separately per inet, inet6.

The kernel option is left for a transition period, marked deprecated,
and will be removed soon.

Initially requested by: phk (1 year 1 day ago)
MFC after: 4 weeks

Change the curvnet variable from a global const struct vnet *,
previously always pointing to the default vnet context, to a
dynamically changing thread-local one. The currvnet context
should be set on entry to networking code via CURVNET_SET() macros,
and reverted to previous state via CURVNET_RESTORE(). Recursions
on curvnet are permitted, though strongly discuouraged.

This change should have no functional impact on nooptions VIMAGE
kernel builds, where CURVNET_* macros expand to whitespace.

The curthread->td_vnet (aka curvnet) variable's purpose is to be an
indicator of the vnet context in which the current network-related
operation takes place, in case we cannot deduce the current vnet
context from any other source, such as by looking at mbuf's
m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so
far curvnet has turned out to be an invaluable consistency checking
aid: it helps to catch cases when sockets, ifnets or any other
vnet-aware structures may have leaked from one vnet to another.

The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros
was a result of an empirical iterative process, whith an aim to
reduce recursions on CURVNET_SET() to a minimum, while still reducing
the scope of CURVNET_SET() to networking only operations - the
alternative would be calling CURVNET_SET() on each system call entry.
In general, curvnet has to be set in three typicall cases: when
processing socket-related requests from userspace or from within the
kernel; when processing inbound traffic flowing from device drivers
to upper layers of the networking stack, and when executing
timer-driven networking functions.

This change also introduces a DDB subcommand to show the list of all
vnet instances.

Approved by: julian (mentor)

Make indentation more uniform accross vnet container structs.

This is a purely cosmetic / NOP change.

Reviewed by: bz
Approved by: julian (mentor)
Verified by: svn diff -x -w producing no output

Permit buiding kernels with options VIMAGE, restricted to only a single
active network stack instance. Turning on options VIMAGE at compile
time yields the following changes relative to default kernel build:

1) V_ accessor macros for virtualized variables resolve to structure
fields via base pointers, instead of being resolved as fields in global
structs or plain global variables. As an example, V_ifnet becomes:

options VIMAGE: ((struct vnet_net *) vnet_net)->_ifnet
default build: vnet_net_0._ifnet
options VIMAGE_GLOBALS: ifnet

2) INIT_VNET_* macros will declare and set up base pointers to be used
by V_ accessor macros, instead of resolving to whitespace:

INIT_VNET_NET(ifp->if_vnet); becomes

struct vnet_net *vnet_net = (ifp->if_vnet)->mod_data[VNET_MOD_NET];

3) Memory for vnet modules registered via vnet_mod_register() is now
allocated at run time in sys/kern/kern_vimage.c, instead of per vnet
module structs being declared as globals. If required, vnet modules
can now request the framework to provide them with allocated bzeroed
memory by filling in the vmi_size field in their vmi_modinfo structures.

4) structs socket, ifnet, inpcbinfo, tcpcb and syncache_head are
extended to hold a pointer to the parent vnet. options VIMAGE builds
will fill in those fields as required.

5) curvnet is introduced as a new global variable in options VIMAGE
builds, always pointing to the default and only struct vnet.

6) struct sysctl_oid has been extended with additional two fields to
store major and minor virtualization module identifiers, oid_v_subs and
oid_v_mod. SYSCTL_V_* family of macros will fill in those fields
accordingly, and store the offset in the appropriate vnet container
struct in oid_arg1.
In sysctl handlers dealing with virtualized sysctls, the
SYSCTL_RESOLVE_V_ARG1() macro will compute the address of the target
variable and make it available in arg1 variable for further processing.

Unused fields in structs vnet_inet, vnet_inet6 and vnet_ipfw have
been deleted.

Reviewed by: bz, rwatson
Approved by: julian (mentor)

Stub out IN6_LOOKUP_MULTI() for GETSPI requests, for now.

This has the effect that IPv6 multicast traffic won't trigger
an SPI allocation when IPSEC is in use, however, this obviously
needs to stomp on locks, and IN6_LOOKUP_MULTI() is about to go away.

This definitely needs to be revisited before 8.x is branched as
a release branch.

key_gettunnel() has been unsued with FAST_IPSEC (now IPSEC).
KAME had explicit checks at one point using it, so just hide it behind
#if 0 for now until we are sure if we can completely dump it or not.

MFC after: 1 month

Introduce vnet module registration / initialization framework with
dependency tracking and ordering enforcement.

With this change, per-vnet initialization functions introduced with
r190787 are no longer directly called from traditional initialization
functions (which cc in most cases inlined to pre-r190787 code), but are
instead registered via the vnet framework first, and are invoked only
after all prerequisite modules have been initialized. In the long run,
this framework should allow us to both initialize and dismantle
multiple vnet instances in a correct order.

The problem this change aims to solve is how to replay the
initialization sequence of various network stack components, which
have been traditionally triggered via different mechanisms (SYSINIT,
protosw). Note that this initialization sequence was and still can be
subtly different depending on whether certain pieces of code have been
statically compiled into the kernel, loaded as modules by boot
loader, or kldloaded at run time.

The approach is simple - we record the initialization sequence
established by the traditional mechanisms whenever vnet_mod_register()
is called for a particular vnet module. The vnet_mod_register_multi()
variant allows a single initializer function to be registered multiple
times but with different arguments - currently this is only used in
kern/uipc_domain.c by net_add_domain() with different struct domain *
as arguments, which allows for protosw-registered initialization
routines to be invoked in a correct order by the new vnet
initialization framework.

For the purpose of identifying vnet modules, each vnet module has to
have a unique ID, which is statically assigned in sys/vimage.h.
Dynamic assignment of vnet module IDs is not supported yet.

A vnet module may specify a single prerequisite module at registration
time by filling in the vmi_dependson field of its vnet_modinfo struct
with the ID of the module it depends on. Unless specified otherwise,
all vnet modules depend on VNET_MOD_NET (container for ifnet list head,
rt_tables etc.), which thus has to and will always be initialized
first. The framework will panic if it detects any unresolved
dependencies before completing system initialization. Detection of
unresolved dependencies for vnet modules registered after boot
(kldloaded modules) is not provided.

Note that the fact that each module can specify only a single
prerequisite may become problematic in the long run. In particular,
INET6 depends on INET being already instantiated, due to TCP / UDP
structures residing in INET container. IPSEC also depends on INET,
which will in turn additionally complicate making INET6-only kernel
configs a reality.

The entire registration framework can be compiled out by turning on the
VIMAGE_GLOBALS kernel config option.

Reviewed by: bz
Approved by: julian (mentor)

First pass at separating per-vnet initializer functions
from existing functions for initializing global state.

At this stage, the new per-vnet initializer functions are
directly called from the existing global initialization code,
which should in most cases result in compiler inlining those
new functions, hence yielding a near-zero functional change.

Modify the existing initializer functions which are invoked via
protosw, like ip_init() et. al., to allow them to be invoked
multiple times, i.e. per each vnet. Global state, if any,
is initialized only if such functions are called within the
context of vnet0, which will be determined via the
IS_DEFAULT_VNET(curvnet) check (currently always true).

While here, V_irtualize a few remaining global UMA zones
used by net/netinet/netipsec networking code. While it is
not yet clear to me or anybody else whether this is the right
thing to do, at this stage this makes the code more readable,
and makes it easier to track uncollected UMA-zone-backed
objects on vnet removal. In the long run, it's quite possible
that some form of shared use of UMA zone pools among multiple
vnets should be considered.

Bump __FreeBSD_version due to changes in layout of structs
vnet_ipfw, vnet_inet and vnet_net.

Approved by: julian (mentor)

Fixed comments so it stays in 80 chars by line
with hard tabs of 8 chars....

Approved by: gnn(mentor)

Spelling fix in a comment

Approved by: gnn(mentor)

Fixed style for some comments

Approved by: gnn(mentor)

Fixed style for some comments

Approved by: gnn(mentor)

Fixed deletion of sav entries in key_delsah()

Approved by: gnn(mentor)
Obtained from: NETASQ
MFC after: 1 month

SAs are valid (but dying) when they reached soft lifetime,
even if they have never been used.

Approved by: gnn(mentor)
MFC after: 2 weeks

Add size-guards evaluated at compile-time to the main struct vnet_*
which are not in a module of their own like gif.

Single kernel compiles and universe will fail if the size of the struct
changes. Th expected values are given in sys/vimage.h.
See the comments where how to handle this.

Requested by: peter

For all files including net/vnet.h directly include opt_route.h and
net/route.h.

Remove the hidden include of opt_route.h and net/route.h from net/vnet.h.

We need to make sure that both opt_route.h and net/route.h are included
before net/vnet.h because of the way MRT figures out the number of FIBs
from the kernel option. If we do not, we end up with the default number
of 1 when including net/vnet.h and array sizes are wrong.

This does not change the list of files which depend on opt_route.h
but we can identify them now more easily.

Shuffle the vimage.h includes or add where missing.

Change the functions to ANSI in those cases where it breaks promotion
to int rule. See ISO C Standard: SS6.7.5.3:15.

Approved by: kib (mentor)
Reviewed by: warner
Tested by: silence on -current

Try to remove/assimilate as much of formerly IPv4/6 specific
(duplicate) code in sys/netipsec/ipsec.c and fold it into
common, INET/6 independent functions.

The file local functions ipsec4_setspidx_inpcb() and
ipsec6_setspidx_inpcb() were 1:1 identical after the change
in r186528. Rename to ipsec_setspidx_inpcb() and remove the
duplicate.

Public functions ipsec[46]_get_policy() were 1:1 identical.
Remove one copy and merge in the factored out code from
ipsec_get_policy() into the other. The public function left
is now called ipsec_get_policy() and callers were adapted.

Public functions ipsec[46]_set_policy() were 1:1 identical.
Rename file local ipsec_set_policy() function to
ipsec_set_policy_internal().
Remove one copy of the public functions, rename the other
to ipsec_set_policy() and adapt callers.

Public functions ipsec[46]_hdrsiz() were logically identical
(ignoring one questionable assert in the v6 version).
Rename the file local ipsec_hdrsiz() to ipsec_hdrsiz_internal(),
the public function to ipsec_hdrsiz(), remove the duplicate
copy and adapt the callers.
The v6 version had been unused anyway. Cleanup comments.

Public functions ipsec[46]_in_reject() were logically identical
apart from statistics. Move the common code into a file local
ipsec46_in_reject() leaving vimage+statistics in small AF specific
wrapper functions. Note: unfortunately we already have a public
ipsec_in_reject().

Reviewed by: sam
Discussed with: rwatson (renaming to *_internal)
MFC after: 26 days
X-MFC: keep wrapper functions for public symbols?

Use NULL rather than 0 when comparing pointers.

MFC after: 2 weeks

Remove remain <= MHLEN restriction in m_makespace(),
which caused assert with big packets

PR: kern/124609
Submitted by: fabien.thomas@netasq.com
Approved by: gnn(mentor)
Obtained from: NetBSD
MFC after: 1 month

Switch the last protosw* structs to C99 initializers.

Reviewed by: ed, julian, Christoph Mallon <christoph.mallon@gmx.de>
MFC after: 2 weeks

Fix non-C99 initialization for protosw initializing pr_ousrreq.

Unlike with struct protosw, several instances of struct ip6protosw
did not use C99-style sparse structure initialization, so remove
NULL assignments for now-removed pr_usrreq function pointers.

Reported by: Chris Ruiz <yr.retarded at gmail.com>

Like in the rest of the file and the network stack use inp as
variable name for the inpcb.
For consistency with the other *_hdrsiz functions use 'size'
instead of 'siz' as variable name.

No functional change.

MFC after: 4 weeks

Non-functional (style) changes:
- Always use round brackets with return ().
- Add empty line to beginning of functions without local variables.
- Comments start with a capital letter and end in a '.'.
While there adapt a few comments.

Reviewed by: rwatson
MFC after: 4 weeks

Convert function definitions to constantly use ANSI-style
parameter declarations.

Reviewed by: rwatson
MFC after: 4 weeks

Rewrite ipsec6_setspidx_inpcb() to match the logic in the
(now) equivalent IPv4 counterpart.

MFC after: 4 weeks

For consistency with ipsec4_setspidx_inpcb() rename file local function
ipsec6_setspidx_in6pcb() to ipsec6_setspidx_inpcb().

MFC after: 4 weeks

Change the in6p variable names to inp to be able to diff
the v4 to the v6 implementations.

MFC after: 4 weeks

Make ipsec_getpolicybysock() static and no longer export it. It has not
been used outside this file since about the FAST_IPSEC -> IPSEC change.

MFC after: 4 weeks

Remove long unused netinet/ipprotosw.h (basically since r82884).

Discussed with: rwatson
MFC after: 4 weeks

Another step assimilating IPv[46] PCB code - directly use
the inpcb names rather than the following IPv6 compat macros:
in6pcb,in6p_sp, in6p_ip6_nxt,in6p_flowinfo,in6p_vflag,
in6p_flags,in6p_socket,in6p_lport,in6p_fport,in6p_ppcb and
sotoin6pcb().

Apart from removing duplicate code in netipsec, this is a pure
whitespace, not a functional change.

Discussed with: rwatson
Reviewed by: rwatson (version before review requested changes)
MFC after: 4 weeks (set the timer and see then)

Second round of putting global variables, which were virtualized
but formerly missed under VIMAGE_GLOBAL.

Put the extern declarations of the virtualized globals
under VIMAGE_GLOBAL as the globals themsevles are already.
This will help by the time when we are going to remove the globals
entirely.

Sponsored by: The FreeBSD Foundation

Conditionally compile out V_ globals while instantiating the appropriate
container structures, depending on VIMAGE_GLOBALS compile time option.

Make VIMAGE_GLOBALS a new compile-time option, which by default will not
be defined, resulting in instatiations of global variables selected for
V_irtualization (enclosed in #ifdef VIMAGE_GLOBALS blocks) to be
effectively compiled out. Instantiate new global container structures
to hold V_irtualized variables: vnet_net_0, vnet_inet_0, vnet_inet6_0,
vnet_ipsec_0, vnet_netgraph_0, and vnet_gif_0.

Update the VSYM() macro so that depending on VIMAGE_GLOBALS the V_
macros resolve either to the original globals, or to fields inside
container structures, i.e. effectively

#ifdef VIMAGE_GLOBALS
#define V_rt_tables rt_tables
#else
#define V_rt_tables vnet_net_0._rt_tables
#endif

Update SYSCTL_V_*() macros to operate either on globals or on fields
inside container structs.

Extend the internal kldsym() lookups with the ability to resolve
selected fields inside the virtualization container structs. This
applies only to the fields which are explicitly registered for kldsym()
visibility via VNET_MOD_DECLARE() and vnet_mod_register(), currently
this is done only in sys/net/if.c.

Fix a few broken instances of MODULE_GLOBAL() macro use in SCTP code,
and modify the MODULE_GLOBAL() macro to resolve to V_ macros, which in
turn result in proper code being generated depending on VIMAGE_GLOBALS.

De-virtualize local static variables in sys/contrib/pf/net/pf_subr.c
which were prematurely V_irtualized by automated V_ prepending scripts
during earlier merging steps. PF virtualization will be done
separately, most probably after next PF import.

Convert a few variable initializations at instantiation to
initialization in init functions, most notably in ipfw. Also convert
TUNABLE_INT() initializers for V_ variables to TUNABLE_FETCH_INT() in
initializer functions.

Discussed at: devsummit Strassburg
Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Rather than using hidden includes (with cicular dependencies),
directly include only the header files needed. This reduces the
unneeded spamming of various headers into lots of files.

For now, this leaves us with very few modules including vnet.h
and thus needing to depend on opt_route.h.

Reviewed by: brooks, gnn, des, zec, imp
Sponsored by: The FreeBSD Foundation

Unhide declarations of network stack virtualization structs from
underneath #ifdef VIMAGE blocks.

This change introduces some churn in #include ordering and nesting
throughout the network stack and drivers but is not expected to cause
any additional issues.

In the next step this will allow us to instantiate the virtualization
container structures and switch from using global variables to their
"containerized" counterparts.

Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Unify ipsec[46]_delete_pcbpolicy in ipsec_delete_pcbpolicy.
Ignoring different names because of macros (in6pcb, in6p_sp) and
inp vs. in6p variable name both functions were entirely identical.

Reviewed by: rwatson (as part of a larger changeset)
MFC after: 6 weeks (*)
(*) possibly need to leave a stub wrappers in 7 to keep the symbols.

Merge more of currently non-functional (i.e. resolving to
whitespace) macros from p4/vimage branch.

Do a better job at enclosing all instantiations of globals
scheduled for virtualization in #ifdef VIMAGE_GLOBALS blocks.

De-virtualize and mark as const saorder_state_alive and
saorder_state_any arrays from ipsec code, given that they are never
updated at runtime, so virtualizing them would be pointless.

Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Unbreak the build without INET6.

Change the initialization methodology for global variables scheduled
for virtualization.

Instead of initializing the affected global variables at instatiation,
assign initial values to them in initializer functions. As a rule,
initialization at instatiation for such variables should never be
introduced again from now on. Furthermore, enclose all instantiations
of such global variables in #ifdef VIMAGE_GLOBALS blocks.

Essentialy, this change should have zero functional impact. In the next
phase of merging network stack virtualization infrastructure from
p4/vimage branch, the new initialization methology will allow us to
switch between using global variables and their counterparts residing in
virtualization containers with minimum code churn, and in the long run
allow us to intialize multiple instances of such container structures.

Discussed at: devsummit Strassburg
Reviewed by: bz, julian
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit

Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.

Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().

Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).

All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).

(*) netipsec/keysock.c did not validate depending on compile time options.

Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).

This is the first in a series of commits over the course
of the next few weeks.

Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.

We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.

Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch

Increase statistic counters for enc0 interface when enabled
and processing IPSec traffic.

Approved by: gnn (mentor)
MFC after: 1 week

Add lifetime informations to generated SPD entries when SPDDUMP

Approved by: gnn (mentor)
MFC after: 4 weeks

Fill in a few sysctl descriptions.

Approved by: rwatson

Document a few sysctls. While here, remove dead code
related to ip4_esp_randpad.

Reviewed by: gnn, bz (older version)
Approved by: gnn
Tested with: make universe

Remove unused support for local and foreign addresses in generic raw
socket support. These utility routines are used only for routing and
pfkey sockets, neither of which have a notion of address, so were
required to mock up fake socket addresses to avoid connection
requirements for applications that did not specify their own fake
addresses (most of them).

Quite a bit of the removed code is #ifdef notdef, since raw sockets
don't support bind() or connect() in practice. Removing this
simplifies the raw socket implementation, and removes two (commented
out) uses of dtom(9).

Fake addresses passed to sendto(2) by applications are ignored for
compatibility reasons, but this is now done in a more consistent way
(and with a comment). Possibly, EINVAL could be returned here in
the future if it is determined that no applications depend on the
semantic inconsistency of specifying a destination address for a
protocol without address support, but this will require some amount
of careful surveying.

NB: This does not affect netinet, netinet6, or other wire protocol
raw sockets, which provide their own independent infrastructure with
control block address support specific to the protocol.

MFC after: 3 weeks
Reviewed by: bz

Enter the 1990s. Use real function declaration.

In addition to the ipsec_osdep.h removal a week ago, now also eliminate
IPSEC_SPLASSERT_SOFTNET which has been 'unused' since FreeBSD 5.0.

Remove last bits of OS adaptation code from the IPSec code.

Reviewed By: bz

Fix a bug that when getting/dumping the soft lifetime we reported
the hard lifetime instead.

MFC after: 3 days

Import change from KAME, rev. 1.362 kame/kame/sys/netkey/key.c

In case of "new SA", we must check the hard lifetime of the old SA
to find out if it is not permanent and we can delete it.

Submitted by: sakane via gnn
MFC after: 3 days

Add ';' missed with the SYSINIT changes.
Not noticed by tb as TCP_SIGNATURE is not in LINT.

MFC after: 1 month

In keeping with style(9)'s recommendations on macros, use a ';'
after each SYSINIT() macro invocation. This makes a number of
lightweight C parsers much happier with the FreeBSD kernel
source, including cflow's prcc and lxr.

MFC after: 1 month
Discussed with: imp, rink

Correct IPsec behaviour with a 'use' level in SP but no SA available.
In that case return an continue processing the packet without IPsec.

PR: 121384
MFC after: 5 days
Reported by: Cyrus Rahman (crahman gmail.com)
Tested by: Cyrus Rahman (crahman gmail.com) [slightly older version]

Remove the "Fast " from the
"Fast IPsec: Initialized Security Association Processing." printf.
People kept asking questions about this after the IPsec shuffle.

This still is the Fast IPsec implementation so no worries that it would
be any slower now. There are no functional changes.

Discussed with: sam
MFC after: 4 days

Fix bugs when allocating and passing information of current lifetime and
soft lifetime [1] introduced in rev. 1.21 of key.c.

Along with that, fix a related problem in key_debug
printing the correct data.
While there replace a printf by panic in a sanity check.

PR: 120751
Submitted by: Kazuaki ODA (kazuaki aliceblue.jp) [1]
MFC after: 5 days

Rather than passing around a cached 'priv', pass in an ucred to
ipsec*_set_policy and do the privilege check only if needed.

Try to assimilate both ip*_ctloutput code blocks calling ipsec*_set_policy.

Reviewed by: rwatson

Add sysctls to if_enc(4) to control whether the firewalls or
bpf will see inner and outer headers or just inner or outer
headers for incoming and outgoing IPsec packets.

This is useful in bpf to not have over long lines for debugging
or selcting packets based on the inner headers.
It also properly defines the behavior of what the firewalls see.

Last but not least it gives you if_enc(4) for IPv6 as well.

[ As some auxiliary state was not available in the later
input path we save it in the tdbi. That way tcpdump can give a
consistent view of either of (authentic,confidential) for both
before and after states. ]

Discussed with: thompsa (2007-04-25, basic idea of unifying paths)
Reviewed by: thompsa, gnn

Adjust a comment that suggest that we might consider a panic.

Make clear that this is not a good idea when called from
tcp_output()->ipsec_hdrsiz_tcp()->ipsec4_hdrsize_tcp()
as we do not know if IPsec processing is needed at that point.

Move the priv check before the malloc call for so_pcb.
In case attach fails because of the priv check we leaked the
memory and left so_pcb as fodder for invariants.

Reported by: Pawel Worach
Reviewed by: rwatson

Add a missing priv check in key_attach to prevent non-su users
from messing with the spdb and sadb.

Problem sneaked in with the fast_ipsec+v6->ipsec merger by no
longer going via raw_usrreqs.pr_attach.

Reported by: Pawel Worach
Identified by: rwatson
Reviewed by: rwatson
MFC after: 3 days

Fix for an infinite loop in processing ESP, IPv6 packets.

The control input routine passes a NULL as its void argument when it
has reached the innermost header, which terminates the loop.

Reported by: Pawel Worach <pawel.worach@gmail.com>
Approved by: re

Remove the now-unused NET_{LOCK,UNLOCK,ASSERT}_GIANT() macros, which
previously conditionally acquired Giant based on debug.mpsafenet. As that
has now been removed, they are no longer required. Removing them
significantly simplifies error-handling in the socket layer, eliminated
quite a bit of unwinding of locking in error cases.

While here clean up the now unneeded opt_net.h, which previously was used
for the NET_WITH_GIANT kernel option. Clean up some related gotos for
consistency.

Reviewed by: bz, csjp
Tested by: kris
Approved by: re (kensmith)

Replace hard coded options by their defined PFIL_{IN,OUT} names.

Approved by: re (hrs)

Commit the change from FAST_IPSEC to IPSEC. The FAST_IPSEC
option is now deprecated, as well as the KAME IPsec code.
What was FAST_IPSEC is now IPSEC.

Approved by: re
Sponsored by: Secure Computing

Commit IPv6 support for FAST_IPSEC to the tree.
This commit includes only the kernel files, the rest of the files
will follow in a second commit.

Reviewed by: bz
Approved by: re
Supported by: Secure Computing

'spi' and the return value of ntohl are unsigned. Remove the extra >=0
check which was always true.
Document the special meaning of spi values of 0 and 1-255 with a comment.

Found with: Coverity Prevent(tm)
CID: 2047

In case of failure we can directly return ENOBUFS because
'result' is still NULL and we do not need to free anything.
That allows us to gc the entire goto parts and a now unused variable.

Found with: Coverity Prevent(tm)
CID: 2519

Add a missing return so that we drop out in case of an error and
do not continue with a NULL pointer. [1]

While here change the return of the error handling code path above.
I cannot see why we should always return 0 there. Neither does KAME
nor do we in here for the similar check in all the other functions.

Found with: Coverity Prevent(tm) [1]
CID: 2521

With the current code 'src' is never NULL. Nevertheless move the check for
NULL before dereferencing the pointer.

Found with: Coverity Prevent(tm)
CID: 2528

Looking at {ah,esp}_input_cb it seems we might be able to end up
without an mtag in ipsec4_common_input_cb.
So in case of !IPCOMP (AH,ESP) only change the m_tag_id if an mtag
was passed to ipsec4_common_input_cb.

Found with: Coverity Prevent(tm)
CID: 2523

s,#,*, in a multi-line comment. This is C.
No functional change.

Though we are only called for the three security protocols we can
handle, document those sprotos using an IPSEC_ASSERT so that it will
be clear that 'spi' will always be initialized when used the first time.

Found with: Coverity Prevent(tm)
CID: 2533

Eliminate now-unused SUSER_ALLOWJAIL arguments to priv_check_cred(); in
some cases, move to priv_check() if it was an operation on a thread and
no other flags were present.

Eliminate caller-side jail exception checking (also now-unused); jail
privilege exception code now goes solely in kern_jail.c.

We can't yet eliminate suser() due to some cases in the KAME code where
a privilege check is performed and then used in many different deferred
paths. Do, however, move those prototypes to priv.h.

Reviewed by: csjp
Obtained from: TrustedBSD Project

In ipsec6_output_tunnel() make sure that the SA contents do not change.

The same would apply to ipsec6_output_trans() but there is a larger patch
around which already corrected that case. Do not interfere with that one.

fix typo: s,applyed,applied,g

Implement ICMPv6 support in ipsec6_get_ulp().
This is needed to make security policies work correctly if ICMPv6 type
and/or code are given. See setkey(8) 'upperspec' para. for details.

Add missing
break;
so when comparing AF_INET6 addresses, scope and ports we do not run into
the default case and return 'no match' instead of 'match'.

Integrate the Camellia Block Cipher. For more information see RFC 4132
and its bibliography.

Submitted by: Tomoyuki Okazaki <okazaki at kick dot gr dot jp>
MFC after: 1 month

Update comment regarding how we check privilege on FreeBSD: we now use
priv_check().

add include now required for crypto flags

Overhaul driver/subsystem api's:
o make all crypto drivers have a device_t; pseudo drivers like the s/w
crypto driver synthesize one
o change the api between the crypto subsystem and drivers to use kobj;
cryptodev_if.m defines this api
o use the fact that all crypto drivers now have a device_t to add support
for specifying which of several potential devices to use when doing
crypto operations
o add new ioctls that allow user apps to select a specific crypto device
to use (previous ioctls maintained for compatibility)
o overhaul crypto subsystem code to eliminate lots of cruft and hide
implementation details from drivers
o bring in numerous fixes from Michale Richardson/hifn; mostly for
795x parts
o add an optional mechanism for mmap'ing the hifn 795x public key h/w
to user space for use by openssl (not enabled by default)
o update crypto test tools to use new ioctl's and add cmd line options
to specify a device to use for tests

These changes will also enable much future work on improving the core
crypto subsystem; including proper load balancing and interposing code
between the core and drivers to dispatch small operations to the s/w
driver as appropriate.

These changes were instigated by the work of Michael Richardson.

Reviewed by: pjd
Approved by: re

s,#if INET6,#ifdef INET6,
This unbreaks the build for FAST_IPSEC && !INET6 and was wrong anyway.

Reported by: Dmitry Pryanishnikov <dmitry atlantis.dp.ua>

MFp4: 92972, 98913 + one more change

In ip6_sprintf no longer use and return one of eight static buffers
for printing/logging ipv6 addresses.
The caller now has to hand in a sufficiently large buffer as first
argument.

Add priv.h include required to build FAST_IPSEC, which is not present in
LINT due to a conflict with KAME IPSEC.

Submitted by: Pawel Worach <pawel dot worach at gmail dot com>

Sweep kernel replacing suser(9) calls with priv(9) calls, assigning
specific privilege names to a broad range of privileges. These may
require some future tweaking.

Sponsored by: nCircle Network Security, Inc.
Obtained from: TrustedBSD Project
Discussed on: arch@
Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri,
Alex Lyashkov <umka at sevcity dot net>,
Skip Ford <skip dot ford at verizon dot net>,
Antoine Brodin <antoine dot brodin at laposte dot net>

Fix build breakage from previous commit which confused key_abort and key_close.

Change semantics of socket close and detach. Add a new protocol switch
function, pru_close, to notify protocols that the file descriptor or
other consumer of a socket is closing the socket. pru_abort is now a
notification of close also, and no longer detaches. pru_detach is no
longer used to notify of close, and will be called during socket
tear-down by sofree() when all references to a socket evaporate after
an earlier call to abort or close the socket. This means detach is now
an unconditional teardown of a socket, whereas previously sockets could
persist after detach of the protocol retained a reference.

This faciliates sharing mutexes between layers of the network stack as
the mutex is required during the checking and removal of references at
the head of sofree(). With this change, pru_detach can now assume that
the mutex will no longer be required by the socket layer after
completion, whereas before this was not necessarily true.

Reviewed by: gnn

- Use suser_cred(9) instead of directly comparing cr_uid.
- Compare pointer with NULL.

Reviewed by: rwatson

Add a pseudo interface for packet filtering IPSec connections before or after
encryption. There are two functions, a bpf tap which has a basic header with
the SPI number which our current tcpdump knows how to display, and handoff to
pfil(9) for packet filtering.

Obtained from: OpenBSD
Based on: kern/94829
No objections: arch, net
MFC after: 1 month

Change '#if INET' and '#if INET6' to '#ifdef INET' and '#ifdef INET6'.
This unbreaks compiling a kernel with FAST_IPSEC and no INET6.

Extend the notdef #ifdef to cover the packet copy as there is no point in doing that if we're not doing the rest of the work.

Submitted by: thompsa
MFC after: 1 week

Prevent disappearing SAD entries by implementing MPsafe refcounting.
"Why didn't he use SECASVAR_LOCK()/SECASVAR_UNLOCK() macros to
synchronize access to the secasvar structure's fields?" one may ask.
There were two reasons:
1. refcount(9) is faster then mutex(9) synchronization (one atomic
operation instead of two).
2. Those macros are not used now at all, so at some point we may decide
to remove them entirely.

OK'ed by: gnn
MFC after: 2 weeks

- The authsize field from auth_hash structure was removed.
- Define that we want to receive only 96 bits of HMAC.
- Names of the structues have no longer _96 suffix.

Reviewed by: sam

Hide net.inet.ipsec.test_{replay,integrity} sysctls under #ifdef REGRESSION.

Requested by: sam, rwatson

Introduce two new sysctls:

net.inet.ipsec.test_replay - When set to 1, IPsec will send packets with
the same sequence number. This allows to verify if the other side
has proper replay attacks detection.

net.inet.ipsec.test_integrity - When set 1, IPsec will send packets with
corrupted HMAC. This allows to verify if the other side properly
detects modified packets.

I used the first one to discover that we don't have proper replay attacks
detection in ESP (in fast_ipsec(4)).

Be consistent with the rest of the code.

Remove unused variables s and error in key_detach. The previous
revision removed their usage but did not remove the declaration. This
caused a warning in my build, which was fatal with -Werror.

Remove unintended DEBUG flag setting.

Chance protocol switch method pru_detach() so that it returns void
rather than an error. Detaches do not "fail", they other occur or
the protocol flags SS_PROTOREF to take ownership of the socket.

soclose() no longer looks at so_pcb to see if it's NULL, relying
entirely on the protocol to decide whether it's time to free the
socket or not using SS_PROTOREF. so_pcb is now entirely owned and
managed by the protocol code. Likewise, no longer test so_pcb in
other socket functions, such as soreceive(), which have no business
digging into protocol internals.

Protocol detach routines no longer try to free the socket on detach,
this is performed in the socket code if the protocol permits it.

In rts_detach(), no longer test for rp != NULL in detach, and
likewise in other protocols that don't permit a NULL so_pcb, reduce
the incidence of testing for it during detach.

netinet and netinet6 are not fully updated to this change, which
will be in an upcoming commit. In their current state they may leak
memory or panic.

MFC after: 3 months

Change protocol switch pru_abort() API so that it returns void rather
than an int, as an error here is not meaningful. Modify soabort() to
unconditionally free the socket on the return of pru_abort(), and
modify most protocols to no longer conditionally free the socket,
since the caller will do this.

This commit likely leaves parts of netinet and netinet6 in a situation
where they may panic or leak memory, as they have not are not fully
updated by this commit. This will be corrected shortly in followup
commits to these components.

MFC after: 3 months

Fix more stack corruptions on amd64.

Vararg functions have a different calling convention than regular
functions on amd64. Casting a varag function to a regular one to
match the function pointer declaration will hide the varargs from
the caller and we will end up with an incorrectly setup stack.

Entirely remove the varargs from these functions and change the
functions to match the declaration of the function pointers.
Remove the now unnecessary casts.

Also change static struct ipprotosw[] to two independent
protosw/ip6protosw definitions to remove an unnecessary cast.

PR: amd64/95008
Submitted and tested by: Mats Palmgren
Reviewed by: rwatson
MFC after: 3 days

First steps towards IPSec cleanup.

Make the kernel side of FAST_IPSEC not depend on the shared
structures defined in /usr/include/net/pfkeyv2.h The kernel now
defines all the necessary in kernel structures in sys/netipsec/keydb.h
and does the proper massaging when moving messages around.

Sponsored By: Secure Computing

Allow to use fast_ipsec(4) on debug.mpsafenet=0 and INVARIANTS-enabled
systems. Without the change it will panic on assertions.

MFC after: 2 weeks

Add missing code needed for the detection of IPSec packet replays. [1]

Correctly identify the user running opiepasswd(1) when the login name
differs from the account name. [2]

Security: FreeBSD-SA-06:11.ipsec [1]
Security: FreeBSD-SA-06:12.opie [2]

promote fast ipsec's m_clone routine for public use; it is renamed
m_unshare and the caller can now control how mbufs are allocated

Reviewed by: andre, luigi, mlaier
MFC after: 1 week

Fix stack corruptions on amd64.

Vararg functions have a different calling convention than regular
functions on amd64. Casting a varag function to a regular one to
match the function pointer declaration will hide the varargs from
the caller and we will end up with an incorrectly setup stack.

Entirely remove the varargs from these functions and change the
functions to match the declaration of the function pointers.
Remove the now unnecessary casts.

Lots of explanations and help from: peter
Reviewed by: peter
PR: amd64/89261
MFC after: 6 days

Fix -Wundef warnings found when compiling i386 LINT, GENERIC and
custom kernels.

Use sparse initializers for "struct domain" and "struct protosw",
so they are easier to follow for the human being.

Retire MT_HEADER mbuf type and change its users to use MT_DATA.

Having an additional MT_HEADER mbuf type is superfluous and redundant
as nothing depends on it. It only adds a layer of confusion. The
distinction between header mbuf's and data mbuf's is solely done
through the m->m_flags M_PKTHDR flag.

Non-native code is not changed in this commit. For compatibility
MT_HEADER is mapped to MT_DATA.

Sponsored by: TCP/IP Optimization Fundraise 2005

Replace custom mbuf writeability test with generic M_WRITABLE() test
covering all edge cases too.

Sponsored by: TCP/IP Optimization Fundraise 2005

Correct typo in a comment describing vshiftl().

correct space check

Submitted by: ume

/* -> /*- for license, minor formatting changes

Initialize struct pr_userreqs in new/sparse style and fill in common
default elements in net_init_domain().

This makes it possible to grep these structures and see any bogosities.

Remove extraneous SECPOLICY_LOCK_DESTROY calls that cause the mutex to be
destroyed twice.

Submitted by: Roselyn Lee

Add missing locking for secpolicy refcnt manipulations.

Submitted by: Roselyn Lee

Correct handling of SADB_UPDATE and SADB_ADD requests. key_align may split
the mbuf due to use of m_pulldown. Discarding the result because of this
does not make sense as no subsequent code depends on the entire msg being
linearized (only the individual pieces). It's likely something else is wrong
here but for now this appears to get things back to a working state.

Submitted by: Roselyn Lee

Protect sockaddr_union definitions with a protecting define. This allows to
build kernels with FAST_IPSEC and PF. This is the least disruptive fix.

PR: kern/71836
Reviewed by: bms, various mailing lists
MFC after: 3 days

Apply error and success logic consistently to the function netisr_queue() and
its users.

netisr_queue() now returns (0) on success and ERRNO on failure. At the
moment ENXIO (netisr queue not functional) and ENOBUFS (netisr queue full)
are supported.

Previously it would return (1) on success but the return value of IF_HANDOFF()
was interpreted wrongly and (0) was actually returned on success. Due to this
schednetisr() was never called to kick the scheduling of the isr. However this
was masked by other normal packets coming through netisr_dispatch() causing the
dequeueing of waiting packets.

PR: kern/70988
Found by: MOROHOSHI Akihiko <moro@remus.dti.ne.jp>
MFC after: 3 days

Get rid of the RANDOM_IP_ID option and make it a sysctl. NetBSD
have already done this, so I have styled the patch on their work:

1) introduce a ip_newid() static inline function that checks
the sysctl and then decides if it should return a sequential
or random IP ID.

2) named the sysctl net.inet.ip.random_id

3) IPv6 flow IDs and fragment IDs are now always random.
Flow IDs and frag IDs are significantly less common in the
IPv6 world (ie. rarely generated per-packet), so there should
be smaller performance concerns.

The sysctl defaults to 0 (sequential IP IDs).

Reviewed by: andre, silby, mlaier, ume
Based on: NetBSD
MFC after: 2 months

Add required includes for post-sorwakeup() change to fix FAST_IPSEC
compilation.

Fix a paste-o in key_cmpspidx_withmask().

PR: misc/67013
Submitted by: Zhenmin <zli4@cs.uiuc.edu>

use correct address for SADB_EXT_ADDRESS_DST in key_do_allocsa_policy
(was using src instead of dst)

Submitted by: Bjoern A. Zeeb
Obtained from: KAME
MFC after: 1 day

correct behaviour of key_getsavbyspi broken in rev 1.7; corrects problems with
removing specific SPIs

Submitted by: Bjoern A. Zeeb

add support to prefer old SA to new SA during allocation
(makes net.key.preferred_oldsa work as for KAME)

Submitted by: gabor@sentex.net
Reviewed by: Bjoern A. Zeeb
MFC after: 1 day

Fix a debugging printf snafu.

use native names for if_link, ifa_link, if_addrhead.
Change for (...) to TAILQ_FOREACH(...)

Ok'ed by: sam

Unbreak FAST_IPSEC build on 64 bit archs with INVARIANTS.

Approved by: sam

This file was erroneously removed from HEAD when TCP-MD5 support was MFC'd;
correct this lameness.

Fix type in a sysctl. It used to be: net.key.prefered_oldsa
and is corrected to net.key.preferred_oldsa
This makes it consistent with the KAME IPsec implementation.

Approved by: sam

Initial import of RFC 2385 (TCP-MD5) digest support.

This is the first of two commits; bringing in the kernel support first.
This can be enabled by compiling a kernel with options TCP_SIGNATURE
and FAST_IPSEC.

For the uninitiated, this is a TCP option which provides for a means of
authenticating TCP sessions which came into being before IPSEC. It is
still relevant today, however, as it is used by many commercial router
vendors, particularly with BGP, and as such has become a requirement for
interconnect at many major Internet points of presence.

Several parts of the TCP and IP headers, including the segment payload,
are digested with MD5, including a shared secret. The PF_KEY interface
is used to manage the secrets using security associations in the SADB.

There is a limitation here in that as there is no way to map a TCP flow
per-port back to an SPI without polluting tcpcb or using the SPD; the
code to do the latter is unstable at this time. Therefore this code only
supports per-host keying granularity.

Whilst FAST_IPSEC is mutually exclusive with KAME IPSEC (and thus IPv6),
TCP_SIGNATURE applies only to IPv4. For the vast majority of prospective
users of this feature, this will not pose any problem.

This implementation is output-only; that is, the option is honoured when
responding to a host initiating a TCP session, but no effort is made
[yet] to authenticate inbound traffic. This is, however, sufficient to
interwork with Cisco equipment.

Tested with a Cisco 2501 running IOS 12.0(27), and Quagga 0.96.4 with
local patches. Patches for tcpdump to validate TCP-MD5 sessions are also
available from me upon request.

Sponsored by: sentex.net

must convert protocol to sa type when preparing a DELETE message

Submitted by: Roselyn Lee <rosel@verniernetworks.com>
MFC after: 1 week

o add missing break
o remove extraneous bzero
o add SYSINIT to properly initialize ip4_def_policy

Submitted by: "Bjoern A. Zeeb" <bzeeb+freebsd@zabbadoz.net>
Submitted by: gnn@neville-neil.com

change SYSINIT starting point to be consistent with other modules

add spdcachelookup and spdcachemiss to our version of struct ipsecstat so
netstat works properly

Submitted by: "Bjoern A. Zeeb" <bzeeb+freebsd@zabbadoz.net>

Fix ipip_output() to always set *mp to NULL on failure, even if 'm'
is NULL, otherwise ipsec4_process_packet() may try to m_freem() a
bad pointer.

In ipsec4_process_packet(), don't try to m_freem() 'm' twice; ipip_output()
already did it.

Obtained from: netbsd

fix build after KAME changes

Push m_apply() and m_getptr() up into the colleciton of standard mbuf
routines, and purge them from opencrypto.

Reviewed by: sam
Obtained from: NetBSD
Sponsored by: spc.org

Introduce a MAC label reference in 'struct inpcb', which caches
the MAC label referenced from 'struct socket' in the IPv4 and
IPv6-based protocols. This permits MAC labels to be checked during
network delivery operations without dereferencing inp->inp_socket
to get to so->so_label, which will eventually avoid our having to
grab the socket lock during delivery at the network layer.

This change introduces 'struct inpcb' as a labeled object to the
MAC Framework, along with the normal circus of entry points:
initialization, creation from socket, destruction, as well as a
delivery access control check.

For most policies, the inpcb label will simply be a cache of the
socket label, so a new protocol switch method is introduced,
pr_sosetlabel() to notify protocols that the socket layer label
has been updated so that the cache can be updated while holding
appropriate locks. Most protocols implement this using
pru_sosetlabel_null(), but IPv4/IPv6 protocols using inpcbs use
the the worker function in_pcbsosetlabel(), which calls into the
MAC Framework to perform a cache update.

Biba, LOMAC, and MLS implement these entry points, as do the stub
policy, and test policy.

Reviewed by: sam, bms
Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

MFp4: portability work, general cleanup, locking fixes

change 38496
o add ipsec_osdep.h that holds os-specific definitions for portability
o s/KASSERT/IPSEC_ASSERT/ for portability
o s/SPLASSERT/IPSEC_SPLASSERT/ for portability
o remove function names from ASSERT strings since line#+file pinpints
the location
o use __func__ uniformly to reduce string storage
o convert some random #ifdef DIAGNOSTIC code to assertions
o remove some debuggging assertions no longer needed

change 38498
o replace numerous bogus panic's with equally bogus assertions
that at least go away on a production system

change 38502 + 38530
o change explicit mtx operations to #defines to simplify
future changes to a different lock type

change 38531
o hookup ipv4 ctlinput paths to a noop routine; we should be
handling path mtu changes at least
o correct potential null pointer deref in ipsec4_common_input_cb

chnage 38685
o fix locking for bundled SA's and for when key exchange is required

change 38770
o eliminate recursion on the SAHTREE lock

change 38804
o cleanup some types: long -> time_t
o remove refrence to dead #define

change 38805
o correct some types: long -> time_t
o add scan generation # to secpolicy to deal with locking issues

change 38806
o use LIST_FOREACH_SAFE instead of handrolled code
o change key_flush_spd to drop the sptree lock before purging
an entry to avoid lock recursion and to avoid holding the lock
over a long-running operation
o misc cleanups of tangled and twisty code

There is still much to do here but for now things look to be
working again.

Supported by: FreeBSD Foundation

os dependency glue file for improving portability

Submitted by: <jonathan@decru.com>
Supported by: FreeBSD Foundation
Obtained from: NetBSD

Locking and misc cleanups; most of which I've been running for >4 months:

o add locking
o strip irrelevant spl's
o split malloc types to better account for memory use
o remove unused IPSEC_NONBLOCK_ACQUIRE code
o remove dead code

Sponsored by: FreeBSD Foundation

make sure the packets contains a complete inner header
for ip{4,6}-in-ip{4,6} encapsulation; fixes panic
for truncated ip-in-ip over ipsec

Submitted by: Markus Friedl <markus@openbsd.org>
Obtained from: OpenBSD (rev 1.66 ipsec_input.c)

consolidate callback optimization check in one location by adding a flag
for crypto operations that indicates the crypto code should do the check
in crypto_done

MFC after: 1 day

correct transfer statistics

Submitted by: Larry Baird <lab@gta.com>
MFC after: 1 day

plug xform memory leaks:

o add missing zeroize op when deleting an SA
o don't re-initialize an xform for an SA that already has one

Submitted by: Doug Ambrisko <ambrisko@verniernetworks.com>
MFC after: 1 day

Check crypto driver capabilities and if the driver operates synchronously
mark crypto requests with ``callback immediately'' to avoid doing a context
switch to return crypto results. This completes the work to eliminate
context switches for using software crypto via the crypto subsystem (with
symmetric crypto ops).

Introduce an M_ASSERTPKTHDR() macro which performs the very common task
of asserting that an mbuf has a packet header. Use it instead of hand-
rolled versions wherever applicable.

Submitted by: Hiten Pandya <hiten@unixdaemons.com>

ovbcopy -> bcopy

Replace ovbcopy() with bcopy().

add missing copyright notices

Noticed by: Robert Watson

Update netisr handling; Each SWI now registers its queue, and all queue
drain routines are done by swi_net, which allows for better queue control
at some future point. Packets may also be directly dispatched to a netisr
instead of queued, this may be of interest at some installations, but
currently defaults to off.

Reviewed by: hsu, silby, jayanth, sam
Sponsored by: DARPA, NAI Labs

fix compilation w/o INET6

Noticed by: "James E. Flemer" <jflemer@acm.jhu.edu>

Update to work with the new timewait state.

Reviewed by: sam

o add a CRYPTO_F_CBIMM flag to symmetric ops to indicate the callback
should be done in crypto_done rather than in the callback thread
o use this flag to mark operations from /dev/crypto since the callback
routine just does a wakeup; this eliminates the last unneeded ctx switch
o change CRYPTO_F_NODELAY to CRYPTO_F_BATCH with an inverted meaning
so "0" becomes the default/desired setting (needed for user-mode
compatibility with openbsd)
o change crypto_dispatch to honor CRYPTO_F_BATCH instead of always
dispatching immediately
o remove uses of CRYPTO_F_NODELAY
o define COP_F_BATCH for ops submitted through /dev/crypto and pass
this on to the op that is submitted

Similar changes and more eventually coming for asymmetric ops.

MFC if re gives approval.

Back out M_* changes, per decision of the TRB.

Approved by: trb

Comment out srandom():
1) Already called in init_main.c:proc0_post()
2) Seed is bad

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

o handle jumbograms in m_clone
o fixup some comments

Submitted by: Doug Ambrisko <ambrisko@verniernetworks.com>

Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup,
especially in troff files.

Correct mbuf packet header propagation. Previously, packet headers
were sometimes propagated using M_COPY_PKTHDR which actually did
something between a "move" and a "copy" operation. This is replaced
by M_MOVE_PKTHDR (which copies the pkthdr contents and "removes" it
from the source mbuf) and m_dup_pkthdr which copies the packet
header contents including any m_tag chain. This corrects numerous
problems whereby mbuf tags could be lost during packet manipulations.

These changes also introduce arguments to m_tag_copy and m_tag_copy_chain
to specify if the tag copy work should potentially block. This
introduces an incompatibility with openbsd which we may want to revisit.

Note that move/dup of packet headers does not handle target mbufs
that have a cluster bound to them. We may want to support this;
for now we watch for it with an assert.

Finally, M_COPYFLAGS was updated to include M_FIRSTFRAG|M_LASTFRAG.

Supported by: Vernier Networks
Reviewed by: Robert Watson <rwatson@FreeBSD.org>

o cannot use M_COPY_PKTHDR on an mbuf that has a cluster; if we need to
do this avoid m_getcl so we can copy the packet header to a clean mbuf
before adding the cluster
o move an assert to the right place

Supported by: Vernier Networks

SMP locking for ifnet list.

o Untangle the confusion with the malloc flags {M_WAITOK, M_NOWAIT} and
the mbuf allocator flags {M_TRYWAIT, M_DONTWAIT}.
o Fix a bpf_compat issue where malloc() was defined to just call
bpf_alloc() and pass the 'canwait' flag(s) along. It's been changed
to call bpf_alloc() but pass the corresponding M_TRYWAIT or M_DONTWAIT
flag (and only one of those two).

Submitted by: Hiten Pandya <hiten@unixdaemons.com> (hiten->commit_count++)

FAST_IPSEC fixups:

o fix #ifdef typo
o must use "bounce functions" when dispatched from the protosw table

don't know how this stuff was missed in my testing; must've committed
the wrong bits

Pointy hat: sam
Submitted by: "Doug Ambrisko" <ambrisko@verniernetworks.com>

"Fast IPsec": this is an experimental IPsec implementation that is derived
from the KAME IPsec implementation, but with heavy borrowing and influence
of openbsd. A key feature of this implementation is that it uses the kernel
crypto framework to do all crypto work so when h/w crypto support is present
IPsec operation is automatically accelerated. Otherwise the protocol
implementations are rather differet while the SADB and policy management
code is very similar to KAME (for the moment).

Note that this implementation is enabled with a FAST_IPSEC option. With this
you get all protocols; i.e. there is no FAST_IPSEC_ESP option.

FAST_IPSEC and IPSEC are mutually exclusive; you cannot build both into a
single system.

This software is well tested with IPv4 but should be considered very
experimental (i.e. do not deploy in production environments). This software
does NOT currently support IPv6. In fact do not configure FAST_IPSEC and
INET6 in the same system.

Obtained from: KAME + openbsd
Supported by: Vernier Networks