Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r260048:

In sys/netgraph/netflow, use __FBSDID() instead of old-style rcs_id[].

MFC r241446,r241501

Add NG_NETFLOW_V9INFO_TYPE command to be able to request netflowv9-specific
data.

Submitted by: Dmitry Luhtionov <dmitryluhtionov at gmail.com>

Merge r237162, r237163, r237164, r237226.

Fix improper L4 header handling for IPv6 packets passed via DLT_RAW.
Set netflow v9 observation domain value to fib number instead of node id.
This fixes multi-fib netflow v9 export.
Use time_uptime instead of getnanotime for accouting integer number of seconds.
Simplify IP pointer recovery in case of mbuf reallocation.

Approved by: ae(mentor)
Approved by: re

MFC r232921:

Use rt_numfibs variable instead of compile-time RT_NUMFIBS.

Approved by: kib(mentor)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

o Eliminate flow6_hash_entry in favor of flow_hash_entry. We don't need
a separate struct to start a slist of semi-opaque structs. This
makes some code more compact.
o Rewrite ng_netflow_flow_show() and its API/ABI:
- Support for IPv6 is added.
- Request and response now use same struct. Structure specifies
version (6 or 4), index of last retrieved hash, and also index
of last retrieved entry in the hash entry.

ng_netflow_cache_init() can be void.

Node constructor methods are supposed to be called in syscall
context always. Convert nodes to consistently use M_WAITOK flag
for memory allocation.

Reviewed by: julian

Add support for NetFlow version 9 into ng_netflow(4) node.

Submitted by: Alexander V. Chernikov <melifaro ipfw.ru>

Remove disabled code. In 99% cases exports are send to ng_ksocket(4), which
already forces queued mode, so what was suggested in disabled code is already
done.

Fix copy-paste bug in NGM_NETFLOW_SETCONFIG argument size verification.

PR: kern/134220
Submitted by: Eugene Mychlo
MFC after: 1 week

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Add ability to generate egress netflow instead or in addition to ingress.
Use mbuf tagging for accounted packets to not account packets twice when
both ingress and egress netflow enabled.
To keep compatibility new "setconfig" message added to control new
functionality. By default node works as before, doing only ingress
accounting without using mbuf tags.

Reviewed by: glebius

Replace callout_init(..., 1) with callout_init(..., CALLOUT_MPSAFE) for
better grep-compliance and to standardize with the rest of the kernel.

Reviewed by: jhb
MFC after: 1 week

Revert previous commit.
glebius@ noticed that it was not a bug, but undocumented feature.

Run expire even without export hook connected.

PR: kern/119839

Bump maximum number of interface hooks to the maximum possible value.
This will increase the memory consumption for more than 1 Mb, but this
is required for operation on multiinterface access concentrators running
mpd.

Requested by: Alexander Motin

Recognize 802.1q frames in Ethernet input and process them.

PR: kern/101162
Submitted by: CoolDavid (Tseng Guo-Fu) <cooldavid cdpa.nsysu.edu.tw>

Correct off-by-one errors.

Found with: Coverity Prevent(tm)

In ng_netflow_disconnect() check whether we are working with "iface"
or with "out" hook, and clear the right pointer.

Reported by: Vitaliy Ovsyannikov <V.Ovsyannikov kr.ru>

Check that we have first fragment before pulling up TCP/UDP header.

A new version of NetFlow node.

The most significant changes are:
- Use UMA zone instead of own chunk of memory.
- Lock each hash entry separately.
- Expire items "actively" - interrupt method can expire flows
from hash slot, when it searches through it.
- Remove global tailqueue. Make callout thread search through
every hash slot.
- Export datagram is detached from private data and filled. If
it is incomplete, it is attached back. Another thread will
continue working with it.

Lesser, but also important speedups:
- Flows in hash slot are stored in tailqueue. Whenever a flow is
hit, it is moved to the begging, so it can be located quicker.
- When callout thread works with hash slot it bails out if
slot mutex is contested.

Add a possibility to bypass unmodified accounted data to special
hook(s). Data received on these hook(s) is sent back to ifaceX hook(s).

Refactor node so that it does not modify mbuf contents. Next step would
be pass-thru mode, when traffic is not copied by ng_tee, but passed thru
ng_netflow.

Changes made:

- In ng_netflow_rcvdata() do all necessary pulluping: Ethernet header,
IP header, and TCP/UDP header.
- Pass only pointer to struct ip to ng_netflow_flow_add(). Any TCP/UDP
headers are guaranteed to by after it.
- Merge make_flow_rec() function into ng_netflow_flow_add().

Refactor node so that it does not modify mbuf contents. Next step would
be pass-thru mode, when traffic is not copied by ng_tee, but passed thru
ng_netflow.

Changes made:

- In ng_netflow_rcvdata() do all necessary pulluping: Ethernet header,
IP header, and TCP/UDP header.
- Pass only pointer to struct ip to ng_netflow_flow_add(). Any TCP/UDP
headers are guaranteed to by after it.
- Merge make_flow_rec() function into ng_netflow_flow_add().

Plug item leak, which occured when m_pullup() failed.

Use log() instead of printf(), to reduce flood on console.

MFC after: 1 week

- Use uint16_t to pass argument for NGM_NETFLOW_IFINFO, bump cookie.
- Always check that index number passed from userland
is <= NG_NETFLOW_MAXIFACES. [1]
- Increase NG_NETFLOW_MAXIFACES up to 512. [2]

Noticed by: Roman Palagin [1]
Requested by: Yuri Y. Bushmelev [2]
MFC after: 1 week

- Remove advertising clause from copyright [1]
- Change my email to glebius@FreeBSD.org

Requested by: ru [1]

A netgraph node implementing Netflow version 5.

Supported by: Bestcom ISP, Rinet ISP
Approved by: julian (mentor)