Major NPF improvements (merge from upstream):

- Switch to the C11-style atomic primitives using atomic_loadstore(9).

- npfkern: introduce the 'state.key.interface' and 'state.key.direction'
settings. Users can now choose whether the connection state should be
strictly per-interface or global at the configuration level. Keep NAT
logic to be always per-interface, though.

- npfkern: rewrite the G/C worker logic and make it self-tuning.

- npfkern and libnpf: multiple bug fixes; add param exporting; introduce
more parameters. Remove npf_nvlist_{copyin,copyout}() functions and
refactor npfctl_load_nvlist() with others; add npfctl_run_op() to have
a single entry point for operations. Introduce npf_flow_t and clean up
some code.

- npfctl: lots of fixes for the 'npfctl show' logic; make 'npfctl list'
more informative; misc usability improvements and more user-friendly
error messages.

- Amend and improve the manual pages.

- npfctl_load_nvlist: simplify the config loading logic.
- Fix a small race condition in npf_nat_getaddr().
- Rework pserialize/EBR wrappers, make it easier to maintain.

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.13.2;
NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.10.6; 1.10.12; 1.10.14;
npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.8.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- npfctl_load_nvlist: simplify the config loading logic.
- Fix a small race condition in npf_nat_getaddr().
- Rework pserialize/EBR wrappers, make it easier to maintain.

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.13.2;
NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.10.6; 1.10.12; 1.10.14;
npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.8.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.13.2;
NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.10.6; 1.10.12; 1.10.14;
npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.8.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.10.6; 1.10.12; 1.10.14;
npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.8.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.10.6; 1.10.12;
npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.8.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

branches: 1.7.2; 1.7.4;
NPF:
- Populate the BPF external memory store with L3 information.
- Eliminate NPF_COP_L3 call and just use the data in the memstore.
- Bump NPF_VERSION.

npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

branches: 1.5.2;
NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.4.2;
npfctl_bpf_cidr: another buf fix in handling IPv6 masks (bug found on ARM).

npfctl_bpf_cidr: fix a bug in handling of smaller IPv6 masks.

npfctl: optimise fetch_l3() to avoid unnecessary call to NPF_COP_L3.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.