npfctl: adapt to new libpcap API

Major NPF improvements (merge from upstream):

- Switch to the C11-style atomic primitives using atomic_loadstore(9).

- npfkern: introduce the 'state.key.interface' and 'state.key.direction'
settings. Users can now choose whether the connection state should be
strictly per-interface or global at the configuration level. Keep NAT
logic to be always per-interface, though.

- npfkern: rewrite the G/C worker logic and make it self-tuning.

- npfkern and libnpf: multiple bug fixes; add param exporting; introduce
more parameters. Remove npf_nvlist_{copyin,copyout}() functions and
refactor npfctl_load_nvlist() with others; add npfctl_run_op() to have
a single entry point for operations. Introduce npf_flow_t and clean up
some code.

- npfctl: lots of fixes for the 'npfctl show' logic; make 'npfctl list'
more informative; misc usability improvements and more user-friendly
error messages.

- Amend and improve the manual pages.

Backport selected NPF fixes from the upstream (to be pulled up):

- npf_conndb_lookup: protect the connection lookup with pserialize(9),
instead of incorrectly assuming that the handler always runs at IPL_SOFNET.
Should fix crashes reported on high load (PR/55182).

- npf_config_destroy: handle partially initialized config; fixes crashes
with some invalid configurations.

- NAT policy creation / destruction: set the initial reference and do not
wait for reference draining on destruction; destroy the policy on the
last reference drop instead. Fixes a lockup with the dynamic NAT rules.

- npf_nat_{export,import}: fix a regression since dynamic NAT rules.

- npfctl: fix a regression and restore the default group behaviour.

- Add npf_cache_tcp() and validate the TCP data offset (from maxv@).

libnpf/npfctl: support dynamic NAT rulesets using a name prefix.

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.

(missed a file in previous commit; cvs is so helpful..)

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.50.2;
npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

Major NPF improvements (merge from upstream):

- Switch to the C11-style atomic primitives using atomic_loadstore(9).

- npfkern: introduce the 'state.key.interface' and 'state.key.direction'
settings. Users can now choose whether the connection state should be
strictly per-interface or global at the configuration level. Keep NAT
logic to be always per-interface, though.

- npfkern: rewrite the G/C worker logic and make it self-tuning.

- npfkern and libnpf: multiple bug fixes; add param exporting; introduce
more parameters. Remove npf_nvlist_{copyin,copyout}() functions and
refactor npfctl_load_nvlist() with others; add npfctl_run_op() to have
a single entry point for operations. Introduce npf_flow_t and clean up
some code.

- npfctl: lots of fixes for the 'npfctl show' logic; make 'npfctl list'
more informative; misc usability improvements and more user-friendly
error messages.

- Amend and improve the manual pages.

Backport selected NPF fixes from the upstream (to be pulled up):

- npf_conndb_lookup: protect the connection lookup with pserialize(9),
instead of incorrectly assuming that the handler always runs at IPL_SOFNET.
Should fix crashes reported on high load (PR/55182).

- npf_config_destroy: handle partially initialized config; fixes crashes
with some invalid configurations.

- NAT policy creation / destruction: set the initial reference and do not
wait for reference draining on destruction; destroy the policy on the
last reference drop instead. Fixes a lockup with the dynamic NAT rules.

- npf_nat_{export,import}: fix a regression since dynamic NAT rules.

- npfctl: fix a regression and restore the default group behaviour.

- Add npf_cache_tcp() and validate the TCP data offset (from maxv@).

libnpf/npfctl: support dynamic NAT rulesets using a name prefix.

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.

(missed a file in previous commit; cvs is so helpful..)

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.50.2;
npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

Backport selected NPF fixes from the upstream (to be pulled up):

- npf_conndb_lookup: protect the connection lookup with pserialize(9),
instead of incorrectly assuming that the handler always runs at IPL_SOFNET.
Should fix crashes reported on high load (PR/55182).

- npf_config_destroy: handle partially initialized config; fixes crashes
with some invalid configurations.

- NAT policy creation / destruction: set the initial reference and do not
wait for reference draining on destruction; destroy the policy on the
last reference drop instead. Fixes a lockup with the dynamic NAT rules.

- npf_nat_{export,import}: fix a regression since dynamic NAT rules.

- npfctl: fix a regression and restore the default group behaviour.

- Add npf_cache_tcp() and validate the TCP data offset (from maxv@).

libnpf/npfctl: support dynamic NAT rulesets using a name prefix.

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.

(missed a file in previous commit; cvs is so helpful..)

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.50.2;
npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

libnpf/npfctl: support dynamic NAT rulesets using a name prefix.

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.

(missed a file in previous commit; cvs is so helpful..)

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.50.2;
npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

NPF: fix BPF byte-code generation for a port-range used in a group.
Resolved PR/52609 and PR/54169.

branches: 1.50.2;
npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

npfctl: fix the bpf.jit parameter handling.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2; 1.45.4;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

remove error(1) output

make this compile again.

- Rework NPF NAT syntax to be more structured and support future additions
of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.

branches: 1.6.2;
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS. Be more defensive in the
packet handler. Change the default policy to block when the config is
loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
of rule procedure might happen in the interrupt handler (under a very rare
condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

branches: 1.4.2;
- Handle NPF rule procedures in the parser and thus re-enable them.
- Few small updates to the man page.

Multiple NPF fixes, add better error reporting from kernel side, add some
asserts, bump the version.

- Expire all sessions on flush.
- Enable checking for zero mask in IP{4,6}MATCH after npfctl changes.
- Make locking symmetric for npf_ruleset_inspect().
- Sync function prototypes in npf(3) man page with reality.
- Rename NPF_TABLE_RBTREE to NPF_TABLE_TREE.

Full rewrite of npfctl(8) parser and rework of n-code generation part.
Fixes most of the known bugs and issues with the utility. Note: rule
procedures are not yet (as we want to make them fully modular).

Huge thanks to Martin Husemann who wrote the parser and Christos Zoulas
who wrote intermediate structures and helped to complete the work.

Summary: Ensure default TCP flags are applied to rules like 'pass stateful all'

The documented default "flags S/SAFR" for stateful rules that affect
TCP packets but don't specify any flags, doesn't actually get applied
to a rule like "pass stateful out all". The big problem with this is
that when you then do a "block return-rst" for an incoming packet, the
generated RST packet will create state for the connection attempt it's
blocking, so that a second attempt from the same source will pass.

This change makes the default flags actually apply to such simple
rules. It also fixes a related bug in the code generation for the
flag matching, where part of the action could erroneously be omitted.

Reviewed by <rmind>
Closes PR bin/54124
Pullup to NetBSD 8

Major NPF improvements:
- Convert NPF connection table to thmap. State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address. Translation addresses can now be
selected from a pool of addresses. There are two selection algorithms,
"ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.45.2;
npfctl: add support for the 'no-ports' flag in the 'map' statements.
This allows us to create a NAT policy without the port translation.

branches: 1.44.4;
npfctl:
- Add protocol filter option for "map".
- Print user-friendly error if table contains an entry with invalid netmask.
- Add support for inline ports.

branches: 1.43.2;
npfctl: dynamic interface address handling; update npf.conf(8).

npf.conf: add support for logical NOT, e.g.: pass from ! 10.0.0.1 to any

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

branches: 1.40.2;
- npfctl: fix the confusion in the parser (0/0 case with no other filter).
- Always populate the error dictionary, not only for DEBUG/DIAGNOSTIC.

npfctl:
- Fix the filter criteria when to/from is omitted but port used.
- Print more user-friendly error if an NPF table has a duplicate entry.

branches: 1.38.2;
npfctl_build_code: generate TCP/UDP check for ports case when other blocks
do not imply L4 check; add an assert in npfctl_bpf_proto() and elsewhere.

NPF: imply SYN-only check for the stateful rules by default (when inspecting
TCP packets). Many users trip here. This behaviour can be overriden with the
explicit "flags" keyword, but other configuration does not really make sense.

branches: 1.36.2;
NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296. Add a unit test. Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

NPF: add support for static (stateless) NAT.

fix vax build.

Add support for CDB based NPF tables.

- npfctl: fix table IDs (breakage since the table naming was added).
- libnpf: remove npf_table_exists_p() from public API.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now. Also, fix table name checking in npfctl.

Simplify parsing of npf.conf elements, create the npfvar_t when a value is
parsed (to be used as a general structured for variables and inlined values),
few misc improvements.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes. Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

- NPF: change the group/ruleset syntax - simplify. Update npf.conf(5) manual.
- Add support for the inline pcap-filter(7) syntax in the rule, e.g.:
block out final pcap-filter "tcp and dst 10.1.1.252"

npfctl: remove some n-code leftovers, fix the build, update the man pages.

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
alg "algname"

- Extend npf.conf syntax to support dynamic NAT policies.
- Imply dynamic group when using "ruleset" keyword.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs. Also, update the man page.

Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

- Add NPF version check in proplist as well, not only ioctl. Bump the version.
- Fix a bug in table entry lookup.
- Updates/fixes to the man pages. Misc.

npfctl: add 'validate' command to check the config, but not load it. Update
the man page. Also add a small note about 'debug' command, PR/47298.

npfctl: extend syntax for extracting interface IP address(es) by the family.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.13.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

teach npf ipv6-icmp
reviewed by rmind@

- Rework NPF tables and fix support for IPv6. Implement tree table type
using radix / Patricia tree. Universal IPv4/IPv6 comparator for ptree(3)
was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.