ipfw.8: synopsis misses nat show form

Document the existing behavior, which is currently only available by
reading third party documentation or the source code itself.

PR: 254617
Submitted by: Oliver Kiddle
Differential Revision: https://reviews.freebsd.org/D30189

(cherry picked from commit c8250c5ada85fec8936e8eb8695d7cb80a3ce8ab)

Git Hash: 95f8bb6d06d9fb65dc382423485f2f636ec20375
Git Author: donner@FreeBSD.org

ipfw.8: Fix table example

Fix some erronous lines in the example section.

PR: 248943
Submitted by: Jose Luis Duran
Reviewers: ae, manpages (gbe)
Differential Revision: https://reviews.freebsd.org/D30191

(cherry picked from commit 802637be06457c7593692bdf79f8466fa5f54d4a)

Git Hash: 0f6a5de5c7f05ac7ed660459944118c9386b2f09
Git Author: donner@FreeBSD.org

^sbin/ipfw: Fix null pointer deference when printing counters

ipfw -[tT] prints statistics of the last access. If the rule was never
used, the counter might be not exist. This happens unconditionally on
inserting a new rule. Avoid printing statistics in this case.

PR: 255491
Reported by: Haisheng Zhouz
Reviewed by: ae
Differential Revision: https://reviews.freebsd.org/D30046

(cherry picked from commit bf7cc0f9cb6603a6bdd6131c8d1939724ce6e62d)

Git Hash: 552885bde60021ed6f1db46a95f9fb2557cbe1b6
Git Author: donner@FreeBSD.org

ipfw: make algo name argument optional for some table types

Most of table types currently supported by ipfw have only one
algorithm implementation. When user creates such tables, allow
to omit algo name in arguments. E.g. now it is possible:
ipfw table T1 create type number
ipfw table T2 create type iface
ipfw table T3 create type flow

PR: 233072
Sponsored by: Yandex LLC

(cherry picked from commit 13ad237a19b7368124483d9d1dc3258c27880fef)

Git Hash: 952ad28223596ef4e323a9a7da81ef3faf1c919a
Git Author: ae@FreeBSD.org

MFC r363904:
Fix SIGSEGV in ipfw(8) when NAT64 prefix length is omitted.

MFC r357092,357787: Add support for RFC 6598/Carrier Grade NAT subnets
to libalias and ipfw.

In libalias, a new flag PKT_ALIAS_UNREGISTERED_RFC6598 is added.
This is like PKT_ALIAS_UNREGISTERED_ONLY, but also is RFC 6598 aware.
Also, we add a new NAT option to ipfw called unreg_cgn, which is like
unreg_only, but also is RFC 6598-aware. The reason for the new
flags/options is to avoid breaking existing networks, especially those
which rely on RFC 6598 as an external address.

Submitted by: Neel Chauhan <neel AT neelc DOT org>
Reviewed by: melifaro, rgrimes, Lutz Donnerhacke
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D22877
Differential Revision: https://reviews.freebsd.org/D23448

MFC r359271:
Use IP_FW_NAT44_DESTROY opcode for IP_FW3 socket option to destroy
NAT instance.

The NAT44 group of opcodes for IP_FW3 socket option is modern way
to control NAT instances and this method can be used in future to
switch from numeric to named NAT instances, like was done for ipfw
tables.
The IP_FW_NAT_DEL opcode is the last remnant of old ipfw_ctl control
plane that doesn't support versioned operations. This interface will
be retired soon.

MFC r353545:
Explicitly initialize the memory buffer to store O_ICMP6TYPE opcode.

By default next_cmd() initializes only first u32 of opcode. O_ICMP6TYPE
opcode has array of bit masks to store corresponding ICMPv6 types.
An opcode that precedes O_ICMP6TYPE, e.g. O_IP6_DST, can have variable
length and during opcode filling it can modify memory that will be used
by O_ICMP6TYPE opcode. Without explicit initialization this leads to
creation of wrong opcode.

Reported by: Boris N. Lytochkin
Obtained from: Yandex LLC

MFC r350576: ipfw: fix jail option after r348215

r348215 changed jail_getid(3) to validate passed-in jids as active jails
(as the function is documented to return -1 if the jail does not exist).
This broke the jail option (in some cases?) as the jail historically hasn't
needed to exist at the time of rule parsing; jids will get stored and later
applied.

Fix this caller to attempt to parse *av as a number first and just use it
as-is to match historical behavior. jail_getid(3) must still be used in
order for name arguments to work, but it's strictly a fallback in case we
weren't given a number.

MFC r349364:
Restore ipfw(8)'s compact output support broken after r331668.

Also modify it a bit. Now -c option omits only 'from any to any' part
and works for different protocols (not just for ip).

Reported by: Dmitry Selivanov <dseliv at gmail>

MFC r348235:
Add `missing` and `or-flush` options to "ipfw table <NAME> create"
command to simplify firewall reloading.

The `missing` option suppresses EEXIST error code, but does check that
existing table has the same parameters as new one. The `or-flush` option
implies `missing` option and additionally does flush for table if it
is already exist.

Submitted by: lev
Differential Revision: https://reviews.freebsd.org/D18339

MFC r348301:
Remove unused token that was added in r348235.

MFC r349267:
Add "tcpmss" opcode to match the TCP MSS value.

With this opcode it is possible to match TCP packets with specified
MSS option, whose value corresponds to configured in opcode value.
It is allowed to specify single value, range of values, or array of
specific values or ranges. E.g.

# ipfw add deny log tcp from any to any tcpmss 0-500

MFC r346885:
Handle HAVE_PROTO flag and print "proto" keyword for O_IP4 and O_IP6
opcodes when it is needed.
This should fix the problem, when printed by `ipfw show` rule can not
be added due to missing "proto" keyword.

MFC r345264:
Add NAT64 CLAT implementation as defined in RFC6877.

CLAT is customer-side translator that algorithmically translates 1:1
private IPv4 addresses to global IPv6 addresses, and vice versa.
It is implemented as part of ipfw_nat64 kernel module. When module
is loaded or compiled into the kernel, it registers "nat64clat" external
action. External action named instance can be created using `create`
command and then used in ipfw rules. The create command accepts two
IPv6 prefixes `plat_prefix` and `clat_prefix`. If plat_prefix is ommitted,
IPv6 NAT64 Well-Known prefix 64:ff9b::/96 will be used.

# ipfw nat64clat CLAT create clat_prefix SRC_PFX plat_prefix DST_PFX
# ipfw add nat64clat CLAT ip4 from IPv4_PFX to any out
# ipfw add nat64clat CLAT ip6 from DST_PFX to SRC_PFX in

Obtained from: Yandex LLC
Submitted by: Boris N. Lytochkin
Relnotes: yes
Sponsored by: Yandex LLC

MFC r345263:
Add SPDX-License-Identifier and update year in copyright.

MFC r345262:
Modify struct nat64_config.

Add second IPv6 prefix to generic config structure and rename another
fields to conform to RFC6877. Now it contains two prefixes and length:
PLAT is provider-side translator that translates N:1 global IPv6 addresses
to global IPv4 addresses. CLAT is customer-side translator (XLAT) that
algorithmically translates 1:1 IPv4 addresses to global IPv6 addresses.
Use PLAT prefix in stateless (nat64stl) and stateful (nat64lsn)
translators.

Modify nat64_extract_ip4() and nat64_embed_ip4() functions to accept
prefix length and use plat_plen to specify prefix length.

Retire net.inet.ip.fw.nat64_allow_private sysctl variable.
Add NAT64_ALLOW_PRIVATE flag and use "allow_private" config option to
configure this ability separately for each NAT64 instance.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r339542:
Retire IPFIREWALL_NAT64_DIRECT_OUTPUT kernel option. And add ability
to switch the output method in run-time. Also document some sysctl
variables that can by changed for NAT64 module.

NAT64 had compile time option IPFIREWALL_NAT64_DIRECT_OUTPUT to use
if_output directly from nat64 module. By default is used netisr based
output method. Now both methods can be used, but they require different
handling by rules.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D16647

MFC r344709 (by ygy):
Fix typos and caps for ipfw(8) man page.

PR: 236030
Submitted by: olgeni

MFC r344665 (by trhodes):
Grammar tweaks in ipfw manual page.

MFC r341471:
Reimplement how net.inet.ip.fw.dyn_keep_states works.

Turning on of this feature allows to keep dynamic states when parent
rule is deleted. But it works only when the default rule is
"allow from any to any".

Now when rule with dynamic opcode is going to be deleted, and
net.inet.ip.fw.dyn_keep_states is enabled, existing states will reference
named objects corresponding to this rule, and also reference the rule.
And when ipfw_dyn_lookup_state() will find state for deleted parent rule,
it will return the pointer to the deleted rule, that is still valid.
This implementation doesn't support O_LIMIT_PARENT rules.

The refcnt field was added to struct ip_fw to keep reference, also
next pointer added to be able iterate rules and not damage the content
when deleted rules are chained.

Named objects are referenced only when states are going to be deleted to
be able reuse kidx of named objects when new parent rules will be
installed.

ipfw_dyn_get_count() function was modified and now it also looks into
dynamic states and constructs maps of existing named objects. This is
needed to correctly export orphaned states into userland.

ipfw_free_rule() was changed to be global, since now dynamic state can
free rule, when it is expired and references counters becomes 1.

External actions subsystem also modified, since external actions can be
deregisterd and instances can be destroyed. In these cases deleted rules,
that are referenced by orphaned states, must be modified to prevent access
to freed memory. ipfw_dyn_reset_eaction(), ipfw_reset_eaction_instance()
functions added for these purposes.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D17532

MFC r341472:
Add ability to request listing and deleting only for dynamic states.

This can be useful, when net.inet.ip.fw.dyn_keep_states is enabled, but
after rules reloading some state must be deleted. Added new flag '-D'
for such purpose.

Retire '-e' flag, since there can not be expired states in the meaning
that this flag historically had.

Also add "verbose" mode for listing of dynamic states, it can be enabled
with '-v' flag and adds additional information to states list. This can
be useful for debugging.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r344018:
Remove `set' field from state structure and use set from parent rule.

Initially it was introduced because parent rule pointer could be freed,
and rule's information could become inaccessible. In r341471 this was
changed. And now we don't need this information, and also it can become
stale. E.g. rule can be moved from one set to another. This can lead
to parent's set and state's set will not match. In this case it is
possible that static rule will be freed, but dynamic state will not.
This can happen when `ipfw delete set N` command is used to delete
rules, that were moved to another set.
To fix the problem we will use the set number from parent rule.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r344870:
Fix the problem with O_LIMIT states introduced in r344018.

dyn_install_state() uses `rule` pointer when it creates state.
For O_LIMIT states this pointer actually is not struct ip_fw,
it is pointer to O_LIMIT_PARENT state, that keeps actual pointer
to ip_fw parent rule. Thus we need to cache rule id and number
before calling dyn_get_parent_state(), so we can use them later
when the `rule` pointer is overrided.

PR: 236292

MFC r340792 (by ygy):
Fix a minor typo in ipfw(8) manual page.

PR: 230747
Submitted by: f.toscan@hotmail.it

MFC r340717 (by ygy):
Fix incorrect DSCP value range from 0..64 to 0..63.

PR: 232786
Submitted by: Sergey Akhmatov <sergey@akhmatov.ru>

MFC r343877,r343880:

r343877:
ipfw table list: Fix showing header outside of 'all'.
r343880:
Fix build of r343877

Relnotes: yes

MFC r342298:
Allow use underscores and dots in service names without escaping.

PR: 234237

MFC r341798:
Use correct size for IPv4 address in gethostbyaddr().
When u_long is 8 bytes, it returns EINVAL and 'ipfw -N show' doesn't work.

Reported by: Claudio Eichenberger <cei at yourshop.com>

MFC r341799:
Rework how protocol number is tracked in rule. Save it when O_PROTO
opcode will be printed. This should solve the problem, when protocol
name is not printed in `ipfw -N show`.

Reported by: Claudio Eichenberger <cei at yourshop.com>

MFC r340394: ipfw.8: Fix part of the SYNOPSIS documenting
LIST OF RULES AND PREPROCESSING that is still referred
as last section of the SYNOPSIS later but was erroneously situated
in the section IN-KERNEL NAT.

MFC r340110: ipfw(8): clarify layer2 processing abilities

Make it clear that ipfw action set for layer2 frames is a bit limited.

PR: 59835
Reviewed by: yuripv
Differential Revision: https://reviews.freebsd.org/D17719

MFC r340978-340979: ipfw.8: new section to EXAMPLES: SELECTIVE MIRRORING

MFC r340689:
Make multiline APPLY_MASK() macro to be function-like.

Reported by: cem

Revert r340541. It requires VNET_DEFINE_STATIC() macro that is not yet
merged into stable/11.

MFC r339542:
Retire IPFIREWALL_NAT64_DIRECT_OUTPUT kernel option. And add ability
to switch the output method in run-time. Also document some sysctl
variables that can by changed for NAT64 module.

NAT64 had compile time option IPFIREWALL_NAT64_DIRECT_OUTPUT to use
if_output directly from nat64 module. By default is used netisr based
output method. Now both methods can be used, but they require different
handling by rules.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D16647

MFC r339539:
Add IPFW_RULE_JUSTOPTS flag, that is used by ipfw(8) to mark rule,
that was added using "new rule format". And then, when the kernel
returns rule with this flag, ipfw(8) can correctly show it.

Reported by: lev
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D17373

MFC r338977:

Add description, parameters, options, sysctl and examples of using AQMs to ipfw man page. CoDel, PIE, FQ-CoDel and FQ-PIE AQM for Dummynet exist in FreeBSD 11 and 10.3.

Submitted by: ralsaadi@swin.edu.au
Reviewed by: AllanJude
Differential Revision: https://reviews.freebsd.org/D12507

MFC r340249: ipfw.8: fix small syntax error in an example

MFC r340100:
Do not use bzero() for the O_ICMP6TYPE opcode.

The buffer is already zeroed in compile_rule() function, and also it
may contain configured F_NOT flag in o.len field. This fixes the
filling for "not icmp6types" opcode.

MFC r340175:
Do not print "ip6" keyword in print_icmp6types() for O_ICMP6TYPE opcode.

It produces incompatibility when rules listing is used again to
restore saved ruleset, because "ip6" keyword produces separate opcode.
The kernel already has the check and only IPv6 packets will be checked
for matching.

PR: 232939

MFC r339740:
Use correct format specificator to print setdscp action.

PR: 232642

MFC r338947:
Add "src-ip" or "dst-ip" keyword to the output, when we are printing the
rest of rule options.

Reported by: lev

MFC r337536:
If -q flag is specified, do not complain when we are trying to delete
nonexistent NAT instance or nonexistent rule.

This allows execute batched `delete` commands and do not fail when
found nonexistent rule.

MFC r337574:
Restore the behaviour changed in r337536, when bad `ipfw delete` command
returns error.

Now -q option only makes it quiet. And when -f flag is specified, the
command will ignore errors and continue executing with next batched
command.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r336132:
Add "record-state", "set-limit" and "defer-action" rule options to ipfw.

"record-state" is similar to "keep-state", but it doesn't produce implicit
O_PROBE_STATE opcode in a rule. "set-limit" is like "limit", but it has the
same feature as "record-state", it is single opcode without implicit
O_PROBE_STATE opcode. "defer-action" is targeted to be used with dynamic
states. When rule with this opcode is matched, the rule's action will
not be executed, instead dynamic state will be created. And when this
state will be matched by "check-state", then rule action will be executed.
This allows create a more complicated rulesets.

Submitted by: lev

MFC r335921:

Allow jail names (not just IDs) to be specified for: cpuset(1), ipfw(8),
sockstat(1), ugidfw(8)
These are the last of the jail-aware userland utilities that didn't work
with names.

PR: 229266
Differential Revision: D16047

MFC r335759:
Remove extra "ipfw" from example.

MFC r333403:
Bring in some last changes in NAT64 implementation:

o Modify ipfw(8) to be able set any prefix6 not just Well-Known,
and also show configured prefix6;
o relocate some definitions and macros into proper place;
o convert nat64_debug and nat64_allow_private variables to be
VNET-compatible;
o add struct nat64_config that keeps generic configuration needed
to NAT64 code;
o add nat64_check_prefix6() function to check validness of specified
by user IPv6 prefix according to RFC6052;
o use nat64_check_private_ip4() and nat64_embed_ip4() functions
instead of nat64_get_ip4() and nat64_set_ip4() macros. This allows
to use any configured IPv6 prefixes that are allowed by RFC6052;
o introduce NAT64_WKPFX flag, that is set when IPv6 prefix is
Well-Known IPv6 prefix. It is used to reduce overhead to check this;
o modify nat64lsn_cfg and nat64stl_cfg structures to use nat64_config
structure. And respectivelly modify the rest of code;
o remove now unused ro argument from nat64_output() function;
o remove __FreeBSD_version ifdef, NAT64 was not merged to older versions;
o add commented -DIPFIREWALL_NAT64_DIRECT_OUTPUT flag to module's Makefile
as example.

MFC r333406:
Update NAT64 documentation, now we support any IPv6 prefixes.

MFC r333458:
Fix the printing of rule comments.

Change uint8_t type of opcode argument to int in the print_opcode()
function. Use negative value to print the rest of opcodes, because
zero value is O_NOP, and it can't be uses for this purpose.

Reported by: lev
Approved by: re (gjb)

MFC r332403:

Fix a typo.

MFC r332451:
Fix indenting in ipv6.c file, use tabs instead of mixing tabs and spaces.

MFC r332449:
Remove printing of "not" keyword from print_ip6() function.

After r331668 handling of F_NOT flag done in one place by
print_instruction() function. Also remove unused argument from
print_ip[6]() functions.

MFC r332448:
Remove printing of "not" keyword from print_ip() function.

After r331668 handling of F_NOT flag done in one place by
print_instruction() function.

MFC r331668:
Rework ipfw rules parsing and printing code.

Introduce show_state structure to keep information about printed opcodes.
Split show_static_rule() function into several smaller functions. Make
parsing and printing opcodes into several passes. Each printed opcode
is marked in show_state structure and will be skipped in next passes.
Now show_static_rule() function is simple, it just prints each part
of rule separately: action, modifiers, proto, src and dst addresses,
options. The main goal of this change is avoiding occurrence of wrong
result of `ifpw show` command, that can not be parsed by ipfw(8).
Also now it is possible to make some simple static optimizations
by reordering of opcodes in the rule.

PR: 222705

MFC r326233:

Add to ipfw support for sending an SCTP packet containing an ABORT chunk.
This is similar to the TCP case. where a TCP RST segment can be sent.

There is one limitation: When sending an ABORT in response to an incoming
packet, it should be tested if there is no ABORT chunk in the received
packet. Currently, it is only checked if the first chunk is an ABORT
chunk to avoid parsing the whole packet, which could result in a DOS attack.

Thanks to Timo Voelker for helping me to test this patch.

MFC r327200:

When adding support for sending SCTP packets containing an ABORT chunk
to ipfw in https://svnweb.freebsd.org/changeset/base/326233,
a dependency on the SCTP stack was added to ipfw by accident.

This was noted by Kevel Bowling in https://reviews.freebsd.org/D13594
where also a solution was suggested. This patch is based on Kevin's
suggestion, but implements the required SCTP checksum computation
without any dependency on other SCTP sources.

While there, do some cleanups and improve comments.

Thanks to Kevin Kevin Bowling for reporting the issue and suggesting
a fix.

MFC r324218:

Whitespace changes: Remove leading spaces followed by a tab.

MFC r324216:

Fix a bug which avoided that rules for matching port numbers for SCTP
packets where actually matched.
While there, make clean in the man-page that SCTP port numbers are
supported in rules.

MFC r331203:
Remove note that `fwd tablearg` is supported only by IPv4. IPv6 is
supported too.

MFC r330792:
Do not try to reassemble IPv6 fragments in "reass" rule.

ip_reass() expects IPv4 packet and will just corrupt any IPv6 packets
that it gets. Until proper IPv6 fragments handling function will be
implemented, pass IPv6 packets to next rule.

PR: 170604

MFC r320268,r320276:

ipfw: dummynet: Add 'G' and 'g' suffix for bandwidth configuration/display

Merge rev. 329197 from HEAD:

DSCP values passed to setdscp need to be lowercase.

MFC r327140:
Fix rule number truncation, use uint16_t type to specify rulenum.
Also sort variable declartions by size.

PR: 224555

MFC r324592:
Return 'errno' value from the table_do_modify_record(), it is expected
by table_modify_record().

This makes quiet operations with tables really quiet.

PR: 222953

MFC r319900:

sbin/ipfw: strcpy, strncpy => strlcpy

Reported by: Coverity
CID: 1356162, 1356166
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D10662

MFC r318400:
Allow zero port specification in table entries with type flow.

PR: 217620

MFC: r317343

In fill_ip6(), the value of the pointer av changes before it is
free(3)ed. Thus, introduce a new variable to track the original
value.

MFC r317666:
Add sets support for ipfw table info/list/flush commands.

PR: 212668

MFC r317667:
In parse_range() validate both range values instead of checking
the top value twice.

PR: 202295

MFC r317663:
Properly initialize ipfw_range_tlv variable to fix possible EINVAL
in case when ipfw delete/zero/resetlog command issued for several rules
in the loop. Also reorder some variables by size.

PR: 218993

MFC r317682:
Add `ipfw table all destroy` support.

PR: 212669

MFC r316435:
Add ipfw_pmod kernel module.

The module is designed for modification of a packets of any protocols.
For now it implements only TCP MSS modification. It adds the external
action handler for "tcp-setmss" action.

A rule with tcp-setmss action does additional check for protocol and
TCP flags. If SYN flag is present, it parses TCP options and modifies
MSS option if its value is greater than configured value in the rule.
Then it adjustes TCP checksum if needed. After handling the search
continues with the next rule.

Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D10150

MFC r316434:
Add O_EXTERNAL_DATA opcode support.

This opcode can be used to attach some data to external action opcode.
And unlike to O_EXTERNAL_INSTANCE opcode, this opcode does not require
creating of named instance to pass configuration arguments to external
action handler. The data is coming just next to O_EXTERNAL_ACTION opcode.

The userlevel part currenly supports formatting for opcode with ipfw_insn
size, by default it expects u16 numeric value in the arg1.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r304041:
Move logging via BPF support into separate file.

* make interface cloner VNET-aware;
* simplify cloner code and use if_clone_simple();
* migrate LOGIF_LOCK() to rmlock;
* add ipfw_bpf_mtap2() function to pass mbuf to BPF;
* introduce new additional ipfwlog0 pseudo interface. It differs from
ipfw0 by DLT type used in bpfattach. This interface is intended to
used by ipfw modules to dump packets with additional info attached.
Currently pflog format is used. ipfw_bpf_mtap2() function uses second
argument to determine which interface use for dumping. If dlen is equal
to ETHER_HDR_LEN it uses old ipfw0 interface, if dlen is equal to
PFLOG_HDRLEN - ipfwlog0 will be used.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r304043:
Add three helper function to manage tables from external modules.

ipfw_objhash_lookup_table_kidx does lookup kernel index of table;
ipfw_ref_table/ipfw_unref_table takes and releases reference to table.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r304046, 304108:
Add ipfw_nat64 module that implements stateless and stateful NAT64.

The module works together with ipfw(4) and implemented as its external
action module.

Stateless NAT64 registers external action with name nat64stl. This
keyword should be used to create NAT64 instance and to address this
instance in rules. Stateless NAT64 uses two lookup tables with mapped
IPv4->IPv6 and IPv6->IPv4 addresses to perform translation.

A configuration of instance should looks like this:
1. Create lookup tables:
# ipfw table T46 create type addr valtype ipv6
# ipfw table T64 create type addr valtype ipv4
2. Fill T46 and T64 tables.
3. Add rule to allow neighbor solicitation and advertisement:
# ipfw add allow icmp6 from any to any icmp6types 135,136
4. Create NAT64 instance:
# ipfw nat64stl NAT create table4 T46 table6 T64
5. Add rules that matches the traffic:
# ipfw add nat64stl NAT ip from any to table(T46)
# ipfw add nat64stl NAT ip from table(T64) to 64:ff9b::/96
6. Configure DNS64 for IPv6 clients and add route to 64:ff9b::/96
via NAT64 host.

Stateful NAT64 registers external action with name nat64lsn. The only
one option required to create nat64lsn instance - prefix4. It defines
the pool of IPv4 addresses used for translation.

A configuration of instance should looks like this:
1. Add rule to allow neighbor solicitation and advertisement:
# ipfw add allow icmp6 from any to any icmp6types 135,136
2. Create NAT64 instance:
# ipfw nat64lsn NAT create prefix4 A.B.C.D/28
3. Add rules that matches the traffic:
# ipfw add nat64lsn NAT ip from any to A.B.C.D/28
# ipfw add nat64lsn NAT ip6 from any to 64:ff9b::/96
4. Configure DNS64 for IPv6 clients and add route to 64:ff9b::/96
via NAT64 host.

Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D6434

MFC r304048:
Replace __noinline with special debug macro NAT64NOINLINE.

MFC r304061:
Use %ju to print unsigned 64-bit value.

MFC r304076:
Make statistics nat64lsn, nat64stl an nptv6 output netstat-like:
"@value @description" and fix build due to -Wformat errors.

MFC r304378 (by bz):
Try to fix gcc compilation errors (which are right).
nat64_getlasthdr() returns an int, which can be -1 in case of error,
storing the result in an uint8_t and then comparing to < 0 is not
helpful. Do what is done in the rest of the code and make proto an
int here as well.

MFC r309187:
Fix ICMPv6 Time Exceeded error message translation.

MFC r314718:
Use new ipfw_lookup_table() in the nat64 too.

MFC r315204,315233:
Use memset with structure size.

MFC r307570:
Add support for non-contiguous IPv6 masks in ipfw(8) rules.

For example fe::640:0:0/ffff::ffff:ffff:0:0 will match
addresses fe:*:*:*:0:640:*:*

Submitted by: Eugene Mamchits <mamchits at yandex-team dot ru>
Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r303012:
Add ipfw_nptv6 module that implements Network Prefix Translation for IPv6
as defined in RFC 6296. The module works together with ipfw(4) and
implemented as its external action module. When it is loaded, it registers
as eaction and can be used in rules. The usage pattern is similar to
ipfw_nat(4). All matched by rule traffic goes to the NPT module.

Reviewed by: hrs
Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D6420

MFC r304049:
Add `stats reset` command implementation to NPTv6 module
to be able reset statistics counters.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r304076:
Make statistics nat64lsn, nat64stl an nptv6 output netstat-like:
"@value @description" and fix build due to -Wformat errors.

MFC r314507:
Fix NPTv6 rule counters when one_pass is not enabled.

Consider the rule matching when both @done and @retval values
returned from ipfw_run_eaction() are zero. And modify ipfw_nptv6()
to return IP_FW_DENY and @done=0 when addresses do not match.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

MFC r303018:
Add named dynamic states support to ipfw(4).

The keep-state, limit and check-state now will have additional argument
flowname. This flowname will be assigned to dynamic rule by keep-state
or limit opcode. And then can be matched by check-state opcode or
O_PROBE_STATE internal opcode. To reduce possible breakage and to maximize
compatibility with old rulesets default flowname introduced.
It will be assigned to the rules when user has omitted state name in
keep-state and check-state opcodes. Also if name is ambiguous (can be
evaluated as rule opcode) it will be replaced to default.

Reviewed by: julian
Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D6674

MFC r304087:
Do not warn about ambiguous state name when we inspect a comment token.

MFC r304089:
Add an ability to attach comment to check-state rules.

MFC r310727 (by marius):
Fix a bug in r272840; given that the optlen parameter of setsockopt(2)
is a 32-bit socklen_t, do_get3() passes the kernel to access the wrong
32-bit half on big-endian LP64 machines when simply casting the 64-bit
size_t optlen to a socklen_t pointer.
While at it and given that the intention of do_get3() apparently is to
hide/wrap the fact that socket options are used for communication with
ipfw(4), change the optlen parameter of do_set3() to be of type size_t
and as such more appropriate than uintptr_t, too.

MFC r315305:
Change the syntax of ipfw's named states.

Since the state name is an optional argument, it often can conflict
with other options. To avoid ambiguity now the state name must be
prefixed with a colon.

Sponsored by: Yandex LLC

MFC r304572 (by bz):
Remove the kernel optoion for IPSEC_FILTERTUNNEL, which was deprecated
more than 7 years ago in favour of a sysctl in r192648.

MFC r305122:
Remove redundant sanity checks from ipsec[46]_common_input_cb().

This check already has been done in the each protocol callback.

MFC r309144,309174,309201 (by fabient):
IPsec RFC6479 support for replay window sizes up to 2^32 - 32 packets.

Since the previous algorithm, based on bit shifting, does not scale
with large replay windows, the algorithm used here is based on
RFC 6479: IPsec Anti-Replay Algorithm without Bit Shifting.
The replay window will be fast to be updated, but will cost as many bits
in RAM as its size.

The previous implementation did not provide a lock on the replay window,
which may lead to replay issues.

Obtained from: emeric.poupon@stormshield.eu
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D8468

MFC r309143,309146 (by fabient):
In a dual processor system (2*6 cores) during IPSec throughput tests,
we see a lot of contention on the arc4 lock, used to generate the IV
of the ESP output packets.

The idea of this patch is to split this mutex in order to reduce the
contention on this lock.

Update r309143 to prevent false sharing.

Reviewed by: delphij, markm, ache
Approved by: so
Obtained from: emeric.poupon@stormshield.eu
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D8130

MFC r313330:
Merge projects/ipsec into head/.

Small summary
-------------

o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
option IPSEC_SUPPORT added. It enables support for loading
and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
support was removed. Added TCP/UDP checksum handling for
inbound packets that were decapsulated by transport mode SAs.
setkey(8) modified to show run-time NAT-T configuration of SA.
o New network pseudo interface if_ipsec(4) added. For now it is
build as part of ipsec.ko module (or with IPSEC kernel).
It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
methods. The only one header file <netipsec/ipsec_support.h>
should be included to declare all the needed things to work
with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
Now these protocols are handled directly via IPsec methods.
o TCP_SIGNATURE support was reworked to be more close to RFC.
o PF_KEY SADB was reworked:
- now all security associations stored in the single SPI namespace,
and all SAs MUST have unique SPI.
- several hash tables added to speed up lookups in SADB.
- SADB now uses rmlock to protect access, and concurrent threads
can do SA lookups in the same time.
- many PF_KEY message handlers were reworked to reflect changes
in SADB.
- SADB_UPDATE message was extended to support new PF_KEY headers:
SADB_X_EXT_NEW_ADDRESS_SRC and SADB_X_EXT_NEW_ADDRESS_DST. They
can be used by IKE daemon to change SA addresses.
o ipsecrequest and secpolicy structures were cardinally changed to
avoid locking protection for ipsecrequest. Now we support
only limited number (4) of bundled SAs, but they are supported
for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
check for full history of applied IPsec transforms.
o References counting rules for security policies and security
associations were changed. The proper SA locking added into xform
code.
o xform code was also changed. Now it is possible to unregister xforms.
tdb_xxx structures were changed and renamed to reflect changes in
SADB/SPDB, and changed rules for locking and refcounting.

Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D9352

MFC r313331:
Add removed headers into the ObsoleteFiles.inc.

MFC r313561 (by glebius):
Move tcp_fields_to_net() static inline into tcp_var.h, just below its
friend tcp_fields_to_host(). There is third party code that also uses
this inline.

MFC r313697:
Remove IPsec related PCB code from SCTP.

The inpcb structure has inp_sp pointer that is initialized by
ipsec_init_pcbpolicy() function. This pointer keeps strorage for IPsec
security policies associated with a specific socket.
An application can use IP_IPSEC_POLICY and IPV6_IPSEC_POLICY socket
options to configure these security policies. Then ip[6]_output()
uses inpcb pointer to specify that an outgoing packet is associated
with some socket. And IPSEC_OUTPUT() method can use a security policy
stored in the inp_sp. For inbound packet the protocol-specific input
routine uses IPSEC_CHECK_POLICY() method to check that a packet conforms
to inbound security policy configured in the inpcb.

SCTP protocol doesn't specify inpcb for ip[6]_output() when it sends
packets. Thus IPSEC_OUTPUT() method does not consider such packets as
associated with some socket and can not apply security policies
from inpcb, even if they are configured. Since IPSEC_CHECK_POLICY()
method is called from protocol-specific input routine, it can specify
inpcb pointer and associated with socket inbound policy will be
checked. But there are two problems:
1. Such check is asymmetric, becasue we can not apply security policy
from inpcb for outgoing packet.
2. IPSEC_CHECK_POLICY() expects that caller holds INPCB lock and
access to inp_sp is protected. But for SCTP this is not correct,
becasue SCTP uses own locks to protect inpcb.

To fix these problems remove IPsec related PCB code from SCTP.
This imply that IP_IPSEC_POLICY and IPV6_IPSEC_POLICY socket options
will be not applicable to SCTP sockets. To be able correctly check
inbound security policies for SCTP, mark its protocol header with
the PR_LASTHDR flag.

Differential Revision: https://reviews.freebsd.org/D9538

MFC r313746:
Add missing check to fix the build with IPSEC_SUPPORT and without MAC.

MFC r313805:
Fix LINT build for powerpc.

Build kernel modules support only when both IPSEC and TCP_SIGNATURE
are not defined.

MFC r313922:
For translated packets do not adjust UDP checksum if it is zero.

In case when decrypted and decapsulated packet is an UDP datagram,
check that its checksum is not zero before doing incremental checksum
adjustment.

MFC r314339:
Document that the size of AH ICV for HMAC-SHA2-NNN should be half of
NNN bits as described in RFC4868.

PR: 215978

MFC r314812:
Introduce the concept of IPsec security policies scope.

Currently are defined three scopes: global, ifnet, and pcb.
Generic security policies that IKE daemon can add via PF_KEY interface
or an administrator creates with setkey(8) utility have GLOBAL scope.
Such policies can be applied by the kernel to outgoing packets and checked
agains inbound packets after IPsec processing.
Security policies created by if_ipsec(4) interfaces have IFNET scope.
Such policies are applied to packets that are passed through if_ipsec(4)
interface.
And security policies created by application using setsockopt()
IP_IPSEC_POLICY option have PCB scope. Such policies are applied to
packets related to specific socket. Currently there is no way to list
PCB policies via setkey(8) utility.

Modify setkey(8) and libipsec(3) to be able distinguish the scope of
security policies in the `setkey -DP` listing. Add two optional flags:
'-t' to list only policies related to virtual *tunneling* interfaces,
i.e. policies with IFNET scope, and '-g' to list only policies with GLOBAL
scope. By default policies from all scopes are listed.

To implement this PF_KEY's sadb_x_policy structure was modified.
sadb_x_policy_reserved field is used to pass the policy scope from the
kernel to userland. SADB_SPDDUMP message extended to support filtering
by scope: sadb_msg_satype field is used to specify bit mask of requested
scopes.

For IFNET policies the sadb_x_policy_priority field of struct sadb_x_policy
is used to pass if_ipsec's interface if_index to the userland. For GLOBAL
policies sadb_x_policy_priority is used only to manage order of security
policies in the SPDB. For IFNET policies it is not used, so it can be used
to keep if_index.

After this change the output of `setkey -DP` now looks like:
# setkey -DPt
0.0.0.0/0[any] 0.0.0.0/0[any] any
in ipsec
esp/tunnel/87.250.242.144-87.250.242.145/unique:145
spid=7 seq=3 pid=58025 scope=ifnet ifname=ipsec0
refcnt=1
# setkey -DPg
::/0 ::/0 icmp6 135,0
out none
spid=5 seq=1 pid=872 scope=global
refcnt=1

Obtained from: Yandex LLC
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D9805

PR: 212018
Relnotes: yes
Sponsored by: Yandex LLC

MFC r309281:

Fix 'ipfw delete set N':
do not emit meaningless 'rule 0 not found' warning if set was already empty.

MFC r308673:
Add missing support of named lookup tables to the IPv6 code.

PR: 214419

MFC r303955:
Restore "nat global" support.

Now zero value of arg1 used to specify "tablearg", use the old "tablearg"
value for "nat global". Introduce new macro IP_FW_NAT44_GLOBAL to replace
hardcoded magic number to specify "nat global". Also replace 65535 magic
number with corresponding macro. Fix typo in comments.

PR: 211256

MFC r303842:
Fix constructing of setdscp opcode with tablearg keyword.

setdscp's argument can have zero value that conflicts with IP_FW_TARG
value. Always set high-order bit if parser doesn't find tablearg keyword.

MFC r303845:
Fix formatting of setfib opcode.

Zero fib is correct value and it conflicts with IP_FW_TARG.
Use bprint_uint_arg() only when opcode contains IP_FW_TARG,
otherwise just print numeric value with cleared high-order bit.

Approved by: re (kib)

MFC r303615:
An old tables implementation had all tables preallocated,
so when user did `ipfw table N flush` it always worked, but now
when table N doesn't exist the kernel returns ESRCH error.
This isn't fatal error for flush and destroy commands. Do not
call err(3) when errno is equal to ESRCH. Also warn only when
quiet mode isn't enabled. This fixes a regression in behavior,
when old rules are loaded from file.
Also use correct value for switch in the table_swap().

Reported by: Kevin Oberman
Approved by: re (kib)

MFC r302561,302565:
Flush buffer after output. This fixes adding new data to already
printed flows.

PR: 210882
Approved by: re (kib)

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Hide warning about non-existent lookup tables and informational messages
about modified table entry when quied mode enabled.

Approved by: re (hrs)
Obtained from: Yandex LLC

Belatedly bump .Dd date for Dummynet AQM import in r300779.

Import Dummynet AQM version 0.2.1 (CoDel, FQ-CoDel, PIE and FQ-PIE).

Centre for Advanced Internet Architectures

Implementing AQM in FreeBSD

* Overview <http://caia.swin.edu.au/freebsd/aqm/index.html>

* Articles, Papers and Presentations
<http://caia.swin.edu.au/freebsd/aqm/papers.html>

* Patches and Tools <http://caia.swin.edu.au/freebsd/aqm/downloads.html>

Overview

Recent years have seen a resurgence of interest in better managing
the depth of bottleneck queues in routers, switches and other places
that get congested. Solutions include transport protocol enhancements
at the end-hosts (such as delay-based or hybrid congestion control
schemes) and active queue management (AQM) schemes applied within
bottleneck queues.

The notion of AQM has been around since at least the late 1990s
(e.g. RFC 2309). In recent years the proliferation of oversized
buffers in all sorts of network devices (aka bufferbloat) has
stimulated keen community interest in four new AQM schemes -- CoDel,
FQ-CoDel, PIE and FQ-PIE.

The IETF AQM working group is looking to document these schemes,
and independent implementations are a corner-stone of the IETF's
process for confirming the clarity of publicly available protocol
descriptions. While significant development work on all three schemes
has occured in the Linux kernel, there is very little in FreeBSD.

Project Goals

This project began in late 2015, and aims to design and implement
functionally-correct versions of CoDel, FQ-CoDel, PIE and FQ_PIE
in FreeBSD (with code BSD-licensed as much as practical). We have
chosen to do this as extensions to FreeBSD's ipfw/dummynet firewall
and traffic shaper. Implementation of these AQM schemes in FreeBSD
will:
* Demonstrate whether the publicly available documentation is
sufficient to enable independent, functionally equivalent implementations

* Provide a broader suite of AQM options for sections the networking
community that rely on FreeBSD platforms

Program Members:

* Rasool Al Saadi (developer)

* Grenville Armitage (project lead)

Acknowledgements:

This project has been made possible in part by a gift from the
Comcast Innovation Fund.

Submitted by: Rasool Al-Saadi <ralsaadi@swin.edu.au>
X-No objection: core
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D6388

Make `ipfw internal olist` output more user friendly.
Print object type as string for known types.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

Make named objects set-aware. Now it is possible to create named
objects with the same name in different sets.

Add optional manage_sets() callback to objects rewriting framework.
It is intended to implement handler for moving and swapping named
object's sets. Add ipfw_obj_manage_sets() function that implements
generic sets handler. Use new callback to implement sets support for
lookup tables.
External actions objects are global and they don't support sets.
Modify eaction_findbyname() to reflect this.
ipfw(8) now may fail to move rules or sets, because some named objects
in target set may have conflicting names.
Note that ipfw_obj_ntlv type was changed, but since lookup tables
actually didn't support sets, this change is harmless.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

sbin: minor spelling fixes.

No functional change.

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Prune bogus svn:mergeinfo from directores on which it should not ever
be present.

Sponsored by: The FreeBSD Foundation

Add External Actions KPI to ipfw(9).

It allows implementing loadable kernel modules with new actions and
without needing to modify kernel headers and ipfw(8). The module
registers its action handler and keyword string, that will be used
as action name. Using generic syntax user can add rules with this
action. Also ipfw(8) can be easily modified to extend basic syntax
for external actions, that become a part base system.
Sample modules will coming soon.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

Fix output formatting of O_UNREACH6 opcode.

Obtained from: Yandex LLC

Set buffer to empty string to prevent duplicated output in some cases.

PR: 193888

Upgrade our copies of clang, llvm, lldb and compiler-rt to 3.8.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang will soon be available here:
<http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.8.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Davide Italiano and Antoine Brodin
for their help.

Relnotes: yes

Fix bug in filling and handling ipfw's O_DSCP opcode.
Due to integer overflow CS4 token was handled as BE.

PR: 207459
MFC after: 1 week

Improve collation string and locales support

Merge collation support from Illumos and DragonflyBSD.

Locales are now generated with the new localedef(1) tool from CLDR POSIX files.
The generated files are now identified as "BSD 1.0" format.

The libc now only read "BSD 1.0" locales definitions, all other version will be
set to "C"
The localedef(1) tool has been imported from Illumos and modified to use tree(3)
instead of the CDDL avl(3)
A set of tool created by edwin@ and extended by marino@ for dragonfly has been
added to be able to generate locales and the Makefiles from the vanilla CLDR
unicode databases + a universal UTF-8 charmap (by marino@)
Update the locales to unicode v27
Given our regex(3) does not support multibyte (yet) it has been forced to always
use locale C
Remove now unused colldef(1) and mklocale(1)
Finish implementing the numeric BSD extension for ctypes
The number of supported locales has grown from 175 to 250 locales. Among the new
locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish
locales.
Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS,
zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components
version when possible.

Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under
BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able
to generate locales definition usable in freebsd sources out of vanilla CLDR
definitions, John Marino (DragonflyBSD) who first merge the Illumos work into
Dragonfly and spent hours tracking down bugs.

Implement `ipfw internal olist` command to list named objects.

Reviewed by: melifaro
Obtained from: Yandex LLC
Sponsored by: Yandex LLC

Merge mpsutil(8) branch

mpsutil(8)/mprutil(8) are new utilities for managing LSI Fusion-MPT
2/3 controllers (mps(4) and mpr(4))

For now only informational commands have been implemented.

This utility has been written by scottl@ [1] and polished by myself[2]

Submitted by: scottl
Discussed with: scottl
Relnotes: yes
Sponsored by: Netflix [1]
Sponsored by: Gandi.net [2]

Fix a ton of speelling errors

arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337

Upgrade our copies of clang, llvm, lldb, compiler-rt and libc++ to 3.7.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.7.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.7.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Andrew Turner and Antoine Brodin for their help.

Exp-run: antoine
Relnotes: yes

Fix possible segmentation fault.

PR: 203494
MFC after: 1 week

Code cleanup unused-but-set-variable spotted by gcc.

Reviewed by: melifaro
Approved by: bapt (mentor)
Differential Revision: D3473

In ipfw2, avoid left-shifting negative integers, which is undefined.
While here, make some other arguments to htonl(3) unsigned too.

MFC after: 3 days

sbin/ipfw fix typo: info -> into

example:

DEPRECATED: inserting data into non-existent table sshguard. (auto-created)

Approved by: bdrewery

Fix ipfw range deletion.

Spotted by: ian,julian

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Bring back support for checking tables via "ipfw -n".

Currently we have different table key types which can easily interfere
with each other (numbers and IPv4 address, interface names and hostnames,
flows and hostnames/addresses).
This conflicts are solved by [auto-]creating _typed_ tables, so after
table is created, only keys of given type can be inserted to that table.
ipfw(8) consults with kernel about key/value type for particular table so
it knows key/value interpretation.
However, we have 2 cases (adding entries to non-existing table and
parsing configuration file via `ipfw -n`) when kernel is unable to
provide us table info we need. Fix the latter case by partially importing
old `table_fill_xentry()` parse function responsible for guessing key type.

Sponsored by: Yandex LLC

Correctly print valtype for empty bitmask.

Generalize object reference handling in ipfw rules.
No ABI changes.

Fix use-after-free.

Reported by: Coverity Scan, pfg
CID: 1245747

Upgrade our copy of clang, llvm and lldb to 3.6.0 release.

Please note that from 3.5.0 onwards, clang/llvm/lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.6.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.6.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste for the lldb part of this upgrade.

Exp-run: antoine

Properly initialize scope zone id when next hop address stored
directly in the O_FORWARD_IP6 opcode. Use getnameinfo(3) to formatting
the IPv6 addresses of such opcodes.

Obtained from: Yandex LLC
Sponsored by: Yandex LLC

Fix `ipfw fwd tablearg'. Use dedicated field nh4 in struct table_value
to obtain IPv4 next hop address in tablearg case.

Add `fwd tablearg' support for IPv6. ipfw(8) uses INADDR_ANY as next hop
address in O_FORWARD_IP opcode for specifying tablearg case. For IPv6 we
still use this opcode, but when packet identified as IPv6 packet, we
obtain next hop address from dedicated field nh6 in struct table_value.

Replace hopstore field in struct ip_fw_args with anonymous union and add
hopstore6 field. Use this field to copy tablearg value for IPv6.

Replace spare1 field in struct table_value with zoneid. Use it to keep
scope zone id for link-local IPv6 addresses. Since spare1 was used
internally, replace spare0 array with two variables spare0 and spare1.

Use getaddrinfo(3)/getnameinfo(3) functions for parsing and formatting
IPv6 addresses in table_value. Use zoneid field in struct table_value
to store sin6_scope_id value.

Since the kernel still uses embedded scope zone id to represent
link-local addresses, convert next_hop6 address into this form before
return from pfil processing. This also fixes in6_localip() check
for link-local addresses.

Differential Revision: https://reviews.freebsd.org/D2015
Obtained from: Yandex LLC
Sponsored by: Yandex LLC

Retrieve counters from kernel if rule timstamping is requested.

PR: kern/197271
Submitted by: lev
Sponsored by: Yandex LLC

natd(8) will work with an unconfigured interface and effectively not do
anything until the interface is assigned an address. This fixes
ipfw_nat to do the same by using an IP of INADDR_ANY instead of
aborting the nat setup if the requested interface is not yet configured.

Differential Revision: https://reviews.freebsd.org/D1539
Reviewed by: melifaro, glebius, gnn
MFC after: 1 week

Upgrade our copy of clang, llvm and lldb to 3.5.0 release.

Please note that this version now requires C++11 support to build; see
UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.5.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Andrew Turner, Justin Hibbits and
Antoine Brodin for their invaluable help with this import.

Approved by: portmgr (antoine)
MFC after: 1 month

mdoc: remove EOL whitespace.

Convert sbin/ to LIBADD
Reduce overlinking

Misc mdoc fixes:

- Remove superfluous paragraph macros.
- Remove/fix empty or incorrect macros.
- Sort sections into conventional order.
- Terminate quoted strings properly.
- Remove EOL whitespace.

Fix documentation issue.

PR: 194581
Submitted by: madpilot

Fix displaying non-contiguous netmasks.

Found by: ae
Sponsored by: Yandex LLC

* Zero rule buffer.
* Rename 'read' variable.

Pointed by: luigi

* Fix table sets handling.
* Simplify formatting.

Suggested by: luigi

Show error when deleting non-existing rule number.

Found by: Oleg Ginzburg

* Fix zeroing individual entries via ipfw(8).
* Report error and return non-zero exit code if zeroing non-matched entries

Found by: Oleg Ginzburg

Partially fix build on !amd64

Pointed by: bz

Merge projects/ipfw to HEAD.

Main user-visible changes are related to tables:

* Tables are now identified by names, not numbers.
There can be up to 65k tables with up to 63-byte long names.
* Tables are now set-aware (default off), so you can switch/move
them atomically with rules.
* More functionality is supported (swap, lock, limits, user-level lookup,
batched add/del) by generic table code.
* New table types are added (flow) so you can match multiple packet fields at once.
* Ability to add different type of lookup algorithms for particular
table type has been added.
* New table algorithms are added (cidr:hash, iface:array, number:array and
flow:hash) to make certain types of lookup more effective.
* Table value are now capable of holding multiple data fields for
different tablearg users

Performance changes:
* Main ipfw lock was converted to rmlock
* Rule counters were separated from rule itself and made per-cpu.
* Radix table entries fits into 128 bytes
* struct ip_fw is now more compact so more rules will fit into 64 bytes
* interface tables uses array of existing ifindexes for faster match

ABI changes:
All functionality supported by old ipfw(8) remains functional.
Old & new binaries can work together with the following restrictions:
* Tables named other than ^\d+$ are shown as table(65535) in
ruleset in old binaries

Internal changes:.
Changing table ids to numbers resulted in format modification for
most sockopt codes. Old sopt format was compact, but very hard to
extend (no versioning, inability to add more opcodes), so
* All relevant opcodes were converted to TLV-based versioned IP_FW3-based codes.
* The remaining opcodes were also converted to be able to eliminate
all older opcodes at once
* All IP_FW3 handlers uses special API instead of calling sooptcopy*
directly to ease adding another communication methods
* struct ip_fw is now different for kernel and userland
* tablearg value has been changed to 0 to ease future extensions
* table "values" are now indexes in special value array which
holds extended data for given index
* Batched add/delete has been added to tables code
* Most changes has been done to permit batched rule addition.
* interface tracking API has been added (started on demand)
to permit effective interface tables operations
* O(1) skipto cache, currently turned off by default at
compile-time (eats 512K).

* Several steps has been made towards making libipfw:
* most of new functions were separated into "parse/prepare/show
and actuall-do-stuff" pieces (already merged).
* there are separate functions for parsing text string into "struct ip_fw"
and printing "struct ip_fw" to supplied buffer (already merged).
* Probably some more less significant/forgotten features

MFC after: 1 month
Sponsored by: Yandex LLC

Improve "reserved keywords" hack:

we can't easily predict (in current parsing model)
if the keyword is ipfw(8) reserved keyword or port name.
Checking proto database via getprotobyname() consumes a lot of
CPU and leads to tens of seconds for parsing large ruleset.
Use list of reserved keywords and check them as pre-requisite
before doing getprotobyname().

Obtained from: Yandex LLC

Whitespace/style changes merged from projects/ipfw.

Merge buffer-printing changes from from projects/ipfw as preparation
for branch merge.

Requested by: luigi

mdoc: remove superfluous paragraph macros.

use .Mt to mark up email addresses consistently (part1)

PR: 191174
Submitted by: Franco Fichtner <franco@lastsummer.de>

ECN marking implenetation for dummynet.
Changes include both DCTCP and RFC 3168 ECN marking methodology.

DCTCP draft: http://tools.ietf.org/html/draft-bensley-tcpm-dctcp-00

Submitted by: Midori Kato (aoimidori27@gmail.com)
Worked with: Lars Eggert (lars@netapp.com)
Reviewed by: luigi, hiren

Fix wrong formatting of 0.0.0.0/X table records in ipfw(8).

Add `flags` u16 field to the hole in ipfw_table_xentry structure.
Kernel has been guessing address family for supplied record based
on xent length size.
Userland, however, has been getting fixed-size ipfw_table_xentry structures
guessing address family by checking address by IN6_IS_ADDR_V4COMPAT().

Fix this behavior by providing specific IPFW_TCF_INET flag for IPv4 records.

PR: bin/189471
Submitted by: Dennis Yusupoff <dyr@smartspb.net>
MFC after: 2 weeks

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Fix world build WITHOUT_PF.

Sponsored by: Nginx, Inc.

Minor grammar fix

PR: docs/185057
Submitted by: Yuri (yuri@rawbw.com)
Approved by: gjb (mentor)

Add net.inet.ip.fw.dyn_keep_states sysctl which
re-links dynamic states to default rule instead of
flushing on rule deletion.
This can be useful while performing ruleset reload
(think about `atomic` reload via changing sets).
Currently it is turned off by default.

MFC after: 2 weeks
Sponsored by: Yandex LLC

Check ipfw table numbers in both user and kernel space before rule addition.

Found by: Saychik Pavel <umka@localka.net>
MFC after: 2 weeks
Sponsored by: Yandex LLC

Fix key lookup in ipfw(8) broken since r232865.
Print warning for IPv4 address strings which are valid in
inet_aton() but not valid in inet_pton(). (1)

Found by: Özkan KIRIK <ozkan.kirik@gmail.com>
Submitted by: Ian Smith <smithi@nimnet.asn.au> (1)
MFC after: 2 weeks
Sponsored by: Yandex LLC

Remove duplicated parapgraph.

MFC after: 3 days

Fix ipfw(8) sets of ipv6 addresses handling.
Conditionally use stack buffer instead of calling strdup().

PR: bin/104921
MFC after: 2 weeks

Minor spelling and grammar fixes.

Remove EOL whitespace.

Remove unused variable.

Add ipfw support for setting/matching DiffServ codepoints (DSCP).

Setting DSCP support is done via O_SETDSCP which works for both
IPv4 and IPv6 packets. Fast checksum recalculation (RFC 1624) is done for IPv4.
Dscp can be specified by name (AFXY, CSX, BE, EF), by value
(0..63) or via tablearg.

Matching DSCP is done via another opcode (O_DSCP) which accepts several
classes at once (af11,af22,be). Classes are stored in bitmask (2 u32 words).

Many people made their variants of this patch, the ones I'm aware of are
(in alphabetic order):

Dmitrii Tejblum
Marcelo Araujo
Roman Bogorodskiy (novel)
Sergey Matveichuk (sem)
Sergey Ryabin

PR: kern/102471, kern/121122
MFC after: 2 weeks

Do not suddenly fail on some rulesets if -n (syntax check only) is specified
and ipfw(4) module is not loaded.

MFC after: 2 weeks

Implement buffer size checking in ipfw(8) add cmd.

PR: bin/65961
Submitted by: Eugene Grosbein <eugen@grosbein.pp.ru>
MFC after: 2 weeks

Fix ipfw table argument parsing/printing.
Fix style.

PR: kern/175909
Submitted by: Daniel Hagerty <hag@linnaean.org>
MFC after: 2 weeks

Remove the recently added sysctl variable net.pfil.forward.
Instead, add protocol specific mbuf flags M_IP_NEXTHOP and
M_IP6_NEXTHOP. Use them to indicate that the mbuf's chain
contains the PACKET_TAG_IPFORWARD tag. And do a tag lookup
only when this flag is set.

Suggested by: andre

Remove the IPFIREWALL_FORWARD kernel option and make possible to turn
on the related functionality in the runtime via the sysctl variable
net.pfil.forward. It is turned off by default.

Sponsored by: Yandex LLC
Discussed with: net@
MFC after: 2 weeks

Whitespace fixes

MFC after: 2 weeks

Permit table to be used as IPv6 address.

Reported by: Serhiy Popov <sergiuspso@ukr.net>
MFC after: 2 weeks

Fix grammar in the portion about FIBs. Also, cross-reference
setfib(2) instead of setfib(1) for the 16-FIB limit.

PR: docs/157452
Approved by: hrs (mentor)

Whitespace cleanup for ipfw.8 -- start each sentence on a new line,
and put a comma after e.g. and i.e.. While here, wrap long lines.

PR: docs/157452
Approved by: hrs (mentor)

Remove unused values

remove the last __unused instance in sbin/ipfw.
This particular function (show_prerequisites() ) we should actually
remove the argument from the callers as well, but i'll do it at a
later time.

Fix some compile errors at high WARNS, including one
for an uninitialized variable.

unused parameters and variables are annotated with
(void)foo; /* UNUSED */
instead of __unused, because this code needs to build
also on linux and windows.

In ipfw(8), make the text about dynamic rules consistent.

PR: docs/120539
Approved by: gabor (mentor)
MFC after: 5 days

Make ipfw0 logging pseudo-interface clonable. It can be created automatically
by $firewall_logif rc.conf(5) variable at boot time or manually by ifconfig(8)
after a boot.

Discussed on: freebsd-ipfw@

- Make ipfw's sched rules case insensitive, for user-friendliness.
- Add a note to the ipfw(8) man page about the rules no longer being
case sensitive.
- Fix some typos in the man page.

PR: docs/164772
Reviewed by: bz
Approved by: gabor (doc mentor, src committer)
MFC after: 2 weeks

Update maximum number of tables available in ipfw to reflect
changes done in r233478.

Approved by: kib(mentor)
MFC after: 3 days

mdoc: use Po and Pc macros instead of parens. Also avoid starting a line
with Ns.

Remove trailing whitespace per mdoc lint warning

Disussed with: gavin
No objection from: doc
Approved by: joel
MFC after: 3 days

- Permit number of ipfw tables to be changed in runtime.

net.inet.ip.fw.tables_max is now read-write.

- Bump IPFW_TABLES_MAX to 65535
Default number of tables is still 128

- Remove IPFW_TABLES_MAX from ipfw(8) code.

Sponsored by Yandex LLC

Approved by: kib(mentor)

MFC after: 2 weeks

Remove superfluous paragraph macro.

- Add ipfw eXtended tables permitting radix to be used for any kind of keys.
- Add support for IPv6 and interface extended tables
- Make number of tables to be loader tunable in range 0..65534.
- Use IP_FW3 opcode for all new extended table cmds

No ABI changes are introduced. Old userland will see valid tables for
IPv4 tables and no entries otherwise. Flush works for any table.

IP_FW3 socket option is used to encapsulate all new opcodes:
/* IP_FW3 header/opcodes */
typedef struct _ip_fw3_opheader {
uint16_t opcode; /* Operation opcode */
uint16_t reserved[3]; /* Align to 64-bit boundary */
} ip_fw3_opheader;

New opcodes added:
IP_FW_TABLE_XADD, IP_FW_TABLE_XDEL, IP_FW_TABLE_XGETSIZE, IP_FW_TABLE_XLIST

ipfw(8) table argument parsing behavior is changed:
'ipfw table 999 add host' now assumes 'host' to be interface name instead of
hostname.

New tunable:
net.inet.ip.fw.tables_max controls number of table supported by ipfw in given
VNET instance. 128 is still the default value.

New syntax:
ipfw add skipto tablearg ip from any to any via table(42) in
ipfw add skipto tablearg ip from any to any via table(4242) out

This is a bit hackish, special interface name '\1' is used to signal interface
table number is passed in p.glob field.

Sponsored by Yandex LLC

Reviewed by: ae
Approved by: ae (mentor)

MFC after: 4 weeks

remove some write-only variables.
There is another block of code that is now useless as the computation
is done in the kernel.

Correct capitalization of "Hz" in user-visible text (manpages, printf(),
etc).

MFC after: 3 days

Merge multi-FIB IPv6 support from projects/multi-fibv6/head/:

Extend the so far IPv4-only support for multiple routing tables (FIBs)
introduced in r178888 to IPv6 providing feature parity.

This includes an extended rtalloc(9) KPI for IPv6, the necessary
adjustments to the network stack, and user land support as in netstat.

Sponsored by: Cisco Systems, Inc.
Reviewed by: melifaro (basically)
MFC after: 10 days

Bump .Dd for r231076.

Submitted by: bz

Make the 'tcpwin' option of ipfw(8) accept ranges and lists.

Submitted by: sem

Spelling fixes for sbin/

Replace index() and rindex() calls with strchr() and strrchr().

The index() and rindex() functions were marked LEGACY in the 2001
revision of POSIX and were subsequently removed from the 2008 revision.
The strchr() and strrchr() functions are part of the C standard.

This makes the source code a lot more consistent, as most of these C
files also call into other str*() routines. In fact, about a dozen
already perform strchr() calls.

- Add fallthrough comment

Approved by: pluknet
Found with: Coverity Prevent(tm)
CID: 10125

Fix parsing of redirect_addr argument.

PR: kern/162739
MFC after: 3 days

- fix duplicate "a a" in some comments

Submitted by: eadler
Approved by: simon
MFC after: 3 days

Note that NAT instance argument can be tablearg.

PR: misc/162265
Submitted by: Paul Procacci <pprocacci gmail.com>

Add support for IPv6 to ipfw fwd:
Distinguish IPv4 and IPv6 addresses and optional port numbers in
user space to set the option for the correct protocol family.
Add support in the kernel for carrying the new IPv6 destination
address and port.
Add support to TCP and UDP for IPv6 and fix UDP IPv4 to not change
the address in the IP header.
Add support for IPv6 forwarding to a non-local destination.
Add a regession test uitilizing VIMAGE to check all 20 possible
combinations I could think of.

Obtained from: David Dolson at Sandvine Incorporated
(original version for ipfw fwd IPv6 support)
Sponsored by: Sandvine Incorporated
PR: bin/117214
MFC after: 4 weeks
Approved by: re (kib)

Fix a regression where a rule containing a source port option after a
destination IP would incorrectly display the source port as a destination
port.

Reviewed by: luigi
Approved by: re (kib)
MFC after: 1 week

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

Add new rule actions "call" and "return" to ipfw. They make
possible to organize subroutines with rules.

The "call" action saves the current rule number in the internal
stack and rules processing continues from the first rule with
specified number (similar to skipto action). If later a rule with
"return" action is encountered, the processing returns to the first
rule with number of "call" rule saved in the stack plus one or higher.

Submitted by: Vadim Goncharov
Discussed by: ipfw@, luigi@

Improve error reporting. Use corresponding error message when file to be
preprocessed is missing. Also suggest to use absolute pathname if -p option
is specified.

PR: bin/156653
MFC after: 2 weeks

Actually, if code had followed style(9), there would be less stupid errors
like the one fixed in r223416.

Noticed by: julian

One more braino from me.

Pointy hat to: glebius
Submitted by: Alexander V. Chernikov <melifaro ipfw.ru>

Fix clang warnings.

Approved by: philip (mentor)

- Fix my braino in the 220835, when I used strtok(). It isn't
applicable here, since modifies the string. Switch to strchr().
- Restore support for undocumented optional parameters of
redir_port and redir_proto, that were disabled in 220835.
- While here, change !isalpha() checks on optinal parameters
for isdigit().

Submitted by: Alexander V. Chernikov <melifaro ipfw.ru>
PR: kern/143653

Implement "global" mode for ipfw nat. It is similar to natd(8)
"globalport" option for multiple NAT instances.

If ipfw rule contains "global" keyword instead of nat_number, then
for each outgoing packet ipfw_nat looks up translation state in all
configured nat instances. If an entry is found, packet aliased
according to that entry, otherwise packet is passed unchanged.

User can specify "skip_global" option in NAT configuration to exclude
an instance from the lookup in global mode.

PR: kern/157867
Submitted by: Alexander V. Chernikov (previous version)
Tested by: Eugene Grosbein

Check nat id a bit more strictly.

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Initialize co.use_set variable before parsing each new rule.

PR: bin/134975
MFC after: 2 weeks

Increase buffer size for the command line.

PR: bin/125370
Submitted by: sem
MFC after: 2 weeks

Add tablearg support for ipfw setfib.

PR: kern/156410
MFC after: 2 weeks

mdoc:
- use a proper macro for interface name ipfw0.
- add missing section number for bpf cross reference.

Rewrite NAT configuration parser, so that memory allocation size is
calculated dynamically.

PR: kern/143653

More whitespace fixes.

Checked with: md5, diff -x -w

Whitespace fixes.

Checked with: md5, diff -w

The first customer of the SO_USER_COOKIE option:
the "sockarg" ipfw option matches packets associated to
a local socket and with a non-zero so_user_cookie value.
The value is made available as tablearg, so it can be used
as a skipto target or pipe number in ipfw/dummynet rules.

Code by Paul Joe, manpage by me.

Submitted by: Paul Joe
MFC after: 1 week

mdoc: make pages render with mandoc

It's a bit more pedantic regarding .Bl list elements. This has an added
benefit of unbreaking the ipfw(8) manpage, where groff was silently
skipping one list element.

document logging through bpf

mdoc: drop redundant .Pp and .LP calls

They have no effect when coming in pairs, or before .Bl/.Bd

Fix incorrect usage of 'assure' and 'insure'.

Approved by: rrs (mentor)

expand_number(3) takes a uint64_t * now.

MFC after: 3 weeks

Fix typos, spelling, formatting and mdoc mistakes found by Nobuyuki while
translating these manual pages. Minor corrections by me.

Submitted by: Nobuyuki Koganemaru <n-kogane@syd.odn.ne.jp>

Document that the "ngtee" action no longer accepts packet, and
thus don't depend on one_pass flag anymore.

This is a POLA violation, but it is quite difficult to restore
the old behavior with new code. Also, the new behavior matches
behavior of the older "tee" action, and this is more intuitive.

better printing of headers when listing flows

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

fix 64-bit build

Reported by: Robert Noland

Slightly different handling of printf/snprintf for unaligned uint64_t,
which should improve readability, and also to ease the port to
platforms that do not support %llu

MFC after: 3 days

fix a buffer overflow with large (100k+) number of input lines.

MFC after: 3 days

Set net.inet6.ip6.fw.enable as well.

fix another bug in "ipfw set N ..."

Submitted by: Marcin Wisnicki

Tweak language to make one point potentially clearer for non-native spekers

PR: bin/121424
Submitted by: "Julian H. Stacey" <jhs berklix.org>

accept lower case m as a synonym for Mega (bit/s or bytes/s).

print correctly commands of the form

ipfw add 100 allow ip from { 1.2.3.4 or 5.6.7.8 }

(note that the above example could be better written as

ipfw add 100 allow dst-ip 1.2.3.4,5.6.7.8

Submitted by: Riccardo Panicucci

+ implement (two lines) the kernel side of 'lookup dscp N' to use the
dscp as a search key in table lookups;

+ (re)implement a sysctl variable to control the expire frequency of
pipes and queues when they become empty;

+ add 'queue number' as optional part of the flow_id. This can be
enabled with the command

queue X config mask queue ...

and makes it possible to support priority-based schedulers, where
packets should be grouped according to the priority and not some
fields in the 5-tuple.
This is implemented as follows:
- redefine a field in the ipfw_flow_id (in sys/netinet/ip_fw.h) but
without changing the size or shape of the structure, so there are
no ABI changes. On passing, also document how other fields are
used, and remove some useless assignments in ip_fw2.c

- implement small changes in the userland code to set/read the field;

- revise the functions in ip_dummynet.c to manipulate masks so they
also handle the additional field;

There are no ABI changes in this commit.

Implement "lookup dscp N" which does a lookup of the DSCP (top 6 bits
of ip->ip_tos) in a table. This can be useful to direct traffic to
different pipes/queues according to the DSCP of the packet, as follows:

ipfw add 100 queue tablearg lookup dscp 3 // table 3 maps dscp->queue

This change is a no-op (but harmless) until the two-line kernel
side is committed, which will happen shortly.

implement listing of a subset of pipes/queues/schedulers.
The filtering of the output is done in the kernel instead of userland
to reduce the amount of data transfered.

add back DPADD (removed by mistake in a previous commit)

more documentation on new dummynet features.

make the listing of queues/pipes/schedulers handle the case of
data size increasing while we fetch the info.

fix handling of sets

reduce diffs with the cross-platform version (windows needs
some extra initialization)

remove stale comment

Bring in the most recent version of ipfw and dummynet, developed
and tested over the past two months in the ipfw3-head branch. This
also happens to be the same code available in the Linux and Windows
ports of ipfw and dummynet.

The major enhancement is a completely restructured version of
dummynet, with support for different packet scheduling algorithms
(loadable at runtime), faster queue/pipe lookup, and a much cleaner
internal architecture and kernel/userland ABI which simplifies
future extensions.

In addition to the existing schedulers (FIFO and WF2Q+), we include
a Deficit Round Robin (DRR or RR for brevity) scheduler, and a new,
very fast version of WF2Q+ called QFQ.

Some test code is also present (in sys/netinet/ipfw/test) that
lets you build and test schedulers in userland.

Also, we have added a compatibility layer that understands requests
from the RELENG_7 and RELENG_8 versions of the /sbin/ipfw binaries,
and replies correctly (at least, it does its best; sometimes you
just cannot tell who sent the request and how to answer).
The compatibility layer should make it possible to MFC this code in a
relatively short time.

Some minor glitches (e.g. handling of ipfw set enable/disable,
and a workaround for a bug in RELENG_7's /sbin/ipfw) will be
fixed with separate commits.

CREDITS:
This work has been partly supported by the ONELAB2 project, and
mostly developed by Riccardo Panicucci and myself.
The code for the qfq scheduler is mostly from Fabio Checconi,
and Marta Carbone and Francesco Magno have helped with testing,
debugging and some bug fixes.

Fixed dependencies (make checkdpadd).

implement a new match option,

lookup {dst-ip|src-ip|dst-port|src-port|uid|jail} N

which searches the specified field in table N and sets tablearg
accordingly.
With dst-ip or src-ip the option replicates two existing options.
When used with other arguments, the option can be useful to
quickly dispatch traffic based on other fields.

Work supported by the Onelab project.

MFC after: 1 week

fix the indentation for addr: values

MFC after: 3 days

restore setting of sin_len (was removed in 1.146 last february) as
it seems that now it is necessary for 'forward' to work outside lo0.
The bug (and fix) was reported on 8.0. This patch probably applies
to RELENG_7 as well.
It seems that 'pf' has a similar bug.

Submitted by: Lytochkin Boris
MFC after: 3 days

fix argument type in the call to expand_number

Submitted by: gcc 4.3
MFC after: 3 days

use qsort_r instead of heapsort;
staticize two functions.

MFC after: 3 days

Fix minor resource leak in a function.

Reviewed by: luigi
MFC after: 1 week

Fix setfib(1) section number.

PR: 133765
Submitted by: Konstantin Zolotukhin <erebus@gorodok.net>
MFC after: 3 days

- 'burst' description rewritten.

Submitted by: Ben Kaduk
Approved by: re (kib)

o Kill grammar nits.

PR: docs/136061
Submitted by: Ben Kaduk
MFC after: 1 week

- fix dummynet 'fast' mode for WF2Q case.
- fix printing of pipe profile data.
- introduce new pipe parameter: 'burst' - how much data can be sent through
pipe bypassing bandwidth limit.

Permit the specification of bandwidth values within
"profile" files (bandwidth is mandatory when using a
profile, so it makes sense to have everything in one place).

Update the manpage accordingly.

Submitted by: Marta Carbone

add a missing format in a printf
Detected building with gcc 4.3.3

MFC after: 3 days

Several ipfw options and actions use a 16-bit argument to indicate
pipes, queues, tags, rule numbers and so on.
These are all different namespaces, and the only thing they have in
common is the fact they use a 16-bit slot to represent the argument.

There is some confusion in the code, mostly for historical reasons,
on how the values 0 and 65535 should be used. At the moment, 0 is
forbidden almost everywhere, while 65535 is used to represent a
'tablearg' argument, i.e. the result of the most recent table() lookup.

For now, try to use explicit constants for the min and max allowed
values, and do not overload the default rule number for that.

Also, make the MTAG_IPFW declaration only visible to the kernel.

NOTE: I think the issue needs to be revisited before 8.0 is out:
the 2^16 namespace limit for rule numbers and pipe/queue is
annoying, and we can easily bump the limit to 2^32 which gives
a lot more flexibility in partitioning the namespace.

MFC after: 5 days

remove a printf that was only useful for debugging.

MFC after: 3 days

Kill hard sentence break added in the previous revision.

Add emulation of delay profiles, which lets you model various
types of MAC overheads such as preambles, link level retransmissions
and more.

Note- this commit changes the userland/kernel ABI for pipes
(but not for ordinary firewall rules) so you need to rebuild
kernel and /sbin/ipfw to use dummynet features.

Please check the manpage for details on the new feature.

The MFC would be trivial but it breaks the ABI, so it will
be postponed until after 7.2 is released.

Interested users are welcome to apply the patch manually
to their RELENG_7 tree.

Work supported by the European Commission, Projects Onelab and
Onelab2 (contract 224263).

o Grammar.

Various cleanup of text, moving a couple of paragraphs
above to avoid referencing undefined terms (humans are not compilers
but still care about these things).

Change some .Sh to .Ss to better reflect the structure of the text.

No new content.

Remove contractions, reword a sentence to avoid a double negative,
and bump document date for previous change.

OKed by: piso

Improve a bit reass documentation:

-document fragment handling sysctls
-mention some caveats about fragments handling (and to deal with it)

Implement an ipfw action to reassemble ip packets: reass.

Mdoc style, spelling, grammar and wording fixes. This manpage needs more work.

move a variable declaration to the beginning of the block
(unfortunately, it is far away; we need to pack this code in
a better way).

remove some signed/unsigned and one const/!const warning

mark a function static, as it is

Add SCTP NAT support.

Submitted by: CAIA (http://caia.swin.edu.au)

Explain that we assume AF_INET and only use the addr and port field
from a struct sockaddr_in, so there is no need to initialize sin_len

remove duplicate #include

put the altq-related functions into a separate file.
Minor cleanup of the includes used by the various source files,
including annotations of why certain headers are used.

Avoid the use of duplicated typedefs -- see the comment for details.

fix printing of uint64_t values, so we can use WARNS=2

fix wrong variable usage...

Put nat and ipv6 support in their own files.

Usual moving of code with no changes from ipfw2.c to the
newly created files, and addition of prototypes to ipfw2.h

I have added forward declarations for ipfw_insn_* in ipfw2.h
to avoid a global dependency on ip_fw.h

Put dummynet-related code in a separate file.
To this purpose, add prototypes for global functions in ipfw2.h
and move there also the list of tokens used in various places in the code.

never mind, for the time being let's stick with WARNS=0 until
we sort out all proper printf formats.

Start splitting the monster file in smaller blocks.

In this episode:
- introduce a common header with a minimal set of common definitions;
- bring the main() function and options parser in main.c
- rename the main functions with an ipfw_ prefix

No code changes except for the introduction of a global variable,
resvd_set_number, which stores the RESVD_SET value from ip_fw.h
and is used to remove the dependency of main.c from ip_fw.h
(and the subtree of dependencies) for just a single constant.

put the usage() function inline, it was only 1 line and used once;
slightly reformat the help() text;
slightly correct the text for the 'extraneous filename' error message;

put all options in a single struct, and document them.

This will allow us to easily restore the original values when processing
commands from a file (where each individual line can have its own options).

I believe this is safe to build with WARNS=2 now

remove a couple of rarely used #define;

change PRINT_UINT from a macro to a function (renaming is
postponed to reduce clutter)

wrap all malloc/calloc/realloc calls so they exit on failure
without having to check in each place.

Remove an wrong strdup from previous commit.

Some implementations of getopt() expect that argv[0] is always the
program name, and ignore that entry. ipfw2.c code instead skips
this entry and starts with options at offset 0, relying on a more
tolerant implementation of the library.

This change fixes the issue by always passing a program name
in the first entry to getopt. The motivation for this change
is to remove a potential compatibility issue should we use
a different getopt() implementation in the future.

No functional changes.

Submitted by: Marta Carbone (parts)
MFC after: 4 weeks

remove some useless #include,
document why timeconv.h is needed

MFC after: 3 days

Fix a number of (innocuous) warnings, and remove a useless test.
There are still several signed/unsigned warnings left, which
require a bit more study for a proper fix.

This file has grown beyond reasonable limits.

We really need to split it into separate components (ipv4, ipv6,
dummynet, nat, table, userland-kernel communication ...) so we can
make mainteinance easier.

MFC after: 1 weeks

Update the ipfw man page to reflect last change (-q option with nat option).

MFC after: 3 days

Honor the quiet (-q) option while adding a nat rule.

Submitted by: Andrey V. Elsukov<bu7cher@yandex.ru>
MFC after: 3 days

o Remove a debug code and restore an accidentally deleted code
in a previous commit.

o Do nothing in show_nat() for a test mode (-n). This prevents
show_nat() from endless loop and makes work ipfw -n nat <...>.

PR: bin/128064
Submitted by: sem
MFC after: 1 month

Fix the build.

Noted by: ganbold@

* add all keyword for table list & flush actions.
* add tables_max sysctl.
* add default_rule sysctl.

PR: 127058 (partially)

Add keyword all in addtion to the table number for the 'list' and the
'flush' actions on tables. Part of PR: 127058.

PR: 127058 (based on)
MFC after: 1 month

Unbreak the build.

Add the check of the table number.

Move table list to a separate function.

Free allocated memory.

Remove some unused variables.

Style(9) the show_nat() function.

Do not do the useless job for an empty table.

MFC after: 1 month

Use IPFW_DEFAULT_RULE instead of hardcoded value since now it is
available.

MFC after: 5 days.

Trivial typo fix.

Approved by: gnn (mentor)

Slight wordsmithing. prompted by danger@

Document the use of the tablearg keyword together with the skipto command.

Note that setfib is not a terminal rule.

Change two variables to size_t to improve portability.
Submitted by: Xin Li

Add code to allow the system to handle multiple routing tables.
This particular implementation is designed to be fully backwards compatible
and to be MFC-able to 7.x (and 6.x)

Currently the only protocol that can make use of the multiple tables is IPv4
Similar functionality exists in OpenBSD and Linux.

From my notes:

-----

One thing where FreeBSD has been falling behind, and which by chance I
have some time to work on is "policy based routing", which allows
different
packet streams to be routed by more than just the destination address.

Constraints:
------------

I want to make some form of this available in the 6.x tree
(and by extension 7.x) , but FreeBSD in general needs it so I might as
well do it in -current and back port the portions I need.

One of the ways that this can be done is to have the ability to
instantiate multiple kernel routing tables (which I will now
refer to as "Forwarding Information Bases" or "FIBs" for political
correctness reasons). Which FIB a particular packet uses to make
the next hop decision can be decided by a number of mechanisms.
The policies these mechanisms implement are the "Policies" referred
to in "Policy based routing".

One of the constraints I have if I try to back port this work to
6.x is that it must be implemented as a EXTENSION to the existing
ABIs in 6.x so that third party applications do not need to be
recompiled in timespan of the branch.

This first version will not have some of the bells and whistles that
will come with later versions. It will, for example, be limited to 16
tables in the first commit.
Implementation method, Compatible version. (part 1)
-------------------------------
For this reason I have implemented a "sufficient subset" of a
multiple routing table solution in Perforce, and back-ported it
to 6.x. (also in Perforce though not always caught up with what I
have done in -current/P4). The subset allows a number of FIBs
to be defined at compile time (8 is sufficient for my purposes in 6.x)
and implements the changes needed to allow IPV4 to use them. I have not
done the changes for ipv6 simply because I do not need it, and I do not
have enough knowledge of ipv6 (e.g. neighbor discovery) needed to do it.

Other protocol families are left untouched and should there be
users with proprietary protocol families, they should continue to work
and be oblivious to the existence of the extra FIBs.

To understand how this is done, one must know that the current FIB
code starts everything off with a single dimensional array of
pointers to FIB head structures (One per protocol family), each of
which in turn points to the trie of routes available to that family.

The basic change in the ABI compatible version of the change is to
extent that array to be a 2 dimensional array, so that
instead of protocol family X looking at rt_tables[X] for the
table it needs, it looks at rt_tables[Y][X] when for all
protocol families except ipv4 Y is always 0.
Code that is unaware of the change always just sees the first row
of the table, which of course looks just like the one dimensional
array that existed before.

The entry points rtrequest(), rtalloc(), rtalloc1(), rtalloc_ign()
are all maintained, but refer only to the first row of the array,
so that existing callers in proprietary protocols can continue to
do the "right thing".
Some new entry points are added, for the exclusive use of ipv4 code
called in_rtrequest(), in_rtalloc(), in_rtalloc1() and in_rtalloc_ign(),
which have an extra argument which refers the code to the correct row.

In addition, there are some new entry points (currently called
rtalloc_fib() and friends) that check the Address family being
looked up and call either rtalloc() (and friends) if the protocol
is not IPv4 forcing the action to row 0 or to the appropriate row
if it IS IPv4 (and that info is available). These are for calling
from code that is not specific to any particular protocol. The way
these are implemented would change in the non ABI preserving code
to be added later.

One feature of the first version of the code is that for ipv4,
the interface routes show up automatically on all the FIBs, so
that no matter what FIB you select you always have the basic
direct attached hosts available to you. (rtinit() does this
automatically).

You CAN delete an interface route from one FIB should you want
to but by default it's there. ARP information is also available
in each FIB. It's assumed that the same machine would have the
same MAC address, regardless of which FIB you are using to get
to it.

This brings us as to how the correct FIB is selected for an outgoing
IPV4 packet.

Firstly, all packets have a FIB associated with them. if nothing
has been done to change it, it will be FIB 0. The FIB is changed
in the following ways.

Packets fall into one of a number of classes.

1/ locally generated packets, coming from a socket/PCB.
Such packets select a FIB from a number associated with the
socket/PCB. This in turn is inherited from the process,
but can be changed by a socket option. The process in turn
inherits it on fork. I have written a utility call setfib
that acts a bit like nice..

setfib -3 ping target.example.com # will use fib 3 for ping.

It is an obvious extension to make it a property of a jail
but I have not done so. It can be achieved by combining the setfib and
jail commands.

2/ packets received on an interface for forwarding.
By default these packets would use table 0,
(or possibly a number settable in a sysctl(not yet)).
but prior to routing the firewall can inspect them (see below).
(possibly in the future you may be able to associate a FIB
with packets received on an interface.. An ifconfig arg, but not yet.)

3/ packets inspected by a packet classifier, which can arbitrarily
associate a fib with it on a packet by packet basis.
A fib assigned to a packet by a packet classifier
(such as ipfw) would over-ride a fib associated by
a more default source. (such as cases 1 or 2).

4/ a tcp listen socket associated with a fib will generate
accept sockets that are associated with that same fib.

5/ Packets generated in response to some other packet (e.g. reset
or icmp packets). These should use the FIB associated with the
packet being reponded to.

6/ Packets generated during encapsulation.
gif, tun and other tunnel interfaces will encapsulate using the FIB
that was in effect withthe proces that set up the tunnel.
thus setfib 1 ifconfig gif0 [tunnel instructions]
will set the fib for the tunnel to use to be fib 1.

Routing messages would be associated with their
process, and thus select one FIB or another.
messages from the kernel would be associated with the fib they
refer to and would only be received by a routing socket associated
with that fib. (not yet implemented)

In addition Netstat has been edited to be able to cope with the
fact that the array is now 2 dimensional. (It looks in system
memory using libkvm (!)). Old versions of netstat see only the first FIB.

In addition two sysctls are added to give:
a) the number of FIBs compiled in (active)
b) the default FIB of the calling process.

Early testing experience:
-------------------------

Basically our (IronPort's) appliance does this functionality already
using ipfw fwd but that method has some drawbacks.

For example,
It can't fully simulate a routing table because it can't influence the
socket's choice of local address when a connect() is done.

Testing during the generating of these changes has been
remarkably smooth so far. Multiple tables have co-existed
with no notable side effects, and packets have been routes
accordingly.

ipfw has grown 2 new keywords:

setfib N ip from anay to any
count ip from any to any fib N

In pf there seems to be a requirement to be able to give symbolic names to the
fibs but I do not have that capacity. I am not sure if it is required.

SCTP has interestingly enough built in support for this, called VRFs
in Cisco parlance. it will be interesting to see how that handles it
when it suddenly actually does something.

Where to next:
--------------------

After committing the ABI compatible version and MFCing it, I'd
like to proceed in a forward direction in -current. this will
result in some roto-tilling in the routing code.

Firstly: the current code's idea of having a separate tree per
protocol family, all of the same format, and pointed to by the
1 dimensional array is a bit silly. Especially when one considers that
there is code that makes assumptions about every protocol having the
same internal structures there. Some protocols don't WANT that
sort of structure. (for example the whole idea of a netmask is foreign
to appletalk). This needs to be made opaque to the external code.

My suggested first change is to add routing method pointers to the
'domain' structure, along with information pointing the data.
instead of having an array of pointers to uniform structures,
there would be an array pointing to the 'domain' structures
for each protocol address domain (protocol family),
and the methods this reached would be called. The methods would have
an argument that gives FIB number, but the protocol would be free
to ignore it.

When the ABI can be changed it raises the possibilty of the
addition of a fib entry into the "struct route". Currently,
the structure contains the sockaddr of the desination, and the resulting
fib entry. To make this work fully, one could add a fib number
so that given an address and a fib, one can find the third element, the
fib entry.

Interaction with the ARP layer/ LL layer would need to be
revisited as well. Qing Li has been working on this already.

This work was sponsored by Ironport Systems/Cisco

Reviewed by: several including rwatson, bz and mlair (parts each)
Obtained from: Ironport systems/Cisco

Dummynet has a limit of 100 slots queue size (or 1MB, if you give
the limit in bytes) hard coded into both the kernel and userland.
Make both these limits a sysctl, so it is easy to change the limit.
If the userland part of ipfw finds that the sysctls don't exist,
it will just fall back to the traditional limits.

(100 packets is quite a small limit these days. If you want to test
TCP at 100Mbps, 100 packets can only accommodate a DBP of 12ms.)

Note these sysctls in the man page and warn against increasing them
without thinking first.

MFC after: 3 weeks

Add table/tablearg support to ipfw's nat.

MFC After: 1 week

-Fix display of nat range.
-Whitespace elimination.

Bug spotted by: Luiz Otavio O Souza
MFC After: 3 days

Fix display of multiple nat rules.

Bug spotted by: Luiz Otavio O Souza
PR: 120734
MFC After: 3 days

Instead of using a heuristic to decide whether to display
table 'values' as IP addresses, use an explicit argument (-i).
This is a 'POLA' issue. This is a low risk change and should be MFC'd
to RELENG_6 and RELENG 7. it might be put as an errata item for 6.3.
(not sure about 6.2).

Fix suggested by: Eugene Grosbein
PR: 120720
MFC After: 3 days

Add a note that ipfw states do not implicitly match ICMP error messages.

Hide ipfw internal data structures behind IPFW_INTERNAL rather than
exposing them to all consumers of ip_fw.h. These structures are
used in both ipfw(8) and ipfw(4), but not part of the user<->kernel
interface for other applications to use, rather, shared
implementation.

MFC after: 3 days
Reported by: Paul Vixie <paul at vix dot com>

o Fix ipfw(8) command line parser bug: "ipfw nat 1 config if" requires an argument.

PR: bin/119815
Submitted by: Dierk Sacher
MFC after: 1 week

Calculate p.fs.lookup_step correctly. This should prevent zeroing of
w_q_lookup table (used in RED algorithm for (1 - w_q)^t computation).

MFC after: 1 months

Polish this manual page a bit:

- refer to the dummynet(4) man page only once, later use rather
the .Nm macro.
- use .Va macro when refering to the sysctl variables
- grammar and markup fixes

Reviewed by: keramida, trhodes, ru (roughly)
MFC-after: 1 week

- New sysctl variable: net.inet.ip.dummynet.io_fast
If it is set to zero value (default) dummynet module will try to emulate
real link as close as possible (bandwidth & latency): packet will not leave
pipe faster than it should be on real link with given bandwidth.
(This is original behaviour of dummynet which was altered in previous commit)
If it is set to non-zero value only bandwidth is enforced: packet's latency
can be lower comparing to real link with given bandwidth.

- Document recently introduced dummynet(4) sysctl variables.

Requested by: luigi, julian
MFC after: 3 month

o Fix indentation. No functional changes.

Change IPTOS_CE to IPTOS_ECN_CE.

Approved by: njl (mentor)

Comply with the removal of IPTOS_CE and IPTOS_ECT.
Discussed on freebsd-net with no objections.

Approved by: njl (mentor), rwatson

o Fix a typo in ipfw table usage example.

PR: docs/117172
Submitted by: novel
MFC after: 1 week

o Cosmetic: fix the issue when "ipfw(8) show" produces "not" twice:

$ ipfw -n add 1 allow layer2 not mac-type ip
00001 allow ip from any to any layer2 not not mac-type 0x0800

PR: bin/115372
Submitted by: Andrey V. Elsukov
Approved by: re (hrs)
MFC after: 3 weeks

o Fix bug I introduced in the previous commit (ipfw set extention):
pack a set number correctly.

Submitted by: oleg

o Plug a memory leak.

Submitted by: oleg and Andrey V. Elsukov
Approved by: re (kensmith)
MFC after: 1 week

Rename option IPSEC_FILTERGIF to IPSEC_FILTERTUNNEL.
Also rename the related functions in a similar way.
There are no functional changes.

For a packet coming in with IPsec tunnel mode, the default is
to only call into the firewall with the "outer" IP header and
payload.

With this option turned on, in addition to the "outer" parts,
the "inner" IP header and payload are passed to the
firewall too when going through ip_input() the second time.

The option was never only related to a gif(4) tunnel within
an IPsec tunnel and thus the name was very misleading.

Discussed at: BSDCan 2007
Best new name suggested by: rwatson
Reviewed by: rwatson
Approved by: re (bmah)

Remove references to mpsafenet. This option no longer exists.

Approved by: re@ (bmah)

o Make ipfw set more robust -- now it is possible:
- to show a specific set: ipfw set 3 show
- to delete rules from the set: ipfw set 9 delete 100 200 300
- to flush the set: ipfw set 4 flush
- to reset rules counters in the set: ipfw set 1 zero

PR: kern/113388
Submitted by: Andrey V. Elsukov
Approved by: re (kensmith)
MFC after: 6 weeks

o Teach get_mac_addr_mask() to not silently accept incorrect MAC
addresses.
o Swap a couple of magic 6s by ETHER_ADDR_LEN.

PR: bin/80913
Submitted by: Andrey V. Elsukov
MFC after: 1 month

Add support for filtering on Routing Header Type 0 and
Mobile IPv6 Routing Header Type 2 in addition to filter
on the non-differentiated presence of any Routing Header.

MFC after: 3 weeks

o Make ipfw(8) show rules with mac/mac-type options correctly.

Before:

$ ipfw -n add 100 count icmp from any to any mac-type 0x01
00100 count icmp 0x0001
$ ipfw -n add 100 count icmp from any to any mac any any
00100 count icmp MAC any any any

After:

$ ipfw -n add 100 count icmp from any to any mac-type 0x01
00100 count icmp from any to any mac-type 0x0001
$ ipfw -n add 100 count icmp from any to any mac any any
00100 count icmp from any to any MAC any any

PR: bin/112244
Submitted by: Andrey V. Elsukov
MFC after: 1 month

o Add missed w/space in the error message.

Spotted by: Ivan Voras
MFC after: 1 week

Mention the nat command in the synopsis and in the action section.

Approved by: glebius (mentor)

Fix a parsing bug when specifying more than one address with dotted decimal
netmask.

Reported by: Igor Anishchuk
PR: kern/107565
MFC after: 3 days

Summer of Code 2005: improve libalias - part 2 of 2

With the second (and last) part of my previous Summer of Code work, we get:

-ipfw's in kernel nat

-redirect_* and LSNAT support

General information about nat syntax and some examples are available
in the ipfw (8) man page. The redirect and LSNAT syntax are identical
to natd, so please refer to natd (8) man page.

To enable in kernel nat in rc.conf, two options were added:

o firewall_nat_enable: equivalent to natd_enable

o firewall_nat_interface: equivalent to natd_interface

Remember to set net.inet.ip.fw.one_pass to 0, if you want the packet
to continue being checked by the firewall ruleset after being
(de)aliased.

NOTA BENE: due to some problems with libalias architecture, in kernel
nat won't work with TSO enabled nic, thus you have to disable TSO via
ifconfig (ifconfig foo0 -tso).

Approved by: glebius (mentor)

Add a note about rule syntax compared to the shell used so users do not get
frustraited when:
ipfw add 201 deny ip from any to table(2) in via xl1
returns "Badly placed ( )'s"

PR: 73638

When addr/mask examples are given, show both a host and network
address, to avoid confusing the users that a full address is
always required.

Submitted by: Josh Paetzel <josh@tcbug.org> (through freebsd-doc)
MFC after: 3 days

o Check for a required "pathname" argument presence.

PR: bin/95146
Submitted by: candy-sendpr@kgc.co.jp
MFC after: 3 weeks

Markup fixes.

Check the length of the ipv4 and ipv6 address lists. It must be less
than F_LEN_MASK.

MFC after: 5 days

Use bzero() to clear the whole ipfw_insn_icmp6 structure in fill_icmp6types(),
otherwise this command

ipfw add allow ipv6-icmp from any to 2002::1 icmp6types 1,2,128,129

turns into icmp6types 1,2,32,33,34,...94,95,128,129

PR: 102422 (part 1)
Submitted by: Andrey V. Elsukov <bu7cher at yandex.ru>
MFC after: 5 days

A pipe bandwidth of 10MBits/s should probably
be understood as 10Mbits/s not 10MBytes/s.

Submitted by: Gavin McCullagh <gavin.mccullagh@nuim.ie>
MFC after: 1 week

Regigle parens to try and get the intended affect. This should fix people
having trouble with the "me6" keyword. Also, we were using inet_pton on
the wrong variable in one place.

Reviewed by: mlaier (previous version of patch)
Obtained from: Sascha Blank (inet_pton change)
MFC after: 1 week

Fix typo.

comply with style police

Submitted by: ru
MFC after: 1 month

Allow ipfw to forward to a destination that is specified by a table.
for example:
fwd tablearg ip from any to table(1)
where table 1 has entries of the form:
1.1.1.0/24 10.2.3.4
208.23.2.0/24 router2

This allows trivial implementation of a secondary routing table implemented
in the firewall layer.

I expect more work (under discussion with Glebius) to follow this to clean
up some of the messy parts of ipfw related to tables.

Reviewed by: Glebius
MFC after: 1 month

Take IP_FIREWALL_EXTENDED out of the man page too.
MFC after: 1 week

Use the SLIST_NEXT macro instead of sle_next.

Checked with: cmp(1)

Specify correct argument range for tag/untag keywords.

Approved by: glebius (mentor)

Add support of 'tablearg' feature for:
- 'tag' & 'untag' action parameters.
- 'tagged' & 'limit' rule options.
Rule examples:
pipe 1 tag tablearg ip from table(1) to any
allow ip from any to table(2) tagged tablearg
allow tcp from table(3) to any 25 setup limit src-addr tablearg

sbin/ipfw/ipfw2.c:
1) new macros
GET_UINT_ARG - support of 'tablearg' keyword, argument range checking.
PRINT_UINT_ARG - support of 'tablearg' keyword.
2) strtoport(): do not silently truncate/accept invalid port list expressions
like: '1,2-abc' or '1,2-3-4' or '1,2-3x4'. style(9) cleanup.

Approved by: glebius (mentor)
MFC after: 1 month

Print dynamic rules for IPv6 as well.

PR: bin/98349
Submitted by: Mark Andrews
MFC after: 2 weeks

Implement internal (i.e. inside kernel) packet tagging using mbuf_tags(9).
Since tags are kept while packet resides in kernelspace, it's possible to
use other kernel facilities (like netgraph nodes) for altering those tags.

Submitted by: Andrey Elsukov <bu7cher at yandex dot ru>
Submitted by: Vadim Goncharov <vadimnuclight at tpu dot ru>
Approved by: glebius (mentor)
Idea from: OpenBSD PF
MFC after: 1 month

For src/dest parsing take off the netmask before checking for AF with
inet_pton. This fixes cases like "fe02::/16".

PR: bin/91245
Reported by: Fredrik Lindberge

Update manpage for net.inet6.ip6.fw.enable sysctl.

Requested by: bz

Amazing.. two screwups in one commit.
I'm piling on thise pointy hats on top of each other.
At least they nest..

I can't believe that no-one noticed that I broke ipfw table del
for over a month!
put {} around if clause with multiple statements

Revert `proto ip' back to the previous behavior. The kernel side of
ipfw2 doesn't allow zero as protocol number.

MFC after: 3 days

oops, mismerge from working sources.. not only add new code,
but remove old code!

Stop ipfw from aborting when asked to delete a table entry that
doesn't exist or add one that is already present, if the -q flag
is set. Useful for "ipfw -q /dev/stdin" when the command above is
invoked from something like python or TCL to feed commands
down the throat of ipfw.
MFC in: 1 week

Fix a markup glitch.

Forget about ipfw1 and ipfw2. We aren't in RELENG_4 anymore.

Document 'tablearg' keyword.

Wording by: emaste

[mdoc] add missing space before a punctuation type argument.

Add a new feature for optimizining ipfw rulesets - substitution of the
action argument with the value obtained from table lookup. The feature
is now applicable only to "pipe", "queue", "divert", "tee", "netgraph"
and "ngtee" rules.

An example usage:

ipfw pipe 1000 config bw 1000Kbyte/s
ipfw pipe 4000 config bw 4000Kbyte/s
ipfw table 1 add x.x.x.x 1000
ipfw table 1 add x.x.x.y 4000
ipfw pipe tablearg ip from table(1) to any

In the example above the rule will throw different packets to different pipes.

TODO:
- Support "skipto" action, but without searching all rules.
- Improve parser, so that it warns about bad rules. These are:
- "tablearg" argument to action, but no "table" in the rule. All
traffic will be blocked.
- "tablearg" argument to action, but "table" searches for entry with
a specific value. All traffic will be blocked.
- "tablearg" argument to action, and two "table" looks - for src and
for dst. The last lookup will match.

Cleanup _FreeBSD_version.

We couldn't specify the rule for filtering tunnel traffic since an
IPv6 support was committed:

- Stop treating `ip' and `ipv6' as special in `proto' option as they
conflict with /etc/protocols.

- Disuse `ipv4' in `proto' option as it is corresponding to `ipv6'.

- When protocol is specified as numeric, treat it as it is even it is
41 (ipv6).

- Allow zero for protocol as it is valid number of `ip'.

Still, we cannot specify an IPv6 over an IPv4 tunnel like before such
as:

pass ipv6 from any to any

But, now, you can specify it like:

pass ip4 from any to any proto ipv6

PR: kern/89472
Reported by: Ga l Roualland <gael.roualland__at__dial.oleane.com>
MFC after: 1 week

Catch up with ip_dummynet.h rev. 1.38 and fix build.

Garbage-collect now unused struct _ipfw_insn_pipe and flush_pipe_ptrs(),
thus removing a few XXXes.
Document the ABI breakage in UPDATING.

-mdoc sweep.

Restore the documentation about uid, gid or prison based rules requiring
that debug.mpsafenet be set to 0. It is still possible for dead locks to
occur while these filtering options are used due to the layering violation
inherent in their implementation.

Discussed: -current, rwatson, glebius

Redirect bridge(4) to if_bridge(4) and rename sysctl accordingly.

Reminded by: ru

* Add dynamic sysctl for net.inet6.ip6.fw.
* Correct handling of IPv6 Extension Headers.
* Add unreach6 code.
* Add logging for IPv6.

Submitted by: sysctl handling derived from patch from ume needed for ip6fw
Obtained from: is_icmp6_query and send_reject6 derived from similar
functions of netinet6,ip6fw
Reviewed by: ume, gnn; silence on ipfw@
Test setup provided by: CK Software GmbH
MFC after: 6 days

Bump document date. Remove EOL whitespace introduced in previous
commit. Start new line at sentence break in previous commit.

Approved by: re (implicit, fixing a commit made 5 minutes ago)

Document some limitations of uid/gid rules.

Approved by: re (rwatson)
MFC after: 3 days

Markup fixes.

Approved by: re (blanket)

add_proto() now fills proto for us so stop to 'guess' the protocol from the
command and rather trust the value add_proto filled in. While here, fix an
oversight in the pretty printing of ip6/4 options.

Better explain, then actually implement the IPFW ALTQ-rule first-match
policy. It may be used to provide more detailed classification of
traffic without actually having to decide its fate at the time of
classification.

MFC after: 1 week

Add support for IPv4 only rules to IPFW2 now that it supports IPv6 as well.
This is the last requirement before we can retire ip6fw.

Reviewed by: dwhite, brooks(earlier version)
Submitted by: dwhite (manpage)
Silence from: -ipfw

Unbreak handling of "ip[v]6" protocol and option flag. No more segfaults
and not every protocol is IPv6.

'ngtee' also depends on net.inet.ip.fw.one_pass.

IPFW version 2 is the only option now in HEAD. Do not confuse
users of future releases with instructions about building IPFW2
on RELENG_4.

Fix a the previous commit. I wanted to remove the if and always run the
body not remove both.

Reported by: ceri
Pointy hat: brooks

Don't force IPv6 proto to be printed numericaly.

Noticed by: ceri

Add IPv6 support to IPFW and Dummynet.

Submitted by: Mariano Tortoriello and Raffaele De Lorenzo (via luigi)

Be more specific when complaining about bit masks.

Bring back the full packet destination manipulation for 'ipfw fwd'
with the kernel compile time option:

options IPFIREWALL_FORWARD_EXTENDED

This option has to be specified in addition to IPFIRWALL_FORWARD.

With this option even packets targeted for an IP address local
to the host can be redirected. All restrictions to ensure proper
behaviour for locally generated packets are turned off. Firewall
rules have to be carefully crafted to make sure that things like
PMTU discovery do not break.

Document the two kernel options.

PR: kern/71910
PR: kern/73129
MFC after: 1 week

Expand *n't contractions.

Sort SEE ALSO.

Submitted by: ru

Document how interaction with ng_ipfw node is configured.

Add a ng_ipfw node, implementing a quick and simple interface between
ipfw(4) and netgraph(4) facilities.

Reviewed by: andre, brooks, julian

Don't print extra " via ", if we have already printed one. While here,
slightly style brackets.

PR: misc/75297
MFC after: 1 week

Sort sections.

Markup nits.

Deprecate unmaintainable uses of strncmp to implement abbreviations.
This commit replaces those with two new functions that simplify the code
and produce warnings that the syntax is deprecated. A small number of
sensible abbreviations may be explicitly added based on user feedback.

There were previously three types of strncmp use in ipfw:
- Most commonly, strncmp(av, "string", sizeof(av)) was used to allow av
to match string or any shortened form of it. I have replaced this
with a new function _substrcmp(av, "string") which returns 0 if av
is a substring of "string", but emits a warning if av is not exactly
"string".

- The next type was two instances of strncmp(av, "by", 2) which allowed
the abbreviation of bytes to "by", "byt", etc. Unfortunately, it
also supported "bykHUygh&*g&*7*ui". I added a second new function
_substrcmp2(av, "by", "bytes") which acts like the strncmp did, but
complains if the user doesn't spell out the word "bytes".

- There is also one correct use of strncmp to match "table(" which might
have another token after it without a space.

Since I changed all the lines anyway, I also fixed the treatment of
strncmp's return as a boolean in many cases. I also modified a few
strcmp cases as well to be fully consistent.

Scheduled mdoc(7) sweep.

Write some bit mask limits in hex rather than decimal so they look less
magic.

Update the IPFW man page to reflect reality. mpsafenet=0 is no longer
required when using ucred based rules.

Pointed out by: seanc (thanks!)
MFC after: 1 month

Remove a duplicate line from an apparent merge error in rev 1.63.

Be more clear that "bridged" is a synonym for "layer2".

PR: docs/44400
Submitted by: Constantin Stefanov <cstef at mail dot ru>

Refuse to unload the ipdivert module unless the 'force' flag is given to kldunload.

Reflect the fact that IPDIVERT is a loadable module in the divert(4) and ipfw(8)
man pages.

Add a note to the man page warning users about possible lock order
reversals+system lock ups if they are using ucred based rules
while running with debug.mpsafenet=1.

I am working on merging a shared locking mechanism into ipfw which
should take care of this problem, but it still requires a bit more
testing and review.

Reference altq(4) instead of pf.conf(5).

Tip of the hat to: mlaier

Commit forgotten documentation for "diverted" rules.

Remove blindly-copied extra include path.

Add support to IPFW for matching by TCP data length.

Add the documentation for IPFW's diverted(-loopback|-output) matches.

Add support to IPFW for classification based on "diverted" status
(that is, input via a divert socket).

Remove accidentally-added O_DIVERTED section.

Add to IPFW the ability to do ALTQ classification/tagging.

Since "d" is an array of 32 bit values, it is more
correct to change the cast from unsigned int to uint32_t.

Pointed out by: luigi

Prepare for 5.x soon becoming -STABLE.

Pointed out by: -current users

Make 'ipfw tee' behave as inteded and designed. A tee'd packet is copied
and sent to the DIVERT socket while the original packet continues with the
next rule. Unlike a normally diverted packet no IP reassembly attemts are
made on tee'd packets and they are passed upwards totally unmodified.

Note: This will not be MFC'd to 4.x because of major infrastucture changes.

PR: kern/64240 (and many others collapsed into that one)

Currently when ipfw(8) generates the micro-instructions for rules which
contain O_UID, O_GID and O_JAIL opcodes, the F_NOT or F_OR logical
operator bits get clobbered. Making it impossible to use the ``NOT'' or
``OR'' operators with uid, gid and jail based constraints.

The ipfw_insn instruction template contains a ``len'' element which
stores two pieces of information, the size of the instruction
(in 32-bit words) in the low 6 bits of "len" with the 2 remaining
bits to implement OR and NOT.

The current code clobbers the OR and NOT bits by initializing the
``len'' element to the size, rather than OR'ing the bits. This change
fixes this by changing the initialization of cmd->len to an OR operation
for the O_UID, O_GID and O_JAIL opcodes.

This may be a MFC candidate for RELENG_5.

Reviewed by: andre
Approved by: luigi
PR: kern/63961 (partially)

o Initialize a local variable and make gcc happy.

PR: bin/71485
Submitted by: Jukka A. Ukkonen

o Restore a historical ipfw1 logamount behaviour: rules with 'log'
keyword but without 'logamount' limit the amount of their log messages
by net.inet.ip.fw.verbose_limit sysctl value.

RELENG_5 candidate.

PR: kern/46080
Submitted by: Dan Pelleg
MFC after: 1 week

Fix 'show' command for pipes and queues.

PR: bin/70311
Submitted by: Pawel Malachowski <pawmal-posting@freebsd.lublin.pl>
MFC after: 3 days

Remove trailing whitespace and change "prisoniD" to "prisonID".

Pointed out by: simon
Approved by: bmilekic (mentor)

Add the ability to associate ipfw rules with a specific prison ID.
Since the only thing truly unique about a prison is it's ID, I figured
this would be the most granular way of handling this.

This commit makes the following changes:

- Adds tokenizing and parsing for the ``jail'' command line option
to the ipfw(8) userspace utility.
- Append the ipfw opcode list with O_JAIL.
- While Iam here, add a comment informing others that if they
want to add additional opcodes, they should append them to the end
of the list to avoid ABI breakage.
- Add ``fw_prid'' to the ipfw ucred cache structure.
- When initializing ucred cache, if the process is jailed,
set fw_prid to the prison ID, otherwise set it to -1.
- Update man page to reflect these changes.

This change was a strong motivator behind the ucred caching
mechanism in ipfw.

A sample usage of this new functionality could be:

ipfw add count ip from any to any jail 2

It should be noted that because ucred based constraints
are only implemented for TCP and UDP packets, the same
applies for jail associations.

Conceptual head nod by: pjd
Reviewed by: rwatson
Approved by: bmilekic (mentor)

New ipfw option "antispoof":

For incoming packets, the packet's source address is checked if it
belongs to a directly connected network. If the network is directly
connected, then the interface the packet came on in is compared to
the interface the network is connected to. When incoming interface
and directly connected interface are not the same, the packet does
not match.

Usage example:

ipfw add deny ip from any to any not antispoof in

Manpage education by: ru

Extend versrcreach by checking against the rt_flags for RTF_REJECT and
RTF_BLACKHOLE as well.

To quote the submitter:

The uRPF loose-check implementation by the industry vendors, at least on Cisco
and possibly Juniper, will fail the check if the route of the source address
is pointed to Null0 (on Juniper, discard or reject route). What this means is,
even if uRPF Loose-check finds the route, if the route is pointed to blackhole,
uRPF loose-check must fail. This allows people to utilize uRPF loose-check mode
as a pseudo-packet-firewall without using any manual filtering configuration --
one can simply inject a IGP or BGP prefix with next-hop set to a static route
that directs to null/discard facility. This results in uRPF Loose-check failing
on all packets with source addresses that are within the range of the nullroute.

Submitted by: James Jun <james@towardex.com>

Mechanically kill hard sentence breaks.

Fixed a bug spotted by compiling with -Wall.

Introduce a new feature to IPFW2: lookup tables. These are useful
for handling large sparse address sets. Initial implementation by
Vsevolod Lobko <seva@ip.net.ua>, refined by me.

MFC after: 1 week

o Move NEED1 macro to the top of the source file.

o Add sanity checking to the firewall delete operation
which tells the user that a firewall rule
specification is required.

The previous behaviour was to exit without reporting any
errors to the user.

Approved by: bmilekic (mentor)

o Fix usage example.

PR: docs/67065
Submitted by: David Syphers