Do the actual pfr_strerror() to pf_strerror() rename

Missed in previous

Enable pfctl(8) to recursively flush rules and tables from PF driver. The
recursive operation ("pfctl -a '*' ...") works for '-s' option already. This
change enables the same thing for '-F' option, so "pfctl -a '*' -Fa" will flush
everything from PF driver.

The idea was discussed with many on tech@ in spring 2019.

OK kn@

When creating tables inside anchors, pfctl warned about namespace
collisions with global tables, but only in certain cases and with
limited information sometimes leaving users clueless.

Deferring the check to process_tabledefs() where tables are eventually
created, both anchor and table name are known which allows for checking
all existing anchors.

With this, warn on all duplicates even in dry-runs (`-n') and print
quoted names so they can be copied to fix configurations right away.

No functional change in parsing or ruleset production.

Discussed with and OK sashan

Error out on missing table command, zap internal wrapper function

Table name and table command require each other as reflected in the
synopsis [-t table -T command [address ...]], so print usage and exit if
only one of them is given.

By moving the inter-dependence check right after option parsing is done,
we can bail out even before opening pf(4) and drop the internal wrapper
pfctl_command_tables() as unneeded indirection with now duplicate checks.

OK sashan

Zap unused segment struct definition

There since import and last used by ALTQ which henning removed in 2004.

OK sashan

Remove unused af argument from unmask()

This has been unused for years.

While here, zap the duplicate function signature from pfctl.h (already
present in pfctl_parser.h); spotted by sashan, thanks.

OK sashan

Move duplicate code into new helper print_addr_str()

This simply puts the wiggle around inet_ntop() from four into one location.

OK benno

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.

When creating tables inside anchors, pfctl warned about namespace
collisions with global tables, but only in certain cases and with
limited information sometimes leaving users clueless.

Deferring the check to process_tabledefs() where tables are eventually
created, both anchor and table name are known which allows for checking
all existing anchors.

With this, warn on all duplicates even in dry-runs (`-n') and print
quoted names so they can be copied to fix configurations right away.

No functional change in parsing or ruleset production.

Discussed with and OK sashan

Error out on missing table command, zap internal wrapper function

Table name and table command require each other as reflected in the
synopsis [-t table -T command [address ...]], so print usage and exit if
only one of them is given.

By moving the inter-dependence check right after option parsing is done,
we can bail out even before opening pf(4) and drop the internal wrapper
pfctl_command_tables() as unneeded indirection with now duplicate checks.

OK sashan

Zap unused segment struct definition

There since import and last used by ALTQ which henning removed in 2004.

OK sashan

Remove unused af argument from unmask()

This has been unused for years.

While here, zap the duplicate function signature from pfctl.h (already
present in pfctl_parser.h); spotted by sashan, thanks.

OK sashan

Move duplicate code into new helper print_addr_str()

This simply puts the wiggle around inet_ntop() from four into one location.

OK benno

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.

Error out on missing table command, zap internal wrapper function

Table name and table command require each other as reflected in the
synopsis [-t table -T command [address ...]], so print usage and exit if
only one of them is given.

By moving the inter-dependence check right after option parsing is done,
we can bail out even before opening pf(4) and drop the internal wrapper
pfctl_command_tables() as unneeded indirection with now duplicate checks.

OK sashan

Zap unused segment struct definition

There since import and last used by ALTQ which henning removed in 2004.

OK sashan

Remove unused af argument from unmask()

This has been unused for years.

While here, zap the duplicate function signature from pfctl.h (already
present in pfctl_parser.h); spotted by sashan, thanks.

OK sashan

Move duplicate code into new helper print_addr_str()

This simply puts the wiggle around inet_ntop() from four into one location.

OK benno

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.

Remove unused af argument from unmask()

This has been unused for years.

While here, zap the duplicate function signature from pfctl.h (already
present in pfctl_parser.h); spotted by sashan, thanks.

OK sashan

Move duplicate code into new helper print_addr_str()

This simply puts the wiggle around inet_ntop() from four into one location.

OK benno

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.

Move duplicate code into new helper print_addr_str()

This simply puts the wiggle around inet_ntop() from four into one location.

OK benno

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.

add option -N (no domain resolution)
manpage wording and reminder about usage() jmc@
ok florian@ henning@

sbin/pfctl: void functions and exit(3) on error

Changes:
voided:
- pfctl_clear_tables
- pfctl_show_tables
- pfctl_show_ifaces

Those functions now exit(3) in case of error instead
of passing it up to the callers (where it was ignored).

OK mikeb@, sashan@

DEFAULT_PRIORITY and DEFAULT_QLIMIT no longer used

remove altq bits here, too
(i was convinced i committed that yesterday already, hrm)

config bits for the bandwidth shaping part of the new queueing subsystem
syntax worked out with many in ljubljana using a whiteboard, testing &
looking over by many, ok phessler sthen

New attempt to make the -P flag work with -ss, so that states can be
printed with port names if desired.

tcpdump's pf_print_state.c has diverged significantly from pfctl's, so
the change to tcpdump's pf_print_state.c is not exactly the same as
pfctl's.

ok henning sthen

revert previous, breaks tcpdump
spotted by jmc@

Make the -P flag work with -ss, so that states can be printed with port
names if desired.

ok henning

Add support for weighted round-robin in load balancing pools and tables.
Diff from zinke@ with a some minor cleanup.
ok henning claudio deraadt

The ioctl to show states returns a pfsync_state which is in network byte
order and therefore a ntohs is needed to show the rdomain correctly.
OK henning@ dlg@

remove -A, -O, -R and -T load
the partial loading of a ruleset (leaving ancors aside) is wrong and
conflicts with the general idea of how pf works. last not least it breaks
with the optimizer generating tables automagically.
ok deraadt sthen krw manpage jmc

rtables are stacked on rdomains (it is possible to have multiple routing
tables on top of a rdomain) but until now our code was a crazy mix so that
it was impossible to correctly use rtables in that case. Additionally pf(4)
only knows about rtables and not about rdomains. This is especially bad when
tracking (possibly conflicting) states in various domains.
This diff fixes all or most of these issues. It adds a lookup function to
get the rdomain id based on a rtable id. Makes pf understand rdomains and
allows pf to move packets between rdomains (it is similar to NAT).
Because pf states now track the rdomain id as well it is necessary to modify
the pfsync wire format. So old and new systems will not sync up.
A lot of help by dlg@, tested by sthen@, jsg@ and probably more
OK dlg@, mpf@, deraadt@

Second half of PF state table rearrangement.
- Mechanical change: Use arrays for state key pointers in pf_state, and
addr/port in pf_state_key, to allow the use of indexes.
- Fix NAT, pfsync, pfctl, and tcpdump to handle the new state structures.
In struct pfsync_state, both state keys are included even when identical.
- Also fix some bugs discovered in the existing code during testing.
(in particular, "block return" for TCP packets was not returning an RST)

ok henning beck deraadt
tested by otto dlg beck laurent

Special thanks to users Manuel Pata and Emilio Perea who did enough testing
to actually find some bugs.

remove unused functions

from tobias@

ok mcbride@ tobias@

Cope with new ioctl interface (use pfsync_state instead of pf_state)

ok henning@ toby@ pyr@

use DIOCGETRULE ioctl & action set to PF_GET_CLR_CNTR to clear counters
with -z instead of DIOCCLRRULECTRS. Unbreaks -z with anchors and makes the
read & reset operation atomic.
innstrument pfctl_show_rules() to clear counters while reading rules and
add a new output format for it, showing nothing, if only resetting counters
without actually displaying them is requested. minor cleanups on the way.
ok dhartmei & agreement from theo and ryan

zap unused function
From: Andrey Matveev <evol@online.ptt.ru>

clean up and rework the interface absraction code big time, rip out multiple
useless layers of indirection and make the code way cleaner overall.
this is just the start, more to come...
worked very hard on by Ryan and me in Montreal last week, on the airplane to
vancouver and yesterday here in calgary. it hurt.
ok ryan theo

Modify pfctl behaviour so that 'set ...' options are no longer "sticky", ie.
they are reset to default values if omitted from a subsequent ruleset load.
Also:
- make sure 'set ...' options are not loaded in anchors.
- add a -m ("merge") flag to pfctl which allows an individual option to be set
without reseting the others, eg:
# echo "set loginterface fxp0" | pfctl -mf -

ok henning@ dhartmei@

Remove unused functions. ok beck@ henning@

Allow recursive anchors (anchors within anchors, up to 64
levels deep). More work required, but this is already
functional. authpf users will need to adjust their anchor
calls, but this will change again soon. ok beck@, cedric@,
henning@, mcbride@

Do not try to load directories. found+ok mpech@

Makes pfctl -ss and pfctl -sq use optional -i argument.
ok dhartmei@ markus@ mcbride@

add -i flag, use it for -sI as a start. ok henning@, ok+test mcbride@

KNF

Clean up 'pfctl -s all' output.

ok deraadt@ henning@

Many improvements to the handling of interfaces in PF.

1) PF should do the right thing when unplugging/replugging or cloning/
destroying NICs.

2) Rules can be loaded in the kernel for not-yet-existing devices
(USB, PCMCIA, Cardbus). For example, it is valid to write:
"pass in on kue0" before kue USB is plugged in.

3) It is possible to write rules that apply to group of interfaces
(drivers), like "pass in on ppp all"

4) There is a new ":peer" modifier that completes the ":broadcast"
and ":network" modifiers.

5) There is a new ":0" modifier that will filter out interface aliases.
Can also be applied to DNS names to restore original PF behaviour.

6) The dynamic interface syntax (foo) has been vastly improved, and
now support multiple addresses, v4 and v6 addresses, and all userland
modifiers, like "pass in from (fxp0:network)"

7) Scrub rules now support the !if syntax.

8) States can be bound to the specific interface that created them or
to a group of interfaces for example:

- pass all keep state (if-bound)
- pass all keep state (group-bound)
- pass all keep state (floating)

9) The default value when only keep state is given can be selected by
using the "set state-policy" statement.

10) "pfctl -ss" will now print the interface scope of the state.

This diff change the pf_state structure slighltly, so you should
recompile your userland tools (pfctl, authpf, pflogd, tcpdump...)

Tested on i386, sparc, sparc64 by Ryan
Tested on macppc, sparc64 by Daniel

ok deraadt@ mcbride@

KNF

fix cedric's breakage:
int is not the same as u_long
caused an integer overflow on our 64 bit archs and thus made pf not working
there

ok mcbride@ marc@ millert@ cedric@

Rearchitecture of the userland/kernel IOCTL interface for transactions.
This brings us close to 100% atomicity for a "pfctl -f pf.conf" command.
(some splxxx work remain in the kernel). Basically, improvements are:

- Anchors/Rulesets cannot disappear unexpectedly anymore.
- No more leftover in the kernel if "pfctl -f" fail.
- Commit is now done in a single atomic IOCTL.

WARNING: The kernel code is fully backward compatible, but the new
pfctl/authpf userland utilities will only run on a new kernel.

The following ioctls are deprecated (i.e. will be deleted sooner or
later, depending on how many 3rd party utilities use them and how soon
they can be upgraded):

- DIOCBEGINRULES
- DIOCCOMMITRULES
- DIOCBEGINALTQS
- DIOCCOMMITALTQS
- DIOCRINABEGIN
- DIOCRINADEFINE

They are replaced by the following ioctls (yes, PF(4) will follow)
which operate on a vector of rulesets:

- DIOCXBEGIN
- DIOCXCOMMIT
- DIOCXROLLBACK

Ok dhartmei@ mcbride@

Document interactions between tables and anchors.
Add a warning on global/anchor name clashes to help prevent mistakes from our
users during the 3.3 -> 3.4 switch.
ok henning@

Make table tickets per-ruleset instead of global.
Make table tickets u_int32_t for consistency with other parts of PF.
Ok dhartmei@ henning@

KNF after cedric (grmpf)

This patch finally cleanup pfctl_table.c. No more global buffer,
and a couple of parsing functions moved to parse.y or pfctl_parser
where they belong.

I also took the opportunity to replace "void" functions with exit(1)
or err() inside by "int" functions, with the caller checking the
return value for errors (much cleaner and an old request from Theo)

ok dhartmei@ henning@

Buffer management functions.
ok dhartmei@

Reorg part I: move 3 functions out of pf_table.c to pf_radix.c
ok dhartmei@

A table in an anchor creates a real anchor: pfctl -sA works.
The following two pfctl functions work with an "-a" option:
- pfctl [-a foo[:bar]] -sT
- pfctl [-a foo[:bar]] -FT
ok dhartmei@

Allow tables to be loaded into anchors.
Most pfctl table commands (excluding 'show' and 'flush') support the "-a"
modifier.
ok dhartmei@

let print_altq and print_queue take a struct node_queue_bw parameter instead
of dintinct bw_percent

qname_to_qid and qname_to_pfaltq can be private functions now; nothing
outside pfctl_altq.c uses them any more, nor should.

kill dead code.
qid_to_qname is not used anywhere. moreover, I cannot think of any
legitimate use; misuse is easy tho.

allow queue specs to be limited to certain interfaces.

altq on { $if0 $if1 $if2 $if3 } priq bandwidth 10Mb queue { one two }
queue one priority 1 priq(default)
queue two on $if0 priority 15
queue two on ! $if0 priority 0

ok dhartmei@

lotsa const char *
from David Hill <david at phobia.ms> a while ago

fix queue assignment on filter rules which are not bound to an interface.
when looking up the queue IDs using qname_to_qid, we do not need to limit
the matching on the interface in question, as it is guaranteed that same
named queues on different interfaces habe the same queue id. moreover, we
must not limit the matches to the interface if we do not have an interface
given on the filter rule to match on ;-)

found after problems reported by Andre Nathan <andre at v2r dot com dot br>

ok dhartmei@ pb@ cedric@

when printing queues at load time that have bandwidth specified in percent,
print the bandwidth in percent instead of the calculated absolute value.
if a queue belongs to more than one interface and they have different
bandwidth the calculated absolute is of course different per interface.
previously the first calculated absolute value was shown; what of course is
incorrect on the second interface. note that only the print was wrong, the
correct values were passed to the kernel.

ok theo daniel

allow macro definition on the command line:
pfctl -Dextif=wi0 -f /etc/pf.conf
command line macro definitions override the ones made in the file (idea
theo), very handy if your notebook has another NIC at some conference, as
well as for debugging etc.
idea rezine@mistrusted.net via pb@
hacked live at FOSDEM

ok pb@ dhartmei@ cedric@

let pfctl -vvsq loop and display measured bandwidth and packets/s per queue.
cbq only for now.

export rate2str

no named parameters in prototypes

ok dhartmei cedric henning

Improve pfctl -vvs{r,n} output with rule containing tables.
Shows the number of entries in the table or if the table is not active.
ok dhartmei@, no objections.

first attack at pfctl queue statistics, to be displayed with pfctl -vsq

This commit is dedicated to the cute KLM girls who made part of this
possible with giving me a seat in the plane where you actually have enough
place to hack. Thanks, girls.

ok markus@ dhartmei@

Add support for active/inactive tablesets in the kernel.
Add table definition/initialisation construct in pfctl parser.
Add and fix documentation for pf.4 and pf.conf.5.
Tested on i386 and sparc64 by myself, macppc by Daniel.
ok dhartmei@

Remove table name hashing (pass the name in each ioctl instead), and
introduce reference counting for tables, they are now automatically
created and deleted through referencing rules. Diff partly from cedric@.
ok mcbride@, henning@, cedric@

minor style

I do not know where this policy of "one .h file for every .c file" comes
from, but whoever thought of it is stupid.