Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Rename getline with get_line to avoid collision with getline(3)

When getline(3) in 2009 was added a _WITH_GETLINE guard has also been added.
This rename is made in preparation for the removal of this guard

Obtained from: OpenBSD

Upgrade our copies of clang, llvm, lldb and compiler-rt to 3.8.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang will soon be available here:
<http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.8.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Davide Italiano and Antoine Brodin
for their help.

Relnotes: yes

Merge r293013 from clang380-import branch:

Fix a clang 3.8.0 warning in pflogd.c:

contrib/pf/pflogd/pflogd.c:769:8: error: logical not is only applied to the left hand side of this comparison [-Werror,-Wlogical-not-parentheses]
if (!if_exists(interface) == -1) {
^ ~~

The if_exists() function does not return -1, and even if it did, it
would not be the correct way to check. Just ditch the == -1 instead.

Obtained from: OpenBSD's pflogd.c 1.49
MFC after: 3 days

Improve collation string and locales support

Merge collation support from Illumos and DragonflyBSD.

Locales are now generated with the new localedef(1) tool from CLDR POSIX files.
The generated files are now identified as "BSD 1.0" format.

The libc now only read "BSD 1.0" locales definitions, all other version will be
set to "C"
The localedef(1) tool has been imported from Illumos and modified to use tree(3)
instead of the CDDL avl(3)
A set of tool created by edwin@ and extended by marino@ for dragonfly has been
added to be able to generate locales and the Makefiles from the vanilla CLDR
unicode databases + a universal UTF-8 charmap (by marino@)
Update the locales to unicode v27
Given our regex(3) does not support multibyte (yet) it has been forced to always
use locale C
Remove now unused colldef(1) and mklocale(1)
Finish implementing the numeric BSD extension for ctypes
The number of supported locales has grown from 175 to 250 locales. Among the new
locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish
locales.
Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS,
zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components
version when possible.

Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under
BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able
to generate locales definition usable in freebsd sources out of vanilla CLDR
definitions, John Marino (DragonflyBSD) who first merge the Illumos work into
Dragonfly and spent hours tracking down bugs.

Merge mpsutil(8) branch

mpsutil(8)/mprutil(8) are new utilities for managing LSI Fusion-MPT
2/3 controllers (mps(4) and mpr(4))

For now only informational commands have been implemented.

This utility has been written by scottl@ [1] and polished by myself[2]

Submitted by: scottl
Discussed with: scottl
Relnotes: yes
Sponsored by: Netflix [1]
Sponsored by: Gandi.net [2]

Upgrade our copies of clang, llvm, lldb, compiler-rt and libc++ to 3.7.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.7.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.7.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Andrew Turner and Antoine Brodin for their help.

Exp-run: antoine
Relnotes: yes

Fix a couple of missing lines that obscured the -p description.

Submitted by: Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.com>
MFC after: 1 week

Upgrade our copy of clang, llvm and lldb to 3.5.0 release.

Please note that this version now requires C++11 support to build; see
UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.5.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Andrew Turner, Justin Hibbits and
Antoine Brodin for their invaluable help with this import.

Approved by: portmgr (antoine)
MFC after: 1 month

Update NetBSD Foundation copyrights to 2-clause BSD

The NetBSD Foundation states "Third parties are encouraged to change the
license on any files which have a 4-clause license contributed to the
NetBSD Foundation to a 2-clause license."

This change removes clauses 3 and 4 from copyright / license blocks that
list The NetBSD Foundation as the only copyright holder.

Sponsored by: The FreeBSD Foundation

Ressurect the local change documenting
authpf's requirement for a mounted fdescfs(5).

PR: docs/186250
MFC after: 1 week

o Create directory sys/netpfil, where all packet filters should
reside, and move there ipfw(4) and pf(4).

o Move most modified parts of pf out of contrib.

Actual movements:

sys/contrib/pf/net/*.c -> sys/netpfil/pf/
sys/contrib/pf/net/*.h -> sys/net/
contrib/pf/pfctl/*.c -> sbin/pfctl
contrib/pf/pfctl/*.h -> sbin/pfctl
contrib/pf/pfctl/pfctl.8 -> sbin/pfctl
contrib/pf/pfctl/*.4 -> share/man/man4
contrib/pf/pfctl/*.5 -> share/man/man5

sys/netinet/ipfw -> sys/netpfil/ipfw

The arguable movement is pf/net/*.h -> sys/net. There are
future plans to refactor pf includes, so I decided not to
break things twice.

Not modified bits of pf left in contrib: authpf, ftp-proxy,
tftp-proxy, pflogd.

The ipfw(4) movement is planned to be merged to stable/9,
to make head and stable match.

Discussed with: bz, luigi

Merge the projects/pf/head branch, that was worked on for last six months,
into head. The most significant achievements in the new code:

o Fine grained locking, thus much better performance.
o Fixes to many problems in pf, that were specific to FreeBSD port.

New code doesn't have that many ifdefs and much less OpenBSDisms, thus
is more attractive to our developers.

Those interested in details, can browse through SVN log of the
projects/pf/head branch. And for reference, here is exact list of
revisions merged:

r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330,
r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656,
r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782,
r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868,
r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223,
r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456,
r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505,
r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168,
r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230,
r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398,
r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548,
r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672,
r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169,
r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442,
r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522,
r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661,
r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212.

I'd like to thank people who participated in early testing:

Tested by: Florian Smeets <flo freebsd.org>
Tested by: Chekaluk Vitaly <artemrts ukr.net>
Tested by: Ben Wilber <ben desync.com>
Tested by: Ian FREISLICH <ianf cloudseed.co.za>

Merge multi-FIB IPv6 support from projects/multi-fibv6/head/:

Extend the so far IPv4-only support for multiple routing tables (FIBs)
introduced in r178888 to IPv6 providing feature parity.

This includes an extended rtalloc(9) KPI for IPv6, the necessary
adjustments to the network stack, and user land support as in netstat.

Sponsored by: Cisco Systems, Inc.
Reviewed by: melifaro (basically)
MFC after: 10 days

Replace an OpenBSDism with a FreeBSDism in the pfctl(8) man page: we put
configuration file man pages in section 5, and we prefer rc.conf to
rc.conf.local.

MFC after: 3 days

Restore a feature that was present in 5.x and 6.x, and was cleared in
7.x, 8.x and 9.x with pf(4) imports: pfsync(4) should suppress CARP
preemption, while it is running its bulk update.

However, reimplement the feature in more elegant manner, that is
partially inspired by newer OpenBSD:

- Rename term "suppression" to "demotion", to match with OpenBSD.
- Keep a global demotion factor, that can be raised by several
conditions, for now these are:
- interface goes down
- carp(4) has problems with ip_output() or ip6_output()
- pfsync performs bulk update
- Unlike in OpenBSD the demotion factor isn't a counter, but
is actual value added to advskew. The adjustment values for
particular error conditions are also configurable, and their
defaults are maximum advskew value, so a single failure bumps
demotion to maximum. This is for POLA compatibility, and should
satisfy most users.
- Demotion factor is a writable sysctl, so user can do
foot shooting, if he desires to.

- Fix examples to show new CARP style.
- Remove OpenBSDisms, add FreeBSDisms.

Correct the description of struct pfioc_state_kill.

PR: kern/158997
Submitted by: ohauer

Note the PF version.

Discussed with: bz

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

Update packet filter (pf) code to OpenBSD 4.5.

You need to update userland (world and ports) tools
to be in sync with the kernel.

Submitted by: mlaier
Submitted by: eri

Add a new option -P to suppress getservbyport(3) calls when printing rules.
This allows one to force consistent printing of numeric port numbers like
we do with -n for other tools like netstat (just that -n was already taken)
rather than the service names.

-P is currently unused in OpenBSD so the change is eligible for upstreaming.

PR: misc/151015
Submitted by: Matt Koivisto (mkoivisto sandvine.com)
Sponsored by: Sandvine Incorporated
MFC after: 1 week

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Enable closefrom(2) here, as we have supported it for some time now.

Discussed with: mlaier
MFC after: 2 weeks

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Adapt OpenBSD pf's "sloopy" TCP state machine which is useful for Direct
Server Return mode, where not all packets would be visible to the load
balancer or gateway.

This commit should be reverted when we merge future pf versions. The
benefit it would provide is that this version does not break any existing
public interface and thus won't be a problem if we want to MFC it to
earlier FreeBSD releases.

Discussed with: mlaier
Obtained from: OpenBSD
Sponsored by: iXsystems, Inc.
MFC after: 1 month

Bootstrap mergeinfo on pf userland

Max's changes got left out of the MRT commit.

Make ALTQ cope with disappearing interfaces (particularly common with mpd
and netgraph in gernal). This also allows to add queues for an interface
that is not yet existing (you have to provide the bandwidth for the
interface, however).

PR: kern/106400, kern/117827
MFC after: 2 weeks

MFOpenBSD rev 1.393 pf.conf.5

do not describe `/' as solidus; from Allen (freebsd pr120484);

PR: 120484
Submitted by: Allen <alandsidel at 1001islington dot com>
MFC After: 3 days

Update for libpcap 0.9.8

Lost these during the import. Hand me the pointy hat.

Approved by: re (implicit)

Commit resolved import of OpenBSD 4.1 pf userland from perforce.

Approved by: re (kensmith)

This commit was generated by cvs2svn to compensate for changes in r171169,
which included commits to RCS files with non-trunk default branches.

Revert my previous change, add an MLINK from securelevel.7 to security.7

Discussed with: brueffer

Change securelevel(7) to security(7). Yes i am aware
that this is within the contrib directory.

PR: docs/104402
Submitted by: Dr. Markus Waldeck <waldeck at gmx dot de>

Discussed with: mlaier

From OpenBSD, rev. 1.379
Document how 'allow-opts' applies to routing headers in IPv6.

MFC after: 1 week
Discussed with: mlaier

From OpenBSD, rev. 1.91:
fix servicecurve check; no point in checking the same sc three times, it
was obviously intended to check all three. has been wrong since the
beginning, 4 years... noticed by Earl Lapus <earl.lapus@gmail.com>, Vasil
Dimov <vd@FreeBSD.org> mailed me then, ok mcbride

MFC after: 3 days

Mention that we do not support route labels in the BUGS section.

PR: docs/93590
Reported by: Niki Denev

- Note that the synchronisation interface needs to be up and have
an IP address assigned.
- Add "quick" keyword to pf.conf example.

PR: docs/85209

Document authpf's requirement for a mounted fdescfs(5).

PR: docs/89635
MFC after: 1 day

Constfy errstr as it is in OpenBSD to unbreak the build.

Pointed out by: Suken Woo, Martin Wilke, Wesley Morgan

Use strtonum now that we have it in libc as well.

Fix build after timeval.tv_sec changed from long to time_t.

Add an rc.d script to start pfsync at the right moment of the
system boot, and hook it up in the system.

The separate script is needed because in the presence of various
interface lists in rc.conf ($network_interfaces, $cloned_interfaces,
$sppp_interfaces, $gif_interfaces, more to come) it is hard to start
them orderly, so that pfsync is brought up after its syncdev, which
is required for the proper startup of pfsync.

Discussed with: mlaier on -pf
MFC after: 5 days

Redirect bridge(4) to if_bridge(4). These should have pointed to if_bridge
from the begining.

Reminded by: ru

FreeBSD now supports BIOCLOCK. So we can use it now.

Reviewed by: mlaier

More tcpdump 8->1 cleanup.

Approved by: mlaier
MFC after: 3 days

- Remove MLINKS to nonexistant manpages
- Change some section numbers to match reality
- For MLINKS to manpages from ports, mention which port installs them

MFC after: 3 days

Resolve conflicts created during the import of pf 3.7 Some features are
missing and will be implemented in a second step. This is functional as is.

Tested by: freebsd-pf, pfsense.org
Obtained from: OpenBSD

This commit was generated by cvs2svn to compensate for changes in r145837,
which included commits to RCS files with non-trunk default branches.

- remove OpenBSDisms, add FreeBSDisms
- comment out feature, we do not have yet: tcpdumping on pfsync,
add a BUGS section
- reference carp.4
- dereference bpf(4), tcpdump(7), hostname.if(5)
- sort references
- tell when pfsync appeared in FreeBSD

Reviewed by: mlaier
MFC after: 1 week

Fix sloppy use of "manpage", bump .Dd where applicable and rename RED to
Random Early Detection (not ... Drop) in order to be consistent with other
documentation on ALTQ

Pointed out by: simon, ru, Brad Davis

Be more verbose about altq SYNOPSIS and add more linkage in the relating pf
documents.

Inspired by: scottl
Reviewed by: Brad Davis <so14kNOso14kSPAMcom>
MFC after: 3 days

Fix a reference from pool(9) -> zone(9), but keep on talking about "memory
pools" as that is what UMA provides.

Submitted by: Jay <jay NO meangrape SPAM com>

Rename the QUEUEING section to QUEUEING/ALTQ to make it easier to find the
appropiate section when redirected from ALTQ(4).

MFC after: 2 days

Make pflogd cope with module unload (and the sudden disappearing of pflog0).
Instead of eating all the available CPU we now shutdown gracefully.

Submitted by: yongari
MFC after: 3 days

Document a problem with user/group filtering. With debug.mpsafenet=1 this
might result in a deadlock. The fix involves critical changes in the PF
locking strategy (which will happen after 5.3R). For now advise users to set
debug.mpsafenet=0 if they use this kind of filtering.

The same problem exists for IPFW.

mdoc help from: simon
MFC after: 2 days

PFIL_HOOKS is no longer an optional item.

Submitted by: Anders Hanssen
MFC after: 1 day

Bring in some examples (and create space for future work here):
- Add OpenBSD example rulesets as advertised in etc/pf.conf and pf.conf(5)
- Tweak the pointer to fit the FreeBSD default location share/examples/pf
- Account for the new directory in BSD.usr.dist (no hier(7) change required
as share/examples is an opaque item there).

Obtained from: OpenBSD
Reminded by: Thomas T. Veldhouse
PR: docs/71691
MFC after: 2 days

Make pflogd(8) store pcap_sf_pkthdr instead of MD timeval contaminated
pcap_pkthdr. This makes /var/log/pflog standart compliant on 64bit archs.

OpenBSD has fixed this by changing the bpf timeval to 32bit in the kernel,
so no need to report this over (again).

PR: bin/71096 (w/ changes)
Submitted by: Ville-Pertti Keinonen
Tested by: amd64(submitter), sparc64(yongari), i386(myself)
MFC after: 3 days

Loopback a fix from Cedric Berger:
Fix table add/replace commands with securelevel=2.
Reported by James J. Lippard.

Discussed with: yongari
MFC after: 5 days

Fix printing of u_int64_t with a cast to unsigned long long.

Found-by: tinderbox(amd64)

Commit userland part of pf version 3.5 from OpenBSD (OPENBSD_3_5_BASE).

This commit was generated by cvs2svn to compensate for changes in r130614,
which included commits to RCS files with non-trunk default branches.

FreeBSD-ify the manpage. Our inetd does not support bind-address:port syntax
Christian will follow up with some additional words about how to protect
this from the outside world.

Submitted-by: brueffer
Approved-by: bms(mentor)

FreeBSD-if .4 manpages for pf/pflog/pfsync.

PR: docs/65687
Submitted by: Sergey Matveychuk
Approved by: bms(mentor)

Fix $FreeBSD$ ids.

Fix some style(9) related issues after discussion with/education from bde:
- Add <sys/param.h> and <limits.h> where required (do not depend on other
headers pulling it in).
- __dead -> __dead2
- #if defined() -> #ifdef
- Remove ugly PRIu64 macros and use %llu w/ (unsigned long long) cast.

All changes looped back to OpenBSD (where applicable) for easier sync in the
future.

Requested by: bde
Approved by: bms(mentor)

Fix two instances of improper NULL/0 use idetified by the changes lately.

Submitted by: Patrick Marie
Approved by: bms(mentor)

Add local define of HTONL() as it was decided to protect this by _KERNEL
in <net/pfvar.h>

This commit was generated by cvs2svn to compensate for changes in r126357,
which included commits to RCS files with non-trunk default branches.

Apply diff from the port.
Rather small diff for the userland (in contrast to the kernel):
- Some header file location/differences
- Clean compilation on 64bit arch (identified by bento a long time ago)
- ALTQ not (yet) available. Leave a switch for patchsets and future ...
- most files can be used from the vendor branch

Approved by: bms(in general)

This commit was generated by cvs2svn to compensate for changes in r126353,
which included commits to RCS files with non-trunk default branches.