MFH (r313975, r315909, r317827, r344533): upgrade to latest OpenPAM

Approved by: re (gjb)

Subset of upstream r902 which fixes custom prompts.

PR: 216172

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Improve collation string and locales support

Merge collation support from Illumos and DragonflyBSD.

Locales are now generated with the new localedef(1) tool from CLDR POSIX files.
The generated files are now identified as "BSD 1.0" format.

The libc now only read "BSD 1.0" locales definitions, all other version will be
set to "C"
The localedef(1) tool has been imported from Illumos and modified to use tree(3)
instead of the CDDL avl(3)
A set of tool created by edwin@ and extended by marino@ for dragonfly has been
added to be able to generate locales and the Makefiles from the vanilla CLDR
unicode databases + a universal UTF-8 charmap (by marino@)
Update the locales to unicode v27
Given our regex(3) does not support multibyte (yet) it has been forced to always
use locale C
Remove now unused colldef(1) and mklocale(1)
Finish implementing the numeric BSD extension for ctypes
The number of supported locales has grown from 175 to 250 locales. Among the new
locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish
locales.
Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS,
zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components
version when possible.

Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under
BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able
to generate locales definition usable in freebsd sources out of vanilla CLDR
definitions, John Marino (DragonflyBSD) who first merge the Illumos work into
Dragonfly and spent hours tracking down bugs.

Merge mpsutil(8) branch

mpsutil(8)/mprutil(8) are new utilities for managing LSI Fusion-MPT
2/3 controllers (mps(4) and mpr(4))

For now only informational commands have been implemented.

This utility has been written by scottl@ [1] and polished by myself[2]

Submitted by: scottl
Discussed with: scottl
Relnotes: yes
Sponsored by: Netflix [1]
Sponsored by: Gandi.net [2]

Upgrade our copies of clang, llvm, lldb, compiler-rt and libc++ to 3.7.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.7.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.7.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Andrew Turner and Antoine Brodin for their help.

Exp-run: antoine
Relnotes: yes

Restore the upstream (and documented) behavior of searching for modules
both in /usr/lib and /usr/local/lib, thus simplifying the use of modules
from ports, without breaking the compat32 case again.

PR: 191151
MFC after: 3 weeks

Upgrade our copy of clang, llvm and lldb to 3.6.0 release.

Please note that from 3.5.0 onwards, clang/llvm/lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.6.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.6.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste for the lldb part of this upgrade.

Exp-run: antoine

Upgrade our copy of clang, llvm and lldb to 3.5.0 release.

Please note that this version now requires C++11 support to build; see
UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.5.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Andrew Turner, Justin Hibbits and
Antoine Brodin for their invaluable help with this import.

Approved by: portmgr (antoine)
MFC after: 1 month

Staticize the test plans.

Merge upstream r825: fix line continuation in whitespace

Upgrade to OpenPAM Ourouparia.

Remove incorrect props

Fix incorrect error handling in PAM policy parser. [SA-14:13]

Merge the projects/clang-sparc64 branch back to head. This brings in
several updates from the llvm and clang trunks to make the sparc64
backend fully functional.

Apart from one patch to sys/sparc64/include/pcpu.h which is still under
discussion, this makes it possible to let clang fully build world and
kernel for sparc64.

Any assistance with testing this on actual sparc64 hardware is greatly
appreciated, as there will unavoidably be bugs left.

Many thanks go to Roman Divacky for his upstream work on getting the
sparc64 backend into shape.

MFC after: 1 month

MFV (r262529): fix is_upper() predicate

MFC after: 1 week

MFV (r255387): undo autoprop damage

Approved by: re (glebius)

Update to OpenPAM Nummularia.

This was a good idea that never went anywhere.

MFV (r255364): move the code around in preparation for Nummularia.

Backport upstream r684 (OPENPAM_DEBUG enables debugging macros but does
not turn debugging on by default) and add OPENPAM_DEBUG to CFLAGS.

Merge upstream r634:646: correctly parse mixed quoted / unquoted text.

OpenPAM Micrampelis was re-rolled due to a showstopper bug.

Update to OpenPAM Micrampelis.

r234173 missed one instance of FREE(line).

Noticed by: glebius@

Keep a copy of the original pointer returned by openpam_readline() so
we can free it later, instead of trying to free a pointer that points
to the end of the buffer.

Committed to head because this code no longer exists upstream.

Submitted by: jasone@

Merge upstream r509: don't log an error in the common case (ENOENT).

Upgrade to OpenPAM Lycopsida.

Add minimal validation of the service name to fend off at least one
attack vector against applications that allow the applicant to specify
which policy to apply.

Submitted by: Matthias Drochner <drochner@netbsd.org>
MFC after: 1 week

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Merge upstream r432:

pam_end() already contains a NULL check, and it is not unreasonable to
call it with a NULL pamh in a cleanup / error-handling situation. Remove
OPENPAM_NONNULL, which may cause gcc to optimize away the NULL check.

This fixes a potential NULL pointer dereference in error-handling code
in passwd(1).

Merge upstream r421: grammar nit in pam.conf(5).

Update and remove CVS-specific items

Approved by: re (kib)

Remove empty directories from the HEAD.

Discussed with: developers, imp

Don't try to auto-detect dynamic linking; it fails on mips. The Makefile
part of the patch is an ugly (and hopefully temporary) hack.

Discussed with: imp@

Bootstrap mergeinfo

As per discussion, commit experimental metadata for my contrib packages.
The idea is to have a FREEBSD-vendor file for every third-party package
in the tree.

This commit was generated by cvs2svn to compensate for changes in r175801,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r174835,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r174832,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r168463,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r147466,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r147464,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r147462,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r147460,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r147455,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r141098,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r132691,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r126854,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r125647,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r123394,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r122912,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r117839,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r117615,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r117610,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r116520,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r115698,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r115619,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r114536,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r114438,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r114436,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r110989,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r110558,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r110556,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r110503,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r109756,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r109686,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r108794,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r108577,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r107937,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r107579,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r105770,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r99160,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r99158,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r97241,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r96364,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r95978,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r95908,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94970,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94968,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94884,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94880,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94878,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94735,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94706,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94670,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94562,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94532,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94213,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r94209,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r93982,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r93694,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r93526,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r92289,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91830,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91799,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91796,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91684,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91100,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91097,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r91094,
which included commits to RCS files with non-trunk default branches.