Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r242960:
Implement the waitid() SUSv4 function using wait6() system call.

PR: standards/170346

MFC r237434:
Use struct vdso_timehands data to implement fast gettimeofday(2) and
clock_gettime(2) functions if supported.

MFC r236695:

Fix two warnings about self-assignment in libc. These normally only
trigger with clang, when you either use -save-temps, or ccache.

Reported by: Sevan / Venture37 <venture37@gmail.com>

MFC r234657:
Take the spinlock around clearing of the fp->_flags in fclose(3), which
indicates the avaliability of FILE, to prevent possible reordering of
the writes as seen by other CPUs.

MFC r231868:
Fetch the aux vector for the static libc, and use the entries to
initialize the cache of the system information as it was done for the
dynamic libc. This removes several sysctls from the static binary
startup.

Use the aux vector to fill the single struct dl_phdr_info describing
the static binary itself, to implement dl_iterate_phdr(3) for the
static binaries.

Fix a problem whereby a corrupt DNS record can cause named to crash. [11:06]

Add an API for alerting internal libc routines to the presence of
"unsafe" paths post-chroot, and use it in ftpd. [11:07]

Fix a buffer overflow in telnetd. [11:08]

Make pam_ssh ignore unpassphrased keys unless the "nullok" option is
specified. [11:09]

Add sanity checking of service names in pam_start. [11:10]

Approved by: so (cperciva)
Approved by: re (bz)
Security: FreeBSD-SA-11:06.bind
Security: FreeBSD-SA-11:07.chroot
Security: FreeBSD-SA-11:08.telnetd
Security: FreeBSD-SA-11:09.pam_ssh
Security: FreeBSD-SA-11:10.pam

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Introduce a non-portable function pthread_getthreadid_np(3) to retrieve
calling thread's unique integral ID, which is similar to AIX function of
the same name. Bump __FreeBSD_version to note its introduction.

Reviewed by: kib

Remove some unneeded spaces from the __sym_compat() macro, since newer
versions of gas are more fussy about spaces surrounding '@' signs in
versioned symbol names.

Revert revision 214007, I realized that MySQL wants to resolve
a silly rwlock deadlock problem, the deadlock is caused by writer
waiters, if a thread has already locked a reader lock, and wants to
acquire another reader lock, it will be blocked by writer waiters,
but we had already fixed it years ago.

Unbreak buildworld by including pthread_rwlockattr_setkind_np and
pthread_rwlockattr_getkind_np.

To support stack unwinding for cancellation points, add -fexceptions flag
for them, two functions _pthread_cancel_enter and _pthread_cancel_leave
are added to let thread enter and leave a cancellation point, it also
makes it possible that other functions can be cancellation points in
libraries without having to be rewritten in libthr.

On shared object unload, in __cxa_finalize, call and clear all installed
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.

Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.

Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.

The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.

Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks

Use aux vector to get values for SSP canary, pagesize, pagesizes array,
number of host CPUs and osreldate.

This eliminates the last sysctl(2) calls from the dynamically linked image
startup.

No objections from: kan
Tested by: marius (sparc64)
MFC after: 1 month

The NetBSD Foundation has granted permission to remove clause 3 and 4 from
their software.

Obtained from: NetBSD

Use umtx to implement process sharable semaphore, to make this work,
now type sema_t is a structure which can be put in a shared memory area,
and multiple processes can operate it concurrently.
User can either use mmap(MAP_SHARED) + sem_init(pshared=1) or use sem_open()
to initialize a shared semaphore.
Named semaphore uses file system and is located in /tmp directory, and its
file name is prefixed with 'SEMD', so now it is chroot or jail friendly.
In simplist cases, both for named and un-named semaphore, userland code
does not have to enter kernel to reduce/increase semaphore's count.
The semaphore is designed to be crash-safe, it means even if an application
is crashed in the middle of operating semaphore, the semaphore state is
still safely recovered by later use, there is no waiter counter maintained
by userland code.
The main semaphore code is in libc and libthr only has some necessary stubs,
this makes it possible that a non-threaded application can use semaphore
without linking to thread library.
Old semaphore implementation is kept libc to maintain binary compatibility.
The kernel ksem API is no longer used in the new implemenation.

Discussed on: threads@

Revert the previous change to pthread_once() stub in libc. It is actually
a feature that libstdc++ depends on to simulate the behavior of libc's
internal '__isthreaded' variable. One benefit of this is that _libc_once()
is now private to _once_stub.c.

Requested by: kan

Add an internal _once() method. This works identical to pthread_once(3)
with the additional property that it is safe for routines in libc to use
in both single-threaded and multi-threaded processes. Multi-threaded
processes use the pthread_once() implementation from the threading library
while single-threaded processes use a simplified "stub" version internal
to libc. The libc stub-version of pthread_once() now also uses the
simplified "stub" version as well instead of being a nop.

Reviewed by: deischen, Matthew Fleming @ Isilon
Suggested by: alc
MFC after: 1 week

Make openat(2) a cancellation point.

This is required by POSIX and matches open(2).

Reviewed by: kib, jhb
MFC after: 1 month

Change the ABI of some of the structures used by the SYSV IPC API:
- The uid/cuid members of struct ipc_perm are now uid_t instead of unsigned
short.
- The gid/cgid members of struct ipc_perm are now gid_t instead of unsigned
short.
- The mode member of struct ipc_perm is now mode_t instead of unsigned short
(this is merely a style bug).
- The rather dubious padding fields for ABI compat with SV/I386 have been
removed from struct msqid_ds and struct semid_ds.
- The shm_segsz member of struct shmid_ds is now a size_t instead of an
int. This removes the need for the shm_bsegsz member in struct
shmid_kernel and should allow for complete support of SYSV SHM regions
>= 2GB.
- The shm_nattch member of struct shmid_ds is now an int instead of a
short.
- The shm_internal member of struct shmid_ds is now gone. The internal
VM object pointer for SHM regions has been moved into struct
shmid_kernel.
- The existing __semctl(), msgctl(), and shmctl() system call entries are
now marked COMPAT7 and new versions of those system calls which support
the new ABI are now present.
- The new system calls are assigned to the FBSD-1.1 version in libc. The
FBSD-1.0 symbols in libc now refer to the old COMPAT7 system calls.
- A simplistic framework for tagging system calls with compatibility
symbol versions has been added to libc. Version tags are added to
system calls by adding an appropriate __sym_compat() entry to
src/lib/libc/incldue/compat.h. [1]

PR: kern/16195 kern/113218 bin/129855
Reviewed by: arch@, rwatson
Discussed with: kan, kib [1]

Revert unnecessary memset after calloc.

Suggested by: jhb
Approved by: dfr (mentor)

Fix an issue when nss fallback routines are used in a multithreaded application.

Reviewed by: bushman
Approved by: dfr (mentor)

Add support for the FPA floating-point format on ARM. The
FPA floating-point format is identical to the VFP format,
but is always stored in big-endian.
Introduce _IEEE_WORD_ORDER to describe the byte-order of
the FP representation.

Obtained from: Juniper Networks, Inc

Merge the resolver part of BIND 9.4.3 into HEAD.
It includes the following fix:

2426. [bug] libbind: inet_net_pton() can sometimes return the
wrong value if excessively large netmasks are
supplied. [RT #18512]

Reported by: Maksymilian Arciemowicz <cxib__at__securityreason.com>

prop change.

delete svn:keywords.

prop change

Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes. The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes. Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.

Turn execvpe() into an internal libc routine.

Adding exevpe() has caused some ports to break. Even though execvpe() is
a useful routine, it does not conform to any standards.

This patch is a little bit different from the patch sent to the mailing
list. I forgot to remove execvpe from the Symbol.map (which does not
seem to miscompile libc, though).

Reviewed by: davidxu
Approved by: philip

Make pthread_cleanup_push() and pthread_cleanup_pop() as a pair of macros,
use stack space to keep cleanup information, this eliminates overhead of
calling malloc() and free() in thread library.

Discussed on: thread@

Add some compatibility code so that software which is built to use the new
struct flock with l_sysid member can work properly on an an old kernel which
doesn't support l_sysid.

Sponsored by: Isilon Systems

Add pthread_setaffinity_np and pthread_getaffinity_np to libc namespace.

Compile libthr with warnings.

(Somehow this file sneaked from initial commit.)

Compile libthr with warnings.

Add pthread_mutex_isowned_np() so there is no need for an additional
prototype next to the implementation.

MFC after: 2 weeks

Change the C wrappers for mmap/lseek/pread/pwrite/truncate/ftruncate to
call the pad-less versions of the corresponding syscalls if the running
kernel supports it. Check kern.osreldate once per program and cache the
result to select the appropriate syscall. This maintains userland
compatability with kernel.old's from quite a while back.

Approved by: re (kensmith)

Merge BIND 9.4.1 into main chunk.

MFC after: 2 weeks

This commit was generated by cvs2svn to compensate for changes in r170242,
which included commits to RCS files with non-trunk default branches.

Remove 3rd clause, renumber, ok per email

Upgrade res_update(3) and the friends to BIND9's one excluding TSIG
support.

X-MFC after: never

This commit was generated by cvs2svn to compensate for changes in r158782,
which included commits to RCS files with non-trunk default branches.

- Extend the nsswitch to support Services, Protocols and Rpc
databases.
- Make nsswitch support caching.

Submitted by: Michael Bushkov <bushman__at__rsu.ru>
Sponsored by: Google Summer of Code 2005

Add semaphore functions, and remove some dupes from the #if 0 section.

Add a bunch of missing pthread functions, and move out-of-order functions.

We have gmtime_r(3).

- Merge our local changes.
- Exclude unnecessary functions for us.

This commit was generated by cvs2svn to compensate for changes in r156952,
which included commits to RCS files with non-trunk default branches.

Add entries for new pthread stubs.

Add some more pthread stubs so that librt can use them.
The thread jump table has been resorted, so you need to
keep libc, libpthread, and libthr in sync.

Submitted by: xu

In preparation for a new malloc implementation:

* Add posix_memalign().

* Move calloc() from calloc.c to malloc.c. Add a calloc() implementation in
rtld-elf in order to make the loader happy (even though calloc() isn't
used in rtld-elf).

* Add _malloc_prefork() and _malloc_postfork(), and use them instead of
directly manipulating __malloc_lock.

Approved by: phk, markm (mentor)

Move the declaration of __cleanup to libc_private.h as it is used in both
stdio/ and stdlib/. Don't define __cleanup twice.

Add namespace #defines for usleep.

Update my email address.

Add support for TLS in statically linked programs.

When a dynamic NSS module is built and linked against a thread
library, it may pull in that thread library at run time. If the
process started out single-threaded, this could cause attempts to
release locks that do not exist. Guard against this possibility by
checking __isthreaded before invoking thread primitives.

A similar problem remains if the process is linked against one thread
library, but the NSS module is linked against another. This can only
be avoided by careful design of the NSS module.

Submitted by: Sean McNeil <sean@mcneil.com> (mostly; bugs are mine)

Make the resolver(3) and many associated interfaces much more reentrant.
The getaddrinfo(3), getipnodebyname(3) and resolver(3) can coincide now
with what should be totally reentrant, and h_errno values will now
be preserved correctly, but this does not affect interfaces such as
gethostbyname(3) which are still mostly non-reentrant.

In all of these relevant functions, the thread-safety has been pushed
down as far as it seems possible right now. This means that operations
that are selected via nsdispatch(3) (i.e. files, yp, dns) are protected
still under global locks that getaddrinfo(3) defines, but where possible
the locking is greatly reduced. The most noticeable improvement is
that multiple DNS lookups can now be run at the same time, and this
shows major improvement in performance of DNS-lookup threaded programs,
and solves the "Mozilla tab serialization" problem.

No single-threaded applications need to be recompiled. Multi-threaded
applications that reference "_res" to change resolver(3) options will
need to be recompiled, and ones which reference "h_errno" will also
if they desire the correct h_errno values. If the applications already
understood that _res and h_errno were not thread-safe and had their own
locking, they will see no performance improvement but will not
actually break in any way.

Please note that when NSS modules are used, or when nsdispatch(3)
defaults to adding any lookups of its own to the individual libc
_nsdispatch() calls, those MUST be reentrant as well.

Add a delta accidentally omitted from the previous commit:
Define DBL_MANH_SIZE and DBL_MANL_SIZE to be the sizes of the
high and low words of the mantissa in bits, respectively.

Remove #include of spinlock.h from libc_private.h. Declare spinlocks as
struct _spinlock. Keep the typedef in for now; another set of changes
may come around to clean up consumers of spinlocks.

Requested by: bde

Externalize malloc's spinlock so that a thread library can take
it around an application's fork() call. Our new thread libraries
(libthr, libpthread) can now have threads running while another
thread calls fork(). In this case, it is possible for malloc
to be left in an inconsistent state in the child. Our thread
libraries, libpthread in particular, need to use malloc internally
after a fork (in the child).

Reviewed by: davidxu

Back out the `hiding' of strlcpy and strlcat. Several people
vocally objected to this safety belt.

`Hide' strlcpy and strlcat (using the namespace.h / __weak_reference
technique) so that we don't wind up calling into an application's
version if the application defines them.

Inspired by: qpopper's interfering and buggy version of strlcpy

Correct a bug that was somehow both obvious and hard-to-see. :-)
An incorrectly-sized allocation was being made due to an incorrect
argument to the `sizeof' operator. Obvious, because it violated the
`foo = malloc(sizeof(*foo))' idiom. Hard-to-see, because it was a
missing `*' (`*p' versus `**p').

Resulting failure was
Reported by: ache

Sponsored by: DARPA, Network Associates Laboratories

= Implement name service switch modules (NSS modules). NSS modules
may be built into libc (`static NSS modules') or dynamically loaded
via dlopen (`dynamic NSS modules'). Modules are loaded/initialized
at configuration time (i.e. when nsdispatch is called and nsswitch.conf
is read or re-read).

= Make the nsdispatch(3) core thread-safe.

= New status code for nsdispatch(3) `NS_RETURN', currently used to
signal ERANGE-type issues.

= syslog(3) problems, don't warn/err/abort.

= Try harder to avoid namespace pollution.

= Implement some shims to assist in porting NSS modules written for
the GNU C Library nsswitch interface.

Sponsored by: DARPA, Network Associates Laboratories

- Define a _spinunlock() function so that threading implementations may do
more complicated things than just setting the lock to 0.
- Implement stubs for this function in libc and the two threading libraries
that are currently in the tree.

Eliminate 19 warnings in libc (at level WARNS=2) of the
`implicit declaration of function' variety.

Eliminate 61 warnings emitted at WARNS=2 (leaving 53 to go).
Only warnings that could be fixed without changing the generated object
code and without restructuring the source code have been handled.

Reviewed by: /sbin/md5

Implement fpclassify():
o Add a MD header private to libc called _fpmath.h; this header
contains bitfield layouts of MD floating-point types.
o Add a MI header private to libc called fpmath.h; this header
contains bitfield layouts of MI floating-point types.
o Add private libc variables to lib/libc/$arch/gen/infinity.c for
storing NaN values.
o Add __double_t and __float_t to <machine/_types.h>, and provide
double_t and float_t typedefs in <math.h>.
o Add some C99 manifest constants (FP_ILOGB0, FP_ILOGBNAN, HUGE_VALF,
HUGE_VALL, INFINITY, NAN, and return values for fpclassify()) to
<math.h> and others (FLT_EVAL_METHOD, DECIMAL_DIG) to <float.h> via
<machine/float.h>.
o Add C99 macro fpclassify() which calls __fpclassify{d,f,l}() based
on the size of its argument. __fpclassifyl() is never called on
alpha because (sizeof(long double) == sizeof(double)), which is good
since __fpclassifyl() can't deal with such a small `long double'.

This was developed by David Schultz and myself with input from bde and
fenner.

PR: 23103
Submitted by: David Schultz <dschultz@uclink.Berkeley.EDU>
(significant portions)
Reviewed by: bde, fenner (earlier versions)

Add waitpid to the list of hidden names for use by wordexp.c and grantpt.c.

Argh, change declaration of two-dimensional array so that it actually
builds.

Make this compile with whatever error-checking is enabled in buildworld
and/or beast.

Use a jump table (a la Solaris) for pthread routines with default
entries in the table being stubs. While I'm here, add macros to
auto-generate the stubs. A conforming threads library can override
the stub routines by filling in the jump table.

Add some entries to namespace.h and sync un-namespace.h to it.
Also add a comment to remind folks to update un-namespace.h
when changing namespace.h.

* Add stubs for pthread_cond_broadcast.
* Fix typos in rwlock stubs.
* Add pthread_XXX counterparts to the _pthread_XXX stubs which libraries
like libX11 can use to ensure thread-safety without requiring the use
of a thread library.

Submitted by: Terry Lambert (pthread_cond_broadcast)
Reviewed by: deischen

Do not use __progname directly (except in [gs]etprogname(3)).
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.

Fix the style of the SCM ID's.
I believe have made all of libc .h's as consistent as possible.

Remove __P() usage.

Fixed namespace pollution related to `err' in libc in the same way as for
`warn'. Now a whole 2 members of the err() family don't cause pollution.

This fixes world breakage in awk for NOSHARED worlds. contrib/awk/msg.c
has had its own version of err() for a long time, but this somehow
didn't cause problems until the update to awk-3.1.0.

signanosleep(2) hasn't existed since 1998.

Fixed namespace pollution related to `warn' in libc (but not in other
libraries or for other members of the err() family).

This fixes world breakage in bc and rcs/* for NOSHARED worlds.

Add entries for the posix1e functions that will be overridden in libc_r.

Approved by: rwatson
Obtained from: TrustedBSD Project

Bring in a hybrid of SunSoft's transport-independent RPC (TI-RPC) and
associated changes that had to happen to make this possible as well as
bugs fixed along the way.

Bring in required TLI library routines to support this.

Since we don't support TLI we've essentially copied what NetBSD
has done, adding a thin layer to emulate direct the TLI calls
into BSD socket calls.

This is mostly from Sun's tirpc release that was made in 1994,
however some fixes were backported from the 1999 release (supposedly
only made available after this porting effort was underway).

The submitter has agreed to continue on and bring us up to the
1999 release.

Several key features are introduced with this update:
Client calls are thread safe. (1999 code has server side thread
safe)
Updated, a more modern interface.

Many userland updates were done to bring the code up to par with
the recent RPC API.

There is an update to the pthreads library, a function
pthread_main_np() was added to emulate a function of Sun's threads
library.

While we're at it, bring in NetBSD's lockd, it's been far too
long of a wait.

New rpcbind(8) replaces portmap(8) (supporting communication over
an authenticated Unix-domain socket, and by default only allowing
set and unset requests over that channel). It's much more secure
than the old portmapper.

Umount(8), mountd(8), mount_nfs(8), nfsd(8) have also been upgraded
to support TI-RPC and to support IPV6.

Umount(8) is also fixed to unmount pathnames longer than 80 chars,
which are currently truncated by the Kernel statfs structure.

Submitted by: Martin Blapp <mb@imp.ch>
Manpage review: ru
Secure RPC implemented by: wpaul

libc MT-safety, part 2.

Add a lock to FILE. flockfile and friends are now implemented
(for the most part) in libc. flockfile_debug is implemented in
libc_r; I suppose it's about time to kill it but will do it in
a future commit.

Fix a potential deadlock in _fwalk in a threaded environment.
A file flag (__SIGN) was added to stdio.h that, when set, tells
_fwalk to ignore it in its walk. This seemed to be needed in
refill.c because each file needs to be locked when flushing.

Add a stub for pthread_self in libc. This is needed by flockfile
which is allowed by POSIX to be recursive.

Make fgetpos() error return value (-1) match man page.

Remove recursive calls to locked functions (stdio); I think I've
got them all, but I may have missed a couple.

A few K&R -> ANSI conversions along with removal of a few instances
of "register".

$Id$ -> $FreeBSD$ in libc/stdio/rget.c

Not objected to: -arch, a few months ago

Remove _THREAD_SAFE and make libc thread-safe by default by
adding (weak definitions to) stubs for some of the pthread
functions. If the threads library is linked in, the real
pthread functions will pulled in.

Use the following convention for system calls wrapped by the
threads library:
__sys_foo - actual system call
_foo - weak definition to __sys_foo
foo - weak definition to __sys_foo

Change all libc uses of system calls wrapped by the threads
library from foo to _foo. In order to define the prototypes
for _foo(), we introduce namespace.h and un-namespace.h
(suggested by bde). All files that need to reference these
system calls, should include namespace.h before any standard
includes, then include un-namespace.h after the standard
includes and before any local includes. <db.h> is an exception
and shouldn't be included in between namespace.h and
un-namespace.h namespace.h will define foo to _foo, and
un-namespace.h will undefine foo.

Try to eliminate some of the recursive calls to MT-safe
functions in libc/stdio in preparation for adding a mutex
to FILE. We have recursive mutexes, but would like to avoid
using them if possible.

Remove uneeded includes of <errno.h> from a few files.

Add $FreeBSD$ to a few files in order to pass commitprep.

Approved by: -arch

$Id$ -> $FreeBSD$

Implement compile time debug support for spinlocks.

Simplify the atomic lock prototype, removing the lock value.

Delete the unlock prototypes that are not required.

Remove leading underscores from the FILE lock functions that POSIX
specifies.

In a threaded library, expect the lock field to be declared volatile,
so provide function prototypes that respect that, avoiding a gcc
warning that `volatile' is being thrown away.

Add a private header file for libc/libc_r/libpthread to contain
definitions for things like locking etc.

Add a private (to libc, libc_r and libpthread) header file containing
prototypes for the spinlock functions that will be used for thread locks.
libc will have stubs declared with weak symbols. libpthread and libc_r
will have functions that really do something.