Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r237434:
Use struct vdso_timehands data to implement fast gettimeofday(2) and
clock_gettime(2) functions if supported.

MFC r234657:
Take the spinlock around clearing of the fp->_flags in fclose(3), which
indicates the avaliability of FILE, to prevent possible reordering of
the writes as seen by other CPUs.

MFC r231868:
Fetch the aux vector for the static libc, and use the entries to
initialize the cache of the system information as it was done for the
dynamic libc. This removes several sysctls from the static binary
startup.

Use the aux vector to fill the single struct dl_phdr_info describing
the static binary itself, to implement dl_iterate_phdr(3) for the
static binaries.

Fix a problem whereby a corrupt DNS record can cause named to crash. [11:06]

Add an API for alerting internal libc routines to the presence of
"unsafe" paths post-chroot, and use it in ftpd. [11:07]

Fix a buffer overflow in telnetd. [11:08]

Make pam_ssh ignore unpassphrased keys unless the "nullok" option is
specified. [11:09]

Add sanity checking of service names in pam_start. [11:10]

Approved by: so (cperciva)
Approved by: re (bz)
Security: FreeBSD-SA-11:06.bind
Security: FreeBSD-SA-11:07.chroot
Security: FreeBSD-SA-11:08.telnetd
Security: FreeBSD-SA-11:09.pam_ssh
Security: FreeBSD-SA-11:10.pam

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

To support stack unwinding for cancellation points, add -fexceptions flag
for them, two functions _pthread_cancel_enter and _pthread_cancel_leave
are added to let thread enter and leave a cancellation point, it also
makes it possible that other functions can be cancellation points in
libraries without having to be rewritten in libthr.

On shared object unload, in __cxa_finalize, call and clear all installed
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.

Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.

Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.

The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.

Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks

Use aux vector to get values for SSP canary, pagesize, pagesizes array,
number of host CPUs and osreldate.

This eliminates the last sysctl(2) calls from the dynamically linked image
startup.

No objections from: kan
Tested by: marius (sparc64)
MFC after: 1 month

Use umtx to implement process sharable semaphore, to make this work,
now type sema_t is a structure which can be put in a shared memory area,
and multiple processes can operate it concurrently.
User can either use mmap(MAP_SHARED) + sem_init(pshared=1) or use sem_open()
to initialize a shared semaphore.
Named semaphore uses file system and is located in /tmp directory, and its
file name is prefixed with 'SEMD', so now it is chroot or jail friendly.
In simplist cases, both for named and un-named semaphore, userland code
does not have to enter kernel to reduce/increase semaphore's count.
The semaphore is designed to be crash-safe, it means even if an application
is crashed in the middle of operating semaphore, the semaphore state is
still safely recovered by later use, there is no waiter counter maintained
by userland code.
The main semaphore code is in libc and libthr only has some necessary stubs,
this makes it possible that a non-threaded application can use semaphore
without linking to thread library.
Old semaphore implementation is kept libc to maintain binary compatibility.
The kernel ksem API is no longer used in the new implemenation.

Discussed on: threads@

Revert the previous change to pthread_once() stub in libc. It is actually
a feature that libstdc++ depends on to simulate the behavior of libc's
internal '__isthreaded' variable. One benefit of this is that _libc_once()
is now private to _once_stub.c.

Requested by: kan

Add an internal _once() method. This works identical to pthread_once(3)
with the additional property that it is safe for routines in libc to use
in both single-threaded and multi-threaded processes. Multi-threaded
processes use the pthread_once() implementation from the threading library
while single-threaded processes use a simplified "stub" version internal
to libc. The libc stub-version of pthread_once() now also uses the
simplified "stub" version as well instead of being a nop.

Reviewed by: deischen, Matthew Fleming @ Isilon
Suggested by: alc
MFC after: 1 week

Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes. The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes. Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.

Turn execvpe() into an internal libc routine.

Adding exevpe() has caused some ports to break. Even though execvpe() is
a useful routine, it does not conform to any standards.

This patch is a little bit different from the patch sent to the mailing
list. I forgot to remove execvpe from the Symbol.map (which does not
seem to miscompile libc, though).

Reviewed by: davidxu
Approved by: philip

Add some compatibility code so that software which is built to use the new
struct flock with l_sysid member can work properly on an an old kernel which
doesn't support l_sysid.

Sponsored by: Isilon Systems

Change the C wrappers for mmap/lseek/pread/pwrite/truncate/ftruncate to
call the pad-less versions of the corresponding syscalls if the running
kernel supports it. Check kern.osreldate once per program and cache the
result to select the appropriate syscall. This maintains userland
compatability with kernel.old's from quite a while back.

Approved by: re (kensmith)

Remove 3rd clause, renumber, ok per email

Add some more pthread stubs so that librt can use them.
The thread jump table has been resorted, so you need to
keep libc, libpthread, and libthr in sync.

Submitted by: xu

In preparation for a new malloc implementation:

* Add posix_memalign().

* Move calloc() from calloc.c to malloc.c. Add a calloc() implementation in
rtld-elf in order to make the loader happy (even though calloc() isn't
used in rtld-elf).

* Add _malloc_prefork() and _malloc_postfork(), and use them instead of
directly manipulating __malloc_lock.

Approved by: phk, markm (mentor)

Move the declaration of __cleanup to libc_private.h as it is used in both
stdio/ and stdlib/. Don't define __cleanup twice.

Add support for TLS in statically linked programs.

Remove #include of spinlock.h from libc_private.h. Declare spinlocks as
struct _spinlock. Keep the typedef in for now; another set of changes
may come around to clean up consumers of spinlocks.

Requested by: bde

Externalize malloc's spinlock so that a thread library can take
it around an application's fork() call. Our new thread libraries
(libthr, libpthread) can now have threads running while another
thread calls fork(). In this case, it is possible for malloc
to be left in an inconsistent state in the child. Our thread
libraries, libpthread in particular, need to use malloc internally
after a fork (in the child).

Reviewed by: davidxu

Eliminate 19 warnings in libc (at level WARNS=2) of the
`implicit declaration of function' variety.

Argh, change declaration of two-dimensional array so that it actually
builds.

Make this compile with whatever error-checking is enabled in buildworld
and/or beast.

Use a jump table (a la Solaris) for pthread routines with default
entries in the table being stubs. While I'm here, add macros to
auto-generate the stubs. A conforming threads library can override
the stub routines by filling in the jump table.

Add some entries to namespace.h and sync un-namespace.h to it.
Also add a comment to remind folks to update un-namespace.h
when changing namespace.h.

Do not use __progname directly (except in [gs]etprogname(3)).
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.

Remove _THREAD_SAFE and make libc thread-safe by default by
adding (weak definitions to) stubs for some of the pthread
functions. If the threads library is linked in, the real
pthread functions will pulled in.

Use the following convention for system calls wrapped by the
threads library:
__sys_foo - actual system call
_foo - weak definition to __sys_foo
foo - weak definition to __sys_foo

Change all libc uses of system calls wrapped by the threads
library from foo to _foo. In order to define the prototypes
for _foo(), we introduce namespace.h and un-namespace.h
(suggested by bde). All files that need to reference these
system calls, should include namespace.h before any standard
includes, then include un-namespace.h after the standard
includes and before any local includes. <db.h> is an exception
and shouldn't be included in between namespace.h and
un-namespace.h namespace.h will define foo to _foo, and
un-namespace.h will undefine foo.

Try to eliminate some of the recursive calls to MT-safe
functions in libc/stdio in preparation for adding a mutex
to FILE. We have recursive mutexes, but would like to avoid
using them if possible.

Remove uneeded includes of <errno.h> from a few files.

Add $FreeBSD$ to a few files in order to pass commitprep.

Approved by: -arch

$Id$ -> $FreeBSD$

Remove leading underscores from the FILE lock functions that POSIX
specifies.

Add a private header file for libc/libc_r/libpthread to contain
definitions for things like locking etc.