Fix two issues related to thread private data in asr.
- setting up asr in single thread mode and then starting threads using asr
would lead to multiple threads sharing the same resolver.
- destruction of a thread that has been using asr would leak data.
Problem originally reported by Alexey Sokolov and Uli Schlachter.
ok kettenis@

New futex(2) based rwlock implementation based on the mutex code.

This implementation reduces contention because threads no longer need
to spin calling sched_yield(2) before going to sleep.

Tested by many, thanks!

ok visa@, pirofti@

Make the "not my pool" searching loop a tiny bit smarter, while
making the number of pools variable. Do not document the malloc
conf settings atm, don't know yet if they will stay. Thanks to all
the testers. ok deraadt@

Implement __cxa_thread_atexit to support C++11 thread_local scope. The
interface is also made available as __cxa_thread_atexit_impl to satisfy the
needs of GNU libstdc++.

ok guenther@, millert@

Revert recent changes to unbreak ports/net/samba

While it is not clear (to me) why that ports ends up with corrupted
shared libs, reverting those changes fixes the issue and should allow us
to close p2k17 more smoothly.

Discussed with a bunch, ok ajacoutot@ guenther@

Change pthread_cleanup_{push,pop} to macros that store the cleanup info
on the stack instead of mallocing the list and move the APIs from libpthread
to libc so that they can be used inside libc.

Note: the standard was explicitly written to permit/support this
"macro with unmatched brace" style and it's what basically everyone
else already does. We xor the info with random cookies with a
random magic to detect/trip-up overwrites.

Major bump to both libc and libpthread due to the API move.

ok mpi@

Move mutex, condvar, and thread-specific data routes, pthread_once, and
pthread_exit from libpthread to libc, along with low-level bits to
support them. Major bump to both libc and libpthread.

Requested by libressl team. Ports testing by naddy@
ok kettenis@

Wrap _malloc_init() so internal calls go directly

prodded by otto@
ok kettenis@ otto@

Less lock contention by using more pools for mult-threaded programs.
tested by many (thanks!) ok tedu, guenther@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Make pthread_atfork() track the DSO that called it like atexit() does,
unregistering callbacks if the DSO is unloaded. Move the callback
handling from libpthread to libc, though libpthread still overrides the
inner call to handle locking and thread-library reinitialization.
Major version bump for both libc and libpthread.

verification that this fixes various ports ajacoutot@
asm assistance miod@; ok millert@ deraadt@

Make consistent the syscall stubs for the syscalls that got special
handling to fix up the alignment of 64bit arguments so that they do
the same dance where _thread_sys_FOO is the real stub and FOO is a
weak alias. For some of them, this is needed for cancellation handling.

From discussions with fgsch@, ok millert@

remove _thread_malloc_init prototype

remove _MALLOC_LOCK_INIT; major bump; ok deraadt@

- make arc4random*() functions thread safe. Use a custom spinlock function
instead of the generic pthread macros since free(3) uses __arc4_getbyte()
when freeing small sized allocations and the generic pthread macros call
malloc(3).
- eliminate passing pointers to a static variable with global scope (rs)
for additional code clarity and reduction.
- shlib minor bumps for libc and libpthread due to new functions.
From andreas@ with some bits from me. okay tedu@ marc@ w/some spot
checking from millert@

fix misleading comment; the _MUTEX_LOCK/_MUTEX_UNLOCK macros work with
both static and non-static mutexs.

_FD_LOCK/UNLOCK() is libpthread specific and not needed for librthread, so
isolate its usage to libpthread only and replace with generic non-static
mutex support in the one place it is needed:
- remove _FD_LOCK/UNLOCK from lseek and ftruncate in libc and make the
functions weak so that libpthread can override with its own new
versions that do the locking.
- remove _thread_fd_lock/unlock() weak functions from libc and adjust
libpthread for the change.
- add generic _thread_mutex_lock/unlock/destroy() weak functions in libc
to support non-static mutexes in libc and add libpthread and librthread
implementations for them. libc can utilize non-static mutexes via the
new _MUTEX_LOCK/UNLOCK/DESTROY() macros. Actually these new macros can
support both static and non-static mutexes but currently only using
them for non-static.
- make opendir/closedir/readdir/readdir_r/seekdir/telldir() thread-safe
for both thread libraries by using a non-static mutex in the struct
_dirdesc (typedef DIR), utilizing it in the *dir functions and remove
remaining and incorrect _FD_LOCK/UNLOCK() use in libc.
- add comments to both thread libraries to indicate libc depends on the
current implementation of static mutex initialization. suggested by
marc@
- major bump libc and libpthread due to function removal, structure
change and weak symbol conversions.
okay marc@, tedu@

Part 2 of file descriptor race and deadlock corrections.

Adjust design of file descriptor table to eliminate races
with both opening and closing of file descriptor entries
and eliminates one class of deadlocks. One nice side effect
of this change in design should be better performance for
applications that open and close many file descriptors due
to reduced fd_table_lock contention and fd entry reuse.

- Add entry states to manage use of entry and eliminate
some closing races. fd entries are not deallocated upon
close() now.
- Call _thread_fd_table_init with one of five discreet
modes to properly initialize an entry and manage the
state transition to open.
- When closing an entry hold the entry spinlock locked
across the state transition and the _thread_sys_close
call to close another race.
- Introduce a new lock type FD_RDWR_CLOSE that transitions
either a closed entry or an open entry into closing state
and then waits for a RDWR lock so that the lock queue can
unwind normally. All subsequent fd lock attempts for that
entry are rejected with EBADF until the fd is fully closed,
or reopened by dup2(). Once a thread holds the FD_RDWR_LOCK
it is safe to close() it or dup2() on it.
- When a thread creates a new fd there is a window of time
when another thread could attempt to use the fd before the
creating thread has initialized the entry for it. This can
result in improper status_flags for the entry, so record
the entries init mode, detect when this has happened and
correct the status_flags when needed.

reviewed by marc@ & brad@, tested by several, okay brad@

Avouid a race in atexit() handling by introducing a lock. Problem
originally reported by Gergely Kovacs; help from dhartmei@;
ok tedu@ millert@

Use a forward declaration of struct timespec instead of assuming
it has already been defined by previous includes. While this
is true at the moment, it will no longer be true after a pending
includes cleanup. OK marc@

major bump to libc and libpthread to break the dependency of a
particular implementation of libpthread for libc. libc no longer
needs pthread.h to compile.
OK millert@, brad@, tedu@

thread safer libc (note: safer, not safe)
Access to the global _res structure replaced by pointers to a
per thread instance. If unthreaded the pointer is to the
global structure.

Also replaced a 64k stack array with malloc-ed memory so
threaded aps (with a default 64k stack) have a chance at working.
ok deraadt@

thread safe libc -- 2nd try. OK miod@, millert@
Thanks to miod@ for m68k and vax fixes

back out previous patch.. there are still some vax/m68k issues

libc changes for thread safety. Tested on:
alpha (millert@), i386 (marc@), m68k (millert@ and miod@),
powerpc (drahn@ and dhartmei@), sparc (millert@ and marc@),
sparc64 (marc@), and vax (millert@ and miod@).
Thanks to millert@, miod@, and mickey@ for fixes along the way.

Part one of userland __P removal. Done with a simple regexp with some minor hand editing to make comments line up correctly. Another pass is forthcoming that handles the cases that could not be done automatically.

put changes back, this time ALL the files.

Back out fgsch@'s tree breaking commits.
Test next time, ok?

fix some const warnings

#(endif|else) foo is incorrect, make it #endif /* foo */
deraadt@ ok

grammer/spelling

typo in comment, AILASES -> ALIASES

oops

rewrite to use the helpers in the thread/ directory and to define weak alias macros

clean

Add thread-safety to libc, so that libc_r will build (on i386 at least).
All POSIX libc api now there (to P1003.1c/D10)
(more md stuff is needed for other libc/arch/*)
(setlogin is no longer a special syscall)
Add -pthread option to gcc (that makes it use -lc_r and -D_POSIX_THREADS).
Doc some re-entrant routines
Add libc_r to intro(3)
dig() uses some libc srcs and an extra -I was needed there.
Add more md stuff to libc_r.
Update includes for the pthreads api
Update libc_r TODO

New futex(2) based rwlock implementation based on the mutex code.

This implementation reduces contention because threads no longer need
to spin calling sched_yield(2) before going to sleep.

Tested by many, thanks!

ok visa@, pirofti@

Make the "not my pool" searching loop a tiny bit smarter, while
making the number of pools variable. Do not document the malloc
conf settings atm, don't know yet if they will stay. Thanks to all
the testers. ok deraadt@

Implement __cxa_thread_atexit to support C++11 thread_local scope. The
interface is also made available as __cxa_thread_atexit_impl to satisfy the
needs of GNU libstdc++.

ok guenther@, millert@

Revert recent changes to unbreak ports/net/samba

While it is not clear (to me) why that ports ends up with corrupted
shared libs, reverting those changes fixes the issue and should allow us
to close p2k17 more smoothly.

Discussed with a bunch, ok ajacoutot@ guenther@

Change pthread_cleanup_{push,pop} to macros that store the cleanup info
on the stack instead of mallocing the list and move the APIs from libpthread
to libc so that they can be used inside libc.

Note: the standard was explicitly written to permit/support this
"macro with unmatched brace" style and it's what basically everyone
else already does. We xor the info with random cookies with a
random magic to detect/trip-up overwrites.

Major bump to both libc and libpthread due to the API move.

ok mpi@

Move mutex, condvar, and thread-specific data routes, pthread_once, and
pthread_exit from libpthread to libc, along with low-level bits to
support them. Major bump to both libc and libpthread.

Requested by libressl team. Ports testing by naddy@
ok kettenis@

Wrap _malloc_init() so internal calls go directly

prodded by otto@
ok kettenis@ otto@

Less lock contention by using more pools for mult-threaded programs.
tested by many (thanks!) ok tedu, guenther@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Make pthread_atfork() track the DSO that called it like atexit() does,
unregistering callbacks if the DSO is unloaded. Move the callback
handling from libpthread to libc, though libpthread still overrides the
inner call to handle locking and thread-library reinitialization.
Major version bump for both libc and libpthread.

verification that this fixes various ports ajacoutot@
asm assistance miod@; ok millert@ deraadt@

Make consistent the syscall stubs for the syscalls that got special
handling to fix up the alignment of 64bit arguments so that they do
the same dance where _thread_sys_FOO is the real stub and FOO is a
weak alias. For some of them, this is needed for cancellation handling.

From discussions with fgsch@, ok millert@

remove _thread_malloc_init prototype

remove _MALLOC_LOCK_INIT; major bump; ok deraadt@

- make arc4random*() functions thread safe. Use a custom spinlock function
instead of the generic pthread macros since free(3) uses __arc4_getbyte()
when freeing small sized allocations and the generic pthread macros call
malloc(3).
- eliminate passing pointers to a static variable with global scope (rs)
for additional code clarity and reduction.
- shlib minor bumps for libc and libpthread due to new functions.
From andreas@ with some bits from me. okay tedu@ marc@ w/some spot
checking from millert@

fix misleading comment; the _MUTEX_LOCK/_MUTEX_UNLOCK macros work with
both static and non-static mutexs.

_FD_LOCK/UNLOCK() is libpthread specific and not needed for librthread, so
isolate its usage to libpthread only and replace with generic non-static
mutex support in the one place it is needed:
- remove _FD_LOCK/UNLOCK from lseek and ftruncate in libc and make the
functions weak so that libpthread can override with its own new
versions that do the locking.
- remove _thread_fd_lock/unlock() weak functions from libc and adjust
libpthread for the change.
- add generic _thread_mutex_lock/unlock/destroy() weak functions in libc
to support non-static mutexes in libc and add libpthread and librthread
implementations for them. libc can utilize non-static mutexes via the
new _MUTEX_LOCK/UNLOCK/DESTROY() macros. Actually these new macros can
support both static and non-static mutexes but currently only using
them for non-static.
- make opendir/closedir/readdir/readdir_r/seekdir/telldir() thread-safe
for both thread libraries by using a non-static mutex in the struct
_dirdesc (typedef DIR), utilizing it in the *dir functions and remove
remaining and incorrect _FD_LOCK/UNLOCK() use in libc.
- add comments to both thread libraries to indicate libc depends on the
current implementation of static mutex initialization. suggested by
marc@
- major bump libc and libpthread due to function removal, structure
change and weak symbol conversions.
okay marc@, tedu@

Part 2 of file descriptor race and deadlock corrections.

Adjust design of file descriptor table to eliminate races
with both opening and closing of file descriptor entries
and eliminates one class of deadlocks. One nice side effect
of this change in design should be better performance for
applications that open and close many file descriptors due
to reduced fd_table_lock contention and fd entry reuse.

- Add entry states to manage use of entry and eliminate
some closing races. fd entries are not deallocated upon
close() now.
- Call _thread_fd_table_init with one of five discreet
modes to properly initialize an entry and manage the
state transition to open.
- When closing an entry hold the entry spinlock locked
across the state transition and the _thread_sys_close
call to close another race.
- Introduce a new lock type FD_RDWR_CLOSE that transitions
either a closed entry or an open entry into closing state
and then waits for a RDWR lock so that the lock queue can
unwind normally. All subsequent fd lock attempts for that
entry are rejected with EBADF until the fd is fully closed,
or reopened by dup2(). Once a thread holds the FD_RDWR_LOCK
it is safe to close() it or dup2() on it.
- When a thread creates a new fd there is a window of time
when another thread could attempt to use the fd before the
creating thread has initialized the entry for it. This can
result in improper status_flags for the entry, so record
the entries init mode, detect when this has happened and
correct the status_flags when needed.

reviewed by marc@ & brad@, tested by several, okay brad@

Avouid a race in atexit() handling by introducing a lock. Problem
originally reported by Gergely Kovacs; help from dhartmei@;
ok tedu@ millert@

Use a forward declaration of struct timespec instead of assuming
it has already been defined by previous includes. While this
is true at the moment, it will no longer be true after a pending
includes cleanup. OK marc@

major bump to libc and libpthread to break the dependency of a
particular implementation of libpthread for libc. libc no longer
needs pthread.h to compile.
OK millert@, brad@, tedu@

thread safer libc (note: safer, not safe)
Access to the global _res structure replaced by pointers to a
per thread instance. If unthreaded the pointer is to the
global structure.

Also replaced a 64k stack array with malloc-ed memory so
threaded aps (with a default 64k stack) have a chance at working.
ok deraadt@

thread safe libc -- 2nd try. OK miod@, millert@
Thanks to miod@ for m68k and vax fixes

back out previous patch.. there are still some vax/m68k issues

libc changes for thread safety. Tested on:
alpha (millert@), i386 (marc@), m68k (millert@ and miod@),
powerpc (drahn@ and dhartmei@), sparc (millert@ and marc@),
sparc64 (marc@), and vax (millert@ and miod@).
Thanks to millert@, miod@, and mickey@ for fixes along the way.

Part one of userland __P removal. Done with a simple regexp with some minor hand editing to make comments line up correctly. Another pass is forthcoming that handles the cases that could not be done automatically.

put changes back, this time ALL the files.

Back out fgsch@'s tree breaking commits.
Test next time, ok?

fix some const warnings

#(endif|else) foo is incorrect, make it #endif /* foo */
deraadt@ ok

grammer/spelling

typo in comment, AILASES -> ALIASES

oops

rewrite to use the helpers in the thread/ directory and to define weak alias macros

clean

Add thread-safety to libc, so that libc_r will build (on i386 at least).
All POSIX libc api now there (to P1003.1c/D10)
(more md stuff is needed for other libc/arch/*)
(setlogin is no longer a special syscall)
Add -pthread option to gcc (that makes it use -lc_r and -D_POSIX_THREADS).
Doc some re-entrant routines
Add libc_r to intro(3)
dig() uses some libc srcs and an extra -I was needed there.
Add more md stuff to libc_r.
Update includes for the pthreads api
Update libc_r TODO

Make the "not my pool" searching loop a tiny bit smarter, while
making the number of pools variable. Do not document the malloc
conf settings atm, don't know yet if they will stay. Thanks to all
the testers. ok deraadt@

Implement __cxa_thread_atexit to support C++11 thread_local scope. The
interface is also made available as __cxa_thread_atexit_impl to satisfy the
needs of GNU libstdc++.

ok guenther@, millert@

Revert recent changes to unbreak ports/net/samba

While it is not clear (to me) why that ports ends up with corrupted
shared libs, reverting those changes fixes the issue and should allow us
to close p2k17 more smoothly.

Discussed with a bunch, ok ajacoutot@ guenther@

Change pthread_cleanup_{push,pop} to macros that store the cleanup info
on the stack instead of mallocing the list and move the APIs from libpthread
to libc so that they can be used inside libc.

Note: the standard was explicitly written to permit/support this
"macro with unmatched brace" style and it's what basically everyone
else already does. We xor the info with random cookies with a
random magic to detect/trip-up overwrites.

Major bump to both libc and libpthread due to the API move.

ok mpi@

Move mutex, condvar, and thread-specific data routes, pthread_once, and
pthread_exit from libpthread to libc, along with low-level bits to
support them. Major bump to both libc and libpthread.

Requested by libressl team. Ports testing by naddy@
ok kettenis@

Wrap _malloc_init() so internal calls go directly

prodded by otto@
ok kettenis@ otto@

Less lock contention by using more pools for mult-threaded programs.
tested by many (thanks!) ok tedu, guenther@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Make pthread_atfork() track the DSO that called it like atexit() does,
unregistering callbacks if the DSO is unloaded. Move the callback
handling from libpthread to libc, though libpthread still overrides the
inner call to handle locking and thread-library reinitialization.
Major version bump for both libc and libpthread.

verification that this fixes various ports ajacoutot@
asm assistance miod@; ok millert@ deraadt@

Make consistent the syscall stubs for the syscalls that got special
handling to fix up the alignment of 64bit arguments so that they do
the same dance where _thread_sys_FOO is the real stub and FOO is a
weak alias. For some of them, this is needed for cancellation handling.

From discussions with fgsch@, ok millert@

remove _thread_malloc_init prototype

remove _MALLOC_LOCK_INIT; major bump; ok deraadt@

- make arc4random*() functions thread safe. Use a custom spinlock function
instead of the generic pthread macros since free(3) uses __arc4_getbyte()
when freeing small sized allocations and the generic pthread macros call
malloc(3).
- eliminate passing pointers to a static variable with global scope (rs)
for additional code clarity and reduction.
- shlib minor bumps for libc and libpthread due to new functions.
From andreas@ with some bits from me. okay tedu@ marc@ w/some spot
checking from millert@

fix misleading comment; the _MUTEX_LOCK/_MUTEX_UNLOCK macros work with
both static and non-static mutexs.

_FD_LOCK/UNLOCK() is libpthread specific and not needed for librthread, so
isolate its usage to libpthread only and replace with generic non-static
mutex support in the one place it is needed:
- remove _FD_LOCK/UNLOCK from lseek and ftruncate in libc and make the
functions weak so that libpthread can override with its own new
versions that do the locking.
- remove _thread_fd_lock/unlock() weak functions from libc and adjust
libpthread for the change.
- add generic _thread_mutex_lock/unlock/destroy() weak functions in libc
to support non-static mutexes in libc and add libpthread and librthread
implementations for them. libc can utilize non-static mutexes via the
new _MUTEX_LOCK/UNLOCK/DESTROY() macros. Actually these new macros can
support both static and non-static mutexes but currently only using
them for non-static.
- make opendir/closedir/readdir/readdir_r/seekdir/telldir() thread-safe
for both thread libraries by using a non-static mutex in the struct
_dirdesc (typedef DIR), utilizing it in the *dir functions and remove
remaining and incorrect _FD_LOCK/UNLOCK() use in libc.
- add comments to both thread libraries to indicate libc depends on the
current implementation of static mutex initialization. suggested by
marc@
- major bump libc and libpthread due to function removal, structure
change and weak symbol conversions.
okay marc@, tedu@

Part 2 of file descriptor race and deadlock corrections.

Adjust design of file descriptor table to eliminate races
with both opening and closing of file descriptor entries
and eliminates one class of deadlocks. One nice side effect
of this change in design should be better performance for
applications that open and close many file descriptors due
to reduced fd_table_lock contention and fd entry reuse.

- Add entry states to manage use of entry and eliminate
some closing races. fd entries are not deallocated upon
close() now.
- Call _thread_fd_table_init with one of five discreet
modes to properly initialize an entry and manage the
state transition to open.
- When closing an entry hold the entry spinlock locked
across the state transition and the _thread_sys_close
call to close another race.
- Introduce a new lock type FD_RDWR_CLOSE that transitions
either a closed entry or an open entry into closing state
and then waits for a RDWR lock so that the lock queue can
unwind normally. All subsequent fd lock attempts for that
entry are rejected with EBADF until the fd is fully closed,
or reopened by dup2(). Once a thread holds the FD_RDWR_LOCK
it is safe to close() it or dup2() on it.
- When a thread creates a new fd there is a window of time
when another thread could attempt to use the fd before the
creating thread has initialized the entry for it. This can
result in improper status_flags for the entry, so record
the entries init mode, detect when this has happened and
correct the status_flags when needed.

reviewed by marc@ & brad@, tested by several, okay brad@

Avouid a race in atexit() handling by introducing a lock. Problem
originally reported by Gergely Kovacs; help from dhartmei@;
ok tedu@ millert@

Use a forward declaration of struct timespec instead of assuming
it has already been defined by previous includes. While this
is true at the moment, it will no longer be true after a pending
includes cleanup. OK marc@

major bump to libc and libpthread to break the dependency of a
particular implementation of libpthread for libc. libc no longer
needs pthread.h to compile.
OK millert@, brad@, tedu@

thread safer libc (note: safer, not safe)
Access to the global _res structure replaced by pointers to a
per thread instance. If unthreaded the pointer is to the
global structure.

Also replaced a 64k stack array with malloc-ed memory so
threaded aps (with a default 64k stack) have a chance at working.
ok deraadt@

thread safe libc -- 2nd try. OK miod@, millert@
Thanks to miod@ for m68k and vax fixes

back out previous patch.. there are still some vax/m68k issues

libc changes for thread safety. Tested on:
alpha (millert@), i386 (marc@), m68k (millert@ and miod@),
powerpc (drahn@ and dhartmei@), sparc (millert@ and marc@),
sparc64 (marc@), and vax (millert@ and miod@).
Thanks to millert@, miod@, and mickey@ for fixes along the way.

Part one of userland __P removal. Done with a simple regexp with some minor hand editing to make comments line up correctly. Another pass is forthcoming that handles the cases that could not be done automatically.

put changes back, this time ALL the files.

Back out fgsch@'s tree breaking commits.
Test next time, ok?

fix some const warnings

#(endif|else) foo is incorrect, make it #endif /* foo */
deraadt@ ok

grammer/spelling

typo in comment, AILASES -> ALIASES

oops

rewrite to use the helpers in the thread/ directory and to define weak alias macros

clean

Add thread-safety to libc, so that libc_r will build (on i386 at least).
All POSIX libc api now there (to P1003.1c/D10)
(more md stuff is needed for other libc/arch/*)
(setlogin is no longer a special syscall)
Add -pthread option to gcc (that makes it use -lc_r and -D_POSIX_THREADS).
Doc some re-entrant routines
Add libc_r to intro(3)
dig() uses some libc srcs and an extra -I was needed there.
Add more md stuff to libc_r.
Update includes for the pthreads api
Update libc_r TODO

Implement __cxa_thread_atexit to support C++11 thread_local scope. The
interface is also made available as __cxa_thread_atexit_impl to satisfy the
needs of GNU libstdc++.

ok guenther@, millert@

Revert recent changes to unbreak ports/net/samba

While it is not clear (to me) why that ports ends up with corrupted
shared libs, reverting those changes fixes the issue and should allow us
to close p2k17 more smoothly.

Discussed with a bunch, ok ajacoutot@ guenther@

Change pthread_cleanup_{push,pop} to macros that store the cleanup info
on the stack instead of mallocing the list and move the APIs from libpthread
to libc so that they can be used inside libc.

Note: the standard was explicitly written to permit/support this
"macro with unmatched brace" style and it's what basically everyone
else already does. We xor the info with random cookies with a
random magic to detect/trip-up overwrites.

Major bump to both libc and libpthread due to the API move.

ok mpi@

Move mutex, condvar, and thread-specific data routes, pthread_once, and
pthread_exit from libpthread to libc, along with low-level bits to
support them. Major bump to both libc and libpthread.

Requested by libressl team. Ports testing by naddy@
ok kettenis@

Wrap _malloc_init() so internal calls go directly

prodded by otto@
ok kettenis@ otto@

Less lock contention by using more pools for mult-threaded programs.
tested by many (thanks!) ok tedu, guenther@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Make pthread_atfork() track the DSO that called it like atexit() does,
unregistering callbacks if the DSO is unloaded. Move the callback
handling from libpthread to libc, though libpthread still overrides the
inner call to handle locking and thread-library reinitialization.
Major version bump for both libc and libpthread.

verification that this fixes various ports ajacoutot@
asm assistance miod@; ok millert@ deraadt@

Make consistent the syscall stubs for the syscalls that got special
handling to fix up the alignment of 64bit arguments so that they do
the same dance where _thread_sys_FOO is the real stub and FOO is a
weak alias. For some of them, this is needed for cancellation handling.

From discussions with fgsch@, ok millert@

remove _thread_malloc_init prototype

remove _MALLOC_LOCK_INIT; major bump; ok deraadt@

- make arc4random*() functions thread safe. Use a custom spinlock function
instead of the generic pthread macros since free(3) uses __arc4_getbyte()
when freeing small sized allocations and the generic pthread macros call
malloc(3).
- eliminate passing pointers to a static variable with global scope (rs)
for additional code clarity and reduction.
- shlib minor bumps for libc and libpthread due to new functions.
From andreas@ with some bits from me. okay tedu@ marc@ w/some spot
checking from millert@

fix misleading comment; the _MUTEX_LOCK/_MUTEX_UNLOCK macros work with
both static and non-static mutexs.

_FD_LOCK/UNLOCK() is libpthread specific and not needed for librthread, so
isolate its usage to libpthread only and replace with generic non-static
mutex support in the one place it is needed:
- remove _FD_LOCK/UNLOCK from lseek and ftruncate in libc and make the
functions weak so that libpthread can override with its own new
versions that do the locking.
- remove _thread_fd_lock/unlock() weak functions from libc and adjust
libpthread for the change.
- add generic _thread_mutex_lock/unlock/destroy() weak functions in libc
to support non-static mutexes in libc and add libpthread and librthread
implementations for them. libc can utilize non-static mutexes via the
new _MUTEX_LOCK/UNLOCK/DESTROY() macros. Actually these new macros can
support both static and non-static mutexes but currently only using
them for non-static.
- make opendir/closedir/readdir/readdir_r/seekdir/telldir() thread-safe
for both thread libraries by using a non-static mutex in the struct
_dirdesc (typedef DIR), utilizing it in the *dir functions and remove
remaining and incorrect _FD_LOCK/UNLOCK() use in libc.
- add comments to both thread libraries to indicate libc depends on the
current implementation of static mutex initialization. suggested by
marc@
- major bump libc and libpthread due to function removal, structure
change and weak symbol conversions.
okay marc@, tedu@

Part 2 of file descriptor race and deadlock corrections.

Adjust design of file descriptor table to eliminate races
with both opening and closing of file descriptor entries
and eliminates one class of deadlocks. One nice side effect
of this change in design should be better performance for
applications that open and close many file descriptors due
to reduced fd_table_lock contention and fd entry reuse.

- Add entry states to manage use of entry and eliminate
some closing races. fd entries are not deallocated upon
close() now.
- Call _thread_fd_table_init with one of five discreet
modes to properly initialize an entry and manage the
state transition to open.
- When closing an entry hold the entry spinlock locked
across the state transition and the _thread_sys_close
call to close another race.
- Introduce a new lock type FD_RDWR_CLOSE that transitions
either a closed entry or an open entry into closing state
and then waits for a RDWR lock so that the lock queue can
unwind normally. All subsequent fd lock attempts for that
entry are rejected with EBADF until the fd is fully closed,
or reopened by dup2(). Once a thread holds the FD_RDWR_LOCK
it is safe to close() it or dup2() on it.
- When a thread creates a new fd there is a window of time
when another thread could attempt to use the fd before the
creating thread has initialized the entry for it. This can
result in improper status_flags for the entry, so record
the entries init mode, detect when this has happened and
correct the status_flags when needed.

reviewed by marc@ & brad@, tested by several, okay brad@

Avouid a race in atexit() handling by introducing a lock. Problem
originally reported by Gergely Kovacs; help from dhartmei@;
ok tedu@ millert@

Use a forward declaration of struct timespec instead of assuming
it has already been defined by previous includes. While this
is true at the moment, it will no longer be true after a pending
includes cleanup. OK marc@

major bump to libc and libpthread to break the dependency of a
particular implementation of libpthread for libc. libc no longer
needs pthread.h to compile.
OK millert@, brad@, tedu@

thread safer libc (note: safer, not safe)
Access to the global _res structure replaced by pointers to a
per thread instance. If unthreaded the pointer is to the
global structure.

Also replaced a 64k stack array with malloc-ed memory so
threaded aps (with a default 64k stack) have a chance at working.
ok deraadt@

thread safe libc -- 2nd try. OK miod@, millert@
Thanks to miod@ for m68k and vax fixes

back out previous patch.. there are still some vax/m68k issues

libc changes for thread safety. Tested on:
alpha (millert@), i386 (marc@), m68k (millert@ and miod@),
powerpc (drahn@ and dhartmei@), sparc (millert@ and marc@),
sparc64 (marc@), and vax (millert@ and miod@).
Thanks to millert@, miod@, and mickey@ for fixes along the way.

Part one of userland __P removal. Done with a simple regexp with some minor hand editing to make comments line up correctly. Another pass is forthcoming that handles the cases that could not be done automatically.

put changes back, this time ALL the files.

Back out fgsch@'s tree breaking commits.
Test next time, ok?

fix some const warnings

#(endif|else) foo is incorrect, make it #endif /* foo */
deraadt@ ok

grammer/spelling

typo in comment, AILASES -> ALIASES

oops

rewrite to use the helpers in the thread/ directory and to define weak alias macros

clean

Add thread-safety to libc, so that libc_r will build (on i386 at least).
All POSIX libc api now there (to P1003.1c/D10)
(more md stuff is needed for other libc/arch/*)
(setlogin is no longer a special syscall)
Add -pthread option to gcc (that makes it use -lc_r and -D_POSIX_THREADS).
Doc some re-entrant routines
Add libc_r to intro(3)
dig() uses some libc srcs and an extra -I was needed there.
Add more md stuff to libc_r.
Update includes for the pthreads api
Update libc_r TODO