Remove svn:mergeinfo carried over from stable/9.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC 250691:
Return one-based key so that user can check if the key is ever allocated
in the first place.

MFC r261354:
In _pthread_kill(), if passed pthread is current thread, do not send
the signal second time, by adding the missed else before if statement.

PR: threads/186309

MFC r258499:
Fix for the spurious signal handler call with zero signo in the threaded
process.

MFC r239347, 240295, 240296 and 253325:

r239347 | davidxu | 2012-08-17 10:26:31 +0800 (Fri, 17 Aug 2012) | 7 lines

Implement syscall clock_getcpuclockid2, so we can get a clock id
for process, thread or others we want to support.
Use the syscall to implement POSIX API clock_getcpuclock and
pthread_getcpuclockid.

PR: 168417

------------------------------------------------------------------------
r240295 | davidxu | 2012-09-10 13:00:29 +0800 (Mon, 10 Sep 2012) | 2 lines

Add missing prototype for clock_getcpuclockid.

------------------------------------------------------------------------
r240296 | davidxu | 2012-09-10 13:09:39 +0800 (Mon, 10 Sep 2012) | 2 lines

Process CPU-Time Clocks option is supported, define _POSIX_CPUTIME.

------------------------------------------------------------------------
r253325 | kib | 2013-07-14 03:32:50 +0800 (Sun, 14 Jul 2013) | 6 lines

Allow to call clock_gettime() on the clock id for zombie process.

Reported by: Petr Salinger <Petr.Salinger@seznam.cz>
PR: threads/180496
Sponsored by: The FreeBSD Foundation

MFC r251985:
When enabling the cancellation, only process the pending cancellation
for asynchronous mode.

MFC r251284:
Unify the code of check_deferred_signal() for all architectures.

MFC r250013 (by davidxu):
Remove extra code for SA_RESETHAND, it is not needed because kernel has
already done this.

Approved by: davidxu

MFC r251047:
Avoid a nested frame for getcontext() call in check_deferred_signal().

MFC r251040:
Partially apply the capitalization of the heading word of the sequence
and fix typo.

Merge r238637,r238640,r238641,r238642:

------------------------------------------------------------------------
r238637 | davidxu | 2012-07-20 09:56:14 +0800 (Fri, 20 Jul 2012) | 6 lines

Don't forget to release a thread reference count,
replace _thr_ref_add() with _thr_find_thread(),
so reference count is no longer needed.

------------------------------------------------------------------------
r238640 | davidxu | 2012-07-20 11:00:41 +0800 (Fri, 20 Jul 2012) | 2 lines

Eliminate duplicated code.

------------------------------------------------------------------------
r238641 | davidxu | 2012-07-20 11:16:52 +0800 (Fri, 20 Jul 2012) | 2 lines

Eliminate duplicated code.

------------------------------------------------------------------------
r238642 | davidxu | 2012-07-20 11:22:17 +0800 (Fri, 20 Jul 2012) | 2 lines

Don't assign same value.

Approved by: re (kib)

MFC: 234424

Set SIGCANCEL to SIGTHR as part of some cleanup of DTrace code.

Reviewed by: davidxu@

MFC: r231619

Add thread-local storage support for arm:

- Switch to Variant I TCB layout
- Use function from rtld for TCB allocation/deallocation

MFC r236135:

Return EBUSY for PTHREAD_MUTEX_ADAPTIVE_NP too when the mutex could not
be acquired.

PR: 168317

Merge 233103, 233912 from head:

233103:
Some software think a mutex can be destroyed after it owned it, for
example, it uses a serialization point like following:
pthread_mutex_lock(&mutex);
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&muetx);
They think a previous lock holder should have already left the mutex and
is no longer referencing it, so they destroy it. To be maximum compatible
with such code, we use IA64 version to unlock the mutex in kernel, remove
the two steps unlocking code.

233912:
umtx operation UMTX_OP_MUTEX_WAKE has a side-effect that it accesses
a mutex after a thread has unlocked it, it event writes data to the mutex
memory to clear contention bit, there is a race that other threads
can lock it and unlock it, then destroy it, so it should not write
data to the mutex memory if there isn't any waiter.
The new operation UMTX_OP_MUTEX_WAKE2 try to fix the problem. It
requires thread library to clear the lock word entirely, then
call the WAKE2 operation to check if there is any waiter in kernel,
and try to wake up a thread, if necessary, the contention bit is set again
by the operation. This also mitgates the chance that other threads find
the contention bit and try to enter kernel to compete with each other
to wake up sleeping thread, this is unnecessary. With this change, the
mutex owner is no longer holding the mutex until it reaches a point
where kernel umtx queue is locked, it releases the mutex as soon as
possible.
Performance is improved when the mutex is contensted heavily. On Intel
i3-2310M, the runtime of a benchmark program is reduced from 26.87 seconds
to 2.39 seconds, it even is better than UMTX_OP_MUTEX_WAKE which is
deprecated now. http://people.freebsd.org/~davidxu/bench/mutex_perf.c

Special code for stable/9:
And add code to detect if the UMTX_OP_MUTEX_WAKE2 is available.

MFC r233516: libthr: In the atfork handlers for signals, do not skip the
last signal.

_SIG_MAXSIG works a bit unexpectedly: signals 1 till _SIG_MAXSIG are valid,
both bounds inclusive.

MFC 231906:

Check both seconds and nanoseconds are zero, only checking nanoseconds
is zero may trigger timeout too early. It seems a copy&paste bug.

MFC r230430:
Use getcontextx(3) internal API instead of getcontext(2) to provide
the signal handlers with the context information in the deferrred
case.

MFC 231106:
Plug a memory leak. When a cached thread is reused, don't clear sleep
queue pointers, just reuse it.

MFC 231503:
Make code more stable by checking NULL pointers.

MFC r228536:

The TCB_GET32() and TCB_GET64() macros in the i386 and amd64-specific
versions of pthread_md.h have a special case of dereferencing a null
pointer. Clang warns about this with:

In file included from lib/libthr/arch/i386/i386/pthread_md.c:36:
lib/libthr/arch/i386/include/pthread_md.h:96:10: error: indirection of non-volatile null pointer will be deleted, not trap [-Werror,-Wnull-dereference]
return (TCB_GET32(tcb_self));
^~~~~~~~~~~~~~~~~~~
lib/libthr/arch/i386/include/pthread_md.h:73:13: note: expanded from:
: "m" (*(u_int *)(__tcb_offset(name)))); \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/libthr/arch/i386/include/pthread_md.h:96:10: note: consider using __builtin_trap() or qualifying pointer with 'volatile'

Since this indirection is done relative to the fs or gs segment, to
retrieve thread-specific data, it is an exception to the rule.

Therefore, add a volatile qualifier to tell the compiler we really want
to dereference a zero address.

MFC r227604:
Pass CVWAIT flags to kernel, this should handle
Timeout correctly for pthread_cond_timedwait when
it uses kernel-based condition variable.

Approved by: re (kib)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Do not set thread name to less than informative 'initial thread'.

Merge from r161730:
o Set TP using inline assembly to avoid dead code elimination.
o Eliminate _tcb.

Merge from r161840:
Stylize: avoid using a global register variable.

Merge from r157461:
Simplify _get_curthread() and _tcb_ctor because libc and rtld now
already allocate thread pointer space in tls block for initial thread.

Merge from r177853:
Replace function _umtx_op with _umtx_op_err, the later function directly
returns errno, because errno can be mucked by user's signal handler and
most of pthread api heavily depends on errno to be correct, this change
should improve stability of the thread library.

MFC after: 1 week

r179417 introduced a bug into pthread_once(). Previously pthread_once()
used a global pthread_mutex_t for synchronization. r179417 replaced that
with an implementation that directly used atomic instructions and thr_*
syscalls to synchronize callers to pthread_once. However, calling
pthread_mutex_lock on the global mutex implicitly ensured that
_thr_check_init() had been called but with r179417 this was no longer
guaranteed. This meant that if you were unlucky enough to have your first
call into libthr be a call to pthread_once(), you would segfault when
trying to access the pointer returned by _get_curthread().

The fix is to explicitly call _thr_check_init() from pthread_once().

Reviewed by: davidxu
Approved by: emaste (mentor)
MFC after: 1 week

Introduce a non-portable function pthread_getthreadid_np(3) to retrieve
calling thread's unique integral ID, which is similar to AIX function of
the same name. Bump __FreeBSD_version to note its introduction.

Reviewed by: kib

Fix a typo.

Submitted by: avg

For the process that already loaded libthr but still not initialized
threading, fall back to libc method of performing
__pthread_map_stacks_exec() job.

Reported and tested by: Mykola Dzham <i levsha me>

Implement the __pthread_map_stacks_exec() for libthr.

Stack creation code is changed to call _rtld_get_stack_prot() to get
the stack protection right. There is a race where thread is created
during dlopen() of dso that requires executable stacks. Then,
_rtld_get_stack_prot() may return PROT_READ | PROT_WRITE, but thread
is still not linked into the thread list. In this case, the callback
misses the thread stack, and rechecks the required protection
afterward.

Reviewed by: davidxu

Add section .note.GNU-stack for assembly files used by 386 and amd64.

Return 0 instead of garbage value.

Found by: clang static analyzer

Because sleepqueue may still being used, we should always check wchan with
queue locked.

Always clear flag PMUTEX_FLAG_DEFERED when unlocking, as it is only
significant for lock owner.

Add sleep queue code.

MFp4:

- Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based
condition variable, this should eliminate an extra system call to get
current time.

- Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single
system call. Create userland sleep queue for condition variable, in most
cases, thread will wait in the queue, the pthread_cond_signal will defer
thread wakeup until the mutex is unlocked, it tries to avoid an extra
system call and a extra context switch in time window of pthread_cond_signal
and pthread_mutex_unlock.

The changes are part of process-shared mutex project.

Use sysctl kern.sched.cpusetsize to retrieve size of kernel cpuset.

Return previous sigaction correctly.

Submitted by: avg

Remove local variable 'first', instead check signal number in memory,
because the variable can be in register, second checking the variable
may still return true, however this is unexpected.

Check small set and reject it, this is how kernel did. Always use the
size kernel is using.

- Revert r214409.
- Use long word to figure out sizeof kernel cpuset, hope it works.

Remove locking and unlock in pthread_mutex_destroy, because
it can not fix race condition in application code, as a result,
the problem described in PR threads/151767 is avoided.

Fix typo.

Get cpuset in pthread_attr_get_np() and free it in pthread_attr_destroy().

MFC after: 7 days

Revert revision 214007, I realized that MySQL wants to resolve
a silly rwlock deadlock problem, the deadlock is caused by writer
waiters, if a thread has already locked a reader lock, and wants to
acquire another reader lock, it will be blocked by writer waiters,
but we had already fixed it years ago.

Set default type to PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP, this
is the type we are using.

sort function name.

s/||/&&

Add pthread_rwlockattr_setkind_np and pthread_rwlockattr_getkind_np, the
functions set or get pthread_rwlock type, current supported types are:
PTHREAD_RWLOCK_PREFER_READER_NP,
PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP,
PTHREAD_RWLOCK_PREFER_WRITER_NP,
default is PTHREAD_RWLOCK_PREFER_WRITER_NONCECURSIVE_NP, this maintains
binary compatible with old code.

Oops, don't remove -fexceptions flag.

unwind.h was imported, gcc directory is no longer needed.

change code to use unwind.h.

Check invalid mutex in _mutex_cv_unlock.

In current code, statically initialized and destroyed object have
same null value, the code can not distinguish between them, to
fix the problem, now a destroyed object is assigned to a non-null
value, and it will be rejected by some pthread functions.
PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP is changed to number 1, so that
adaptive mutex can be statically initialized correctly.

Report death event to debugger before moving to gc list, otherwise
debugger may can not find it on thread list.

Only access unwind_disabled when _PTHREAD_FORCED_UNWIND is defined.

Add missing field.

Because old _pthread_cleanup_push/pop do not have frame address,
it is incompatible with stack unwinding code, if they are invoked,
disable stack unwinding for current thread, and when thread is
exiting, print a warning message.

Simplify code, and in while loop, fix operator to match the unwinding
direction.

To support stack unwinding for cancellation points, add -fexceptions flag
for them, two functions _pthread_cancel_enter and _pthread_cancel_leave
are added to let thread enter and leave a cancellation point, it also
makes it possible that other functions can be cancellation points in
libraries without having to be rewritten in libthr.

inline testcancel() into thr_cancel_leave(), because cancel_pending is
almost false, this makes a slight better branch predicting.

In most cases, cancel_point and cancel_async needn't be checked again,
because cancellation is almostly checked at cancellation points.

If we are at cancellation point, always work as deferred mode despite
whether asynchronous mode is turned on or not, this always gives us a
chance to decide whether thread should be canceled or not in
cancellation points.

Because atfork lock is held while forking, a thread cancellation triggered
by atfork handler is unsafe, use intenal flag no_cancel to disable it.

Fix typo.

- _Unwind_Resume function is not used, remove it.
- Use a store barrier to make sure uwl_forcedunwind is lastest thing
other threads can see.
- Add some comments.

Fix a race condition when finding stack unwinding functions.

add code to support stack unwinding when thread exits. note that only
defer-mode cancellation works, asynchrnous mode does not work because
it lacks of libuwind's support. stack unwinding is not enabled unless
LIBTHR_UNWIND_STACK is defined in Makefile.

Move back IN_GCLIST flag into field tlflags, since thread list and gc list
still share same lock.

Don't compare thread pointers again.

Fix copy&paste problem.

Update symbol.

PS_DEAD state needs not be checked because _thr_find_thread() has already
checked it.

Convert thread list lock from mutex to rwlock.

Merge from tbemd, with a small amount of rework:
For all libthr contexts, use ${MACHINE_CPUARCH}
for all libc contexts, use ${MACHINE_ARCH} if it exists, otherwise use
${MACHINE_CPUARCH}
Move some common code up a layer (the .PATH statement was the same in
all the arch submakefiles).

# Hope she hasn't busted powerpc64 with this...

Because POSIX does not allow EINTR to be returned from sigwait(),
add a wrapper for it in libc and rework the code in libthr, the
system call still can return EINTR, we keep this feature.

Discussed on: thread
Reviewed by: jilles

To avoid possible race condition, SIGCANCEL is always sent except the
thread is dead.

Fix off-by-one error in function _thr_sigact_unload, also disable the
function, it seems some gnome application tends to crash if we
unregister sigaction automatically.

Remove incorrect comments, also make sure signal is
disabled when unregistering sigaction.

In function __pthread_cxa_finalize(), also make code for removing
atfork handler be async-signal safe.

pthread_atfork should acquire writer lock and protect the code
with critical region.

Change atfork lock from mutex to rwlock, also make mutexes used by malloc()
module private type, when private type mutex is locked/unlocked, thread
critical region is entered or leaved. These changes makes fork()
async-signal safe which required by POSIX. Note that user's atfork handler
still needs to be async-signal safe, but it is not problem of libthr, it
is user's responsiblity.

Add signal handler wrapper, the reason to add it becauses there are
some cases we want to improve:
1) if a thread signal got a signal while in cancellation point,
it is possible the TDP_WAKEUP may be eaten by signal handler
if the handler called some interruptibly system calls.
2) In signal handler, we want to disable cancellation.
3) When thread holding some low level locks, it is better to
disable signal, those code need not to worry reentrancy,
sigprocmask system call is avoided because it is a bit expensive.
The signal handler wrapper works in this way:
1) libthr installs its signal handler if user code invokes sigaction
to install its handler, the user handler is recorded in internal
array.
2) when a signal is delivered, libthr's signal handler is invoke,
libthr checks if thread holds some low level lock or is in critical
region, if it is true, the signal is buffered, and all signals are
masked, once the thread leaves critical region, correct signal
mask is restored and buffered signal is processed.
3) before user signal handler is invoked, cancellation is temporarily
disabled, after user signal handler is returned, cancellation state
is restored, and pending cancellation is rescheduled.

Unregister thread specific data destructor when a corresponding dso
is unloaded.

clear lock to zero state if it is destroyed.

eliminate unused code.

Decrease rdlock count only when thread unlocked a reader lock.

MFC after: 3 days

Unify 32-bit and 64-bit PowerPC libthr support. This reduces code
duplication, and simplifies the TBEMD import.

Requested by: imp

Remove unused source.

MFC after: 2 weeks

The __hidden definition is provided by sys/cdefs.h.

MFC after: 2 weeks

Add wrapper for setcontext() and swapcontext(), the wrappers
unblock SIGCANCEL which is needed by thread cancellation.

On shared object unload, in __cxa_finalize, call and clear all installed
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.

Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.

Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.

The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.

Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks

Reduce redundant code.

Submitted by: kib

In current implementation, thread cancellation is done in signal handler,
which does not know what is the state of interrupted system call, for
example, open() system call opened a file and the thread is still cancelled,
result is descriptor leak, there are other problems which can cause resource
leak or undeterminable side effect when a thread is cancelled. However, this
is no longer true in new implementation.

In defering mode, a thread is canceled if cancellation request is pending and
later the thread enters a cancellation point, otherwise, a later
pthread_cancel() just causes SIGCANCEL to be sent to the target thread, and
causes target thread to abort system call, userland code in libthr then checks
cancellation state, and cancels the thread if needed. For example, the
cancellation point open(), the thread may be canceled at start,
but later, if it opened a file descriptor, it is not canceled, this avoids
file handle leak. Another example is read(), a thread may be canceled at start
of the function, but later, if it read some bytes from a socket, the thread
is not canceled, the caller then can decide if it should still enable cancelling
or disable it and continue reading data until it thinks it has read all
bytes of a packet, and keeps a protocol stream in health state, if user ignores
partly reading of a packet without disabling cancellation, then second iteration
of read loop cause the thread to be cancelled.
An exception is that the close() cancellation point always closes a file handle
despite whether the thread is cancelled or not.

The old mechanism is still kept, for a functions which is not so easily to
fix a cancellation problem, the rough mechanism is used.

Reviewed by: kib@

According to specification, function fcntl() is a cancellation point only
when cmd argument is F_SETLKW.

Tweak code a bit to be POSIX compatible, when a cancellation request
is acted upon, or when a thread calls pthread_exit(), the thread first
disables cancellation by setting its cancelability state to
PTHREAD_CANCEL_DISABLE and its cancelability type to
PTHREAD_CANCEL_DEFERRED. The cancelability state remains set to
PTHREAD_CANCEL_DISABLE until the thread has terminated.

It has no effect if a cancellation cleanup handler or thread-specific
data destructor routine changes the cancelability state to
PTHREAD_CANCEL_ENABLE.

Use _SIG_VALID instead of expanded form of the macro.

Submitted by: Garrett Cooper <yanegomi gmail com>
MFC after: 1 week

Fix SVN mismerge. We somehow ended up with the 32-bit powerpc version
in arch/powerpc64 instead of the 64-bit one.

Powerpc64 thread libraries support.

Coalesce one more broken line.

Coalesce a couple of broken lines since they can fit within 80
characters. Little nit found while looking at a bug report.

mdoc: order prologue macros consistently by Dd/Dt/Os

Although groff_mdoc(7) gives another impression, this is the ordering
most widely used and also required by mdocml/mandoc.

Reviewed by: ru
Approved by: philip, ed (mentors)

Merge r195129 from project/mips to head by hand:

r195129 | gonzo | 2009-06-27 17:28:56 -0600 (Sat, 27 Jun 2009) | 2 lines
- Use sysarch(2) in MIPS version of _tcb_set/_tcb_get

remove file thr_sem_new.c.

Remove extra new semaphore stubs, because libc already has them, and
ld can find the newest version which is default.

Poked by: kan@

Use umtx to implement process sharable semaphore, to make this work,
now type sema_t is a structure which can be put in a shared memory area,
and multiple processes can operate it concurrently.
User can either use mmap(MAP_SHARED) + sem_init(pshared=1) or use sem_open()
to initialize a shared semaphore.
Named semaphore uses file system and is located in /tmp directory, and its
file name is prefixed with 'SEMD', so now it is chroot or jail friendly.
In simplist cases, both for named and un-named semaphore, userland code
does not have to enter kernel to reduce/increase semaphore's count.
The semaphore is designed to be crash-safe, it means even if an application
is crashed in the middle of operating semaphore, the semaphore state is
still safely recovered by later use, there is no waiter counter maintained
by userland code.
The main semaphore code is in libc and libthr only has some necessary stubs,
this makes it possible that a non-threaded application can use semaphore
without linking to thread library.
Old semaphore implementation is kept libc to maintain binary compatibility.
The kernel ksem API is no longer used in the new implemenation.

Discussed on: threads@

Work-around a race condition on ia64 while unlocking a contested lock.
The race condition is believed to be in UMTX_OP_MUTEX_WAKE. On ia64,
we simply go to the kernel to unlock.
The big question is why this is only a race condition on ia64...

MFC after: 3 days

Revert r199830 for now. Too many ports dlopen() libraries linked with
libthr, but forgot to link main binary with it.

Libthr cannot be dynamically loaded into the running process.
Mark it with -z nodlopen for now.

Discussed with: jhb, kan
MFC after: 3 weeks

Current pselect(3) is implemented in usermode and thus vulnerable to
well-known race condition, which elimination was the reason for the
function appearance in first place. If sigmask supplied as argument to
pselect() enables a signal, the signal might be delivered before thread
called select(2), causing lost wakeup. Reimplement pselect() in kernel,
making change of sigmask and sleep atomic.

Since signal shall be delivered to the usermode, but sigmask restored,
set TDP_OLDMASK and save old mask in td_oldsigmask. The TDP_OLDMASK
should be cleared by ast() in case signal was not gelivered during
syscall execution.

Reviewed by: davidxu
Tested by: pho
MFC after: 1 month

Implement _umtx_op_err() for ia64.

Make openat(2) a cancellation point.

This is required by POSIX and matches open(2).

Reviewed by: kib, jhb
MFC after: 1 month

don't report error if key was deleted.

PR: threads/135462

rwlock implemented from libthr need to fall through the 'hard path' and
query umtx also if the shared waiters bit is set on a shared lock.
The writer starvation avoidance technique, infact, can lead to shared
waiters on a shared lock which can bring to a missed wakeup and thus
to a deadlock if the right bit is not checked (a notable case is the
writers counterpart to be handled through expired timeouts).

Fix that by checking for the shared waiters bit also when unlocking the
shared locks.

That bug was causing a reported MySQL deadlock.
Many thanks go to Nick Esborn and his employer DesertNet which provided
time and machines to identify and fix this issue.

PR: thread/135673
Reported by: Nick Esborn <nick at desert dot net>
Tested by: Nick Esborn <nick at desert dot net>
Reviewed by: jeff

In the current code, rdlock_count is not correctly handled for some cases.
The most notable is that it is not bumped in rwlock_rdlock_common() when
the hard path (__thr_rwlock_rdlock()) returns successfully.
This can lead to deadlocks in libthr when rwlocks recursion in read mode
happens.
Fix the interested parts by correctly handling rdlock_count.

PR: threads/136345
Reported by: rink
Tested by: rink
Reviewed by: jeff
Approved by: re (kib)
MFC: 2 weeks

These are some cosmetic changes to improve the clarity of libthr's fork implementation.

Now that the kernel defines CACHE_LINE_SIZE in machine/param.h, use
that definition in the custom locking code for the run-time linker
rather than local definitions.

Pointed out by: tinderbox
MFC after: 2 weeks

Turn on nodelete linker flag because libthr can not be unloaded safely,
it does hook on to libc.

Forcibly unlock the malloc() locks in the child process after fork(),
by temporary pretending that the process is still multithreaded.
Current malloc lock primitives do nothing for singlethreaded process.

Reviewed by: davidxu, deischen

Don't ignore other fcntl functions, directly call __sys_fcntl if
WITHOUT_SYSCALL_COMPAT is not defined.

Reviewed by: deischen

Don't reference non-existent __fcntl_compat if WITHOUT_SYSCALL_COMPAT is defined.

Submitted by: Pawel Worach "pawel dot worach at gmail dot com"

With only one threading library, simplify the logic of setting SHLIBDIR.

Fix build when WITH_SSP is set explicitly.

Submitted by: Jeremie Le Hen

Honor WITHOUT_INSTALLLIB in some places.

When libthr and rtld start up, there are a number of magic spells cast
in order to get the symbol binding state "just so". This is to allow
locking to be activated and not run into recursion problems later.

However, one of the magic bits involves an explicit call to _umtx_op()
to force symbol resolution. It does a wakeup operation on a fake,
uninitialized (ie: random contents) umtx. Since libthr isn't active, this
is harmless. Nothing can match the random wakeup.

However, valgrind finds this and is not amused. Normally I'd just
write a suppression record for it, but the idea of passing random
args to syscalls (on purpose) just doesn't feel right.

Provide custom simple allocator for rtld locks in libthr. The allocator
does not use any external symbols, thus avoiding possible recursion into
rtld to resolve symbols, when called.

Reviewed by: kan, davidxu
Tested by: rink
MFC after: 1 month

Invoke _rtld_atfork_post earlier, before we reinitialize rtld locks
by switching into single-thread mode.

libthr ignores broken use of lock bitmaps used by default rtld locking
implementation, this in turn turns lock handoff in _rtld_thread_init
into NOP. This in turn makes child processes of forked multi-threaded
programs to run with _thr_signal_block still in effect, with most
signals blocked.

Reported by: phk, kib

Unlock the malloc() locks in the child process after fork(). This gives
us working malloc in the fork child of the multithreaded process.

Although POSIX requires that only async-signal safe functions shall be
operable after fork in multithreaded process, not having malloc lower
the quality of our implementation.

Tested by: rink
Discussed with: kan, davidxu
Reviewed by: kan
MFC after: 1 month

Add two rtld exported symbols, _rtld_atfork_pre and _rtld_atfork_post.
Threading library calls _pre before the fork, allowing the rtld to
lock itself to ensure that other threads of the process are out of
dynamic linker. _post releases the locks.

This allows the rtld to have consistent state in the child. Although
child may legitimately call only async-safe functions, the call may
need plt relocation resolution, and this requires working rtld.

Reported and debugging help by: rink
Reviewed by: kan, davidxu
MFC after: 1 month (anyway, not before 7.1 is out)

Allow psaddr_t to be widened by using thr_pread_{int,long,ptr},
where critical. Some places still use ps_pread/ps_pwrite directly,
but only need changed when byte-order comes into the picture.
Also, change th_p in td_event_msg_t from a pointer type to
psaddr_t, so that events also work when psaddr_t is widened.

Move call to _malloc_thread_cleanup() so that if this is the last thread,
the call never happens. This is necessary because malloc may be used
during exit handler processing.

Submitted by: davidxu

Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes. The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes. Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.

In function pthread_condattr_getpshared, store result correctly.

PR: kern/126128

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Add two commands to _umtx_op system call to allow a simple mutex to be
locked and unlocked completely in userland. by locking and unlocking mutex
in userland, it reduces the total time a mutex is locked by a thread,
in some application code, a mutex only protects a small piece of code, the
code's execution time is less than a simple system call, if a lock contention
happens, however in current implemenation, the lock holder has to extend its
locking time and enter kernel to unlock it, the change avoids this disadvantage,
it first sets mutex to free state and then enters kernel and wake one waiter
up. This improves performance dramatically in some sysbench mutex tests.

Tested by: kris
Sounds great: jeff

Make pthread_cleanup_push() and pthread_cleanup_pop() as a pair of macros,
use stack space to keep cleanup information, this eliminates overhead of
calling malloc() and free() in thread library.

Discussed on: thread@

Call the fcntl compatiblity wrapper from the thread library fcntl wrappers
so that they get the benefit of the (limited) forward ABI compatibility.

MFC after: 1 week

Eliminate global mutex by using pthread_once's state field as
a semaphore.

- Reduce function call overhead for uncontended case.
- Remove unused flags MUTEX_FLAGS_* and their code.
- Check validity of the timeout parameter in mutex_self_lock().

Commit missing mips libthr support that I thought I'd committed earlier

Remove libc_r's remnant code.

Use UMTX_OP_WAIT_UINT_PRIVATE and UMTX_OP_WAKE_PRIVATE to save
time in kernel(avoid VM lookup).

Increase the default MUTEX_ADAPTIVE_SPINS to 2000, after further
testing it turns out 200 was too short to give good adaptive
performance.

Reviewed by: jeff
MFC after: 1 week

Bring in mips threads support from perforce mips2-jnpr branch.

Avoid various shadowed variables. libthr is now almost WARNS=4 clean except
for some const dequalifiers that needs more careful investigation.

Ok'ed by: davidxu

Use native rwlock.

_vfork is not in libthr, remove the reference.

don't include pthread_np.h, it is not used.

put THR_CRITICAL_LEAVE into do .. while statement.

add __hidden suffix to _umtx_op_err, this eliminates PLT.

Non-portable functions are in pthread_np.h, fix compiling problem.

Add pthread_setaffinity_np and pthread_getaffinity_np to libc namespace.

Remove unused functions.

Replace function _umtx_op with _umtx_op_err, the later function directly
returns errno, because errno can be mucked by user's signal handler and
most of pthread api heavily depends on errno to be correct, this change
should improve stability of the thread library.

Replace userland rwlock with a pure kernel based rwlock, the new
implementation does not switch pointers when it resumes waiters.

Asked by: jeff

Restore normal pthread_cond_signal path to avoid some obscure races.

return EAGAIN early rather than running bunch of code later, micro optimize
static branch prediction.

Rewrite rwlock to user atomic operations to change rwlock state, this
eliminates internal mutex lock contention when most rwlock operations
are read.

Orignal patch provided by: jeff

Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse. This should fix WITHOUT_LIBTHR builds as a side effect.

Compile libthr with warnings.

Fixed mis-implementation of pthread_mutex_get{spin,yield}loops_np().

Reviewed by: davidxu

Add POSIX pthread API pthread_getcpuclockid() to get a thread's cpu
time clock id.

Resolve __error()'s PLT early so that it needs not to be resolved again,
otherwise rwlock is recursivly called when signal happens and the __error
was never resolved before.

pthread_mutexattr_destroy() was accidentally broken in last revision,
unbreak it. We should really start compiling this with warnings.

Preserve application code's errno in rtld locking code, it attemps to keep
any case safe.

Make pthread_mutexattr_settype to return error number directly and
conformant to POSIX specification.

Bug reported by: modelnine at modelnine dt org

don't reduce new thread's refcount if current thread can not set cpuset
for it, since the new thread will reduce it by itself.

- Trim trailing spaces.
- Use a different sigmask variable name to avoid confusing.

if passed thread pointer is equal to current thread, pass -1 to kernel
to speed up searching.

- Copy signal mask out before THR_UNLOCK(), because THR_UNLOCK() may call
_thr_suspend_check() which messes sigmask saved in thread structure.
- Don't suspend a thread has force_exit set.
- In pthread_exit(), if there is a suspension flag set, wake up waiting-
thread after setting PS_DEAD, this causes waiting-thread to break loop
in suspend_common().

Actually delete SIGCANCEL mask for suspended thread, so the signal will not
be masked when it is resumed.

If a thread is cancelled, it may have already consumed a umtx_wake,
check waiter and semphore counter to see if we may wake up next thread.

Fix a bug when calculating remnant size.

Don't report death event to debugger if it is a forced exit.

Restore code setting new thread's scheduler parameters, I was thinking
that there might be starvations, but because we have already locked the
thread, the cpuset settings will always be done before the new thread
does real-world work.

Increase and decrease in_sigcancel_handler accordingly to avoid possible
error caused by nested SIGCANCEL stack, it is a bit complex.

Use cpuset defined in pthread_attr for newly created thread, for now,
we set scheduling parameters and cpu binding fully in userland, and
because default scheduling policy is SCHED_RR (time-sharing), we set
default sched_inherit to PTHREAD_SCHED_INHERIT, this saves a system
call.

Add more cpu affinity function's symbols.

Check actual size of cpuset kernel is using and define underscore version
of API.

If a new thread is created, it inherits current thread's signal masks,
however if current thread is executing cancellation handler, signal
SIGCANCEL may have already been blocked, this is unexpected, unblock the
signal in new thread if this happens.

MFC after: 1 week

Include cpuset.h, unbreak compiling.

implement pthread_attr_getaffinity_np and pthread_attr_setaffinity_np.

Implement functions pthread_getaffinity_np and pthread_setaffinity_np to
get and set thread's cpu affinity mask.

_pthread_mutex_isowned_np(): use a more reliable method; the current code
will work in simple cases, but may fail in more complicated ones.

Reviewed by: davidxu

style.Makefile(5)

style(9)

Yet another pointy hat: when I zapped FBSDprivate_1.1, I forgot to move
its contents to FBSDprivate_1.0.

Remove unnecessary prototype.

Convert pthread.map to the format expected by version_gen.awk, and modify
the Makefile accordingly; libthr now explicitly uses libc's Versions.def.

MFC after: 2 weeks

Remove incorrectly added FBSDprivate_1.1 namespace, and move symbols which
are new in FreeBSD 8 to the appropriate namespace.

Per discussion on -threads, rename _islocked_np() to _isowned_np().

After careful consideration (and a brief discussion with attilio@), change
the semantics of pthread_mutex_islocked_np() to return true if and only if
the mutex is held by the current thread.

Obviously, change the regression test to match.

MFC after: 2 weeks

Add pthread_mutex_islocked_np(), a cheap way to verify that a mutex is
locked. This is intended primarily to support the userland equivalent
of the various *_ASSERT_LOCKED() macros we have in the kernel.

MFC after: 2 weeks

SYSTEM_SCOPE_ONLY flag is no longer needed, it is the only mode libthr
supports.

sem_post() requires to return -1 on error.

call underscore version of pthread_cleanup_pop instead.

Remove vfork() overloading, it is no longer needed.

Add function prototypes.

1. Add function pthread_mutex_setspinloops_np to turn a mutex's spin
loop count.
2. Add function pthread_mutex_setyieldloops_np to turn a mutex's yield
loop count.
3. Make environment variables PTHREAD_SPINLOOPS and PTHREAD_YIELDLOOPS
to be only used for turnning PTHREAD_MUTEX_ADAPTIVE_NP mutex.

Enclose all code for macro ENQUEUE_MUTEX in do while statement, and
add missing brackets.

MFC: after 1 day

Fix pointer dereferencing problems in _pthread_mutex_init_calloc_cb() that
were obscured by pseudo-opaque pthreads API pointer casting.

Add _pthread_mutex_init_calloc_cb() to libthr and libkse, so that malloc(3)
(part of libc) can use pthreads mutexes without causing infinite recursion
during initialization.

Simplify code, fix a thread cancellation bug in sem_wait and sem_timedwait.

Reuse nwaiter member field to record number of waiters, in sem_post(),
this should reduce the chance having to do a syscall when there is no
waiter in the semaphore.

Remove warning level and aliasing restrictions.

Convert ceiling type to unsigned integer before comparing, fix compiler
warnings.

Add some function prototypes.

Remove umtx_t definition, use type long directly, add wrapper function
_thr_umtx_wait_uint() for umtx operation UMTX_OP_WAIT_UINT, use the
function in semaphore operations, this fixed compiler warnings.

These are the things that the tinderbox has problems with because it
doesn't use the default CFLAGS which contain -fno-strict-aliasing.

Until the code is cleaned up, just add -fno-strict-aliasing to the
CFLAGS of these for the tinderboxes' sake, allowing the rest of the
tree to have -Werror enabled again.

In _pthread_key_create() ensure that libthr is initialized. This
fixes a NULL-dereference of curthread when libstdc+ initializes
the exception handling globals on archs we can't use GNU TLS due
to lack of support in binutils 2.15 (i.e. arm and sparc64), yet,
thus making threaded C++ programs compiled with GCC 4.2.1 work
again on these archs.

Reviewed by: davidxu
MFC after: 3 days

Avoid doing adaptive spinning for priority protected mutex, current
implementation always does lock in kernel.

Don't do adaptive spinning if it is running on UP kernel.

Restore revision 1.55, the kris's adaptive mutex type.

Adaptive mutexes should have the same deadlock detection properties that
default (errorcheck) mutexes do.

Noticed by: davidxu

Add my recent work of adaptive spin mutex code. Use two environments variable
to tune pthread mutex performance:
1. LIBPTHREAD_SPINLOOPS
If a pthread mutex is being locked by another thread, this environment
variable sets total number of spin loops before the current thread
sleeps in kernel, this saves a syscall overhead if the mutex will be
unlocked very soon (well written application code).
2. LIBPTHREAD_YIELDLOOPS
If a pthread mutex is being locked by other threads, this environment
variable sets total number of sched_yield() loops before the currrent
thread sleeps in kernel. if a pthread mutex is locked, the current thread
gives up cpu, but will not sleep in kernel, this means, current thread
does not set contention bit in mutex, but let lock owner to run again
if the owner is on kernel's run queue, and when lock owner unlocks the
mutex, it does not need to enter kernel and do lots of work to resume
mutex waiters, in some cases, this saves lots of syscall overheads for
mutex owner.

In my practice, sometimes LIBPTHREAD_YIELDLOOPS can massively improve performance
than LIBPTHREAD_SPINLOOPS, this depends on application. These two environments
are global to all pthread mutex, there is no interface to set them for each
pthread mutex, the default values are zero, this means spinning is turned off
by default.

Add a new "non-portable" mutex type, PTHREAD_MUTEX_ADAPTIVE_NP. This
is also implemented in glibc and is used by a number of existing
applications (mysql, firefox, etc).

This mutex type is a default mutex with the additional property that
it spins briefly when attempting to acquire a contested lock, doing
trylock operations in userland before entering the kernel to block if
eventually unsuccessful.

The expectation is that applications requesting this mutex type know
that the mutex is likely to be only held for very brief periods, so it
is faster to spin in userland and probably succeed in acquiring the
mutex, than to enter the kernel and sleep, only to be woken up almost
immediately. This can help significantly in certain cases when
pthread mutexes are heavily contended and held for brief durations
(such as mysql).

Spin up to 200 times before entering the kernel, which represents only
a few us on modern CPUs. No performance degradation was observed with
this value and it is sufficient to avoid a large performance drop in
mysql performance in the heavily contended pthread mutex case.

The libkse implementation is a NOP.

Reviewed by: jeff
MFC after: 3 days

- Stop calling libthr alternative as it's now the default
threading library.

- Now that libpthread is a symlink, it's no longer possible
to link applications with libpthread and have libmap.conf(5)
select the desired threading library; applications will be
linked to the default threading library, libkse or libthr.
Remove an obsolete paragraph.

- Mention that improvements can be seen compared to libkse.

Reviewed by: deischen, davidxu

Use macro THR_CLEANUP_PUSH/POP, they are cheaper than pthread_cleanup_push/pop.

Reverse the logic of UP and SMP.

Submitted by: jasone

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Always install libpthread.* symlinks if at least one of
the threading libraries is built. This simplifies the
logic in makefiles that need to check if the pthreads
support is present. It also fixes a bug where we would
build a threading library that we shouldn't have built:
for example, building with WITHOUT_LIBTHR and the default
value of DEFAULT_THREADING_LIB (libthr) would mistakenly
build the libthr library, but not install it.

Approved by: re (kensmith)

Output error message to STDERR_FILENO.

Approved by: re (bmah)

Set warning level to 2.

Bump library versions in preparation for 7.0.

Ok'd by: kan

Fix a logic bug I re-introduced in my patch I sent to Daniel
that would cause the selected shared threading library to be
overwritten with its 32-bit version on amd64.

PR: amd64/112509

Allow DEFAULT_THREAD_LIB to be set from /etc/src.conf.

Submitted by: ru

Enable symbol versioning by default. Use WITHOUT_SYMVER to disable it.
Warning, after symbol versioning is enabled, going back is not easy
(use WITHOUT_SYMVER at your own risk).

Change the default thread library to libthr.

There most likely still needs to be a version bump for at least the
thread libraries. If necessary, this will happen later.

backout experimental adaptive spinning mutex for product use.

Use C comments since we now preprocess these files with CPP.

If a thread who's name is being set is not the current thread, use macros
THR_THREAD_LOCK and THR_THREAD_UNLOCK instead, this should fix wrong
lock level problem.

Bug reported by: ed dot maste at gmail dot com

Remove 3rd clause, renumber, ok per email

Insert mutex at tail if it has highest ceiling.

Oops, don't corrupt the list.

Check if the PP mutex is recursive, if we have already locked it, place the
mutex in right order sorted by priority ceiling.

get LIBPTHREAD_ADAPTIVE_SPIN early, so it can be used for some global
mutexes.

Check environment variable PTHREAD_ADAPTIVE_SPIN, if it is set, use
it as a default spin cycle count.

- Remove variable _thr_scope_system, all threads are system scope.
- Rename _thr_smp_cpus to boolean variable _thr_is_smp.
- Define CPU_SPINWAIT macro for each arch, only X86 supports it.

Create inline function _thr_umutex_trylock2 to only try one atomic
operation, if it is failed, we call syscall directly, this saves
one atomic operation per lock contention.

Correctly check failed syscall.

Move checking for c_has_waiters into low level _thr_ucond_signal and
_thr_ucond_broadcast, clear condition variable pointer in cancellation
info after returing from _thr_ucond_wait, since kernel has already
dropped the internal lock, so we don't need to unlock it in cancellation
handler again.

test cancel_pending to save a thr_wake call in some specical cases.

_thr_ucond_wait drops lock, we should pick it up again.

the c_has_waiters is lazily updated, temporarily disable the false
alarm code.

Use ucond to implement barrier.

Add _thr_ucond_init().

Tweak _thr_cancel_leave_defer a bit to fix a possible race.

Fix typo, I was using a wrong header file, and the typo is not detected
by compiler.

Use kernel provided userspace condition variable to implement pthread
condition variable.

If a thread was detached, return EINVAL instead, the error code
is also returned by pthread_detach() if a thread was already
detached, the error code was already documented:

> [EINVAL] The implementation has detected that the value speci-
> fied by thread does not refer to a joinable thread.

- When building world WITHOUT_LIBPTHREAD, link libthr to libpthread.
- Don't build ngctl(8) and cached(8) if threading libs aren't built.
- Fix various issues in a cached(8) makefile.

Eliminate atomic operations in thread cancellation functions, it should
reduce overheads of cancellation points.

Move code calculating new inherited priority into single function.

Don't inherit THR_FLAGS_NEED_SUSPEND for child process, child process
only has one thread, setting the flag can cause the thread to be
suspended and no another thread will resume it.

o Make _thr_umutex_init a function.
o Eliminate unused parameter for some functions.
o Convert type of first parameter to void * for _thr_umtx_wait
and _thr_umtx_wake.

Use type pthread_state for thread state.

use rtprio_thread system call to get or set thread priority.

Use return value of _thr_umutex_lock instead of using zero.

Replace internal usage of struct umtx with umutex which can supports
real-time if we want, no functionality is changed.

Same as pthread_setschedparam, use sizeof(struct sched_param) instead.

Pass correct parameter size.

Stylize: avoid using a global register variable.

Rename TLS_TP_OFFSET back to TP_OFFSET. The former clashes with rtld.

Stylize.

Stylize.

TLS fixes:
o The TLS pointer (r2) points 0x7000 after the *end* of the TCB.
o _rtld_allocate_tls() gets a pointer to the current TCB, not the
current TLS pointer.
o _rtld_free_tls() gets the size of the TCB structure.

Fix harmless bug: sizeof(tcb) equals sizeof(void*) not sizeof(struct tcb).
The argument is (currently) not used by _rtld_free_tls().

o Set TP using inline assembly to avoid dead code elimination.
o Eliminate _tcb.

Remove unused file.

pthread_sigmask is in thr_sig.c, remove this file.

Kill unused files.

Use umutex APIs to implement pthread_mutex, member pp_mutexq is added
into pthread structure to keep track of locked PTHREAD_PRIO_PROTECT mutex,
no real mutex code is changed, the mutex locking and unlocking code should
has same performance as before.

Add umutex APIs.

Remove alpha left-overs.

Axe unused member field.

Get number of CPUs and ignore spin count on single processor machine.

1. Don't override underscore version of aio_suspend(), system(),
wait(), waitpid() and usleep(), they are internal versions and
should not be cancellation points.
2. Make wait3() as a cancellation point.
3. Move raise() and pause() into file thr_sig.c.
4. Add functions _sigsuspend, _sigwait, _sigtimedwait and _sigwaitinfo,
remove SIGCANCEL bit in wait-set for those functions, the signal is
used internally to implement thread cancellation.

Unexpand two TAILQ_FOREACH_SAFE cases.

Ok'ed by: davidxu

Caching scheduling policy and priority in userland, a critical but baddly
written application is frequently changing thread priority for SCHED_OTHER
policy.

Use thr_setscheduler, thr_getscheduler and thr_setschedparam to implement
pthread functions.

Use kernel facilities to support real-time scheduling.

__error could be called too early before libthr is initialized, test
this case and return global varible errno instead.

Remove unused member.

Remove unused member field m_queue.

Explicitly request pre-zeroed memory instead of memset'ing our
own.

Ok'ed by: davidxu

Add symbol versions: FBSD_1.0 and FBSDprivate.

s/long/int.

- Use same priority range returned by kernel's sched_get_priority_min()
and sched_get_priority_max() syscalls.
- Remove unused fields from structure pthread_attr.

Remove multiple _get_curthread() calls.

Install shared libpthread library into /lib; needed by some
/sbin programs.

Do not check validity of timeout if a mutex can be acquired immediately.
Completly drop recursive mutex in pthread_cond_wait and restore recursive
after resumption. Reorganize code to make gcc to generate better code.

Remove declaration of _thr_initial from MD header file, it is no longer
needed.

Simplify _get_curthread() and _tcb_ctor because libc and rtld now
already allocate thread pointer space in tls block for initial thread.
Only i386 and amd64 have been done, others still have to be tested.

WARNS level 4 cleanup.

Fix prototype mismatch.

Remove priority mutex code because it does not work correctly,
to make it work, turnstile like mechanism to support priority
propagating and other realtime scheduling options in kernel
should be available to userland mutex, for the moment, I just
want to make libthr be simple and efficient thread library.

Discussed with: deischen, julian

Comment out -g compiler option, found by 'FreeBSD Build Options Survey'.

Check cancellation state carefully to see we really need to call
_pthread_testcancel(). Preserve errno in _thr_suspend_check().

Don't cancel thread if it is in critical region.

Only wake up writer if the lock is free.

Compile thr_rtld.c

Add locking support for rtld.

set an upper limit for thread structures which can be allocated.

Set default contention scope to system.

Convert NO_PROFILE and NO_LIB32 to new style.

Remove mqueue and timer, now they are in librt.

Add some more pthread stubs so that librt can use them.
The thread jump table has been resorted, so you need to
keep libc, libpthread, and libthr in sync.

Submitted by: xu

Reimplement mutex_init to get rid of compile warning.

Disable POSIX SIGEV_THREAD notification support, soon we will have a common
library which can be reused both for libthr and libpthread.

Read cycle count before sending signal, this fixes a userland thread
suspension bug.

MFC after: 3 days

Rework last change of pthread_once, create a function _thr_once_init to
reinitialize its internal locks.

After fork(), reinitialize internal locks for pthread_once().

Now, thread name is stored in kernel, userland no longer has to keep it.

use syscall thr_set_name to implement pthread_set_name_np.

Add missing symbol pthread_set_name_np.

Eliminate unused code.

Enable mutex inheritance code in mutex_fork, I forgot to turn on it.
while here, add some comments about process shared mutex.

Increase the number of spinlocks, since libc's malloc implementation is
about to significantly increase the number of spinlocks used.

Approved by: markm (mentor)

In preparation for a new malloc implementation:

* Add posix_memalign().

* Move calloc() from calloc.c to malloc.c. Add a calloc() implementation in
rtld-elf in order to make the loader happy (even though calloc() isn't
used in rtld-elf).

* Add _malloc_prefork() and _malloc_postfork(), and use them instead of
directly manipulating __malloc_lock.

Approved by: phk, markm (mentor)

The thr_new sysscall was already in libc, don't generate it.

Use macro STATIC_LIB_REQUIRE to declare a symbol should be linked into
static binary.

Rescue pthread_set_name_np for compatible reason, remove unused code.

Tweak macro THR_LOCK_RELEASE a bit for non-PTHREAD_INVARIANTS case.

Return real detached state.

Fix a bug recently introduced, the _thread_active_count should be
decreased if thread can not be created.

Allow background threads to be suspended.

Try to reduce total time needed for suspending all threads,
first broadcast signals to all threads, then enter a wait loop.

Remove functions i386_get_gsbase and i386_set_gsbase, they were already
in libc.

Refine thread suspension code, now thread suspension is a blockable
operation, the caller is blocked util target threads are really
suspended, also avoid suspending a thread when it is holding a
critical lock.
Fix a bug in _thr_ref_delete which tests a never set flag.

1. Add SIGEV_THREAD notification for mq_notify.
2. Reuse current timer code and abstract some common code to
to support both timer and mqueue.

Remove in-progress wait code to sync with libpthread's behavior.

Let _mutex_cv_lock call internal functiona mutex_lock_common.

Hide umtx API symbols as well.

1. Retire macro SCLASS, instead simply use language keyword and
put variables in thr_init.c.
2. Hide all global symbols which won't be exported.

Follow the mistake in libpthread, the first version name in libpthread
is LIBTHREAD_1_0, but really it should be LIBPTHREAD_1_0.
Fix it so libmap.conf works again (it was broken by recent versioning
code in rtld_elf).

Clear return code to zero if joiner successfully waited joinee.

Bug reported by: jasone at connonware when using ports lang/onyx
MFC after: 3 days

Update copyright.

Remove unused _get_curthread() call.

Fix name compatible problem with POSIX standard. the sigval_ptr and
sigval_int really should be sival_ptr and sival_int.
Also sigev_notify_function accepts a union sigval value but not a
pointer.

Remove a redundant _get_curthread() call.

In raise(), use a shortcut to directly send signal to current thread.

Fix some comments, eliminate a memory leak.

Use TIMERS_UNLOCK.

Add code to handle timer_delete(). The timer wrapper code is completely
rewritten, now timers created with same sigev_notify_attributes will
run in same thread, this allows user to organize which timers can
run in same thread to save some thread resource.

Add thread exit handler in timer_loop to handle broken buggy code which
could lead to memory leak.

Add timer_create wrapper.

Remove unused variable.

Reviewed by: cognet

Kill unused variable declaration.

Link libthr to libpthread on Alpha and Sparc.

Add experiment code to implement POSIX timer's SIGEV_THREAD notification.

Follow the change in kernel, joiner thread just waits at thread id
address, let kernel wake it up.

Put pthread_condattr_init sorted order.

Export following functions:
_pthread_mutexattr_getpshared
_pthread_mutexattr_setpshared
pthread_condattr_getpshared
pthread_condattr_setpshared
pthread_mutexattr_getpshared
pthread_mutexattr_setpshared

Add functions pthread_mutexattr_setpshared and pthread_mutexattr_getpshared.

The pthread_attr_set_createsuspend_np was broken, fix it by
replacing THR_FLAGS_SUSPENDED with THR_FLAGS_NEED_SUSPEND.

Sort function names.

Add function pthread_timedjoin_np, the function is similar with pthread_join
except the function will return ETIMEDOUT if target thread does not exit
before specified absolute time passes.

Include needed headers that were obtained through <pthread.h>. Sort headers
while here.

- Prefix MUTEX_TYPE_MAX with PTHREAD_ to avoid namespace pollution.
- Remove the macros MUTEX_TYPE_FAST and MUTEX_TYPE_COUNTING_FAST.

OK'ed by: deischen

Add usleep to the map files.

Noticed by: davidxu

Add a cancellation point for usleep().

While here, fix sleep() so that it is also a cancellation point (a
missing weak reference prevented that).

Cast to uintptr_t to avoid compiler warning, it was broken by
the recent atomic_ptr() change.

Bump the shared library version number of all libraries that have not
been bumped since RELENG_5.

Reviewed by: ru
Approved by: re (not needed for commit check but in principle...)

Fix off-by-one nanosecond bug in macro TIMESPEC_ADD.

Reviewed by: deischen
Approved by: re (dwhite)
MFC after : 4 days

Markup fixes.

Approved by: re

Remove incorrect BUGS entry: libthr does support process-scope thread
scheduling.

MFC after: 1 week
Submitted by: davidxu
Approved by: re (scottl)

Add a stub libthr(3) man page to document what it is and why, as well as
to point at libmap.conf(5). This will help answer questions about what
and why it is, although not in great detail.

Approved by: re (scottl)
MFC after: 1 week
MFC note: When MFC'd, don't MFC mention of work not yet MFC'd.

Remove COMPAT_32BIT, it is no longer needed.

Remove unused variable.

Now libthr only uses GDT based tls on i386. using LDT can only increase
clock cycles and has 8191 threads limitation.

Use thr_new syscall to create a new thread, obscure context operations
is no longer needed.

Add i386_get_gsbase, i386_set_gsbase since old libc doesn't have the
functions, otherwise user ports have to be rebuilt.

Adapt the libpthread patch for using i386_set_gsbase() to libthr.

Conditionally report initial thread event.

Add missing event reporting code.

Add debugger event reporting support, current only TD_CREATE and TD_DEATH
events are reported.

Use the new atomic_cmpset_32().

Adjust hash function for smaller pthread structure size.

Remove unique id field which is no longer used by debugger.

Remove debug symbol from installed library, one can always
use compiled version in libthr directory.

Pass exact number of threads.

Export pthread_condattr_getclock, pthread_condattr_setclock.

Import my recent 1:1 threading working. some features improved includes:
1. fast simple type mutex.
2. __thread tls works.
3. asynchronous cancellation works ( using signal ).
4. thread synchronization is fully based on umtx, mainly, condition
variable and other synchronization objects were rewritten by using
umtx directly. those objects can be shared between processes via
shared memory, it has to change ABI which does not happen yet.
5. default stack size is increased to 1M on 32 bits platform, 2M for
64 bits platform.
As the result, some mysql super-smack benchmarks show performance is
improved massivly.

Okayed by: jeff, mtm, rwatson, scottl

Increase the default stacksizes:

32-bit 64-bit
main thread 2 MB 4 MB
other threads 1 MB 2 MB

Approved by: mtm
Adapted from: libpthread

Fix inverted #ifdef that I added. Who had the pointy hat last?

Submitted by: kan

Don't include sys/user.h merely for its side-effect of recursively
including other headers.

Use the recently exposed fs/gs set functions when compiling libthr to
run as a 32 bit support library for an amd64 kernel. 32 bit consumers of
libthr have zero chance of running on an amd64 kernel since we don't
implement the i386_set_ldt() family of functions. Note that this commit
doesn't make it actually work, it just removes one more obstacle.

For variables that are only checked with defined(), don't provide
any fake value.

1. Now that it's a thread's state is changed from within the kernel, where
no userland locks are heald, the dead thread lock can no longer protect
access to it. Therefore, instead of using an if (!dead)...else clause
after walking the active threads list test the thread pointer before
deciding not to walk the dead threads list. If the thread pointer is null
it means it was not found in the active threads list and the dead threads
list should be checked.

2. Do not free the stack of a thread that is not marked dead. This is the
2nd and final part of eliminating the race to free a thread's stack.

MFC after: 3 days

Remove a reference to a non-existent syscall: _thr_exit(). The
actual name is thr_exit(). How this ever worked is beyond me.

Close a race between a thread exiting and the freeing of it's stack.
After some discussion the best option seems to be to signal the thread's
death from within the kernel. This requires that thr_exit() take an
argument.

Discussed with: davidxu, deischen, marcel
MFC after: 3 days

Remove vestiges of libthr's signal mangling past. This fixes that last
known problem with mysql on libthr: not being able to kill mysqld.

The SUSv3 function say that the affected functions MAY FAIL, if the
specified mutex is invalid. In spec parlance 'MAY FAIL' means it's
up to the implementor. So, remove the check for NULL pointers for two
reasons:
1. A mutex may be invalid without necessarily being NULL.
2. If the pointer to the mutex is NULL core-dumping in the
vicinity of the problem is much much much better than failing
in some other part of the code (especially when the application
doesn't check the return value of the function that you oh so
helpfully set to EINVAL).

Implement cancellation points in libc interfaces, as specified by POSIX.

Adjust code to support AMD64, on AMD64, thread needs to set fsbase by
itself before it can execute any other code, so new thread should be
created with all signals are masked until after fsbase is set.

Add AMD64 support code.

Add rtld-elf to the include path for the rtld to pthread TLS interface.

Add TLS support for libthr on i386.

o Assertions to catch that stuff that shouldn't happen is not happening.
o In the rwlock code: move a duplicated check inside an if..else to after
the if...else clause.
o When initializing a static rwlock move the initialization check
inside the lock.
o In thr_setschedparam.c: When breaking out of the trylock...retry if busy
loop make sure to reset the mtx pointer to null if the mutex is nolonger
in a queue.

Define _libthr_debug for use by libthread_db.

Record the offset of thr_id in the thread structure. Required for
debugging.

Change the thread ID (thr_id_t) used for 1:1 threading from being a
pointer to the corresponding struct thread to the thread ID (lwpid_t)
assigned to that thread. The primary reason for this change is that
libthr now internally uses the same ID as the debugger and the kernel
when referencing to a kernel thread. This allows us to implement the
support for debugging without additional translations and/or mappings.

To preserve the ABI, the 1:1 threading syscalls, including the umtx
locking API have not been changed to work on a lwpid_t. Instead the
1:1 threading syscalls operate on long and the umtx locking API has
not been changed except for the contested bit. Previously this was
the least significant bit. Now it's the most significant bit. Since
the contested bit should not be tested by userland, this change is
not expected to be visible. Just to be sure, UMTX_CONTESTED has been
removed from <sys/umtx.h>.

Reviewed by: mtm@
ABI preservation tested on: i386, ia64

When a thread is created suspended have libthr suspend it explicitly
instead of asking the kernel to do it when we create the thread.

Implement pthread_atfork in libthr. This is mostly from deichen's
work in libpthread.

Submitted by: Dan Nelson <dnelson@allantgroup.com>

In the case that the global thread list is being re-initialized after
a fork, make sure that the current thread isn't detached and freed. As
a consequence the thread should be inserted into the head of the
active list only once (in the beginning).

Make libthr async-signal-safe without costly signal masking. The guidlines I
followed are: Only 3 functions (pthread_cancel, pthread_setcancelstate,
pthread_setcanceltype) are required to be async-signal-safe by POSIX. None of
the rest of the pthread api is required to be async-signal-safe. This means
that only the three mentioned functions are safe to use from inside
signal handlers.
However, there are certain system/libc calls that are
cancellation points that a caller may call from within a signal handler,
and since they are cancellation points calls have to be made into libthr
to test for cancellation and exit the thread if necessary. So, the
cancellation test and thread exit code paths must be async-signal-safe
as well. A summary of the changes follows:

o Almost all of the code paths that masked signals, as well as locking the
pthread structure now lock only the pthread structure.
o Signals are masked (and left that way) as soon as a thread enters
pthread_exit().
o The active and dead threads locks now explicitly require that signals
are masked.
o Access to the isdead field of the pthread structure is protected by both
the active and dead list locks for writing. Either one is sufficient for
reading.
o The thread state and type fields have been combined into one three-state
switch to make it easier to read without requiring a lock. It doesn't need
a lock for writing (and therefore for reading either) because only the
current thread can write to it and it is an integer value.
o The thread state field of the pthread structure has been eliminated. It
was an unnecessary field that mostly duplicated the flags field, but
required additional locking that would make a lot more code paths require
signal masking. Any truly unique values (such as PS_DEAD) have been
reborn as separate members of the pthread structure.
o Since the mutex and condvar pthread functions are not async-signal-safe
there is no need to muck about with the wait queues when handling
a signal ...
o ... which also removes the need for wrapping signal handlers and sigaction(2).
o The condvar and mutex async-cancellation code had to be revised as a result
of some of these changes, which resulted in semi-unrelated changes which
would have been difficult to work on as a separate commit, so they are
included as well.

The only part of the changes I am worried about is related to locking for
the pthread joining fields. But, I will take a closer look at them once this
mega-patch is committed.

q§?\022

Unconditionaly initialize any spin lock passed to pthread_spin_init(). While
makeing sure the spinlock isn't already in use might be a nice feature to
have in theory, it's hard to implement in practice since the passed in
pointer may not be NULL, but still be an invalid value (i.e. 1..2..3.. etc).

o Also check that the mutex type is not less than the minimum allowable value.
o Don't check attribute for NULL. It's the callers responsibility.

Make the minimum implementation of pthread_kill conform to the
functionality spelled out in SUSv3.
o Signal of 0 means do everything except send the signal
o Check that the signal is not invalid
o Check that the target thread is not dead/invalid

o Don't explicitly check the thread for NULL. That is the caller's
responsibility.
o If a thread is not joinable, the correct return value is EINVAL.

o If a thread is marked as detached AND on the dead threads list
the correct return value is ESRCH.
o Don't check the attribute for NULL. It's the caller's responsibility.
o Make the bitwise comparison explicit.

If a condition variable is statically initialized don't return
an error. Return successfully without doing anything.

The thread suspend function now returns ETIMEDOUT, not EAGAIN.

o Remove more references to SIGTHR
o Remove clock resolution information left over from libc_r

Remove the garbage collector thread. All resources are freed
in-line. If the exiting thread cannot release a resource, then
the next thread to exit will release it.

o Since we're not using signals for thread synchronization anymore,
sigprocmask no longer needs to be wrapped.
o raise(3) is applied to the calling thread in a threaded program.
o In the sigaction wrapper reference the correct structure.
o Don't treat SIGTHR especially anymore (infact it won't exist in
a little while).

Stop using signals for synchronizing threads. The performance penalty
was too much.

o The mutex locking functions aren't normally cancellation points. But,
we still have to DTRT when an asynchronously cancellable thread is
cancelled while waiting for a mutex.
o While dequeueing a waiting mutex don't skip a thread if it has
a cancel pending. Only skip it if it is also async cancellable.

o Initialize a local variable before referencing it. This was not
the cause of any bugs because it is *always* indirectly set
in the for...loop, but better to be explicit about it.
o Check the magic number of the passed in thread only after it has
been found in the active thread list. Otherwise, if the check is done
at the very beginning we may end up pointing to garbage if the
thread was once a valid thread, but has now been destroyed.

Make NULL a (void*)0 whereever possible, and fix the warnings(-Werror)
that this provokes. "Wherever possible" means "In the kernel OR NOT
C++" (implying C).

There are places where (void *) pointers are not valid, such as for
function pointers, but in the special case of (void *)0, agreement
settles on it being OK.

Most of the fixes were NULL where an integer zero was needed; many
of the fixes were NULL where ascii <nul> ('\0') was needed, and a
few were just "other".

Tested on: i386 sparc64

libthr powerpc support.

Submitted by: Suleiman Souhlal <refugee@segfaulted.com>
Tested with: most libpthread tests, Apache 'worker' MDM

Implement PThreads barriers and barrier attributes.

Don't wake up the thread after the signal handler
has been executed. On return from the signal handler
the call will either be restarted or EINTR will be returned,
but it will not go back to its previous state. So, it is
sufficient to simply change the state to 'running' without
actually trying to wake up the thread.

Remove thr_getschedparam.c since it's contents have been moved into
thr_setschedparam.c

There are consumers of rwlocks, inluding our own libc, that depend on
a PTHREAD_RWLOCK_INITIALIZER to do for rwlocks what
a similarly named symbol does for statically initialized mutexes.
This symbol was dropped in The Open Group Base Specifications Issue 6
and does not exist in IEEE Std 1003.1, 2003, but it should still be
supported for backwards compatibility.

Pointy hat: mtm

o Catch up with the mutex priority protocol fixes.
o Move pthread_getschedparam() into the same file with it's
pthread_set* counterpart. Copyright on both files is identical.

o Style
o Instead of checking both the passed in pointer and its value
for NULL, only check the latter. Any caller that passes in
a NULL pointer is obviously wrong.

o Refactor and, among other things, get rid of insane nesting levels.
o Fix mutex priority protocols. Keep separate counts of priority
inheritance and protection mutexes to make things easier.
This will not have much affect since this is only the
userland side, and the rest involves kernel scheduling.

Move the initialization of thread priority to a common function.

Move the weak references to the top of the file to conform
to the format of other similar files in libthr.

style cleanup: Remove duplicate $FreeBSD$ tags.

These files had tags after the copyright notice,
inside the comment block (incorrect, removed),
and outside the comment block (correct).

Approved by: rwatson (mentor)

Remove the band-aid (#include <time.h>).

Add <time.h> -- bandaid to unbreak world in <semaphore.h>.

Bump up the maximum number concurrent threads on x86.

I update the rwlock code in libthr to be more standards compliant and
what do I get for my troubles? libc breaks offcourse!

Reimplement a hack (in libthr) that allows libc to use
rwlocks without initializing them first. The hack was reimplemented
so that only a private libc version of the rwlock locking functions
initializes an uninitialized rwlock. The application version will
correctly fail.

When suspending a thread if the timeout was very short or
the system call got interrupted and the absolute timeout is
converted to a relative timeout, it may happen that we get a
negative number. In such a case, simply set the timeout to
zero so that if the event that the thread wants to wait for has
happened it can still return successfully, but if it hasn't
happened then the thread doesn't suspend indefinitely. This should
fix certain applications (including mozilla) that seem to hang
indefinitely sometimes.

Noticed and debugged by: Morten Johansen <root@morten-johansen.net>

o Implement the pthread_spin_* functions in libthr.
o Man pages

Refactor _pthread_mutex_init
o Simplify the logic by removing a lot of unnecesary nesting
o Reduce the amount of local variables
o Zero-out the allocated structure and get rid of
all the unnecessary setting to 0 and NULL;

Refactor _pthread_mutex_destroy
o Simplify the logic by removing a lot of unnecesary nesting
o No need to check pointer that the mutex attributes points
to. Checking passed in pointer is enough.

Implement reference counting of read-write locks. This uses
a list in the thread structure to keep track of the locks and
how many times they have been locked. This list is checked
on every lock and unlock. The traversal through the list is
O(n). Most applications don't hold so many locks at once that
this will become a problem. However, if it does become a problem
it might be a good idea to review this once libthr is
off probation and in the optimization cycle.
This fixes:
o deadlock when a thread tries to recursively acquire a
read lock when a writer is waiting on the lock.
o a thread could previously successfully unlock a lock it did not own
o deadlock when a thread tries to acquire a write lock on
a lock it already owns for reading or writing [ this is admittedly
not required by POSIX, but is nice to have ]

Add an implementation of pthread_rwlock_timed{rd,wr}lock() to libthr with
attendant documentation.

o We are not required to initialize an invalid rwlock. So axe all that
code and simply return EINVAL (which is allowed by the standard) in
all those pthread functions that previously initialized it.

o Refactor the pthread_rwlock_[try]rdlock() and pthread_rwlock_[try]wrlock()
functions. They are now completeley condensed into rwlock_rdlock_common()
and rwlock_wrlock_common(), respectively.

o If the application tries to destroy an rwlock that is currently
held by a thread return EBUSY where it previously went ahead and
freed all resources associated with the lock.

o Refactor _pthread_rwlock_init() to make it look (relatively) sane.

o When obtaining a read lock on an rwlock the check for whether it
would exceed the maximum allowed read locks should happen *before*
we obtain the lock.

o The pthread_rwlock_* functions shall *never* return EINTR, so make
sure to requeue/resuspend the thread if it encounters such an error.

o Make a note that pthread_rwlock_unlock() needs to ensure it holds a
lock on an rwlock it tries to unlock. It will be implemented in a
separate commit because it requires some additional rwlock infrastructure.

Return ENOTSUP instead of -1.

o Implement pthread_mutex_timedlock(), which does not block indefinitely on
a mutex locked by another thread.
o document it: pthread_mutex_timedlock(3)

Make it possible for the library to specify a timeout value when
waiting on a locked mutex. This involves passing a struct timespec
from the pthread mutex locking interfaces all the way down to the
function that suspends the thread until the mutex is released.
The timeout is assumed to be an absolute time (i.e. not relative to
the current time).

Also, in _thread_suspend() make the passed in timespec const.

Don't block SIGTRAP - it makes it hard to debug programs with gdb.

Reviewed by: mtm

Preparations to make libthr work in multi-threaded fork()ing applications.

o Remove some code duplication between _thread_init(), which is run once
to initialize libthr and the intitial thread, and pthread_create(), which
initializes newly created threads, into a new function called from both
places: init_td_common()
o Move initialization of certain parts of libthr into a separate
function. These include:
- Active threads list and it's lock
- Dead threads list and it's lock & condition variable
- Naming and insertion of the initial thread into the
active threads list.

Remove _giant_mutex and its associated macros.

Comment out most of pthread_setschedparam. Pthread priorities didn't
work before anyways, and I didn't want to fix broken code I had no
way of testing. It was necessary however, in order to get rid of GIANT_LOCK.
Pthread priorities will have to wait a little longer to get fixed.

When creating a pthread in the suspended state their were two
problems: (1) The wrong flag was being checked for in the attribute
(2) The pthread's state was not being set to indicate it was
suspended.

Noticed by: Igor Sysoev <is@rambler-co.ru>

Doh! Lock the thread passed in by the caller, not the current thread.

Remove uses of GIANT_LOCK and replace with appropriate thread
and thread list locks.

Take a stab at fixing some of the macro-nightmare.
PTHREAD_NEW_STATE should work as expected now: a thread
marked PS_RUNNING will get sent a SIGTHR.
Still more cleanups necessary.

Fix the wrapper function around signals so that a signal handling
thread on one of the mutex or condition variable queues is removed
from those queues before the real signal handler is called.

Ugghh, cvs add the functions necessary to lock the global signal action
table.

o Add a wrapper around sigaction(2), so we can insert our own wrapper
around signals.
o Lock the process global signal action table.

Enable cancellation points around some syscalls.

Use dynamic instead of static LDT allocation.

Approved by: re (scottl)

Relink libc_r.a, libc_r.so and libc_r_p.so from libthr to libkse.
On ia64, where there's no libc_r at all, libkse is now the default
thread library by virtue of these links.

The reasons for this change are:
1. libkse is slated to become the default thread library anyway,
2. active development and maintenance is only present for libkse,
3. GNOME and KDE, both in the process of being supported on ia64,
work better with KSE; even on ia64.

Implement _get_curthread and _set_curthread. We use GCCs builtin
function this, which expands to PAL calls (rduniq and wruniq).
This needs adjustment when TLS is implemented.

This commit was generated by cvs2svn to compensate for changes in r117783,
which included commits to RCS files with non-trunk default branches.

When _PTHREADSINVARIANTS is defined SIGABRT is not included
in the set of signals to block.
Also, make the PANIC macro call abort() instead of simply
exiting.

Change all instances of THR_LOCK/UNLOCK, etc to UMTX_*.
It is a more acurate description of the locks they
operate on.

There's no need for _umtxtrylock to be a separate function.
Roll it into the pre-existing macro that's used to call it.

_pthread_mutex_trylock() is another internal libc function that must block
signals.

Begin making libthr async signal safe.

Create a private, single underscore, version of pthread_mutex_unlock for libc.
pthread_mutex_lock already has one. These versions are different from the
ones that applications will link against because they block all signals
from the time a call to lock the mutex is made until it is successfully
unlocked.

Do not attempt to reque a thread on a mutex queue. It may be that
a thread receives a spurious wakeup from sigtimedwait(), so make sure
that the call to the queueing code is called only once before entering
the loop (not in the loop). This should fix some fatal errors people
are seeing with messages stating the thread is already on the mutex queue.
These errors may still be triggered from signal handlers; however, since
that part of the code is not locked down yet.

Axe AINC.

Submitted by: bde

Catchup with _thread_suspend() changes.

Sweep through pthread locking and use the new locking primitives for
libthr.

Locking primitives and operations in libthr should use struct umtx,
not spinlock_t. Spinlock_t and the associated functions and macros may
require blocking signals in order for async-safe libc functions to behave
appropriately in libthr. This is undesriable for libthr internal locking.
So, this is the first step in completely separating libthr from libc's
locking primitives.

Three new macros should be used for internal libthr locking from now on:
THR_LOCK, THR_TRYLOCK, THR_UNLOCK.

In a critical section, separate the aquisition of the thread lock
and the disabling of signals. What we are really interested in is
keeping track of recursive disabling of signals. We should not
be recursively acquiring thread locks. Any such situations should
be reorganized to not require a recursive lock.

Separating the two out also allows us to block signals independent of
acquiring thread locks. This will be needed in libthr in the near future when
we put the pieces together to protect libc functions that use pthread mutexes
and low level locks.

Make _thread_suspend work with both the old broken sigtimedwait
implementation and the new improved one. We now precompute the
signal set passed to sigtimedwait, using an inverted set when
necessary for compatibility with older kernels.

The move to _retire() a thread in the GC instead of in the thread's
exit function has invalidated the need for _spin[un]lock_pthread().
The _spin[un]lock() functions can now dereference curthread without
the danger that the ldtentry containing the pointer to the thread
has been cleared out from under them.

Create compatibility links for libc_r on ia64 to prevent build-time
breakages. Note that runtime compatibility is not guaranteed. Future
changes to setjmp/longjmp in libc will break threaded applications
linked against libc_r.so.5 on ia64. We pull our "tier 2" card once
more...

Reviewed by: ru

_thread_printf() is only used for debugging or in cases where something's
screwed beyond all help, so it can just skip the pthreads wrapper
for write(2) and call directly into it.

Make C applications statically compiled with libthr work. Previously,
an application compiled -static with libthr would dump core in
malloc(3) because the stub thread initialization routine in libc would
be used instead of the libthr supplied one.

Teach recent changes in the umtx structure in the kernel to the libthr
initialiazer.

Found by: tinderbox

Unwind the _giant_mutex from pthread_detach(). When detaching a joiner thread
it's important the correct lock order is observed: lock first the joined and
then the joiner.

Consolidate static_init() and static_init_private into one function.
The behaviour of this function is controlled by the argument: private.

.S comments must be C comments, not ASM ones.

I botched one of my committs in the last round. Fix it.

Make the mutex static initializers look more like the one for
condition variables. Cosmetic.

Explicitly compare against PTHREAD_MUTEX_INITIALIZER. We shouldn't
encourage calls to the mutex functions with null pointers to mutexes.

Approved by: re/jhb

Use a static lock to ake sure pthread_cond_* functions called
from multiple threads don't initialze the same condition variable
more than once.

Explicitly compare cond pointers with PTHREAD_COND_INITIALIZER instead
of NULL. Just because it happens to be defined as NULL is no reason
to encourage the idea that people can call those functions with
NULL pointers to a condition variable.

Approved by: re/jhb

Missing unlock.

Approved by: re/jhb

Don't hold the active thread list lock when signaling the gc thread.
The dead list thread is sufficient for synchronization.

Retire the arch_id (ldt array slot) in the gc thread instead of the
doing it in the thread itself.

Approved by: re/jhb

It's unnecessary to lock the thread during creation. Simply extend
the scope of the active thread list lock.

Approved by: re/jhb

Minimize the potential for deadlocks between an exiting thread and it's
joiner by making sure all locks and unlocks occur in the same order. For
the record the lock order is: DEAD_LIST, THREAD_LIST, exiting thread, joiner
thread.

Approved by: re/rwatson

Revert part of the last commit. I don't know what I was smoking.

Approved by: re/rwatson

Decouple the thread stack [de]allocating functions from the 'dead threads list'
lock. It's not really necessary and we don't need the added complexity
or potential for deadlocks.

Approved by: re/blanket libthr

Revise the unlock order in _pthread_join(). Also, if the joined
thread is not dead, the join loop is guaranteed to execute at least
once, so there is no need to pick up the thread list lock after
we return from suspenstion only to release it after the loop.

Approved by: re/blanket libthr

Return gracefully, rather than aborting, when the maximum concurrent
threads per process has been reached. Return EAGAIN, as per spec.

Approved by: re/blanket libthr

_pthread_cancel() breaks the normal lock order of first locking the
joined and then the joiner thread. There isn't an easy (sane?) way
to make it use the correct order without introducing races involving
the target thread and finding which (active or dead) list it is on. So,
after locking the canceled thread it will try to lock the joined thread
and if it fails release the first lock and try again from the top.

Introduce a new function, _spintrylock, which is simply a wrapper arround
umtx_trylock(), to help accomplish this.

Approved by: re/blanket libthr

Part of the last patch.
Modify the thread creation and thread searching routine
to lock the thread lists with the new locks instead of GIANT_LOCK.

Approved by: re/blanket libthr

Start locking up the active and dead threads lists. The active threads
list is protected by a spinlock_t, but the dead list uses a pthread_mutex
because it is necessary to synchronize other threads with the garbage
collector thread. Lock/Unlock macros are used so it's easier to make
changes to the locks in the future.

The 'dead thread list' lock is intended to replace the gc mutex.
This doesn't have any practical ramifications. It simply makes it
clearer what the purpose of the lock is. The gc will use this lock,
instead of the gc mutex, to synchronize access to the dead list with
other threads.

Modify _pthread_exit() to use these two new locks instead of GIANT_LOCK,
and also to properly lock and protect thread state changes,
especially with respect to a joining thread.

The gc thread was also re-arranged to be more organized and less nested.

_pthread_join() was also modified to use the thread list locks. However,
locking and unlocking here needs special care because a thread could find
itself in a position where it's joining an exiting thread that is
waiting on the dead list lock, which this thread (joiner) holds. If the
joiner doesn't take care to lock *and* unlock in the same order they
(the joiner and the joinee) could deadlock against each other.

Approved by: re/blanket libthr

The libthr code makes use of higher-level primitives (pthread_mutex_t and
pthread_cond_t) internaly in addition to the low-level spinlock_t. The
garbage collector mutex and condition variable are two such examples. This
might lead to critical sections nested within critical sections. Implement
a reference counting mechanism so that signals are masked only on the first
entry and unmasked on the last exit.

I'm not sure I like the idea of nested critical sections, but if
the library is going to use the pthread primitives it might be necessary.

Approved by: re/blanket libthr

The struct mcontext has changed. It's using the register sets. Bring
this in line.

Lock the cond queue (condition variables):
Access to the thread's flags and state is protected by
_thread_critical_enter/exit(). When a thread is signaled with a condition
its state must be protected by locking it and disabling
signals before it is taken of the waiters' queue.

Move the implementation of pthread_cond_signal() and pthread_cond_broadcast()
into one function, cond_signal(). Its behaviour is determined by the
last argument, int broadcast. If this is set to 1 it will remove all
waiters, otherwise it will wake up only the first waiter thread.

Remove an extraneous call to pthread_testcancel().

Approved by: re/blanket libthr

Add two functions: _spinlock_pthread() and _spinunlock_pthread()
that take the address of a struct pthread as their first argument.
_spin[un]lock() just become wrappers arround these two functions.
These new functions are for use in situations where curthread can't be
used. One example is _thread_retire(), where we invalidate the array index
curthread uses to get its pointer..

Approved by: re/blanket libthr

EDOOFUS
Prevent one thread from messing up another thread's saved signal
mask by saving it in struct pthread instead of leaving it as a
global variable. D'oh!

Approved by: re/blanket libthr

Make WARNS2 clean. The fixes mostly included:
o removed unused variables
o explicit inclusion of header files
o prototypes for externally defined functions

Approved by: re/blanket libthr

note to self: do not confuse void* with int.

Approved by: re/blanket libthr

o Make the defenition of _set_curthread() match its declaration
in thr_private.h

o Lock down the ldt_entries array and ldt_free, which points to
the next free slot. As noted in the comments, it's necessary
to special case the initial_thread because %gs is not setup
for it yet. This is ok because that early in the program there
won't be any reentrancy issues anyways.

Approved by: re/blanket libthr

Insert a debugging aid:
When in either the mutex or cond queue we notice that the thread
is already on one of the queues, don't just simply abort(). Print
out the thread's identifiers and what queue it was on.

Approved by: markm/mentor, re/blanket libthr

Re-enable the garbage collector thread in anticipation of further
locking work. I can't see anything obviously wrong with it (other than
the need to update the locking).

Approved by: markm/mentor, re/blanket libthr

When a thread exits it does not return from the kernel unless it
is the *only* remaining thread in the application, in which case we
should not core dump, and instead exit gracefully.

Approved by: markm/mentor, re/blanket libthr

The thread id was being set *before* zeroing out the thread. Reverse
the order.

Approved by: markm/mentor, re/blanket libthr

Move a misplaced comment.

Approved by: markm/mentor (implicit), re/blanket libthr

Do some cleanup with respect to condition variables. The implementation
of pthread_cond_timedwait() is moved into cond_wait_common().
Pthread_cond_wait() and pthread_cond_timedwait() are now wrappers around
this function. Previously, the former called the latter with the abstime
pointing to 0 time. This violated Posix semantics should an application
have reason to call it with that argument because instead or returning
immediately it would have waited indefinitely for the cv to be signaled.

Approved by: markm/mentor, re/blanket libthr
Reviewed by: jeff