History log of /freebsd-current/lib/libthr/thread/thr_umtx.h
Revision Date Author Comments
# ef9871c6 17-Jan-2024 Brooks Davis <brooks@FreeBSD.org>

libthr: move _umtx_op_err() to libsys

Declare in sys/umtx.h and implement in libsys. Explicitly link libthr
with libsys.

When building libthr static include _umtx_op_err so we don't break static
linkage with -lpthread.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908


# b3e76948 16-Aug-2023 Warner Losh <imp@FreeBSD.org>

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/


# 4d846d26 10-May-2023 Warner Losh <imp@FreeBSD.org>

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix


# 5e53a4f9 25-Nov-2017 Pedro F. Giffuni <pfg@FreeBSD.org>

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.


# d2335a57 19-May-2017 Eric van Gyzen <vangyzen@FreeBSD.org>

libthr: fix warnings at WARNS=6

Fix warnings about the following when WARNS=6 (which I will commit soon):

- casting away const
- no previous 'extern' declaration for non-static variable
- others as explained by #pragmas and comments
- unused parameters

The last is the only functional change.

Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10808


# 2a339d9e 17-May-2016 Konstantin Belousov <kib@FreeBSD.org>

Add implementation of robust mutexes, hopefully close enough to the
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.

A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held. The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.

The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths. Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.

The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive). Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.

Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot. When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.

The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.

Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
the lifetime of the shared mutex associated with a vnode' page.

Reviewed by: jilles (previous version, supposedly the objection was fixed)
Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by: pho
Sponsored by: The FreeBSD Foundation


# 3cb14a89 27-Apr-2013 Jilles Tjoelker <jilles@FreeBSD.org>

libthr: Fix a parameter name in an internal header file.


# e220a13a 11-Aug-2012 David Xu <davidxu@FreeBSD.org>

MFp4:
Further decreases unexpected context switches by defering mutex wakeup
until internal sleep queue lock is released.


# 17ce6063 04-Apr-2012 David Xu <davidxu@FreeBSD.org>

umtx operation UMTX_OP_MUTEX_WAKE has a side-effect that it accesses
a mutex after a thread has unlocked it, it event writes data to the mutex
memory to clear contention bit, there is a race that other threads
can lock it and unlock it, then destroy it, so it should not write
data to the mutex memory if there isn't any waiter.
The new operation UMTX_OP_MUTEX_WAKE2 try to fix the problem. It
requires thread library to clear the lock word entirely, then
call the WAKE2 operation to check if there is any waiter in kernel,
and try to wake up a thread, if necessary, the contention bit is set again
by the operation. This also mitgates the chance that other threads find
the contention bit and try to enter kernel to compete with each other
to wake up sleeping thread, this is unnecessary. With this change, the
mutex owner is no longer holding the mutex until it reaches a point
where kernel umtx queue is locked, it releases the mutex as soon as
possible.
Performance is improved when the mutex is contensted heavily. On Intel
i3-2310M, the runtime of a benchmark program is reduced from 26.87 seconds
to 2.39 seconds, it even is better than UMTX_OP_MUTEX_WAKE which is
deprecated now. http://people.freebsd.org/~davidxu/bench/mutex_perf.c


# 24c20949 27-Feb-2012 David Xu <davidxu@FreeBSD.org>

Follow changes made in revision 232144, pass absolute timeout to kernel,
this eliminates a clock_gettime() syscall.


# d1078b0b 21-Dec-2010 David Xu <davidxu@FreeBSD.org>

MFp4:

- Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based
condition variable, this should eliminate an extra system call to get
current time.

- Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single
system call. Create userland sleep queue for condition variable, in most
cases, thread will wait in the queue, the pthread_cond_signal will defer
thread wakeup until the mutex is unlocked, it tries to avoid an extra
system call and a extra context switch in time window of pthread_cond_signal
and pthread_mutex_unlock.

The changes are part of process-shared mutex project.


# a7d5f7eb 19-Oct-2010 Jamie Gritton <jamie@FreeBSD.org>

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.


# ada33a6e 31-Aug-2010 David Xu <davidxu@FreeBSD.org>

Change atfork lock from mutex to rwlock, also make mutexes used by malloc()
module private type, when private type mutex is locked/unlocked, thread
critical region is entered or leaved. These changes makes fork()
async-signal safe which required by POSIX. Note that user's atfork handler
still needs to be async-signal safe, but it is not problem of libthr, it
is user's responsiblity.


# 02c3c858 31-Aug-2010 David Xu <davidxu@FreeBSD.org>

Add signal handler wrapper, the reason to add it becauses there are
some cases we want to improve:
1) if a thread signal got a signal while in cancellation point,
it is possible the TDP_WAKEUP may be eaten by signal handler
if the handler called some interruptibly system calls.
2) In signal handler, we want to disable cancellation.
3) When thread holding some low level locks, it is better to
disable signal, those code need not to worry reentrancy,
sigprocmask system call is avoided because it is a bit expensive.
The signal handler wrapper works in this way:
1) libthr installs its signal handler if user code invokes sigaction
to install its handler, the user handler is recorded in internal
array.
2) when a signal is delivered, libthr's signal handler is invoke,
libthr checks if thread holds some low level lock or is in critical
region, if it is true, the signal is buffered, and all signals are
masked, once the thread leaves critical region, correct signal
mask is restored and buffered signal is processed.
3) before user signal handler is invoked, cancellation is temporarily
disabled, after user signal handler is returned, cancellation state
is restored, and pending cancellation is rescheduled.


# fe0506d7 09-Mar-2010 Marcel Moolenaar <marcel@FreeBSD.org>

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.


# 917c07ca 24-Sep-2009 Attilio Rao <attilio@FreeBSD.org>

MFC r197445:
Let fall down in the hard path (thus handling shared waiters wakeup
correctly) for the shared waiters also in the rwlock held in shared mode
as well, fixing possible deadlocks.

Please note that this is a special condition as we want this fix in
before RC2 as we assume it is critical and so it has been handled
as an instant-merge. For the STABLE_7 branch, 1 week before the MFC
is assumed.

Approved by: re (kib)


# b13c5f28 23-Sep-2009 Attilio Rao <attilio@FreeBSD.org>

rwlock implemented from libthr need to fall through the 'hard path' and
query umtx also if the shared waiters bit is set on a shared lock.
The writer starvation avoidance technique, infact, can lead to shared
waiters on a shared lock which can bring to a missed wakeup and thus
to a deadlock if the right bit is not checked (a notable case is the
writers counterpart to be handled through expired timeouts).

Fix that by checking for the shared waiters bit also when unlocking the
shared locks.

That bug was causing a reported MySQL deadlock.
Many thanks go to Nick Esborn and his employer DesertNet which provided
time and machines to identify and fix this issue.

PR: thread/135673
Reported by: Nick Esborn <nick at desert dot net>
Tested by: Nick Esborn <nick at desert dot net>
Reviewed by: jeff


# d7f03759 19-Oct-2008 Ulf Lilleengen <lulf@FreeBSD.org>

- Import the HEAD csup code which is the basis for the cvsmode work.


# 7de1ecef 24-Jun-2008 David Xu <davidxu@FreeBSD.org>

Add two commands to _umtx_op system call to allow a simple mutex to be
locked and unlocked completely in userland. by locking and unlocking mutex
in userland, it reduces the total time a mutex is locked by a thread,
in some application code, a mutex only protects a small piece of code, the
code's execution time is less than a simple system call, if a lock contention
happens, however in current implemenation, the lock holder has to extend its
locking time and enter kernel to unlock it, the change avoids this disadvantage,
it first sets mutex to free state and then enters kernel and wake one waiter
up. This improves performance dramatically in some sysbench mutex tests.

Tested by: kris
Sounds great: jeff


# 8d6a11a0 28-Apr-2008 David Xu <davidxu@FreeBSD.org>

Use UMTX_OP_WAIT_UINT_PRIVATE and UMTX_OP_WAKE_PRIVATE to save
time in kernel(avoid VM lookup).


# 8bf1a48c 01-Apr-2008 David Xu <davidxu@FreeBSD.org>

Replace userland rwlock with a pure kernel based rwlock, the new
implementation does not switch pointers when it resumes waiters.

Asked by: jeff


# 6fdfcacb 20-Nov-2007 David Xu <davidxu@FreeBSD.org>

Remove umtx_t definition, use type long directly, add wrapper function
_thr_umtx_wait_uint() for umtx operation UMTX_OP_WAIT_UINT, use the
function in semaphore operations, this fixed compiler warnings.


# 842a092b 19-Dec-2006 David Xu <davidxu@FreeBSD.org>

Check environment variable PTHREAD_ADAPTIVE_SPIN, if it is set, use
it as a default spin cycle count.


# 8a8178c0 14-Dec-2006 David Xu <davidxu@FreeBSD.org>

Create inline function _thr_umutex_trylock2 to only try one atomic
operation, if it is failed, we call syscall directly, this saves
one atomic operation per lock contention.


# 670b44d6 04-Dec-2006 David Xu <davidxu@FreeBSD.org>

Add _thr_ucond_init().


# 2bd2c907 04-Dec-2006 David Xu <davidxu@FreeBSD.org>

Use kernel provided userspace condition variable to implement pthread
condition variable.


# 8042f26d 13-Oct-2006 David Xu <davidxu@FreeBSD.org>

o Make _thr_umutex_init a function.
o Eliminate unused parameter for some functions.
o Convert type of first parameter to void * for _thr_umtx_wait
and _thr_umtx_wake.


# bddd24cd 05-Sep-2006 David Xu <davidxu@FreeBSD.org>

Replace internal usage of struct umtx with umutex which can supports
real-time if we want, no functionality is changed.


# cf13ecda 27-Aug-2006 David Xu <davidxu@FreeBSD.org>

Add umutex APIs.


# 8429e734 20-Dec-2005 David Xu <davidxu@FreeBSD.org>

Hide umtx API symbols as well.


# 2ff77b92 27-Jul-2005 David Xu <davidxu@FreeBSD.org>

Cast to uintptr_t to avoid compiler warning, it was broken by
the recent atomic_ptr() change.


# a091d823 01-Apr-2005 David Xu <davidxu@FreeBSD.org>

Import my recent 1:1 threading working. some features improved includes:
1. fast simple type mutex.
2. __thread tls works.
3. asynchronous cancellation works ( using signal ).
4. thread synchronization is fully based on umtx, mainly, condition
variable and other synchronization objects were rewritten by using
umtx directly. those objects can be shared between processes via
shared memory, it has to change ABI which does not happen yet.
5. default stack size is increased to 1M on 32 bits platform, 2M for
64 bits platform.
As the result, some mysql super-smack benchmarks show performance is
improved massivly.

Okayed by: jeff, mtm, rwatson, scottl