#
259065 |
|
07-Dec-2013 |
gjb |
- Copy stable/10 (r259064) to releng/10.0 as part of the 10.0-RELEASE cycle. - Update __FreeBSD_version [1] - Set branch name to -RC1
[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so start releng/10.0 at '100' so the branch is started with a value ending in zero.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
256281 |
|
10-Oct-2013 |
gjb |
Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation
|
#
233912 |
|
05-Apr-2012 |
davidxu |
umtx operation UMTX_OP_MUTEX_WAKE has a side-effect that it accesses a mutex after a thread has unlocked it, it event writes data to the mutex memory to clear contention bit, there is a race that other threads can lock it and unlock it, then destroy it, so it should not write data to the mutex memory if there isn't any waiter. The new operation UMTX_OP_MUTEX_WAKE2 try to fix the problem. It requires thread library to clear the lock word entirely, then call the WAKE2 operation to check if there is any waiter in kernel, and try to wake up a thread, if necessary, the contention bit is set again by the operation. This also mitgates the chance that other threads find the contention bit and try to enter kernel to compete with each other to wake up sleeping thread, this is unnecessary. With this change, the mutex owner is no longer holding the mutex until it reaches a point where kernel umtx queue is locked, it releases the mutex as soon as possible. Performance is improved when the mutex is contensted heavily. On Intel i3-2310M, the runtime of a benchmark program is reduced from 26.87 seconds to 2.39 seconds, it even is better than UMTX_OP_MUTEX_WAKE which is deprecated now. http://people.freebsd.org/~davidxu/bench/mutex_perf.c
|
#
232144 |
|
25-Feb-2012 |
davidxu |
In revision 231989, we pass a 16-bit clock ID into kernel, however according to POSIX document, the clock ID may be dynamically allocated, it unlikely will be in 64K forever. To make it future compatible, we pack all timeout information into a new structure called _umtx_time, and use fourth argument as a size indication, a zero means it is old code using timespec as timeout value, but the new structure also includes flags and a clock ID, so the size argument is different than before, and it is non-zero. With this change, it is possible that a thread can sleep on any supported clock, though current kernel code does not have such a POSIX clock driver system.
|
#
231989 |
|
22-Feb-2012 |
davidxu |
Use unused fourth argument of umtx_op to pass flags to kernel for operation UMTX_OP_WAIT. Upper 16bits is enough to hold a clock id, and lower 16bits is used to pass flags. The change saves a clock_gettime() syscall from libthr.
|
#
228220 |
|
03-Dec-2011 |
pho |
Add declaration of umtx_copyin_timeout()
In collaboration with: kib MFC after: 1 week
|
#
218970 |
|
23-Feb-2011 |
jhb |
Use umtx_key objects to uniquely identify futexes. Private futexes in different processes that happen to use the same user address in the separate processes will now be treated as distinct futexes rather than the same futex. We can now honor shared futexes properly by mapping them to a PROCESS_SHARED umtx_key. Private futexes use THREAD_SHARED umtx_key objects.
In conjunction with: dchagin Reviewed by: kib MFC after: 1 week
|
#
218969 |
|
23-Feb-2011 |
jhb |
Expose the umtx_key structure and API to the rest of the kernel.
MFC after: 3 days
|
#
216641 |
|
22-Dec-2010 |
davidxu |
MFp4:
- Add flags CVWAIT_ABSTIME and CVWAIT_CLOCKID for umtx kernel based condition variable, this should eliminate an extra system call to get current time.
- Add sub-function UMTX_OP_NWAKE_PRIVATE to wake up N channels in single system call. Create userland sleep queue for condition variable, in most cases, thread will wait in the queue, the pthread_cond_signal will defer thread wakeup until the mutex is unlocked, it tries to avoid an extra system call and a extra context switch in time window of pthread_cond_signal and pthread_mutex_unlock.
The changes are part of process-shared mutex project.
|
#
201472 |
|
04-Jan-2010 |
davidxu |
Add user-level semaphore synchronous type, this change allows multiple processes to share semaphore by using shared memory area, in simplest case, only one atomic operation is needed in userland, waiter flag is maintained by kernel and userland only checks the flag, if the flag is set, user code enters kernel and does a wakeup() call. Move type definitions into file _umtx.h to minimize compiling time. Also type names need to be prefixed with underline character, this would reduce name conflict (still in progress).
|
#
179970 |
|
24-Jun-2008 |
davidxu |
Add two commands to _umtx_op system call to allow a simple mutex to be locked and unlocked completely in userland. by locking and unlocking mutex in userland, it reduces the total time a mutex is locked by a thread, in some application code, a mutex only protects a small piece of code, the code's execution time is less than a simple system call, if a lock contention happens, however in current implemenation, the lock holder has to extend its locking time and enter kernel to unlock it, the change avoids this disadvantage, it first sets mutex to free state and then enters kernel and wake one waiter up. This improves performance dramatically in some sysbench mutex tests.
Tested by: kris Sounds great: jeff
|
#
178646 |
|
29-Apr-2008 |
davidxu |
Introduce command UMTX_OP_WAIT_UINT_PRIVATE and UMTX_OP_WAKE_PRIVATE to allow userland to specify that an address is not shared by multiple processes.
|
#
177848 |
|
02-Apr-2008 |
davidxu |
Introduce kernel based userland rwlock. Each umtx chain now has two lists, one for readers and one for writers, other types of synchronization object just use first list.
Asked by: jeff
|
#
173800 |
|
21-Nov-2007 |
davidxu |
Add function UMTX_OP_WAIT_UINT, the function causes thread to wait for an integer to be changed.
|
#
170368 |
|
06-Jun-2007 |
davidxu |
Backout experimental adaptive-spin umtx code.
|
#
165369 |
|
20-Dec-2006 |
davidxu |
Add a lwpid field into per-cpu structure, the lwpid represents current running thread's id on each cpu. This allow us to add in-kernel adaptive spin for user level mutex. While spinning in user space is possible, without correct thread running state exported from kernel, it hardly can be implemented efficiently without wasting cpu cycles, however exporting thread running state unlikely will be implemented soon as it has to design and stablize interfaces. This implementation is transparent to user space, it can be disabled dynamically. With this change, mutex ping-pong program's performance is improved massively on SMP machine. performance of mysql super-smack select benchmark is increased about 7% on Intel dual dual-core2 Xeon machine, it indicates on systems which have bunch of cpus and system-call overhead is low (athlon64, opteron, and core-2 are known to be fast), the adaptive spin does help performance.
Added sysctls: kern.threads.umtx_dflt_spins if the sysctl value is non-zero, a zero umutex.m_spincount will cause the sysctl value to be used a spin cycle count. kern.threads.umtx_max_spins the sysctl sets upper limit of spin cycle count.
Tested on: Athlon64 X2 3800+, Dual Xeon 5130
|
#
164876 |
|
04-Dec-2006 |
davidxu |
if a thread blocked on userland condition variable is pthread_cancel()ed, it is expected that the thread will not consume a pthread_cond_signal(), therefor, we use thr_wake() to mark a flag, the flag tells a thread calling do_cv_wait() in umtx code to not block on a condition variable. Thread library is expected that once a thread detected itself is in pthread_cond_wait, it will call the thr_wake() for itself in its SIGCANCEL handler.
|
#
164839 |
|
02-Dec-2006 |
davidxu |
Introduce userspace condition variable, since we have already POSIX priority mutex implemented, it is the time to introduce this stuff, now we can use umutex and ucond together to implement pthread's condition wait/signal.
|
#
164179 |
|
11-Nov-2006 |
davidxu |
define UMUTEX_CONTESTED as an unsigned integer.
|
#
163449 |
|
17-Oct-2006 |
davidxu |
o Add keyword volatile for user mutex owner field. o Fix type consistent problem by using type long for old umtx and wait channel. o Rename casuptr to casuword.
|
#
162536 |
|
21-Sep-2006 |
davidxu |
Add umtx support for 32bit process on AMD64 machine.
|
#
161678 |
|
28-Aug-2006 |
davidxu |
This is initial version of POSIX priority mutex support, a new userland mutex structure is added as following: struct umutex { __lwpid_t m_owner; uint32_t m_flags; uint32_t m_ceilings[2]; uint32_t m_spare[4]; }; The m_owner represents owner thread, it is a thread id, in non-contested case, userland can simply use atomic_cmpset_int to lock the mutex, if the mutex is contested, high order bit will be set, and userland should do locking and unlocking via kernel syscall. Flag UMUTEX_PRIO_INHERIT represents pthread's PTHREAD_PRIO_INHERIT mutex, which when contention happens, kernel should do priority propagating. Flag UMUTEX_PRIO_PROTECT indicates it is pthread's PTHREAD_PRIO_PROTECT mutex, userland should initialize m_owner to contested state UMUTEX_CONTESTED, then atomic_cmpset_int will be failure and kernel syscall should be invoked to do locking, this becauses for such a mutex, kernel should always boost the thread's priority before it can lock the mutex, m_ceilings is used by PTHREAD_PRIO_PROTECT mutex, the first element is used to boost thread's priority when it locked the mutex, second element is used when the mutex is unlocked, the PTHREAD_PRIO_PROTECT mutex's link list is kept in userland, the m_ceiling[1] is managed by thread library so kernel needn't allocate memory to keep the link list, when such a mutex is unlocked, kernel reset m_owner to UMUTEX_CONTESTED. Flag USYNC_PROCESS_SHARED indicate if the synchronization object is process shared, if the flag is not set, it saves a vm_map_lookup() call.
The umtx chain is still used as a sleep queue, when a thread is blocked on PTHREAD_PRIO_INHERIT mutex, a umtx_pi is allocated to support priority propagating, it is dynamically allocated and reference count is used, it is not optimized but works well in my tests, while the umtx chain has its own locking protocol, the priority propagating protocol are all protected by sched_lock because priority propagating function is called with sched_lock held from scheduler.
No visible performance degradation is found which these changes. Some parameter names in _umtx_op syscall are renamed.
|
#
161599 |
|
25-Aug-2006 |
davidxu |
Add user priority loaning code to support priority propagation for 1:1 threading's POSIX priority mutexes, the code is no-op unless priority-aware umtx code is committed.
|
#
157456 |
|
04-Apr-2006 |
davidxu |
WARNS level 4 cleanup, still has work to do.
|
#
157211 |
|
28-Mar-2006 |
des |
Revert previous commit at davidxu's insistance. Instead, use __DECONST (argh!) and rearrange the prototypes to make it clear that _umtx_op() is not deprecated.
|
#
157206 |
|
28-Mar-2006 |
des |
The undocumented and deprecated system call _umtx_op() takes two pointer arguments. The first one is never used (all callers pass in 0); the second is sometimes used to pass in a struct timespec * which is used as a timeout and never modified. Constify that argument so callers can pass a const struct timespec * without jumping through hoops.
|
#
151692 |
|
26-Oct-2005 |
davidxu |
do umtx_wake at userland thread exit address, so that others userland threads can wait for a thread to exit, and safely assume that the thread has left userland and is no longer using its userland stack, this is necessary for pthread_join when a thread is waiting for another thread to exit which has user customized stack, after pthread_join returns, the userland stack can be reused for other purposes, without this change, the joiner thread has to spin at the address to ensure the thread is really exited.
|
#
143149 |
|
05-Mar-2005 |
davidxu |
Allocate umtx_q from heap instead of stack, this avoids page fault panic in kernel under heavy swapping.
|
#
140858 |
|
26-Jan-2005 |
davidxu |
unbreak libthr binary compatibility.
Reported by: kris
|
#
140421 |
|
18-Jan-2005 |
davidxu |
Revert my previous errno hack, that is certainly an issue, and always has been, but the system call itself returns errno in a register so the problem is really a function of libc, not the system call.
Discussed with : Matthew Dillion <dillon@apollo.backplane.com>
|
#
140273 |
|
15-Jan-2005 |
scottl |
Add a cast to fix a warning.
|
#
140245 |
|
14-Jan-2005 |
davidxu |
make umtx timeout relative so userland can select different clock type, e.g, CLOCK_REALTIME or CLOCK_MONOTONIC. merge umtx_wait and umtx_timedwait into single function.
|
#
140102 |
|
12-Jan-2005 |
davidxu |
Let _umtx_op directly return error code rather than from errno because errno can be tampered potentially by nested signal handle. Now all error codes are returned in negative value, positive value are reserved for future expansion.
|
#
139825 |
|
07-Jan-2005 |
imp |
/* -> /*- for license, minor formatting changes
|
#
139427 |
|
30-Dec-2004 |
davidxu |
Make umtx_wait and umtx_wake more like linux futex does, it is more general than previous. It also lets me implement cancelable point in thread library. Also in theory, umtx_lock and umtx_unlock can be implemented by using umtx_wait and umtx_wake, all atomic operations can be done in userland without kernel's casuptr() function.
|
#
139292 |
|
25-Dec-2004 |
davidxu |
Make _umtx_op() as more general interface, the final parameter needn't be timespec pointer, every parameter will be interpreted by its opcode.
|
#
139291 |
|
25-Dec-2004 |
davidxu |
1. introduce umtx_owner to get an owner of a umtx. 2. add const qualifier to umtx_timedlock and umtx_timedwait. 3. add missing blackets in umtx do_unlock_and_wait.
|
#
139257 |
|
24-Dec-2004 |
davidxu |
1. Fix race condition between umtx lock and unlock, heavy testing on SMP can explore the bug. 2. Let umtx_wake returns number of threads have been woken.
|
#
139013 |
|
18-Dec-2004 |
davidxu |
1. make umtx sharable between processes, the way is two or more processes call mmap() to create a shared space, and then initialize umtx on it, after that, each thread in different processes can use the umtx same as threads in same process. 2. introduce a new syscall _umtx_op to support timed lock and condition variable semantics. also, orignal umtx_lock and umtx_unlock inline functions now are reimplemented by using _umtx_op, the _umtx_op can use arbitrary id not just a thread id.
|
#
131431 |
|
01-Jul-2004 |
marcel |
Change the thread ID (thr_id_t) used for 1:1 threading from being a pointer to the corresponding struct thread to the thread ID (lwpid_t) assigned to that thread. The primary reason for this change is that libthr now internally uses the same ID as the debugger and the kernel when referencing to a kernel thread. This allows us to implement the support for debugging without additional translations and/or mappings.
To preserve the ABI, the 1:1 threading syscalls, including the umtx locking API have not been changed to work on a lwpid_t. Instead the 1:1 threading syscalls operate on long and the umtx locking API has not been changed except for the contested bit. Previously this was the least significant bit. Now it's the most significant bit. Since the contested bit should not be tested by userland, this change is not expected to be visible. Just to be sure, UMTX_CONTESTED has been removed from <sys/umtx.h>.
Reviewed by: mtm@ ABI preservation tested on: i386, ia64
|
#
123740 |
|
23-Dec-2003 |
peter |
Catch a few places where NULL (pointer) was used where 0 (integer) was expected.
|
#
115765 |
|
03-Jun-2003 |
jeff |
- Remove the blocked pointer from the umtx structure. - Use a hash of umtx queues to queue blocked threads. We hash on pid and the virtual address of the umtx structure. This eliminates cases where we previously held a lock across a casuptr call.
Reviwed by: jhb (quickly)
|
#
112904 |
|
31-Mar-2003 |
jeff |
- Add an api for doing smp safe locks in userland. - umtx_lock() is defined as an inline in umtx.h. It tries to do an uncontested acquire of a lock which falls back to the _umtx_lock() system-call if that fails. - umtx_unlock() is also an inline which falls back to _umtx_unlock() if the uncontested unlock fails. - Locks are keyed off of the thr_id_t of the currently running thread which is currently just the pointer to the 'struct thread' in kernel. - _umtx_lock() uses the proc pointer to synchronize access to blocked thread queues which are stored in the first blocked thread.
|