#
267654 |
|
19-Jun-2014 |
gjb |
Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
255862 |
|
24-Sep-2013 |
jhb |
MFC 240424,244582: Improve check coverage about idle threads.
Idle threads are not allowed to acquire any lock but spinlocks. Deny any attempt to do so by panicing at the locking operation when INVARIANTS is on. Then, remove the check on blocking on a turnstile. The check in sleepqueues is left because they are not allowed to use tsleep() either which could happen still.
On entering KDB backends, the hijacked thread to run interrupt context can still be idlethread. At that point, without the panic condition, it can still happen that idlethread then will try to acquire some locks to carry on some operations.
Skip the idlethread check on block/sleep lock operations when KDB is active.
|
#
250581 |
|
12-May-2013 |
hiren |
MFC: r240475
Remove all the checks on curthread != NULL with the exception of some MD trap checks (eg. printtrap()).
Generally this check is not needed anymore, as there is not a legitimate case where curthread != NULL, after pcpu 0 area has been properly initialized.
Reviewed by: attilio Approved by: sbruno (mentor)
|
#
236238 |
|
29-May-2012 |
fabient |
MFC r233628, r234598, r235229, r235831, r226986.
Add software PMC support.
New kernel events can be added at various location for sampling or counting. This will for example allow easy system profiling whatever the processor is with known tools like pmcstat(8).
Simultaneous usage of software PMC and hardware PMC is possible, for example looking at the lock acquire failure, page fault while sampling on instructions.
Sponsored by: NETASQ
|
#
235404 |
|
13-May-2012 |
avg |
MFC r228424,228448: panic: add a switch and infrastructure for stopping other CPUs in SMP case
|
#
230167 |
|
15-Jan-2012 |
avg |
MFC r228430,228433: put sys/systm.h at its proper place or add it if missing
|
#
226255 |
|
11-Oct-2011 |
attilio |
Adaptive spinning for locking primitives, in read-mode, have some tuning SYSCTLs which are inappropriate for a daily use of the machine (mostly useful only by a developer which wants to run benchmarks on it). Remove them before the release as long as we do not want to ship with them in.
Now that the SYSCTLs are gone, instead than use static storage for some constants, use real numeric constants in order to avoid eventual compiler dumbiness and the risk to share a storage (and then a cache-line) among CPUs when doing adaptive spinning together.
Please note that this patch is not a MFC, but an 'edge case' as commit directly to STABLE_9.
Approved by: re (kib)
|
#
225736 |
|
22-Sep-2011 |
kensmith |
Copy head to stable/9 as part of 9.0-RELEASE release cycle.
Approved by: re (implicit)
|
#
219819 |
|
21-Mar-2011 |
jeff |
- Merge changes to the base system to support OFED. These include a wider arg2 for sysctl, updates to vlan code, IFT_INFINIBAND, and other miscellaneous small features.
|
#
217326 |
|
12-Jan-2011 |
mdf |
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.
Commit the kernel changes.
|
#
217265 |
|
11-Jan-2011 |
jhb |
Remove unneeded includes of <sys/linker_set.h>. Other headers that use it internally contain nested includes.
Reviewed by: bde
|
#
208912 |
|
08-Jun-2010 |
jhb |
Fix a sign bug that caused adaptive spinning in sx_xlock() to not work properly. Among other things it did not drop Giant while spinning leading to livelocks.
Reviewed by: rookie, kib, jmallett MFC after: 3 days
|
#
200447 |
|
12-Dec-2009 |
attilio |
In current code, threads performing an interruptible sleep (on both sxlock, via the sx_{s, x}lock_sig() interface, or plain lockmgr), will leave the waiters flag on forcing the owner to do a wakeup even when if the waiter queue is empty. That operation may lead to a deadlock in the case of doing a fake wakeup on the "preferred" (based on the wakeup algorithm) queue while the other queue has real waiters on it, because nobody is going to wakeup the 2nd queue waiters and they will sleep indefinitively.
A similar bug, is present, for lockmgr in the case the waiters are sleeping with LK_SLEEPFAIL on. In this case, even if the waiters queue is not empty, the waiters won't progress after being awake but they will just fail, still not taking care of the 2nd queue waiters (as instead the lock owned doing the wakeup would expect).
In order to fix this bug in a cheap way (without adding too much locking and complicating too much the semantic) add a sleepqueue interface which does report the actual number of waiters on a specified queue of a waitchannel (sleepq_sleepcnt()) and use it in order to determine if the exclusive waiters (or shared waiters) are actually present on the lockmgr (or sx) before to give them precedence in the wakeup algorithm. This fix alone, however doesn't solve the LK_SLEEPFAIL bug. In order to cope with it, add the tracking of how many exclusive LK_SLEEPFAIL waiters a lockmgr has and if all the waiters on the exclusive waiters queue are LK_SLEEPFAIL just wake both queues.
The sleepq_sleepcnt() introduction and ABI breakage require __FreeBSD_version bumping.
Reported by: avg, kib, pho Reviewed by: kib Tested by: pho
|
#
197643 |
|
30-Sep-2009 |
attilio |
When releasing a read/shared lock we need to use a write memory barrier in order to avoid, on architectures which doesn't have strong ordered writes, CPU instructions reordering.
Diagnosed by: fabio Reviewed by: jhb Tested by: Giovanni Trematerra <giovanni dot trematerra at gmail dot com>
|
#
196772 |
|
02-Sep-2009 |
attilio |
Fix some bugs related to adaptive spinning:
In the lockmgr support: - GIANT_RESTORE() is just called when the sleep finishes, so the current code can ends up into a giant unlock problem. Fix it by appropriately call GIANT_RESTORE() when needed. Note that this is not exactly ideal because for any interation of the adaptive spinning we drop and restore Giant, but the overhead should be not a factor. - In the lock held in exclusive mode case, after the adaptive spinning is brought to completition, we should just retry to acquire the lock instead to fallthrough. Fix that. - Fix a style nit
In the sx support: - Call GIANT_SAVE() before than looping. This saves some overhead because in the current code GIANT_SAVE() is called several times.
Tested by: Giovanni Trematerra <giovanni dot trematerra at gmail dot com>
|
#
196334 |
|
17-Aug-2009 |
attilio |
* Change the scope of the ASSERT_ATOMIC_LOAD() from a generic check to a pointer-fetching specific operation check. Consequently, rename the operation ASSERT_ATOMIC_LOAD_PTR(). * Fix the implementation of ASSERT_ATOMIC_LOAD_PTR() by checking directly alignment on the word boundry, for all the given specific architectures. That's a bit too strict for some common case, but it assures safety. * Add a comment explaining the scope of the macro * Add a new stub in the lockmgr specific implementation
Tested by: marcel (initial version), marius Reviewed by: rwatson, jhb (comment specific review) Approved by: re (kib)
|
#
196226 |
|
14-Aug-2009 |
bz |
Add a new macro to test that a variable could be loaded atomically. Check that the given variable is at most uintptr_t in size and that it is aligned.
Note: ASSERT_ATOMIC_LOAD() uses ALIGN() to check for adequate alignment -- however, the function of ALIGN() is to guarantee alignment, and therefore may lead to stronger alignment enforcement than necessary for types that are smaller than sizeof(uintptr_t).
Add checks to mtx, rw and sx locks init functions to detect possible breakage. This was used during debugging of the problem fixed with r196118 where a pointer was on an un-aligned address in the dpcpu area.
In collaboration with: rwatson Reviewed by: rwatson Approved by: re (kib)
|
#
193307 |
|
02-Jun-2009 |
attilio |
Handle lock recursion differenty by always checking against LO_RECURSABLE instead the lock own flag itself.
Tested by: pho
|
#
193025 |
|
29-May-2009 |
attilio |
The patch for r193011 was partially rejected when applied, complete it.
|
#
193011 |
|
28-May-2009 |
attilio |
Reverse the logic for ADAPTIVE_SX option and enable it by default. Introduce for this operation the reverse NO_ADAPTIVE_SX option. The flag SX_ADAPTIVESPIN to be passed to sx_init_flags(9) gets suppressed and the new flag, offering the reversed logic, SX_NOADAPTIVE is added.
Additively implements adaptive spininning for sx held in shared mode. The spinning limit can be handled through sysctls in order to be tuned while the code doesn't reach the release, after which time they should be dropped probabilly.
This change has made been necessary by recent benchmarks where it does improve concurrency of workloads in presence of high contention (ie. ZFS).
KPI breakage is documented by __FreeBSD_version bumping, manpage and UPDATING updates.
Requested by: jeff, kmacy Reviewed by: jeff Tested by: pho
|
#
192853 |
|
26-May-2009 |
sson |
Add the OpenSolaris dtrace lockstat provider. The lockstat provider adds probes for mutexes, reader/writer and shared/exclusive locks to gather contention statistics and other locking information for dtrace scripts, the lockstat(1M) command and other potential consumers.
Reviewed by: attilio jhb jb Approved by: gnn (mentor)
|
#
189846 |
|
15-Mar-2009 |
jeff |
- Wrap lock profiling state variables in #ifdef LOCK_PROFILING blocks.
|
#
182914 |
|
10-Sep-2008 |
jhb |
Teach WITNESS about the interlocks used with lockmgr. This removes a bunch of spurious witness warnings since lockmgr grew witness support. Before this, every time you passed an interlock to a lockmgr lock WITNESS treated it as a LOR.
Reviewed by: attilio
|
#
181334 |
|
05-Aug-2008 |
jhb |
If a thread that is swapped out is made runnable, then the setrunnable() routine wakes up proc0 so that proc0 can swap the thread back in. Historically, this has been done by waking up proc0 directly from setrunnable() itself via a wakeup(). When waking up a sleeping thread that was swapped out (the usual case when waking proc0 since only sleeping threads are eligible to be swapped out), this resulted in a bit of recursion (e.g. wakeup() -> setrunnable() -> wakeup()).
With sleep queues having separate locks in 6.x and later, this caused a spin lock LOR (sleepq lock -> sched_lock/thread lock -> sleepq lock). An attempt was made to fix this in 7.0 by making the proc0 wakeup use the ithread mechanism for doing the wakeup. However, this required grabbing proc0's thread lock to perform the wakeup. If proc0 was asleep elsewhere in the kernel (e.g. waiting for disk I/O), then this degenerated into the same LOR since the thread lock would be some other sleepq lock.
Fix this by deferring the wakeup of the swapper until after the sleepq lock held by the upper layer has been locked. The setrunnable() routine now returns a boolean value to indicate whether or not proc0 needs to be woken up. The end result is that consumers of the sleepq API such as *sleep/wakeup, condition variables, sx locks, and lockmgr, have to wakeup proc0 if they get a non-zero return value from sleepq_abort(), sleepq_broadcast(), or sleepq_signal().
Discussed with: jeff Glanced at by: sam Tested by: Jurgen Weber jurgen - ish com au MFC after: 2 weeks
|
#
179025 |
|
15-May-2008 |
attilio |
- Embed the recursion counter for any locking primitive directly in the lock_object, using an unified field called lo_data. - Replace lo_type usage with the w_name usage and at init time pass the lock "type" directly to witness_init() from the parent lock init function. Handle delayed initialization before than witness_initialize() is called through the witness_pendhelp structure. - Axe out LO_ENROLLPEND as it is not really needed. The case where the mutex init delayed wants to be destroyed can't happen because witness_destroy() checks for witness_cold and panic in case. - In enroll(), if we cannot allocate a new object from the freelist, notify that to userspace through a printf(). - Modify the depart function in order to return nothing as in the current CVS version it always returns true and adjust callers accordingly. - Fix the witness_addgraph() argument name prototype. - Remove unuseful code from itismychild().
This commit leads to a shrinked struct lock_object and so smaller locks, in particular on amd64 where 2 uintptr_t (16 bytes per-primitive) are gained.
Reviewed by: jhb
|
#
177085 |
|
12-Mar-2008 |
jeff |
- Pass the priority argument from *sleep() into sleepq and down into sched_sleep(). This removes extra thread_lock() acquisition and allows the scheduler to decide what to do with the static boost. - Change the priority arguments to cv_* to match sleepq/msleep/etc. where 0 means no priority change. Catch -1 in cv_broadcastpri() and convert it to 0 for now. - Set a flag when sleeping in a way that is compatible with swapping since direct priority comparisons are meaningless now. - Add a sysctl to ule, kern.sched.static_boost, that defaults to on which controls the boost behavior. Turning it off gives better performance in some workloads but needs more investigation. - While we're modifying sleepq, change signal and broadcast to both return with the lock held as the lock was held on enter.
Reviewed by: jhb, peter
|
#
174629 |
|
15-Dec-2007 |
jeff |
- Re-implement lock profiling in such a way that it no longer breaks the ABI when enabled. There is no longer an embedded lock_profile_object in each lock. Instead a list of lock_profile_objects is kept per-thread for each lock it may own. The cnt_hold statistic is now always 0 to facilitate this. - Support shared locking by tracking individual lock instances and statistics in the per-thread per-instance lock_profile_object. - Make the lock profiling hash table a per-cpu singly linked list with a per-cpu static lock_prof allocator. This removes the need for an array of spinlocks and reduces cache contention between cores. - Use a seperate hash for spinlocks and other locks so that only a critical_enter() is required and not a spinlock_enter() to modify the per-cpu tables. - Count time spent spinning in the lock statistics. - Remove the LOCK_PROFILE_SHARED option as it is always supported now. - Specifically drop and release the scheduler locks in both schedulers since we track owners now.
In collaboration with: Kip Macy Sponsored by: Nokia
|
#
173733 |
|
18-Nov-2007 |
attilio |
Expand lock class with the "virtual" function lc_assert which will offer an unified way for all the lock primitives to express lock assertions. Currenty, lockmgrs and rmlocks don't have assertions, so just panic in that case. This will be a base for more callout improvements.
Ok'ed by: jhb, jeff
|
#
173600 |
|
14-Nov-2007 |
julian |
generally we are interested in what thread did something as opposed to what process. Since threads by default have teh name of the process unless over-written with more useful information, just print the thread name instead.
|
#
172416 |
|
02-Oct-2007 |
pjd |
Fix sx_try_slock(), so it only fails when there is an exclusive owner. Before that fix, it was possible for the function to fail if number of sharers changes between 'x = sx->sx_lock' step and atomic_cmpset_acq_ptr() call.
This fixes ZFS problem when ZFS returns strange EIO errors under load. In ZFS there is a code that depends on the fact that sx_try_slock() can only fail if there is an exclusive owner.
Discussed with: attilio Reviewed by: jhb Approved by: re (kensmith)
|
#
171277 |
|
06-Jul-2007 |
attilio |
Fix some problems with lock_profiling in sx locks: - Adjust lock_profiling stubs semantic in the hard functions in order to be more accurate and trustable - Disable shared paths for lock_profiling. Actually, lock_profiling has a subtle race which makes results caming from shared paths not completely trustable. A macro stub (LOCK_PROFILING_SHARED) can be actually used for re-enabling this paths, but is currently intended for developing use only. - Use homogeneous names for automatic variables in hard functions regarding lock_profiling - Style fixes - Add a CTASSERT for some flags building
Discussed with: kmacy, kris Approved by: jeff (mentor) Approved by: re
|
#
170149 |
|
31-May-2007 |
attilio |
Add functions sx_xlock_sig() and sx_slock_sig(). These functions are intended to do the same actions of sx_xlock() and sx_slock() but with the difference to perform an interruptible sleep, so that sleep can be interrupted by external events. In order to support these new featueres, some code renstruction is needed, but external API won't be affected at all.
Note: use "void" cast for "int" returning functions in order to avoid tools like Coverity prevents to whine.
Requested by: rwatson Tested by: rwatson Reviewed by: jhb Approved by: jeff (mentor)
|
#
170115 |
|
29-May-2007 |
attilio |
style(9) fixes for sx locks.
Approved by: jeff (mentor)
|
#
170113 |
|
29-May-2007 |
attilio |
Add a small fix for lock profiling in sx locks. "0" cannot be a correct value since when the function is entered at least one shared holder must be present and since we want the last one "1" is the correct value. Note that lock_profiling for sx locks is far from being perfect. Expect further fixes for that.
Approved by: jeff (mentor)
|
#
169780 |
|
19-May-2007 |
jhb |
Rename the macros for assertion flags passed to sx_assert() from SX_* to SA_* to match mutexes and rwlocks. The old flags still exist for backwards compatiblity.
Requested by: attilio
|
#
169776 |
|
19-May-2007 |
jhb |
Expose sx_xholder() as a public macro. It returns a pointer to the thread that holds the current exclusive lock, or NULL if no thread holds an exclusive lock.
Requested by: pjd
|
#
169774 |
|
19-May-2007 |
jhb |
Oops, didn't include SX_ADAPTIVESPIN in the list of valid flags for the assert in sx_init_flags().
Submitted by: attilio
|
#
169769 |
|
19-May-2007 |
jhb |
Add a new SX_RECURSE flag to make support for recursive exclusive locks conditional. By default, sx(9) locks are back to not supporting recursive exclusive locks.
Submitted by: attilio
|
#
169676 |
|
18-May-2007 |
jhb |
Fix a comment.
|
#
169675 |
|
18-May-2007 |
jhb |
Move lock_profile_object_{init,destroy}() into lock_{init,destroy}().
|
#
169394 |
|
08-May-2007 |
jhb |
Add destroyed cookie values for sx locks and rwlocks as well as extra KASSERTs so that any lock operations on a destroyed lock will panic or hang.
|
#
168333 |
|
03-Apr-2007 |
kmacy |
fix typo
|
#
168332 |
|
03-Apr-2007 |
kmacy |
style fixes and make sure that the lock is treated as released in the sharers == 0 case not that this is somewhat racy because a new sharer can come in while we're updating stats
|
#
168330 |
|
03-Apr-2007 |
kmacy |
Fixes to sx for newsx - fix recursed case and move out of inline
Submitted by: Attilio Rao <attilio@freebsd.org>
|
#
168191 |
|
31-Mar-2007 |
jhb |
Optimize sx locks to use simple atomic operations for the common cases of obtaining and releasing shared and exclusive locks. The algorithms for manipulating the lock cookie are very similar to that rwlocks. This patch also adds support for exclusive locks using the same algorithm as mutexes.
A new sx_init_flags() function has been added so that optional flags can be specified to alter a given locks behavior. The flags include SX_DUPOK, SX_NOWITNESS, SX_NOPROFILE, and SX_QUITE which are all identical in nature to the similar flags for mutexes.
Adaptive spinning on select locks may be enabled by enabling the ADAPTIVE_SX kernel option. Only locks initialized with the SX_ADAPTIVESPIN flag via sx_init_flags() will adaptively spin.
The common cases for sx_slock(), sx_sunlock(), sx_xlock(), and sx_xunlock() are now performed inline in non-debug kernels. As a result, <sys/sx.h> now requires <sys/lock.h> to be included prior to <sys/sx.h>.
The new kernel option SX_NOINLINE can be used to disable the aforementioned inlining in non-debug kernels.
The size of struct sx has changed, so the kernel ABI is probably greatly disturbed.
MFC after: 1 month Submitted by: attilio Tested by: kris, pjd
|
#
167787 |
|
21-Mar-2007 |
jhb |
Rename the 'mtx_object', 'rw_object', and 'sx_object' members of mutexes, rwlocks, and sx locks to 'lock_object'.
|
#
167368 |
|
09-Mar-2007 |
jhb |
Add two new function pointers 'lc_lock' and 'lc_unlock' to lock classes. These functions are intended to be used to drop a lock and then reacquire it when doing an sleep such as msleep(9). Both functions accept a 'struct lock_object *' as their first parameter. The 'lc_unlock' function returns an integer that is then passed as the second paramter to the subsequent 'lc_lock' function. This can be used to communicate state. For example, sx locks and rwlocks use this to indicate if the lock was share/read locked vs exclusive/write locked.
Currently, spin mutexes and lockmgr locks do not provide working lc_lock and lc_unlock functions.
|
#
167365 |
|
09-Mar-2007 |
jhb |
Use C99-style struct member initialization for lock classes.
|
#
167163 |
|
02-Mar-2007 |
kmacy |
lock stats updates need to be protected by the lock
|
#
167136 |
|
01-Mar-2007 |
kmacy |
Evidently I've overestimated gcc's ability to peak inside inline functions and optimize away unused stack values. The 48 bytes that the lock_profile_object adds to the stack evidently has a measurable performance impact on certain workloads.
|
#
167054 |
|
27-Feb-2007 |
kmacy |
Further improvements to LOCK_PROFILING: - Fix missing initialization in kern_rwlock.c causing bogus times to be collected - Move updates to the lock hash to after the lock is released for spin mutexes, sleep mutexes, and sx locks - Add new kernel build option LOCK_PROFILE_FAST - only update lock profiling statistics when an acquisition is contended. This reduces the overhead of LOCK_PROFILING to increasing system time by 20%-25% which on "make -j8 kernel-toolchain" on a dual woodcrest is unmeasurable in terms of wall-clock time. Contrast this to enabling lock profiling without LOCK_PROFILE_FAST and I see a 5x-6x slowdown in wall-clock time.
|
#
167012 |
|
26-Feb-2007 |
kmacy |
general LOCK_PROFILING cleanup
- only collect timestamps when a lock is contested - this reduces the overhead of collecting profiles from 20x to 5x
- remove unused function from subr_lock.c
- generalize cnt_hold and cnt_lock statistics to be kept for all locks
- NOTE: rwlock profiling generates invalid statistics (and most likely always has) someone familiar with that should review
|
#
164246 |
|
13-Nov-2006 |
kmacy |
track lock class name in a way that doesn't break WITNESS
|
#
164159 |
|
11-Nov-2006 |
kmacy |
MUTEX_PROFILING has been generalized to LOCK_PROFILING. We now profile wait (time waited to acquire) and hold times for *all* kernel locks. If the architecture has a system synchronized TSC, the profiling code will use that - thereby minimizing profiling overhead. Large chunks of profiling code have been moved out of line, the overhead measured on the T1 for when it is compiled in but not enabled is < 1%.
Approved by: scottl (standing in for mentor rwatson) Reviewed by: des and jhb
|
#
161337 |
|
15-Aug-2006 |
jhb |
Add a new 'show sleepchain' ddb command similar to 'show lockchain' except that it operates on lockmgr and sx locks. This can be useful for tracking down vnode deadlocks in VFS for example. Note that this command is a bit more fragile than 'show lockchain' as we have to poke around at the wait channel of a thread to see if it points to either a struct lock or a condition variable inside of a struct sx. If td_wchan points to something unmapped, then this command will terminate early due to a fault, but no harm will be done.
|
#
160771 |
|
27-Jul-2006 |
jhb |
Adjust td_locks for non-spin mutexes, rwlocks, and sx locks so that it is a count of all non-spin locks, not just lockmgr locks. This can give us a much cheaper way to see if we have any locks held (such as when returning to userland via userret()) without requiring WITNESS.
MFC after: 1 week
|
#
154484 |
|
17-Jan-2006 |
jhb |
Add a new file (kern/subr_lock.c) for holding code related to struct lock_obj objects: - Add new lock_init() and lock_destroy() functions to setup and teardown lock_object objects including KTR logging and registering with WITNESS. - Move all the handling of LO_INITIALIZED out of witness and the various lock init functions into lock_init() and lock_destroy(). - Remove the constants for static indices into the lock_classes[] array and change the code outside of subr_lock.c to use LOCK_CLASS to compare against a known lock class. - Move the 'show lock' ddb function and lock_classes[] array out of kern_mutex.c over to subr_lock.c.
|
#
154077 |
|
06-Jan-2006 |
jhb |
Trim another pointer from struct lock_object (and thus from struct mtx and struct sx). Instead of storing a direct pointer to a our lock_class struct in lock_object, reserve 4 bits in the lo_flags field to serve as an index into a global lock_classes array that contains pointers to the lock classes. Only debugging code such as WITNESS or INVARIANTS checks and KTR logging need to access the lock_class member, so this shouldn't add any overhead to production kernels. It might add some slight overhead to kernels using those debug options however.
As with the previous set of changes to lock_object, this is going to completely obliterate the kernel ABI, so be sure to recompile all your modules.
|
#
153395 |
|
13-Dec-2005 |
jhb |
Add a new 'show lock' command to ddb. If the argument has a valid lock class, then it displays various information about the lock and calls a new function pointer in lock_class (lc_ddb_show) to dump class-specific information about the lock as well (such as the owner of a mutex or xlock'ed sx lock). This is easier than staring at hex dumps of locks to figure out who owns the lock, etc. Note that extending lock_class doesn't affect the ABI for any kernel modules as the only code that deals with lock_class structures directly is kern_mutex.c, kern_sx.c, and witness.
MFC after: 1 week
|
#
139804 |
|
06-Jan-2005 |
imp |
/* -> /*- for copyright notices, minor format tweaks as necessary
|
#
126316 |
|
27-Feb-2004 |
jhb |
Fix _sx_assert() to panic() rather than printf() when an assertion fails and ignore assertions if we have already paniced.
|
#
126003 |
|
19-Feb-2004 |
pjd |
Simplify check. We are only able to check exclusive lock and if 2nd condition is true, first one is true for sure.
Approved by: jhb, scottl (mentor)
|
#
125421 |
|
04-Feb-2004 |
pjd |
Allow assert that the current thread does not hold the sx(9) lock.
Reviewed by: jhb In cooperation with: juli, jhb Approved by: jhb, scottl (mentor)
|
#
125160 |
|
28-Jan-2004 |
jhb |
Rework witness_lock() to make it slightly more useful and flexible. - witness_lock() is split into two pieces: witness_checkorder() and witness_lock(). Witness_checkorder() determines if acquiring a specified lock at the time it is called would result in a lock order. It optionally adds a new lock order relationship as well. witness_lock() updates witness's data structures to assume that a lock has been acquired by stick a new lock instance in the appropriate lock instance list. - The mutex and sx lock functions now call checkorder() prior to trying to acquire a lock and continue to call witness_lock() after the acquire is completed. This will let witness catch a deadlock before it happens rather than trying to do so after the threads have deadlocked (i.e. never actually report it). - A new function witness_defineorder() has been added that adds a lock order between two locks at runtime without having to acquire the locks. If the lock order cannot be added it will return an error. This function is available to programmers via the WITNESS_DEFINEORDER() macro which accepts either two mutexes or two sx locks as its arguments. - A few simple wrapper macros were added to allow developers to call witness_checkorder() anywhere as a way of enforcing locking assertions in code that might acquire a certain lock in some situations. The macros are: witness_check_{mutex,shared_sx,exclusive_sx} and take an appropriate lock as the sole argument. - The code to remove a lock instance from a lock list in witness_unlock() was unnested by using a goto to vastly improve the readability of this function.
|
#
117494 |
|
12-Jul-2003 |
truckman |
Extend the mutex pool implementation to permit the creation and use of multiple mutex pools with different options and sizes. Mutex pools can be created with either the default sleep mutexes or with spin mutexes. A dynamically created mutex pool can now be destroyed if it is no longer needed.
Create two pools by default, one that matches the existing pool that uses the MTX_NOWITNESS option that should be used for building higher level locks, and a new pool with witness checking enabled.
Modify the users of the existing mutex pool to use the appropriate pool in the new implementation.
Reviewed by: jhb
|
#
116182 |
|
10-Jun-2003 |
obrien |
Use __FBSDID().
|
#
93812 |
|
04-Apr-2002 |
jhb |
Set the lock type equal to the lock name for now as all of the current sx locks don't use very specific lock names.
|
#
93672 |
|
02-Apr-2002 |
arr |
- Add MTX_SYSINIT and SX_SYSINIT as macro glue for allowing sx and mtx locks to be able to setup a SYSINIT call. This helps in places where a lock is needed to protect some data, but the data is not truly associated with a subsystem that can properly initialize it's lock. The macros use the mtx_sysinit() and sx_sysinit() functions, respectively, as the handler argument to SYSINIT().
Reviewed by: alfred, jhb, smp@
|
#
89496 |
|
18-Jan-2002 |
tanimura |
Invert the test of sx_xholder for SX_LOCKED. We need to warn if a thread other than the curthread holds an sx.
While I am here, break a line at the end of warning.
|
#
87594 |
|
10-Dec-2001 |
obrien |
Update to C99, s/__FUNCTION__/__func__/.
|
#
86333 |
|
13-Nov-2001 |
dillon |
Create a mutex pool API for short term leaf mutexes. Replace the manual mutex pool in kern_lock.c (lockmgr locks) with the new API. Replace the mutexes embedded in sxlocks with the new API.
|
#
85412 |
|
24-Oct-2001 |
jhb |
Fix this to actually compile in the !INVARIANTS case.
Reported by: Maxime Henrion <mux@qualys.com>
|
#
85388 |
|
23-Oct-2001 |
jhb |
Change the sx(9) assertion API to use a sx_assert() function similar to mtx_assert(9) rather than several SX_ASSERT_* macros.
|
#
85205 |
|
19-Oct-2001 |
jhb |
The mtx_init() and sx_init() functions bzero'd locks before handing them off to witness_init() making the check for double intializating a lock by testing the LO_INITIALIZED flag moot. Workaround this by checking the LO_INITIALIZED flag ourself before we bzero the lock structure.
|
#
83366 |
|
12-Sep-2001 |
julian |
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process.
Sorry john! (your next MFC will be a doosie!)
Reviewed by: peter@freebsd.org, dillon@freebsd.org
X-MFC after: ha ha ha ha
|
#
82246 |
|
23-Aug-2001 |
jhb |
Use witness_upgrade/downgrade for sx_try_upgrade/downgrade.
|
#
82212 |
|
23-Aug-2001 |
jhb |
Clear the sx_xholder pointer when downgrading an exclusive lock.
|
#
81599 |
|
13-Aug-2001 |
jasone |
Add sx_try_upgrade() and sx_downgrade().
Submitted by: Alexander Kabaev <ak03@gte.com>
|
#
78872 |
|
27-Jun-2001 |
jhb |
- Add trylock variants of shared and exclusive locks. - The sx assertions don't actually need the internal sx mutex lock, so don't bother doing so. - Add a new assertion SX_ASSERT_LOCKED() that asserts that either a shared or exclusive lock should be held. This assertion should be used instead of SX_ASSERT_SLOCKED() in almost all cases. - Adjust some KASSERT()'s to include file and line information. - Use the new witness_assert() function in the WITNESS case for sx slock asserts to verify that the current thread actually owns a slock.
|
#
76272 |
|
04-May-2001 |
jhb |
- Move state about lock objects out of struct lock_object and into a new struct lock_instance that is stored in the per-process and per-CPU lock lists. Previously, the lock lists just kept a pointer to each lock held. That pointer is now replaced by a lock instance which contains a pointer to the lock object, the file and line of the last acquisition of a lock, and various flags about a lock including its recursion count. - If we sleep while holding a sleepable lock, then mark that lock instance as having slept and ignore any lock order violations that occur while acquiring Giant when we wake up with slept locks. This is ok because of Giant's special nature. - Allow witness to differentiate between shared and exclusive locks and unlocks of a lock. Witness will now detect the case when a lock is acquired first in one mode and then in another. Mutexes are always locked and unlocked exclusively. Witness will also now detect the case where a process attempts to unlock a shared lock while holding an exclusive lock and vice versa. - Fix a bug in the lock list implementation where we used the wrong constant to detect the case where a lock list entry was full.
|
#
74912 |
|
28-Mar-2001 |
jhb |
Rework the witness code to work with sx locks as well as mutexes. - Introduce lock classes and lock objects. Each lock class specifies a name and set of flags (or properties) shared by all locks of a given type. Currently there are three lock classes: spin mutexes, sleep mutexes, and sx locks. A lock object specifies properties of an additional lock along with a lock name and all of the extra stuff needed to make witness work with a given lock. This abstract lock stuff is defined in sys/lock.h. The lockmgr constants, types, and prototypes have been moved to sys/lockmgr.h. For temporary backwards compatability, sys/lock.h includes sys/lockmgr.h. - Replace proc->p_spinlocks with a per-CPU list, PCPU(spinlocks), of spin locks held. By making this per-cpu, we do not have to jump through magic hoops to deal with sched_lock changing ownership during context switches. - Replace proc->p_heldmtx, formerly a list of held sleep mutexes, with proc->p_sleeplocks, which is a list of held sleep locks including sleep mutexes and sx locks. - Add helper macros for logging lock events via the KTR_LOCK KTR logging level so that the log messages are consistent. - Add some new flags that can be passed to mtx_init(): - MTX_NOWITNESS - specifies that this lock should be ignored by witness. This is used for the mutex that blocks a sx lock for example. - MTX_QUIET - this is not new, but you can pass this to mtx_init() now and no events will be logged for this lock, so that one doesn't have to change all the individual mtx_lock/unlock() operations. - All lock objects maintain an initialized flag. Use this flag to export a mtx_initialized() macro that can be safely called from drivers. Also, we on longer walk the all_mtx list if MUTEX_DEBUG is defined as witness performs the corresponding checks using the initialized flag. - The lock order reversal messages have been improved to output slightly more accurate file and line numbers.
|
#
73901 |
|
06-Mar-2001 |
jhb |
In order to avoid recursing on the backing mutex for sx locks in the INVARIANTS case, define the actual KASSERT() in _SX_ASSERT_[SX]LOCKED macros that are used in the sx code itself and convert the SX_ASSERT_[SX]LOCKED macros to simple wrappers that grab the mutex for the duration of the check.
|
#
73863 |
|
06-Mar-2001 |
bmilekic |
- Add sx_descr description member to sx lock structure - Add sx_xholder member to sx struct which is used for INVARIANTS-enabled assertions. It indicates the thread that presently owns the xlock. - Add some assertions to the sx lock code that will detect the fatal API abuse: xlock --> xlock xlock --> slock which now works thanks to sx_xholder. Notice that the remaining two problematic cases: slock --> xlock slock --> slock (a little less problematic, but still recursion) will need to be handled by witness eventually, as they are more involved.
Reviewed by: jhb, jake, jasone
|
#
73782 |
|
05-Mar-2001 |
jasone |
Implement shared/exclusive locks.
Reviewed by: bmilekic, jake, jhb
|