History log of /freebsd-current/sys/powerpc/include/pcpu.h
Revision Date Author Comments
# 95ee2897 16-Aug-2023 Warner Losh <imp@FreeBSD.org>

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/


# 4d846d26 10-May-2023 Warner Losh <imp@FreeBSD.org>

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix


# d22883d7 09-Mar-2021 Jason A. Harmening <jah@FreeBSD.org>

Remove PCPU_INC

e4b8deb22227 removed the last in-tree uses of PCPU_INC(). Its
potential benefit is also practically nonexistent. Non-x86
platforms already implement it as PCPU_ADD(..., 1), and according
to [0] there are no recent x86 processors for which the 'inc'
instruction provides a performance benefit over the equivalent
memory-operand form of the 'add' instruction. The only remaining
benefit of 'inc' is smaller instruction size, which in this case
is inconsequential given the limited number of per-CPU data consumers.

[0]: https://www.agner.org/optimize/instruction_tables.pdf

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D29308


# 3f24b505 05-Jun-2020 Justin Hibbits <jhibbits@FreeBSD.org>

powerpc: Add a (CPU/runtime features) flags set to pcpu struct

Summary:
The point of this addition is to cache CPU behavior 'features', to avoid
having to recompute based on CPU, etc.

The first such use case is to avoid the unnecessary manipulation of the
SLBs (Segment Lookaside Buffers) when using the Radix pmap on POWER9.
Since we already get the PCPU pointer wherever we swap the SLB entries,
we can use a cached flag to check if it's necessary to perform the
operation anyway, and skip it when not.

Reviewed by: bdragon
Differential Revision: https://reviews.freebsd.org/D24908


# 9367fb30 29-Dec-2019 Brandon Bergren <bdragon@FreeBSD.org>

[PowerPC] Fix panic when attempting to handle an HMI from an idle thread

In IRC, sfs_ finally managed to get a good trace of a kernel panic that was
happening when attempting to use webengine.

As it turns out, we were using vtophys() from interrupt context on an idle
thread in opal_hmi_handler2().

Since this involves locking the kernel pmap on PPC64 at the moment, this
ended up tripping a KASSERT in mtx_lock(), which then caused a parallel
panic stampede.

So, avoid this by preallocating the flags variable and storing it in PCPU.

Fixes "panic: mtx_lock() by idle thread 0x... on sleep mutex kernelpmap".

Differential Revision: https://reviews.freebsd.org/D22962


# d3895bff 07-Nov-2019 Justin Hibbits <jhibbits@FreeBSD.org>

powerpc/booke: Make the TLB save area and mask match

Since TLB_MAXNEST is 3, the insert mask should only be 2 bits. Given that 2
bits counts to 4, and that we already have plenty of space wasted in
padding, make the nest level 4 to match the mask.


# 03e83a83 06-Sep-2018 Breno Leitao <leitao@FreeBSD.org>

powerpc64: Add initial support for HTM (kABI)

This patch adds the very initial support for HTM that might come at FreeBSD
version 12.1. This basic support defines a new kABI, so, we do not need to change
it later during 12.1 time frame, when the full implementation will come.

Reviewed by: jhibbits
Approved by: re(marius), jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D16889


# bce6d88b 17-Feb-2018 Justin Hibbits <jhibbits@FreeBSD.org>

Merge AIM and Book-E PCPU fields

This is part of a long-term goal of merging Book-E and AIM into a single GENERIC
kernel. As more work is done, the struct may be optimized further.

Reviewed by: nwhitehorn


# 3972f4c1 31-Dec-2017 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Remove PIR from PCPU data. It has an implementation-defined meaning that
is of limited utility outside of platform-specific code and can vary
at runtime when running as a hypervisor guest, so does not even have the
virtue of being a static identifier.

Reviewed by: jhibbits


# 71e3c308 27-Nov-2017 Pedro F. Giffuni <pfg@FreeBSD.org>

sys/powerpc: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.


# 83c9dea1 17-Apr-2017 Gleb Smirnoff <glebius@FreeBSD.org>

- Remove 'struct vmmeter' from 'struct pcpu', leaving only global vmmeter
in place. To do per-cpu stats, convert all fields that previously were
maintained in the vmmeters that sit in pcpus to counter(9).
- Since some vmmeter stats may be touched at very early stages of boot,
before we have set up UMA and we can do counter_u64_alloc(), provide an
early counter mechanism:
o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter.
o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter,
so that at early stages of boot, before counters are allocated we already
point to a counter that can be safely written to.
o For sparc64 that required a whole dummy pcpu[MAXCPU] array.

Further related changes:
- Don't include vmmeter.h into pcpu.h.
- vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit,
to match kernel representation.
- struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion.

This is based on benno@'s 4-year old patch:
https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html

Reviewed by: kib, gallatin, marius, lidl
Differential Revision: https://reviews.freebsd.org/D10156


# e683c328 17-Mar-2017 Justin Hibbits <jhibbits@FreeBSD.org>

Introduce 64-bit PowerPC Book-E support

Extend the Book-E pmap to support 64-bit operation. Much of this was taken from
Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory
list -- PP2D, page directory, page table), but has gaps in the page directory
list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap
between the two parts of the index). In practice this may not be a problem
given the expanded address space. However, an alternative to this would be to
use a 4-level page table, like Linux, and possibly reduce the available address
space; Linux appears to use a 46-bit address space. Alternatively, a cache of
page directory pointers could be used to keep the overall design as-is, but
remove the gaps in the address space.

This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with
the following notes:
* The DPAA driver has not yet been ported to 64-bit so is not included in the
kernel config.
* This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in
(total size kernel+mdroot must be under 64MB).
* This can run both 32-bit and 64-bit processes, and has even been tested to run
a 32-bit init with 64-bit children.

Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open
sourced to be used here, and to stevek for reviewing, and providing some
historical contexts on quirks of the code.

Reviewed by: stevek
Obtained from: Juniper (in part)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D9433


# e2a8d178 18-Feb-2017 Jason A. Harmening <jah@FreeBSD.org>

Bring back r313037, with fixes for mips:

Implement get_pcpu() for amd64/sparc64/mips/powerpc, and use it to
replace pcpu_find(curcpu) in MI code.

Reviewed by: andreast, kan, lidl
Tested by: lidl(mips, sparc64), andreast(powerpc)
Differential Revision: https://reviews.freebsd.org/D9587


# ad62ba6e 03-Feb-2017 Jason A. Harmening <jah@FreeBSD.org>

Revert r313037

The switch to get_pcpu() in MI code seems to cause hangs on MIPS.
Back out until we can get a better idea of what's happening there.

Reported by: kan, lidl


# 65ed4836 31-Jan-2017 Jason A. Harmening <jah@FreeBSD.org>

Implement get_pcpu() for the remaining architectures and use it to
replace pcpu_find(curcpu) in MI code.


# a1748875 24-Nov-2016 Justin Hibbits <jhibbits@FreeBSD.org>

Fix the build post-r309017 for MPC85XX/MPC85XXSPE

r309017 removed two fields from struct vmmeter, which is embedded in struct
pcpu. This caused the struct size to change, triggering the CTASSERT in
sys/pcpu.h. Add the extra 8 bytes back in as padding.


# 713841af 04-Aug-2015 Jason A. Harmening <jah@FreeBSD.org>

Add two new pmap functions:
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)

These will create and destroy a temporary, CPU-local KVA mapping of a specified page.

Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread

Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page

The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.

NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.

Reviewed by: kib
Approved by: kib (mentor)
Differential Revision: http://reviews.freebsd.org/D3013


# 4702d987 12-Dec-2013 Justin Hibbits <jhibbits@FreeBSD.org>

Add PMU-based CPU frequency scaling. This method is used on most Titanium
PowerBooks.

MFC after: 1 month


# bdac4360 11-Nov-2013 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Follow up r223485, which made AIM use the ABI thread pointer instead of
PCPU fields for curthread, by doing the same to Book-E. This closes
some potential races switching between CPUs. As a side effect, it turns out
the AIM and Book-E swtch.S implementations were the same to within a few
registers, so move that to powerpc/powerpc.

MFC after: 3 months


# 05bd8e50 03-Jun-2013 Justin Hibbits <jhibbits@FreeBSD.org>

Pad the PCPU MD struct, to satisfy an assert added with the projects/counters
branch import.

PR: ports/179173,ports/179164


# 85d592ad 10-Apr-2013 Gleb Smirnoff <glebius@FreeBSD.org>

Since UMA_ZONE_PCPU zones put a constraint on sizeof(struct pcpu), declared
as CTASSERT in MI pcpu.h, stop including all possible mutually exclusive
PCPU_MD_FIELDS fields into LINT kernels, due to brekaing
aforementioned CTASSERT.


# 17dece86 08-Apr-2013 Gleb Smirnoff <glebius@FreeBSD.org>

Merge from projects/counters:

Pad struct pcpu so that its size is denominator of PAGE_SIZE. This
is done to reduce memory waste in UMA_PCPU_ZONE zones.

Sponsored by: Nginx, Inc.


# 17f4cae4 27-May-2012 Rafal Jaworowski <raj@FreeBSD.org>

Let us manage differences of Book-E PowerPC variations i.e. vendor /
implementation specific vs. the common architecture definition.

Bring PPC4XX defines (PSL, SPR, TLB). Note the new definitions under
BOOKE_PPC4XX are not used in the code yet.

This change set is not supposed to affect existing E500 support, it's just
another reorg step before bringing support for E500mc, E5500 and PPC465.

Obtained from: AppliedMicro, Freescale, Semihalf


# ae09ab8f 14-Jan-2012 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Rework SLB trap handling so that double-faults into an SLB trap handler are
possible, and double faults within an SLB trap handler are not. The result
is that it possible to take an SLB fault at any time, on any address, for
any reason, at any point in the kernel.

This lets us do two important things. First, it removes the (soft) 16 GB RAM
ceiling on PPC64 as well as any architectural limitations on KVA space.
Second, it lets the kernel tolerate poorly designed hypervisors that
have a tendency to fail to restore the SLB properly after a hypervisor
context switch.

MFC after: 6 weeks


# a8972989 17-Nov-2011 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Use a global __pure2 function instead of a global register variable for
curthread, like on x86 and sparc64. This makes the kernel somewhat more
clang friendly, which doesn't support global register variables.


# e69dff49 23-Jun-2011 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Use the ABI-mandated thread pointer register (r2 for ppc32, r13 for ppc64)
instead of a PCPU field for curthread. This averts a race on SMP systems
with a high interrupt rate where the thread looking up the value of
curthread could be preempted and migrated between obtaining the PCPU
pointer and reading the value of pc_curthread, resulting in curthread being
observed to be the current thread on the thread's original CPU. This played
merry havoc with the system, in particular with mutexes. Many thanks to
jhb for helping me work this one out.

Note that Book-E is in principle susceptible to the same problem, but has
not been modified yet due to lack of Book-E hardware.

MFC after: 2 weeks


# a7d5f7eb 19-Oct-2010 Jamie Gritton <jamie@FreeBSD.org>

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.


# 6416b9a8 15-Sep-2010 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Split the SLB mirror cache into two kinds of object, one for kernel maps
which are similar to the previous ones, and one for user maps, which
are arrays of pointers into the SLB tree. This changes makes user SLB
updates atomic, closing a window for memory corruption. While here,
rearrange the allocation functions to make context switches faster.


# c3e289e1 12-Jul-2010 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

MFppc64:

Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.


# 0b5ac7b6 28-Nov-2009 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

MFC r198212,198378,198427,198428,198723,198724,198725,198731:
SMP support for PowerPC G5 systems.

r198724:
Fix a race in casuword() exposed by csup. casuword() non-atomically
read the current value of its argument before atomically replacing it,
which could occasionally return the wrong value on an SMP system. This
resulted in user mutex operations hanging when using threaded applications.

r198723,198725,198731:
Loop on blocked threads when using ULE scheduler, removing an
XXX MP comment.

r198427:
Add some more paranoia to setting HID registers, and update the AIM
clock routines to work better with SMP.

r198378:
Add SMP support on U3-based G5 systems. While here, correct the
64-bit tlbie function to set the CPU to 64-bit mode correctly.

r198212:
Don't assume that physical addresses are identity mapped. This
allows the second processor on G5 systems to start.


# 999987e5 22-Oct-2009 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Add SMP support on U3-based G5 systems. This does not yet work perfectly:
at least on my Xserve, getting the decrementer and timebase on APs to tick
requires setting up a clock chip over I2C, which is not yet done.

While here, correct the 64-bit tlbie function to set the CPU to 64-bit
mode correctly.

Hardware donated by: grehan


# 28bb01e5 21-May-2009 Rafal Jaworowski <raj@FreeBSD.org>

Initial support for SMP on PowerPC MPC85xx.

Tested with Freescale dual-core MPC8572DS development system.

Obtained from: Freescale, Semihalf


# 1ac37bcb 20-Feb-2009 Nathan Whitehorn <nwhitehorn@FreeBSD.org>

Add Altivec support for supported CPUs. This is derived from the FPU support
code, and also reducing the size of trapcode to fit inside a 32 byte handler
slot.

Reviewed by: grehan
MFC after: 2 weeks


# b2b734e7 13-Jan-2009 Rafal Jaworowski <raj@FreeBSD.org>

Rework BookE pmap towards multi-core support.

o Eliminate tlb0[] (a s/w copy of TLB0)
- The table contents cannot be maintained reliably in multiple MMU
environments, where asynchronous events (invalidations from other cores)
can change our local TLB0 contents underneath.
- Simplify and optimize TLB flushing: system wide invalidations are
performed using tlbivax instruction (propagates to other cores), for
local MMU invalidations a new optimized routine (assembly) is introduced.

o Improve and simplify TID allocation and management.
- Let each core keep track of its TID allocations.
- Simplify TID recycling, eliminate dead code.
- Drop the now unused powerpc/booke/support.S file.

o Improve page tables management logic.

o Simplify TLB1 manipulation routines.

o Other improvements and polishing.

Obtained from: Freescale, Semihalf


# d7f03759 19-Oct-2008 Ulf Lilleengen <lulf@FreeBSD.org>

- Import the HEAD csup code which is the basis for the cvsmode work.


# 70d12a18 19-Aug-2008 John Baldwin <jhb@FreeBSD.org>

Export 'struct pcpu' to userland w/o requiring _KERNEL. A few ports
already define _KERNEL to get to this and I'm about to add hooks to
libkvm to access per-CPU data.

MFC after: 1 week


# 12640815 27-Apr-2008 Marcel Moolenaar <marcel@FreeBSD.org>

MFp4: SMP support


# d5bbdb25 06-Mar-2008 Marcel Moolenaar <marcel@FreeBSD.org>

Add a catch-all for PCPU_MD_FIELDS. While we expect this to be
used in the kernel only (by virtue of checking for _KERNEL),
ports like lsof (part of gtop) cheat. It sets _KERNEL, but does
not set either AIM or E500. As such, PCPU_MD_FIELDS didn't get
defined and the build broke.
The catch-all is to define PCPU_MD_FIELDS with a dummy integer
when at the end of line we ended up without a definition for it.


# 786e4a1b 02-Mar-2008 Rafal Jaworowski <raj@FreeBSD.org>

Unify and generalize PowerPC headers, adjust AIM code accordingly.

Rework of this area is a pre-requirement for importing e500 support (and
other PowerPC core variations in the future). Mainly the following
headers are refactored so that we can cover for low-level differences between
various machines within PowerPC architecture:

<machine/pcpu.h>
<machine/pcb.h>
<machine/kdb.h>
<machine/hid.h>
<machine/frame.h>

Areas which use the above are adjusted and cleaned up.

Credits for this rework go to marcel@

Approved by: cognet (mentor)
MFp4: e500


# 67596082 04-Jun-2007 Attilio Rao <attilio@FreeBSD.org>

Rework the PCPU_* (MD) interface:
- Rename PCPU_LAZY_INC into PCPU_INC
- Add the PCPU_ADD interface which just does an add on the pcpu member
given a specific value.

Note that for most architectures PCPU_INC and PCPU_ADD are not safe.
This is a point that needs some discussions/work in the next days.

Reviewed by: alc, bde
Approved by: jeff (mentor)


# c640357f 10-Mar-2007 Alan Cox <alc@FreeBSD.org>

Push down the implementation of PCPU_LAZY_INC() into the machine-dependent
header file. Reimplement PCPU_LAZY_INC() on amd64 and i386 making it
atomic with respect to interrupts.

Reviewed by: bde, jhb


# 4539337d 12-Jul-2004 Peter Grehan <grehan@FreeBSD.org>

Gratuitous namechange to avoid low-level association with ddb.


# 10df017f 03-Feb-2004 Peter Grehan <grehan@FreeBSD.org>

Move temporary register save area from exception-vector memory to
per-CPU memory. This allows for interrupt handling on multiple CPUs.

Obtained from: NetBSD


# eeaa8979 13-May-2002 Benno Rice <benno@FreeBSD.org>

FPU support.

Obtained from: NetBSD (portions)


# b57e802a 28-Apr-2002 Benno Rice <benno@FreeBSD.org>

Commit of stuff that's been sitting in my tree for a while.

Highlights include:
- New low-level trap code from NetBSD. The high level code still needs a lot
of work.
- Fixes for some pmap handling in thread switching.
- The kernel will now get to attempting to jump into init in user mode. There
are some pmap/trap issues which prevent it from actually getting there though.

Obtained from: NetBSD (parts)


# 0bbc8826 11-Dec-2001 John Baldwin <jhb@FreeBSD.org>

Overhaul the per-CPU support a bit:

- The MI portions of struct globaldata have been consolidated into a MI
struct pcpu. The MD per-CPU data are specified via a macro defined in
machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the
interface would be cleaner (PCPU_GET(my_md_field) vs.
PCPU_GET(md.md_my_md_field)).
- All references to globaldata are changed to pcpu instead. In a UP kernel,
this data was stored as global variables which is where the original name
came from. In an SMP world this data is per-CPU and ideally private to each
CPU outside of the context of debuggers. This also included combining
machine/globaldata.h and machine/globals.h into machine/pcpu.h.
- The pointer to the thread using the FPU on i386 was renamed from
npxthread to fpcurthread to be identical with other architectures.
- Make the show pcpu ddb command MI with a MD callout to display MD
fields.
- The globaldata_register() function was renamed to pcpu_init() and now
init's MI fields of a struct pcpu in addition to registering it with
the internal array and list.
- A pcpu_destroy() function was added to remove a struct pcpu from the
internal array and list.

Tested on: alpha, i386
Reviewed by: peter, jake


# b9c1b06b 13-Nov-2001 John Baldwin <jhb@FreeBSD.org>

The interrupt nesting level is per-thread not per-CPU on FreeBSD.


# fd54558a 18-Sep-2001 John Baldwin <jhb@FreeBSD.org>

- If we ever do the per-cpu KTR stuff, the index won't be volatile as it
will be private to each CPU.
- Re-style(9) the globaldata structures. There really needs to be a MI
struct pcpu that has a MD struct mdpcpu member at some point.


# 63077d16 18-Sep-2001 John Baldwin <jhb@FreeBSD.org>

- Fix a missed idleproc -> idlethread conversion.
- Remove redundany fpucurproc (fpucurthread already existed)


# b40ce416 12-Sep-2001 Julian Elischer <julian@FreeBSD.org>

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha


# 589278db 16-Aug-2001 David E. O'Brien <obrien@FreeBSD.org>

style(9) and make consistent across platforms


# f5b0911c 04-Aug-2001 John Baldwin <jhb@FreeBSD.org>

Axe unused and invalid astpending globaldata member.


# d27f1d4c 16-Jun-2001 Benno Rice <benno@FreeBSD.org>

This commit (along with one pending in sys/dev/ofw and one in sys/conf) give
us our first minimal glimpse of PowerPC support.

With this code we can get to the "mountroot>" prompt on my Apple iMac. We
can't get any further due to lack of clock and interrupt handling, among other
things. This does however mean that pmap and VM are initialising.

We're fairly dependant on OpenFirmware at this point, but I hope to add
support for other classes of firmware at a later stage.

Reviewed by: obrien, dfr


# 6caa8a15 27-Apr-2001 John Baldwin <jhb@FreeBSD.org>

Overhaul of the SMP code. Several portions of the SMP kernel support have
been made machine independent and various other adjustments have been made
to support Alpha SMP.

- It splits the per-process portions of hardclock() and statclock() off
into hardclock_process() and statclock_process() respectively. hardclock()
and statclock() call the *_process() functions for the current process so
that UP systems will run as before. For SMP systems, it is simply necessary
to ensure that all other processors execute the *_process() functions when the
main clock functions are triggered on one CPU by an interrupt. For the alpha
4100, clock interrupts are delievered in a staggered broadcast fashion, so
we simply call hardclock/statclock on the boot CPU and call the *_process()
functions on the secondaries. For x86, we call statclock and hardclock as
usual and then call forward_hardclock/statclock in the MD code to send an IPI
to cause the AP's to execute forwared_hardclock/statclock which then call the
*_process() functions.
- forward_signal() and forward_roundrobin() have been reworked to be MI and to
involve less hackery. Now the cpu doing the forward sets any flags, etc. and
sends a very simple IPI_AST to the other cpu(s). AST IPIs now just basically
return so that they can execute ast() and don't bother with setting the
astpending or needresched flags themselves. This also removes the loop in
forward_signal() as sched_lock closes the race condition that the loop worked
around.
- need_resched(), resched_wanted() and clear_resched() have been changed to take
a process to act on rather than assuming curproc so that they can be used to
implement forward_roundrobin() as described above.
- Various other SMP variables have been moved to a MI subr_smp.c and a new
header sys/smp.h declares MI SMP variables and API's. The IPI API's from
machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h.
- The globaldata_register() and globaldata_find() functions as well as the
SLIST of globaldata structures has become MI and moved into subr_smp.c.
Also, the globaldata list is only available if SMP support is compiled in.

Reviewed by: jake, peter
Looked over by: eivind


# 19284646 28-Mar-2001 John Baldwin <jhb@FreeBSD.org>

Rework the witness code to work with sx locks as well as mutexes.
- Introduce lock classes and lock objects. Each lock class specifies a
name and set of flags (or properties) shared by all locks of a given
type. Currently there are three lock classes: spin mutexes, sleep
mutexes, and sx locks. A lock object specifies properties of an
additional lock along with a lock name and all of the extra stuff needed
to make witness work with a given lock. This abstract lock stuff is
defined in sys/lock.h. The lockmgr constants, types, and prototypes have
been moved to sys/lockmgr.h. For temporary backwards compatability,
sys/lock.h includes sys/lockmgr.h.
- Replace proc->p_spinlocks with a per-CPU list, PCPU(spinlocks), of spin
locks held. By making this per-cpu, we do not have to jump through
magic hoops to deal with sched_lock changing ownership during context
switches.
- Replace proc->p_heldmtx, formerly a list of held sleep mutexes, with
proc->p_sleeplocks, which is a list of held sleep locks including sleep
mutexes and sx locks.
- Add helper macros for logging lock events via the KTR_LOCK KTR logging
level so that the log messages are consistent.
- Add some new flags that can be passed to mtx_init():
- MTX_NOWITNESS - specifies that this lock should be ignored by witness.
This is used for the mutex that blocks a sx lock for example.
- MTX_QUIET - this is not new, but you can pass this to mtx_init() now
and no events will be logged for this lock, so that one doesn't have
to change all the individual mtx_lock/unlock() operations.
- All lock objects maintain an initialized flag. Use this flag to export
a mtx_initialized() macro that can be safely called from drivers. Also,
we on longer walk the all_mtx list if MUTEX_DEBUG is defined as witness
performs the corresponding checks using the initialized flag.
- The lock order reversal messages have been improved to output slightly
more accurate file and line numbers.


# 142ba5f3 09-Feb-2001 John Baldwin <jhb@FreeBSD.org>

- Make astpending and need_resched process attributes rather than CPU
attributes. This is needed for AST's to be properly posted in a preemptive
kernel. They are backed by two new flags in p_sflag: PS_ASTPENDING and
PS_NEEDRESCHED. They are still accesssed by their old macros:
aston(), astoff(), etc. For completeness, an astpending() macro has been
added to check for a pending AST, and clear_resched() has been added to
clear need_resched().
- Rename syscall2() on the x86 back to syscall() to be consistent with
other architectures.


# c6f6b729 24-Jan-2001 John Baldwin <jhb@FreeBSD.org>

- Rename the gd_cpuno member of struct globaldata to gd_cpuid.
- Add a globaldata_register() prototype in the SMP case.


# a448b62a 21-Jan-2001 Jake Burkholder <jake@FreeBSD.org>

Make intr_nesting_level per-process, rather than per-cpu. Setup
interrupt threads to run with it always >= 1, so that malloc can
detect M_WAITOK from "interrupt" context. This is also necessary
in order to context switch from sched_ithd() directly.

Reviewed By: peter


# 75869092 12-Jan-2001 Jake Burkholder <jake@FreeBSD.org>

Remove unused per-cpu variables inside_intr and ss_eflags.


# 25b53bb4 09-Dec-2000 Matt Jacob <mjacob@FreeBSD.org>

Store in globaldata our CPU ID#. Provide a lock for panics - only one
CPU can panic at a time.
Obtained from:Andrew Gallatin <gallatin@cs.duke.edu>


# 4b2c46fa 15-Nov-2000 John Baldwin <jhb@FreeBSD.org>

Add the 'witness_spin_check' per-CPU variable.


# 0384fff8 06-Sep-2000 Jason Evans <jasone@FreeBSD.org>

Major update to the way synchronization is done in the kernel. Highlights
include:

* Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The
alpha port is still in transition and currently uses both.)

* Per-CPU idle processes.

* Interrupts are run in their own separate kernel threads and can be
preempted (i386 only).

Partially contributed by: BSDi (BSD/OS)
Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh