| 368823 |
30-Dec-2020 |
bdragon |
MFC r364447:
[PowerPC] Fix translation-related crashes during startup
After spending a lot of time trying to track down what was going on, I have
isolated the "black screen" failures when using boot1 to boot a G4 machine.
It turns out we were replacing the traps before installing the temporary
BAT entry for the bottom of physical memory. That meant that until the MMU
was bootstrapped, the cached translations were the only thing keeping us
from losing.
Throwing boot1 into the mix was affecting execution flow enough to cause us
to hit an uncached page and crash.
Fix this by properly setting up the initial BAT entry at the same time we
are replacing the OpenFirmware traps, so we can continue executing in
segment 0 until the rest of the DMAP has been set up.
A second thing discovered while researching this is that we were entering a
BAT region for segment 16. It turns out this range was a) considered part
of KVA, and b) has firmware mappings with varying attributes.
If we ever accessed an unmapped page in segment 16, it would cause a BAT
entry to be installed for the whole segment, which would bypass the
existing mappings until it was flushed out again.
Instead, translate the OFW memory attributes into VM memory attributes and
install the ranges into the kernel address space properly.
Reviewed by: adalava
MFC after: 3 weeks
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D25547
(cherry picked from commit f10baa404986adb5733fd5745d467a8f69409ce3)
Git Hash: a6922d706d5295177593b0cb4c1c2364b28ba1a8
Git Author: bdragon@FreeBSD.org |
| 331722 |
29-Mar-2018 |
eadler |
Revert r330897:
This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.
Revert with prejudice.
This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.
Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.
Requested by: gjb (re) |
| 330897 |
14-Mar-2018 |
eadler |
Partial merge of the SPDX changes
These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.
No objections from: pfg |
| 327785 |
10-Jan-2018 |
markj |
MFC r325530 (jeff), r325566 (kib), r325588 (kib):
Replace many instances of VM_WAIT with blocking page allocation flags. |
| 323968 |
24-Sep-2017 |
markj |
MFC r323676:
Implement mmu_page_init for AIM platforms. |
| 316369 |
01-Apr-2017 |
jhibbits |
MFC r310146,r311912,r312369,r312617,r312614,r312659,r312974,r312977,r313005,r314826:
A series of Clang-related powerpc commits
r310146:
Use the right bitwise OR operation for clearing single-step at trap time.
r311912:
Force all TOC references in asm to include '@toc'
r312369:
Use the explicit expanded form of cmp.
r312617:
Hide the 'MOREARGS' macro, it conflicts with contrib code, and is only used
in one file.
r312614:
Don't pass -Wa,-many through clang, the integrated as doesn't support it.
r312659:
Avoid using non-zero argument for __builtin_frame_address().
r312974:
Add a INTR_TRIG_INVALID, and use it in the powerpc interrupt code.
r312977:
Force the setting of bit 7 in the sysmouse packet byte 1 to be unsigned.
r313005:
Update CFLAGS for clang compatibility
r314826:
Clang in base now supports -mlongcall, so remove this hack |
| 302408 |
08-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.
Additional commits post-branch will follow.
Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation |
| 302279 |
29-Jun-2016 |
nwhitehorn |
Do not rely on firmware having pre-enabled the MMU in a reasonable way for
late boot: enable it explicitly after installing the page tables. If booting
from an FDT, also make sure to escape the firmware's MMU context early
before overwriting firmware page tables.
Approved by: re (gjb)
|
| 302214 |
26-Jun-2016 |
nwhitehorn |
Enter 64-bit mode as early as possible in the 64-bit PowerPC boot sequence.
Most of the effect of setting MSR[SF] is that the CPU will stop ignoring
the high 32 bits of registers containing addresses in load/store
instructions. As such, the kernel was setting it only when it began to
need access to high memory. MSR[SF] also affects the operation of some
conditional instructions, however, and so setting it at late times could
subtly break code at very early times. This fixes use of the FDT mode in
loader, and FDT boot more generally, on 64-bit PowerPC systems.
Hardware provided by: IBM LTC
Approved by: re (kib)
|
| 298433 |
21-Apr-2016 |
pfg |
sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.
This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
|
| 296245 |
01-Mar-2016 |
jhibbits |
Add VM_MEMATTR_CACHEABLE support for AIM, for parity with Book-E.
Not used right now, but may be in the future anyway.
|
| 295880 |
22-Feb-2016 |
skra |
As <machine/pmap.h> is included from <vm/pmap.h>, there is no need to
include it explicitly when <vm/pmap.h> is already included.
Reviewed by: alc, kib
Differential Revision: https://reviews.freebsd.org/D5373
|
| 293641 |
10-Jan-2016 |
nwhitehorn |
Remove dead code and dead comments, most notably the implemenation of the
now-obsolete setfault(). No NetBSD code exists in the AIM locore files, so
update the copyrights there.
|
| 293062 |
02-Jan-2016 |
andreast |
Fix booting of 32-bit kernels on 64-bit G5 hardware.
For rs6000, most memory insns and addi/addis do not allow GPR0 for RA
(they use literal zero there instead). So use a 'b' constraint to make
sure to have a base register other than GPR0.
GCC-4.7 and up handles this with allocating r9 instead of r0.
|
| 293051 |
02-Jan-2016 |
nwhitehorn |
Switch setting MSR[SF] to C code. This removes any CPU-specific code
(MSF[SF] is a Book 3-S thing) in the 64-bit locore64.S.
|
| 291442 |
29-Nov-2015 |
nwhitehorn |
Make ELFv2 powerpc64 kernels build and run. Loader support will come in a
separate commit.
|
| 291262 |
24-Nov-2015 |
nwhitehorn |
Use what we really mean (powerpc_lwsync()) rather than the Linux-compat
mb() here and provide some more documentation on what, exactly, makes this
code safe.
Requested by and discussed with: kib, alc
|
| 290990 |
17-Nov-2015 |
nwhitehorn |
Make native page table access endian-safe. Even on CPUs running in
little-endian mode, the hardware page table is big-endian. This is a
no-op on all currently supported systems.
MFC after: 1 month
|
| 290989 |
17-Nov-2015 |
nwhitehorn |
Where appropriate, use the endian-flipping OF_getencprop() instead of
OF_getprop() to get encode-int encoded values from the OF tree. This is
a no-op at present, since all existing PowerPC ports are big-endian, but
it is a correctness improvement and will be required if we have a
little-endian kernel at some future point.
Where it is totally impossible for the code ever to be used on a
little-endian system (much of powerpc/powermac, for instance), I have not
necessarily made the appropriate changes.
MFC after: 1 month
|
| 286584 |
10-Aug-2015 |
kib |
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the
initial thread stack is not adjusted by the tunable, the stack is
allocated too early to get access to the kernel environment. See
TD0_KSTACK_PAGES for the thread0 stack sizing on i386.
The tunable was tested on x86 only. From the visual inspection, it
seems that it might work on arm and powerpc. The arm
USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already
incorrect for the threads with non-default kstack size. I only
changed the macros to use variable instead of constant, since I cannot
test.
On arm64, mips and sparc64, some static data structures are sized by
KSTACK_PAGES, so the tunable is disabled.
Sponsored by: The FreeBSD Foundation
MFC after: 2 week
|
| 286296 |
04-Aug-2015 |
jah |
Add two new pmap functions:
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)
These will create and destroy a temporary, CPU-local KVA mapping of a specified page.
Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread
Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page
The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.
NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.
Reviewed by: kib
Approved by: kib (mentor)
Differential Revision: http://reviews.freebsd.org/D3013
|
| 285620 |
16-Jul-2015 |
jhibbits |
Fix userland program exception handling for powerpc64.
It appears that the linker will not handle 64-bit relocations at addresses that
are not aligned to 8-byte boundaries. Prior to this change the line:
.llong generictrap
was aligned to a 4-byte address, and the linker replaced that with an 8-byte
0x0. Aligning that address to 8 bytes caused the linker to generate the proper
relocation. As a follow-through, the dblow from trap_subr33.S used the code
sequence 'lwz %r1, TRAP_GENTRAP(0)', so this reproduces the analogue of that for
64-bit.
|
| 285195 |
06-Jul-2015 |
jhibbits |
Merge booke and aim interrupt.c files.
Summary:
Both booke and AIM interrupt.c files contain nearly identical code. This merges
the two files, to reduce duplication.
Reviewers: #powerpc, marcel
Reviewed By: marcel
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D2991
|
| 285148 |
04-Jul-2015 |
jhibbits |
Use the correct type for physical addresses.
On Book-E, physical addresses are actually 36-bits, not 32-bits. This is
currently worked around by ignoring the top bits. However, in some cases, the
boot loader configures CCSR to something above the 32-bit mark. This is stage 1
in updating the pmap to handle 36-bit physaddr.
|
| 282264 |
30-Apr-2015 |
jhibbits |
Unify booke and AIM machdep.
Much of the code was common to begin with. There is one nit, which is likely
not an issue at all. With the old code, the AIM machdep would __syncicache()
the entire kernel core at setup. However, in the unified setup, that seems to
hang on the MPC7455, perhaps because it's running later than before. Removing
this allows it to boot just fine. Examining the code, the FreeBSD loader
already does syncicache of the full kernel, and each module loaded, so this
doesn't appear to be an actual problem.
Initial code by Nathan Whitehorn.
|
| 281096 |
05-Apr-2015 |
jhibbits |
Unify Book-E and AIM trap.c
Summary:
Book-E and AIM trap.c are almost identical, except for a few bits. This is step
1 in unifying them.
This also renumbers EXC_DEBUG, to not conflict with AIM vector numbers. Since
this is the only one thus far that is used in the switch statement in trap(),
it's the only one renumbered. If others get added to the switch, which conflict
with AIM numbers, they should also be renumbered.
Reviewers: #powerpc, marcel, nwhitehorn
Reviewed By: marcel
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D2215
|
| 280957 |
01-Apr-2015 |
rstone |
Fix integer truncation bug in malloc(9)
A couple of internal functions used by malloc(9) and uma truncated
a size_t down to an int. This could cause any number of issues
(e.g. indefinite sleeps, memory corruption) if any kernel
subsystem tried to allocate 2GB or more through malloc. zfs would
attempt such an allocation when run on a system with 2TB or more
of RAM.
Note to self: When this is MFCed, sparc64 needs the same fix.
Differential revision: https://reviews.freebsd.org/D2106
Reviewed by: kib
Reported by: Michael Fuckner <michael@fuckner.net>
Tested by: Michael Fuckner <michael@fuckner.net>
MFC after: 2 weeks
|
| 279940 |
13-Mar-2015 |
nwhitehorn |
Deallocate any leftover page table entries in the LPAR at boot. This
prevents contamination from a previous kernel (e.g. after shutdown -r).
|
| 279751 |
07-Mar-2015 |
nwhitehorn |
Make assembly slightly more idiomatic (and able to be handled by clang's
integrated assembler).
|
| 279750 |
07-Mar-2015 |
nwhitehorn |
Make 32-bit PowerPC kernels, like 64-bit PowerPC kernels, position-independent
executables. The goal here, not yet accomplished, is to let the e500 kernel
run under QEMU by setting KERNBASE to something that fits in low memory and
then having the kernel relocate itself at runtime.
|
| 279601 |
04-Mar-2015 |
nwhitehorn |
The AIM DAR (data access fault address register) and Book-E DEAR registers
have the same meaning and occupy the same memory address in the trapframe
courtesy of union. Avoid some pointless #ifdef by spelling them both 'DAR'
in the trapframe.
|
| 279594 |
04-Mar-2015 |
nwhitehorn |
Move Book-E/AIM dependent bits for setting user PMAP during thread switch
out of cpu_switch() and into pmap_activate() where they belong. This also
removes all the #ifdef from cpu_switch().
|
| 279371 |
27-Feb-2015 |
nwhitehorn |
Fix unitialized variable.
|
| 279252 |
24-Feb-2015 |
nwhitehorn |
New pmap implementation for 64-bit PowerPC processors. The main focus of
this change is to improve concurrency:
- Drop global state stored in the shadow overflow page table (and all other
global state)
- Remove all global locks
- Use per-PTE lock bits to allow parallel page insertion
- Reconstruct state when requested for evicted PTEs instead of buffering
it during overflow
This drops total wall time for make buildworld on a 32-thread POWER8 system
by a factor of two and system time by a factor of three, providing performance
20% better than similarly clocked Core i7 Xeons per-core. Performance on
smaller SMP systems, where PMAP lock contention was not as much of an issue,
is nearly unchanged.
Tested on: POWER8, POWER5+, G5 UP, G5 SMP (64-bit and 32-bit kernels)
Merged from: user/nwhitehorn/ppc64-pmap-rework
Looked over by: jhibbits, andreast
MFC after: 3 months
Relnotes: yes
Sponsored by: FreeBSD Foundation
|
| 279189 |
22-Feb-2015 |
nwhitehorn |
Kernel support for the Vector-Scalar eXtension (VSX) found on the POWER7
and POWER8. This instruction set unifies the 32 64-bit scalar floating
point registers with the 32 128-bit vector registers into a single bank
of 64 128-bit registers. Kernel support mostly amounts to saving and
restoring the wider version of the floating point registers and making
sure that both scalar FP and vector registers are enabled once a VSX
instruction is executed. get_mcontext() and friends currently cannot
see the high bits, which will require a little more work.
As the system compiler (GCC 4.2) does not support VSX, making use of this
from userland requires either newer GCC or clang.
Relnotes: yes
Sponsored by: FreeBSD Foundation
|
| 278495 |
10-Feb-2015 |
rpaulo |
Remove FreeBSD/wii.
This port failed to gain traction and probably only a couple Wii consoles
ran FreeBSD all the way to single user mode with an md(4). IPC
support was never implemented, so it was impossible to use any peripheral
Any further development, if any, will happen at https://github.com/rpaulo/wii.
Discussed with: nathanw (a long time ago), jhibbits
|
| 278481 |
10-Feb-2015 |
nwhitehorn |
Set thread priorities on multithreaded CPUs so that threads holding a
spinlock are high-priority and threads waiting for a spinlock are set to
low priority.
|
| 278427 |
09-Feb-2015 |
nwhitehorn |
Simplify trapcode setup by placing a copy of the generic trap handler at
every possible trap address by default. This also makes sure the kernel
notices (and panics at) traps from newer CPUs that the kernel was not
expecting rather than executing gibberish memory.
|
| 277633 |
24-Jan-2015 |
nwhitehorn |
Fix typo in r277561.
|
| 277561 |
23-Jan-2015 |
nwhitehorn |
Use relocation-safe methods to determine the sizes of the exception handlers.
A "size" symbol with its address set to the length of handler would be
shifted forward with all other addresses when relocations are processed.
Instead, just note the end and do the subtraction at runtime.
|
| 277500 |
21-Jan-2015 |
nwhitehorn |
Add POWER7+ and POWER8 to the list of CPUs with 32 SLB slots. This is
mostly a no-op since all currently-supported instances of these CPUs give
the number of SLB slots in the device tree, but keep it here as well just
in case.
|
| 277499 |
21-Jan-2015 |
nwhitehorn |
Make sure to relocate tmpstk with everything else and avoid processing
non-relative relocations that the UART code makes for absent modules.
|
| 277498 |
21-Jan-2015 |
nwhitehorn |
Make 64-bit AIM trap handlers relocatable by changing all absolute branch
instructions to call through pointers instead. In general, these are set
implicitly through relocation processing. One has to be set explicitly in
machdep.c, however, to fit one handler in the tiny (8 instruction) space
available.
Reviewed by: andreast
Differential revision: D1554
Tested on: UP and SMP G5, Cell, POWER5+
|
| 277468 |
21-Jan-2015 |
nwhitehorn |
On 64-bit PowerPC, use more native forms of the PPC 970 HID restore
sequences, like are used to read the HIDs. This is both easier to read
and avoids a miscompilation by GCC in certain circumstances. Also avoid
double restoration of HID4 and HID5.
MFC after: 2 weeks
|
| 277411 |
20-Jan-2015 |
nwhitehorn |
Zero BSS explicitly if not started by loader(8). Add a check for the magic
values that ePAPR-compliant loaders (like skiboot) put in the register
loader uses for the metadata pointer to avoid confusing them.
|
| 277392 |
19-Jan-2015 |
nwhitehorn |
Add some initial infrastructure for relocating the kernel in place.
MFC after: 2 months
Differential revision: D1554
|
| 277356 |
19-Jan-2015 |
nwhitehorn |
Provide a tunable (machdep.moea64_bpvo_pool_size) to set the bootstrap
PVO pool size. The default errs on the exceedingly large side, so absent
any intelligent automatic tuning, at least let the user set it to save
RAM on memory-constrained systems.
MFC after: 2 weeks
|
| 277335 |
18-Jan-2015 |
nwhitehorn |
Use TOC to look up all kernel globals on powerpc64 instead of doing the
non-relocatable lis @ha, ori @l dance and hoping they are below 4 GB.
MFC after: 2 months
|
| 277334 |
18-Jan-2015 |
nwhitehorn |
Refactor PowerPC (especially AIM) init sequence to be less baroque.
MFC after: 2 months
|
| 277157 |
14-Jan-2015 |
nwhitehorn |
Do not remap Open Firmware mappings covered by the direct map. It's
pointless and wastes resources.
MFC after: 1 week
|
| 276772 |
07-Jan-2015 |
markj |
Factor out duplicated code from dumpsys() on each architecture into generic
code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly
identical and simply redefine a number of constants and helper subroutines;
a generic implementation will make it easier to implement features around
kernel core dumps. This change does not alter any minidump code and should
have no functional impact.
PR: 193873
Differential Revision: https://reviews.freebsd.org/D904
Submitted by: Conrad Meyer <conrad.meyer@isilon.com>
Reviewed by: jhibbits (earlier version)
Sponsored by: EMC / Isilon Storage Division
|
| 276515 |
01-Jan-2015 |
nwhitehorn |
Allow booting with both a real Open Firmware tree and a flattened version of
the Open Firmware, as provided by petitboot, for example. Note that this is
not quite complete, since RTAS instantiation still depends on callable
firmware.
MFC after: 2 weeks
|
| 276142 |
23-Dec-2014 |
markj |
Restore the trap type argument to the DTrace trap hook, removed in r268600.
It's redundant at the moment since it can be obtained from the trapframe
on the architectures where DTrace is supported, but this won't be the case
with ARM.
|
| 275683 |
10-Dec-2014 |
andreast |
Fix build for powerpc(32|64) kernels.
|
| 275268 |
29-Nov-2014 |
jhibbits |
Add support for dtrace:fbt on modules for PowerPC
Summary:
Revert the initial FBT-with-KDB changes for trap_subr*.S, and instead use the
db_trap filter function to handle dtrace trap filtering. With this, the MMU is
enabled by the support code, simplifying the codepath altogether.
Test Plan: Tested on my G4 PowerBook
Reviewers: #powerpc, nwhitehorn
Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D1207
MFC after: 3 weeks
|
| 274743 |
20-Nov-2014 |
jhibbits |
cpudep_ap_early_bootstrap() takes no arguments, so no need to give it one.
MFC after: 3 weeks
|
| 273174 |
16-Oct-2014 |
davide |
Follow up to r225617. In order to maximize the re-usability of kernel code
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.
Submitted by: kmacy
Tested by: make universe
|
| 272098 |
25-Sep-2014 |
royger |
ddb: allow specifying the exact address of the symtab and strtab
When the FreeBSD kernel is loaded from Xen the symtab and strtab are
not loaded the same way as the native boot loader. This patch adds
three new global variables to ddb that can be used to specify the
exact position and size of those tables, so they can be directly used
as parameters to db_add_symbol_table. A new helper is introduced, so callers
that used to set ksym_start and ksym_end can use this helper to set the new
variables.
It also adds support for loading them from the Xen PVH port, that was
previously missing those tables.
Sponsored by: Citrix Systems R&D
Reviewed by: kib
ddb/db_main.c:
- Add three new global variables: ksymtab, kstrtab, ksymtab_size that
can be used to specify the position and size of the symtab and
strtab.
- Use those new variables in db_init in order to call db_add_symbol_table.
- Move the logic in db_init to db_fetch_symtab in order to set ksymtab,
kstrtab, ksymtab_size from ksym_start and ksym_end.
ddb/ddb.h:
- Add prototype for db_fetch_ksymtab.
- Declate the extern variables ksymtab, kstrtab and ksymtab_size.
x86/xen/pv.c:
- Add support for finding the symtab and strtab when booted as a Xen
PVH guest. Since Xen loads the symtab and strtab as NetBSD expects
to find them we have to adapt and use the same method.
amd64/amd64/machdep.c:
arm/arm/machdep.c:
i386/i386/machdep.c:
mips/mips/machdep.c:
pc98/pc98/machdep.c:
powerpc/aim/machdep.c:
powerpc/booke/machdep.c:
sparc64/sparc64/machdep.c:
- Use the newly introduced db_fetch_ksymtab in order to set ksymtab,
kstrtab and ksymtab_size.
|
| 272014 |
23-Sep-2014 |
nwhitehorn |
We should have an isync after switching MSR[SF] in bootstrap.
Submitted by: Mark Millard
MFC after: 3 days
|
| 269728 |
08-Aug-2014 |
kib |
Change pmap_enter(9) interface to take flags parameter and superpage
mapping size (currently unused). The flags includes the fault access
bits, wired flag as PMAP_ENTER_WIRED, and a new flag
PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep.
For powerpc aim both 32 and 64 bit, fix implementation to ensure that
the requested mapping is created when PMAP_ENTER_NOSLEEP is not
specified, in particular, wait for the available memory required to
proceed.
In collaboration with: alc
Tested by: nwhitehorn (ppc aim32 and booke)
Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division
MFC after: 2 weeks
|
| 269701 |
08-Aug-2014 |
jhibbits |
Set the si_code appropriately for exception-caused signals.
LLDB checks the si_code, and aborts if a code isn't known.
MFC after: 2 weeks
Relnotes: yes
|
| 269485 |
03-Aug-2014 |
alc |
Retire pmap_change_wiring(). We have never used it to wire virtual pages.
We continue to use pmap_enter() for that. For unwiring virtual pages, we
now use pmap_unwire(), which unwires a range of virtual addresses instead
of a single virtual page.
Sponsored by: EMC / Isilon Storage Division
|
| 269388 |
01-Aug-2014 |
alc |
Simplify the selection of the pvo_head and pvo allocation zone in
moea_enter_locked() and moea64_enter().
Eliminate an unused variable from moea64_enter().
|
| 269368 |
01-Aug-2014 |
alc |
Retire PVO_EXECUTABLE. It's neither used nor set correctly.
|
| 269365 |
01-Aug-2014 |
alc |
Correct a long-standing problem in moea{,64}_pvo_enter() that was revealed
by the combination of r268591 and r269134: When we attempt to add the
wired attribute to an existing mapping, moea{,64}_pvo_enter() do nothing.
(They only set the wired attribute on newly created mappings.)
Tested by: andreast
|
| 269339 |
31-Jul-2014 |
alc |
Correct a defect in r268591. In the implementation of the new function
pmap_unwire(), the call to MOEA64_PVO_TO_PTE() must be performed before
any changes are made to the PVO. Otherwise, MOEA64_PVO_TO_PTE() will
panic.
Reported by: andreast
|
| 268880 |
19-Jul-2014 |
nwhitehorn |
Allow mappings of memory not previously direct-mapped by the kernel when
calling mmap on /dev/mem and add a handler for the possible userland
machine checks that may result. Remove some pointless and wrong copy/paste
that has been in here for a decade as well.
This results in a /dev/mem with identical semantics to the x86 version.
MFC after: 1 week
|
| 268600 |
14-Jul-2014 |
markj |
Invoke the DTrace trap handler before calling trap() on amd64. This matches
the upstream implementation and helps ensure that a trap induced by tracing
fbt::trap:entry is handled without recursively generating another trap.
This makes it possible to run most (but not all) of the DTrace tests under
common/safety/ without triggering a kernel panic.
Submitted by: Anton Rang <anton.rang@isilon.com> (original version)
Phabric: D95
|
| 268596 |
13-Jul-2014 |
alc |
Eliminate an unused variable. Refresh two comments.
|
| 268591 |
13-Jul-2014 |
alc |
Implement pmap_unwire(). See r268327 for the motivation behind this change.
|
| 268504 |
10-Jul-2014 |
alc |
Correct the accounting code for wired mappings. The wrong field of the PVO
entry was being tested. We were incrementing and decrementing the pmap's
wired mapping count based on whether the physical page being mapped or
unmapped was cache coherent, not whether it was a wired mapping.
Reviewed by: nwhitehorn
|
| 266827 |
29-May-2014 |
markj |
Commit the rest of the changes that were intended to be part of r266826.
X-MFC-with: r266826
|
| 266136 |
15-May-2014 |
jhibbits |
oea64 uses 4k pages, too.
MFC after: 1 week
X-MFC-with: r266116
|
| 266116 |
15-May-2014 |
jhibbits |
A page mask size is 12-bits, not 11.
MFC after: 1 week
|
| 263620 |
22-Mar-2014 |
bdrewery |
Rename global cnt to vm_cnt to avoid shadowing.
To reduce the diff struct pcu.cnt field was not renamed, so
PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in
kvm(3) and vmstat(8). The goal was to not affect externally used KPI.
Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the
the global cnt variable.
Exp-run revealed no ports using it directly.
No objection from: arch@
Sponsored by: EMC / Isilon Storage Division
|
| 263289 |
18-Mar-2014 |
emaste |
Update NetBSD Foundation copyrights to 2-clause BSD
The NetBSD Foundation states "Third parties are encouraged to change the
license on any files which have a 4-clause license contributed to the
NetBSD Foundation to a 2-clause license."
This change removes clauses 3 and 4 from copyright / license blocks that
list The NetBSD Foundation as the only copyright holder.
Sponsored by: The FreeBSD Foundation
|
| 261309 |
31-Jan-2014 |
jhibbits |
Unbreak non-SMP builds. This was broken by r259284. Also, reorganize the
code introduced in that revision a bit.
Reviewed by: nwhitehorn
MFC after: 3 weeks
|
| 261258 |
29-Jan-2014 |
jhibbits |
Use a loop of dcbz, instead of calling bzero() to zero a page. This matches
what is done in mmu_oea64.c.
MFC after: 1 month
|
| 259421 |
15-Dec-2013 |
jhibbits |
Save r3 before using it for the trap check, else we end up saving the new r3,
containing the trap instruction encoding (0x7c810808), and restoring it back
with the frame on return. This caused it to panic on my ppc32 machine, but
somehow my ppc64 machine overlooked it, because I was using such a simple
dtrace probe.
X-MFC-with: r259245
MFC after: 2 weeks
|
| 259284 |
13-Dec-2013 |
jhibbits |
Add PMU-based CPU frequency scaling. This method is used on most Titanium
PowerBooks.
MFC after: 1 month
|
| 259245 |
12-Dec-2013 |
jhibbits |
FBT now does work fully on PowerPC.
MFC after: 2 weeks
|
| 258800 |
01-Dec-2013 |
nwhitehorn |
The kernel stack guard pages are only below the stack pointer, not above.
Prevent erroneous detection of stack overflows on legitimate faults on the
page after this thread's stack.
MFC after: 3 days
|
| 258696 |
27-Nov-2013 |
nwhitehorn |
badaddr() is used only in the grackle PCI driver, so move its definition
there. Clean up a spurious setfault() declaration as well.
|
| 258541 |
25-Nov-2013 |
attilio |
- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
version of the releasing functions for mutex, rwlock and sxlock.
Failing to do so skips the lockstat_probe_func invokation for
unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
kernel compiled without lock debugging options, potentially every
consumer must be compiled including opt_kdtrace.h.
Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
is linked there and it is only used as a compile-time stub [0].
[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested. As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while. Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].
Sponsored by: EMC / Isilon storage division
Discussed with: rstone
[0] Reported by: rstone
[1] Discussed with: philip
|
| 258504 |
23-Nov-2013 |
andreast |
Save and restore the trap vectors when doing OF calls on pSeries machines.
It turned out that on pSeries machines the call into OF modified the trap
vectors and this made further behaviour unpredictable.
With this commit I'm now able to boot multi user on a network booted
environment on my IntelliStation 285. This is a POWER5+ machine.
Discussed with: nwhitehorn
MFC after: 1 week
|
| 258424 |
21-Nov-2013 |
jhibbits |
Remove stale comment. The PID provider is handled elsewhere already.
|
| 258268 |
17-Nov-2013 |
nwhitehorn |
Do not assume a value for #address-cells when parsing the OF translations
map. This allows the kernel to get farther with OpenBIOS on 64-bit CPUs.
|
| 258259 |
17-Nov-2013 |
nwhitehorn |
Unify handling of illegal instruction faults between AIM and Book-E. This
allows FPU emulation on AIM as well as providing support for the mfpvr
and lwsync instructions from userland on e500 cores. lwsync, in particular,
is required for many C++ programs to work correctly.
MFC after: 1 week
|
| 258119 |
14-Nov-2013 |
jhibbits |
Fix copy+paste-o, OEA64 uses LPTE, not PTE.
X-MFC with: r257941
|
| 258024 |
11-Nov-2013 |
nwhitehorn |
Use the same implementation of copyinout.c for both AIM and Book-E. This
fixes some bugs in both implementations related to validity checks on
mapping bounds.
|
| 258002 |
11-Nov-2013 |
nwhitehorn |
Follow up r223485, which made AIM use the ABI thread pointer instead of
PCPU fields for curthread, by doing the same to Book-E. This closes
some potential races switching between CPUs. As a side effect, it turns out
the AIM and Book-E swtch.S implementations were the same to within a few
registers, so move that to powerpc/powerpc.
MFC after: 3 months
|
| 257941 |
11-Nov-2013 |
jhibbits |
Add the necessary bits for dumps on ppc64.
MFC after: 2 weeks
|
| 257417 |
31-Oct-2013 |
markj |
Remove references to an unused fasttrap probe hook, and remove the
corresponding x86 trap type. Userland DTrace probes are currently handled
by the other fasttrap hooks (dtrace_pid_probe_ptr and
dtrace_return_probe_ptr).
Discussed with: rpaulo
|
| 257180 |
26-Oct-2013 |
nwhitehorn |
Add some extra sanity checking and checks to printf format specifiers.
|
| 257178 |
26-Oct-2013 |
nwhitehorn |
Interrelated improvements to early boot mappings:
- Remove explicit requirement that the SOC registers be found except as an
optimization (although the MPC85XX LAW drivers still require they be found
externally, which should change).
- Remove magic CCSRBAR_VA value.
- Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and
systems not in real-mode or global 1:1 maps in early boot.
- Allow pmap_mapdev() on Book-E to reissue previous addresses if the
area is already mapped. Additionally have it check all mappings, not
just the CCSR area.
This allows the console on e500 systems to actually work on systems where
the boot loader was not kind enough to set up a 1:1 mapping before starting
the kernel.
|
| 257175 |
26-Oct-2013 |
nwhitehorn |
Clean up missed header references.
|
| 256799 |
20-Oct-2013 |
nwhitehorn |
Since the PS3 port was committed, the AIM nexus device works perfectly fine
on all PowerPC platforms, whether or not they have Open Firmware. Remove
some more duplication and have there be only one nexus driver.
|
| 256793 |
20-Oct-2013 |
nwhitehorn |
Replace the two almost-exactly-identical AIM and Book-E clock.c
implementations with a single one after the application of a very small
amount of #ifdef.
|
| 256792 |
20-Oct-2013 |
nwhitehorn |
Unify the AIM and Book-E vm_machdep.c implementations, which previously
differed only with respect to the AIM version not following style(9) and
some additional features for 64-bit systems and machines with direct maps
in the AIM implementation that are no-ops on Book-E (at least for now).
|
| 256581 |
16-Oct-2013 |
jhibbits |
Fix the Wii build, and remove an extraneous critical_enter().
|
| 256543 |
15-Oct-2013 |
jhibbits |
Add fasttrap for PowerPC. This is the last piece of the dtrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.
MFC after: 1 month
|
| 256542 |
15-Oct-2013 |
jhibbits |
Move the PMC handling to the first level interrupt handler where it belongs.
Also add the pmc_hook use, to handle callchain tracing.
MFC after: 1 week
|
| 255786 |
22-Sep-2013 |
glebius |
- Create kern.ipc.sendfile namespace, and put the new "readhead" OID
there as "kern.ipc.sendfile.readahead".
- Push all nsfbuf related tunables into MD code. Don't move them
to new namespace in favor of POLA.
Reviewed by: scottl
Approved by: re (gjb)
|
| 255724 |
20-Sep-2013 |
alc |
The pmap function pmap_clear_reference() is no longer used. Remove it.
pmap_clear_reference() has had exactly one caller in the kernel for
several years, more precisely, since FreeBSD 8. Now, that call no
longer exists.
Approved by: re (kib)
Sponsored by: EMC / Isilon Storage Division
|
| 255503 |
13-Sep-2013 |
nwhitehorn |
Change VM object lock assertion to match locking higher in the call
chain. This repairs a panic observed during pageout on some 64-bit
PowerPC systems.
Submitted by: grehan
Approved by: re (kib)
MFC after: 2 weeks
Revisit after: 10.0
|
| 255419 |
09-Sep-2013 |
nwhitehorn |
Raise artificial limits on number of CPUs and number of interrupts.
Approved by: re (kib)
|
| 255418 |
09-Sep-2013 |
nwhitehorn |
Add POWER CPUs to the kernel's knowledge. This does not imply we currently
actually run on any machines with POWER CPUs but avoids closing that door
unnecessarily.
Approved by: re (kib)
|
| 255273 |
05-Sep-2013 |
nwhitehorn |
Align stacks of kernel threads correctly at 16-byte boundaries rather than
making sure they are all misaligned at +8 bytes. This fixes clang builds
of powerpc64 kernels (aside from a required increase in KSTACK_PAGES which
will come later).
This commit from FreeBSD/powerpc64 with a clang-built kernel.
MFC after: 2 weeks
|
| 255165 |
03-Sep-2013 |
jhibbits |
Enable PMC interrupt handling, and fix a DTrace trap handling bug.
|
| 254667 |
22-Aug-2013 |
kib |
Revert r254501. Instead, reuse the type stability of the struct pmap
which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE
zone. Initialize the pmap lock in the vmspace zone init function, and
remove pmap lock initialization and destruction from pmap_pinit() and
pmap_release().
Suggested and reviewed by: alc (previous version)
Tested by: pho
Sponsored by: The FreeBSD Foundation
|
| 254138 |
09-Aug-2013 |
attilio |
The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.
Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag
The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.
Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl
|
| 254025 |
07-Aug-2013 |
jeff |
Replace kernel virtual address space allocation with vmem. This provides
transparent layering and better fragmentation.
- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.
Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division
|
| 253978 |
06-Aug-2013 |
jhibbits |
Remove an unnecessary panic. The PVO's PTE entry and the PTEG's PTE entry may
not match, if the PVO's PTE is invalid.
|
| 253976 |
06-Aug-2013 |
jhibbits |
Evict pages from the PTEG when it's full and trying to insert a new PTE,
rather than panicking.
Reviewed by: nwhitehorn
MFC after: 3 weeks
|
| 253351 |
15-Jul-2013 |
ae |
Introduce new structure sfstat for collecting sendfile's statistics
and remove corresponding fields from struct mbstat. Use PCPU counters
and SFSTAT_INC() macro for update these statistics.
Discussed with: glebius
|
| 253272 |
12-Jul-2013 |
nwhitehorn |
Fix check: bitwise and has only one &.
MFC after: 1 week
|
| 250884 |
21-May-2013 |
attilio |
o Relax locking assertions for vm_page_find_least()
o Relax locking assertions for pmap_enter_object() and add them also
to architectures that currently don't have any
o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade
operation on the per-object rwlock
o Use all the mechanisms above to make vm_map_pmap_enter() to work
mostl of the times only with readlocks.
Sponsored by: EMC / Isilon storage division
Reviewed by: alc
|
| 250747 |
17-May-2013 |
alc |
Relax the object locking assertion in pmap_enter_locked().
Reviewed by: attilio
Sponsored by: EMC / Isilon Storage Division
|
| 249918 |
26-Apr-2013 |
jhibbits |
Remove a comment that shouldn't have gone in.
X-MFC-with: r249864
|
| 249864 |
25-Apr-2013 |
jhibbits |
Introduce kernel coredumps to ppc32 AIM. Leeched from the booke code.
MFC after: 2 weeks
|
| 249129 |
05-Apr-2013 |
jhibbits |
Print out DSISR in a fatal DSI trap.
Sponsored by:
|
| 248508 |
19-Mar-2013 |
kib |
Implement the concept of the unmapped VMIO buffers, i.e. buffers which
do not map the b_pages pages into buffer_map KVA. The use of the
unmapped buffers eliminate the need to perform TLB shootdown for
mapping on the buffer creation and reuse, greatly reducing the amount
of IPIs for shootdown on big-SMP machines and eliminating up to 25-30%
of the system time on i/o intensive workloads.
The unmapped buffer should be explicitely requested by the GB_UNMAPPED
flag by the consumer. For unmapped buffer, no KVA reservation is
performed at all. The consumer might request unmapped buffer which
does have a KVA reserve, to manually map it without recursing into
buffer cache and blocking, with the GB_KVAALLOC flag.
When the mapped buffer is requested and unmapped buffer already
exists, the cache performs an upgrade, possibly reusing the KVA
reservation.
Unmapped buffer is translated into unmapped bio in g_vfs_strategy().
Unmapped bio carry a pointer to the vm_page_t array, offset and length
instead of the data pointer. The provider which processes the bio
should explicitely specify a readiness to accept unmapped bio,
otherwise g_down geom thread performs the transient upgrade of the bio
request by mapping the pages into the new bio_transient_map KVA
submap.
The bio_transient_map submap claims up to 10% of the buffer map, and
the total buffer_map + bio_transient_map KVA usage stays the
same. Still, it could be manually tuned by kern.bio_transient_maxcnt
tunable, in the units of the transient mappings. Eventually, the
bio_transient_map could be removed after all geom classes and drivers
can accept unmapped i/o requests.
Unmapped support can be turned off by the vfs.unmapped_buf_allowed
tunable, disabling which makes the buffer (or cluster) creation
requests to ignore GB_UNMAPPED and GB_KVAALLOC flags. Unmapped
buffers are only enabled by default on the architectures where
pmap_copy_page() was implemented and tested.
In the rework, filesystem metadata is not the subject to maxbufspace
limit anymore. Since the metadata buffers are always mapped, the
buffers still have to fit into the buffer map, which provides a
reasonable (but practically unreachable) upper bound on it. The
non-metadata buffer allocations, both mapped and unmapped, is
accounted against maxbufspace, as before. Effectively, this means that
the maxbufspace is forced on mapped and unmapped buffers separately.
The pre-patch bufspace limiting code did not worked, because
buffer_map fragmentation does not allow the limit to be reached.
By Jeff Roberson request, the getnewbuf() function was split into
smaller single-purpose functions.
Sponsored by: The FreeBSD Foundation
Discussed with: jeff (previous version)
Tested by: pho, scottl (previous version), jhb, bf
MFC after: 2 weeks
|
| 248457 |
18-Mar-2013 |
jhibbits |
Add FBT for PowerPC DTrace. Also, clean up the DTrace assembly code,
much of which is not necessary for PowerPC.
The FBT module can likely be factored into 3 separate files: common,
intel, and powerpc, rather than duplicating most of the code between
the x86 and PowerPC flavors.
All DTrace modules for PowerPC will be MFC'd together once Fasttrap is
completed.
|
| 248280 |
14-Mar-2013 |
kib |
Add pmap function pmap_copy_pages(), which copies the content of the
pages around, taking array of vm_page_t both for source and
destination. Starting offsets and total transfer size are specified.
The function implements optimal algorithm for copying using the
platform-specific optimizations. For instance, on the architectures
were the direct map is available, no transient mappings are created,
for i386 the per-cpu ephemeral page frame is used. The code was
typically borrowed from the pmap_copy_page() for the same
architecture.
Only i386/amd64, powerpc aim and arm/arm-v6 implementations were
tested at the time of commit. High-level code, not committed yet to
the tree, ensures that the use of the function is only allowed after
explicit enablement.
For sparc64, the existing code has known issues and a stab is added
instead, to allow the kernel linking.
Sponsored by: The FreeBSD Foundation
Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6)
MFC after: 2 weeks
|
| 248084 |
09-Mar-2013 |
attilio |
Switch the vm_object mutex to be a rwlock. This will enable in the
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.
The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
(in order to avoid visibility of implementation details)
- The read-mode operations are added:
VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.
The KPI results heavilly broken by this commit. Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).
Sponsored by: EMC / Isilon storage division
Reviewed by: jeff
Reviewed by: pjd (ZFS specific review)
Discussed with: alc
Tested by: pho
|
| 247463 |
28-Feb-2013 |
mav |
MFcalloutng:
Switch eventtimers(9) from using struct bintime to sbintime_t.
Even before this not a single driver really supported full dynamic range of
struct bintime even in theory, not speaking about practical inexpediency.
This change legitimates the status quo and cleans up the code.
|
| 247400 |
27-Feb-2013 |
attilio |
Merge from vmobj-rwlock:
VM_OBJECT_LOCKED() macro is only used to implement a custom version
of lock assertions right now (which likely spread out thanks to
copy and paste).
Remove it and implement actual assertions.
Sponsored by: EMC / Isilon storage division
Reviewed by: alc
Tested by: pho
|
| 247297 |
26-Feb-2013 |
attilio |
Merge from vmobj-rwlock branch:
Remove unused inclusion of vm/vm_pager.h and vm/vnode_pager.h.
Sponsored by: EMC / Isilon storage division
Tested by: pho
Reviewed by: alc
|
| 243370 |
21-Nov-2012 |
adrian |
Setup BAT0 and BAT1 on the Wii.
This is the missing piece for FreeBSD/Wii, but there's still a lot of
work ahead. We have to reset the MMU in locore before continuing
the boot process because we don't know how the boot loaders might
have setup the BATs. We also disable the PCI BAT because there's no PCI
bus on the Wii.
Thanks to Nathan Whitehorn and Peter Grenhan for their help.
Submitted by: Margarida Gouveia
|
| 243040 |
14-Nov-2012 |
kib |
Flip the semantic of M_NOWAIT to only require the allocation to not
sleep, and perform the page allocations with VM_ALLOC_SYSTEM
class. Previously, the allocation was also allowed to completely drain
the reserve of the free pages, being translated to VM_ALLOC_INTERRUPT
request class for vm_page_alloc() and similar functions.
Allow the caller of malloc* to request the 'deep drain' semantic by
providing M_USE_RESERVE flag, now translated to VM_ALLOC_INTERRUPT
class. Previously, it resulted in less aggressive VM_ALLOC_SYSTEM
allocation class.
Centralize the translation of the M_* malloc(9) flags in the single
inline function malloc2vm_flags().
Discussion started by: "Sears, Steven" <Steven.Sears@netapp.com>
Reviewed by: alc, mdf (previous version)
Tested by: pho (previous version)
MFC after: 2 weeks
|
| 242723 |
07-Nov-2012 |
jhibbits |
Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.
There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"
I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.
MFC after: 1 month
|
| 242534 |
03-Nov-2012 |
attilio |
Rework the known rwlock to benefit about staying on their own
cache line in order to avoid manual frobbing but using
struct rwlock_padalign.
Reviewed by: alc, jimharris
|
| 241020 |
28-Sep-2012 |
alc |
Eliminate a stale comment. It describes another use case for the pmap in
Mach that doesn't exist in FreeBSD.
|
| 240244 |
08-Sep-2012 |
attilio |
userret() already checks for td_locks when INVARIANTS is enabled, so
there is no need to check if Giant is acquired after it.
Reviewed by: kib
MFC after: 1 week
|
| 239682 |
25-Aug-2012 |
rpaulo |
Unbreak tinderbox.
|
| 239666 |
25-Aug-2012 |
rpaulo |
Set mdp only under #ifdef WII.
|
| 239480 |
21-Aug-2012 |
adrian |
On Nintendo Wii CPUs, the mdp value will be garbage. Set it to NULL
so as to not confuse things.
Submitted by: Margarida Gouveia
|
| 238357 |
10-Jul-2012 |
alc |
Avoid recursion on the pvh global lock in the aim oea pmap.
Correct the return type of the pmap_ts_referenced() implementations.
Reported by: jhibbits [1]
Tested by: andreast
|
| 238159 |
06-Jul-2012 |
alc |
Replace all uses of the vm page queues lock by a r/w lock that is private
to this pmap.
Tested by: andreast, jhibbits
|
| 237737 |
29-Jun-2012 |
rpaulo |
The `end' symbol doesn't match the end of the kernel image because it's
relative to the start address (unless the start address is 0, which is
not the case).
This is currently not a problem because all powerpc architectures are
using loader(8) which passes metadata to the kernel including the
correct `endkernel' address. If we don't use loader(8), register 4
and 5 will have the size of the kernel ELF file, not its end address.
We fix that simply by adding `kernel_text' to `end' to compute
`endkernel'.
Discussed with: nathanw
|
| 236019 |
25-May-2012 |
raj |
Fix physical address type to vm_paddr_t also for powerpc64.
|
| 235936 |
24-May-2012 |
raj |
Fix physical address type to vm_paddr_t.
|
| 235689 |
20-May-2012 |
nwhitehorn |
Replace the list of PVOs owned by each PMAP with an RB tree. This simplifies
range operations like pmap_remove() and pmap_protect() as well as allowing
simple operations like pmap_extract() not to involve any global state.
This substantially reduces lock coverages for the global table lock and
improves concurrency.
|
| 235013 |
04-May-2012 |
nwhitehorn |
Fix final bugs in memory barriers on PowerPC:
- Use isync/lwsync unconditionally for acquire/release. Use of isync
guarantees a complete memory barrier, which is important for serialization
of bus space accesses with mutexes on multi-processor systems.
- Go back to using sync as the I/O memory barrier, which solves the same
problem as above with respect to mutex release using lwsync, while not
penalizing non-I/O operations like a return to sync on the atomic release
operations would.
- Place an acquisition barrier around thread lock acquisition in
cpu_switchin().
|
| 234760 |
28-Apr-2012 |
nwhitehorn |
Fix build on 32-bit systems.
|
| 234745 |
28-Apr-2012 |
nwhitehorn |
After switching mutexes to use lwsync, they no longer provide sufficient
guarantees on acquire for the tlbie mutex. Conversely, the TLB invalidation
sequence provides guarantees that do not need to be redundantly applied on
release. Roll a small custom lock that is just right. Simultaneously,
convert the SLB tree changes back to lwsync, as changing them to sync
was a misdiagnosis of the tlbie barrier problem this commit actually fixes.
|
| 234652 |
24-Apr-2012 |
nwhitehorn |
Revert r234581 for this file. The lockless SLB tree code does in fact need
a heavyweight sync instead of a lightweight sync to function properly.
Thanks to mdf for the clarification.
|
| 234581 |
22-Apr-2012 |
nwhitehorn |
Use lwsync to provide memory barriers on systems that support it instead
of sync (lwsync is an alternate encoding of sync on systems that do not
support it, providing graceful fallback). This provides more than an order
of magnitude reduction in the time required to acquire or release a mutex.
MFC after: 2 months
|
| 234576 |
22-Apr-2012 |
nwhitehorn |
Avoid a lock order reversal in pmap_extract_and_hold() from relocking
the page. This PMAP requires an additional lock besides the PMAP lock
in pmap_extract_and_hold(), which vm_page_pa_tryrelock() did not release.
Suggested by: kib
MFC after: 4 days
|
| 234517 |
20-Apr-2012 |
nwhitehorn |
Make sure all pending operations have completed on the existing thread
before (potentially) migrating it to a different CPU.
MFC after: 5 days
|
| 234156 |
11-Apr-2012 |
nwhitehorn |
We don't need kcopy() in any of the remaining places it is used, so
remove it.
MFC after: 2 weeks
|
| 234155 |
11-Apr-2012 |
nwhitehorn |
Only manipulate the PGA_EXECUTABLE flag on managed pages. This is a proxy
for whether the page is physical. On dense phys mem systems (32-bit),
VM_PHYS_TO_PAGE will not return NULL for device memory pages if device
memory is above physical memory even if there is no allocated vm_page.
Attempting to use the returned page could then cause either memory
corruption or a page fault.
|
| 234149 |
11-Apr-2012 |
nwhitehorn |
Fix error in r233949. Synchronizing icaches on uncacheable pages turns out
not to be a good idea, and of course the PV entry list for a page is never
empty after the page has been mapped.
|
| 233964 |
06-Apr-2012 |
nwhitehorn |
Execute an initial ptesync if and only if the PTE is actually being
invalidated, as opposed to a ref/changed bit update.
|
| 233957 |
06-Apr-2012 |
nwhitehorn |
Substantially reduce the scope of the locks held in pmap_enter(), which
improves concurrency slightly.
|
| 233949 |
06-Apr-2012 |
nwhitehorn |
Reduce the frequency that the PowerPC/AIM pmaps invalidate instruction
caches, by invalidating kernel icaches only when needed and not flushing
user caches for shared pages.
Suggested by: kib
MFC after: 2 weeks
|
| 233618 |
28-Mar-2012 |
nwhitehorn |
More PMAP performance improvements: skip 256 MB segments entirely if they
are are not mapped during ranged operations and reduce the scope of the
tlbie lock only to the actual tlbie instruction instead of the entire
sequence. There are a few more optimization possibilities here as well.
|
| 233530 |
27-Mar-2012 |
nwhitehorn |
Make sure to call vm_page_dirty() before the pmap lock is released to
prevent a race where another process could conclude the page was clean.
Submitted by: alc
|
| 233529 |
27-Mar-2012 |
nwhitehorn |
More PMAP concurrency improvements: replace the table lock and (almost) all
uses of the page queues mutex with a new rwlock that protects the page
table and the PV lists. This reduces system time during a parallel
buildworld by 35%.
Reviewed by: alc
|
| 233454 |
25-Mar-2012 |
nwhitehorn |
More PMAP performance improvements: on powerpc64, when TLBIE can be run
with exceptions enabled, leave them enabled and use a regular mutex to
guard TLB invalidations instead of a spinlock.
|
| 233436 |
24-Mar-2012 |
nwhitehorn |
Only call vm_page_dirty() on pages that are writable in order not to
confuse the VM.
|
| 233434 |
24-Mar-2012 |
nwhitehorn |
Following suggestions from alc, skip wired mappings in pmap_remove_pages()
and remove moea64_attr_*() in favor of direct calls to vm_page_dirty()
and friends.
|
| 233117 |
18-Mar-2012 |
nwhitehorn |
Remove acquisition of VM page queues lock from pmap_protect(). Any actual
manipulation of the pvo_vlink and pvo_olink entries is already protected
by the table lock, so most remaining instances of the acquisition of the
page queues lock can likely be replaced with the table lock, or removed
if the table lock is already held.
Reviewed by: alc
|
| 233017 |
15-Mar-2012 |
nwhitehorn |
Implement pmap_remove_pages(). This will be added later to the 32-bit MMU
module.
Suggested by: alc
|
| 233011 |
15-Mar-2012 |
nwhitehorn |
Improve algorithm for deciding whether to loop through all process pages
or look them up individually in pmap_remove() and apply the same logic
in the other ranged operation (pmap_protect). This speeds up make
installworld by a factor of 2 on powerpc64.
MFC after: 1 week
|
| 232980 |
14-Mar-2012 |
nwhitehorn |
Use LIST_FOREACH_SAFE() instead of LIST_FOREACH() in pmap_remove(), since
the point of this loop is to remove elements. This worked by accident before.
MFC after: 2 days
|
| 231019 |
05-Feb-2012 |
andreast |
Revert the _NOPROF entries on cpu_throw, cpu_switch and savectx. They can be
profiled too now.
MFC after: 2 weeks
|
| 230779 |
30-Jan-2012 |
kib |
Fix build for the case of powerpc64 kernel without COMPAT_FREEBSD32.
MFC after: 2 months
|
| 230767 |
30-Jan-2012 |
kib |
Finally, try to enable the nxstacks on amd64 and powerpc64 for both 64bit
and 32bit ABIs. Also try to enable nxstacks for PAE/i386 when supported,
and some variants of powerpc32.
MFC after: 2 months (if ever)
|
| 230400 |
20-Jan-2012 |
andreast |
This commit adds profiling support for powerpc64. Now we can do application
profiling and kernel profiling. To enable kernel profiling one has to build
kgmon(8). I will enable the build once I managed to build and test powerpc
(32-bit) kernels with profiling support.
- add a powerpc64 PROF_PROLOGUE for _mcount.
- add macros to avoid adding the PROF_PROLOGUE in certain assembly entries.
- apply these macros where needed.
- add size information to the MCOUNT function.
MFC after: 3 weeks, together with r230291
|
| 230123 |
15-Jan-2012 |
nwhitehorn |
Rework SLB trap handling so that double-faults into an SLB trap handler are
possible, and double faults within an SLB trap handler are not. The result
is that it possible to take an SLB fault at any time, on any address, for
any reason, at any point in the kernel.
This lets us do two important things. First, it removes the (soft) 16 GB RAM
ceiling on PPC64 as well as any architectural limitations on KVA space.
Second, it lets the kernel tolerate poorly designed hypervisors that
have a tendency to fail to restore the SLB properly after a hypervisor
context switch.
MFC after: 6 weeks
|
| 228869 |
24-Dec-2011 |
jhibbits |
Implement hwpmc counting PMC support for PowerPC G4+ (MPC745x/MPC744x).
Sampling is in progress.
Approved by: nwhitehorn (mentor)
MFC after: 9.0-RELEASE
|
| 228609 |
17-Dec-2011 |
nwhitehorn |
Allow this to work on embedded systems without Open Firmware by making
lack of a /chosen non-fatal, and manually removing memory in use by the
kernel from the physical memory map.
Submitted by: rpaulo
|
| 228605 |
16-Dec-2011 |
nwhitehorn |
Zero BSS on start, in case the ELF loader that started the kernel did not
do this for us. This can happen on some embedded systems.
Submitted by: rpaulo
|
| 228522 |
15-Dec-2011 |
alc |
Eliminate vestiges of page coloring.
|
| 228412 |
11-Dec-2011 |
nwhitehorn |
Keep track of PVO entries in each pmap, which allows much faster
pmap_remove() for large sparse requests. This can prevent pmap_remove()
operations on 64-bit process destruction or swapout that would take
several hundred times the lifetime of the universe to complete. This
behavior is largely indistinguishable from a hang.
|
| 227843 |
22-Nov-2011 |
marius |
- There's no need to overwrite the default device method with the default
one. Interestingly, these are actually the default for quite some time
(bus_generic_driver_added(9) since r52045 and bus_generic_print_child(9)
since r52045) but even recently added device drivers do this unnecessarily.
Discussed with: jhb, marcel
- While at it, use DEVMETHOD_END.
Discussed with: jhb
- Also while at it, use __FBSDID.
|
| 227628 |
17-Nov-2011 |
nwhitehorn |
Use a global __pure2 function instead of a global register variable for
curthread, like on x86 and sparc64. This makes the kernel somewhat more
clang friendly, which doesn't support global register variables.
|
| 227627 |
17-Nov-2011 |
nwhitehorn |
Add an extra invariant here which was useful on 64-bit CPUs.
|
| 227568 |
16-Nov-2011 |
alc |
Refactor the code that performs physically contiguous memory allocation,
yielding a new public interface, vm_page_alloc_contig(). This new function
addresses some of the limitations of the current interfaces, contigmalloc()
and kmem_alloc_contig(). For example, the physically contiguous memory that
is allocated with those interfaces can only be allocated to the kernel vm
object and must be mapped into the kernel virtual address space. It also
provides functionality that vm_phys_alloc_contig() doesn't, such as wiring
the returned pages. Moreover, unlike that function, it respects the low
water marks on the paging queues and wakes up the page daemon when
necessary. That said, at present, this new function can't be applied to all
types of vm objects. However, that restriction will be eliminated in the
coming weeks.
From a design standpoint, this change also addresses an inconsistency
between vm_phys_alloc_contig() and the other vm_phys_alloc*() functions.
Specifically, vm_phys_alloc_contig() manipulated vm_page fields that other
functions in vm/vm_phys.c didn't. Moreover, vm_phys_alloc_contig() knew
about vnodes and reservations. Now, vm_page_alloc_contig() is responsible
for these things.
Reviewed by: kib
Discussed with: jhb
|
| 227386 |
09-Nov-2011 |
nwhitehorn |
Fix a bug where the pmap_cpu_bootstrap() ap argument could be clobbered.
Luckily, it mostly wasn't important, so this didn't cause major problems.
Also improve register reuse when setting up trap frames very slightly.
Submitted by: Justin Hibbits <chmeeedalf at gmail dot com>
MFC after: 5 days
|
| 225474 |
11-Sep-2011 |
kib |
Inline the syscallenter() and syscallret(). This reduces the time measured
by the syscall entry speed microbenchmarks by ~10% on amd64.
Submitted by: jhb
Approved by: re (bz)
MFC after: 2 weeks
|
| 225418 |
06-Sep-2011 |
kib |
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic
flags field. Updates to the atomic flags are performed using the atomic
ops on the containing word, do not require any vm lock to be held, and
are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9)
functions are provided to modify afalgs.
Document the changes to flags field to only require the page lock.
Introduce vm_page_reference(9) function to provide a stable KPI and
KBI for filesystems like tmpfs and zfs which need to mark a page as
referenced.
Reviewed by: alc, attilio
Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64)
Approved by: re (bz)
|
| 224746 |
09-Aug-2011 |
kib |
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag
to VPO_UNMANAGED (and also making the flag protected by the vm object
lock, instead of vm page queue lock).
- Mark the fake pages with both PG_FICTITIOUS (as it is now) and
VPO_UNMANAGED. As a consequence, pmap code now can use use just
VPO_UNMANAGED to decide whether the page is unmanaged.
Reviewed by: alc
Tested by: pho (x86, previous version), marius (sparc64),
marcel (arm, ia64, powerpc), ray (mips)
Sponsored by: The FreeBSD Foundation
Approved by: re (bz)
|
| 224400 |
25-Jul-2011 |
andreast |
This a follow up commit from r224216 for powerpc 32-bit. Increase
the storage size for sintrcnt/sintrnames to .long.
Reviewed by: nwhitehorn
Approved by: re (kib)
|
| 224216 |
19-Jul-2011 |
attilio |
On 64 bit architectures size_t is 8 bytes, thus it should use an 8 bytes
storage.
Fix the sintrcnt/sintrnames specification.
No MFC is previewed for this patch.
Reported, reviewed and tested by: marcel
Approved by: re (kib)
|
| 224187 |
18-Jul-2011 |
attilio |
- Remove the eintrcnt/eintrnames usage and introduce the concept of
sintrcnt/sintrnames which are symbols containing the size of the 2
tables.
- For amd64/i386 remove the storage of intr* stuff from assembly files.
This area can be widely improved by applying the same to other
architectures and likely finding an unified approach among them and
move the whole code to be MI. More work in this area is expected to
happen fairly soon.
No MFC is previewed for this patch.
Tested by: pluknet
Reviewed by: jhb
Approved by: re (kib)
|
| 223758 |
04-Jul-2011 |
attilio |
With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.
Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).
This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.
MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast
|
| 223570 |
26-Jun-2011 |
nwhitehorn |
Revert r223479. It is unnecessary and served only to slightly ameliorate
some manifestations of the bug actually fixed in r223485.
|
| 223485 |
23-Jun-2011 |
nwhitehorn |
Use the ABI-mandated thread pointer register (r2 for ppc32, r13 for ppc64)
instead of a PCPU field for curthread. This averts a race on SMP systems
with a high interrupt rate where the thread looking up the value of
curthread could be preempted and migrated between obtaining the PCPU
pointer and reading the value of pc_curthread, resulting in curthread being
observed to be the current thread on the thread's original CPU. This played
merry havoc with the system, in particular with mutexes. Many thanks to
jhb for helping me work this one out.
Note that Book-E is in principle susceptible to the same problem, but has
not been modified yet due to lack of Book-E hardware.
MFC after: 2 weeks
|
| 223479 |
23-Jun-2011 |
nwhitehorn |
Clear any outstanding atomic reservations when traps are taken. This fixes
some interesting bugs (mostly on SMP systems) with atomic operations
silently failing in interrupt heavy situations, especially when using
overflow pages.
|
| 223471 |
23-Jun-2011 |
andreast |
Fix merge typo.
|
| 222813 |
07-Jun-2011 |
attilio |
etire the cpumask_t type and replace it with cpuset_t usage.
This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.
Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.
Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).
Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.
People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.
Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.
(I really hope I didn't forget anyone, if it happened I apologize in
advance).
|
| 222666 |
04-Jun-2011 |
nwhitehorn |
Fix a typo derived from a mismerge from mmu_oea that would cause
pmap_sync_icache() to sync random (possibly uncached or nonexisting!)
memory, causing kernel page faults or machine checks, most easily
triggered by using GDB. While here, add an additional safeguard to only
sync cacheable memory.
MFC after: 2 days
|
| 222632 |
03-Jun-2011 |
nwhitehorn |
Quantities stored on the stack on ppc64 tend to be twice as large as on
ppc32, so make the early stack correspondingly twice as big.
|
| 222620 |
02-Jun-2011 |
nwhitehorn |
The POWER7 has only 32 SLB slots instead of 64, like other supported
64-bit PowerPC CPUs. Add infrastructure to support variable numbers of
SLB slots and move the user slot from 63 to 0, so that it is always
available.
|
| 222618 |
02-Jun-2011 |
nwhitehorn |
If running under a hypervisor, don't yell at the user about starting
unknown CPU types, instead relying on the hypervisor to have given us a
reasonable environment.
|
| 222616 |
02-Jun-2011 |
nwhitehorn |
Explicitly initialize the first thread's MSR to PSL_KERNSET.
|
| 222615 |
02-Jun-2011 |
nwhitehorn |
Include the modules area in the mapped kernel code. This fixes the kernel's
access to modules and loader metadata when started from real mode, but
without a direct map.
|
| 222614 |
02-Jun-2011 |
nwhitehorn |
Remove some dead code: unnecessary isyncs and memory sorting, which are
handled in mtmsr() and mem_regions(), respectively.
|
| 222613 |
02-Jun-2011 |
nwhitehorn |
MFpseries:
Renovate and improve the AIM Open Firmware support:
- Add RTAS (Run-Time Abstraction Services) support, found on all IBM systems
and some Apple ones
- Improve support for 32-bit real mode Open Firmware systems
- Pull some more OF bits over from the AIM directory
- Fix memory detection on IBM LPARs and systems with more than one /memory
node (by andreast@)
|
| 222309 |
26-May-2011 |
nwhitehorn |
Add a missing isync.
|
| 221981 |
16-May-2011 |
nwhitehorn |
Remove a useless check that served only to make 64-bit PPC systems
unbootable after r221855.
Submitted by: andreast
MFC after: 1 week
|
| 220818 |
19-Apr-2011 |
andreast |
Add leading zeros when printing the physical memory chunks on __powerpc64__.
Approved by: nwhitehorn (mentor)
|
| 220642 |
14-Apr-2011 |
andreast |
Adjust debugging string to match the actual function.
Approved by: nwhitehorn (mentor)
|
| 220639 |
14-Apr-2011 |
andreast |
The macro MOEA_PVO_CHECK is empty and not used. It is a left over from the
NetBSD import. Remove the definition and all its occurrences.
Approved by: nwhitehorn (mentor)
|
| 219523 |
11-Mar-2011 |
mdf |
Mostly revert r219468, as I had misremembered the C standard regarding
the size of an extern array.
Keep one change from strncpy to strlcpy.
|
| 219468 |
10-Mar-2011 |
mdf |
Use MAXPATHLEN rather than the size of an extern array when copying the
kernel name. Also consistenly use strlcpy().
Suggested by: Warner Losh
|
| 218824 |
18-Feb-2011 |
nwhitehorn |
Turn off default generation of userland dot symbols on powerpc64 now that
we have a binutils that supports it. Kernel dot symbols remain on to assist
DDB.
|
| 217896 |
26-Jan-2011 |
dchagin |
Add macro to test the sv_flags of any process. Change some places to test
the flags instead of explicit comparing with address of known sysentvec
structures.
MFC after: 1 month
|
| 217688 |
21-Jan-2011 |
pluknet |
Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize.
Submitted by: perryh pluto.rain.com (previous version)
Reviewed by: jhb
Approved by: kib (mentor)
Tested by: universe
|
| 217561 |
18-Jan-2011 |
kib |
For architectures not using direct map , and requiring real KVA page for
sf buf allocation, use wakeup() instead of wakeup_one() to notify sf
buffer waiters about free buffer.
sf_buf_alloc() calls msleep(PCATCH) when SFB_CATCH flag was given,
and for simultaneous wakeup and signal delivery, msleep() returns
EINTR/ERESTART despite the thread was selected for wakeup_one(). As
result, we loose a wakeup, and some other waiter will not be woken up.
Reported and tested by: az
Reviewed by: alc, jhb
MFC after: 1 week
|
| 217451 |
15-Jan-2011 |
andreast |
Remove unused variables. Spotted by a cppcheck
(devel/cppcheck, http://sourceforge.net/projects/cppcheck) run.
Approved by: nwhitehorn (mentor)
|
| 217341 |
13-Jan-2011 |
nwhitehorn |
Fix handling of NX pages on capable CPUs. Thanks to kib for prodding me
in the right direction.
|
| 217065 |
06-Jan-2011 |
andreast |
Remove unused variables. Spotted by a cppcheck
(devel/cppcheck, http://sourceforge.net/projects/cppcheck) run.
Approved by: nwhitehorn (mentor)
|
| 216765 |
28-Dec-2010 |
nwhitehorn |
Only keep track of PTE validity statistics for pages not locked in the
table. The 'locked' attribute is used to circumvent the regular page table
locking for some special pages, with the result that including locked pages
here causes races when updating the stats.
|
| 216563 |
19-Dec-2010 |
nwhitehorn |
Garbage-collect unused variable.
|
| 216383 |
11-Dec-2010 |
nwhitehorn |
Add some isync()s related to the 64-bit MMU scratch page to avoid race
conditions on its invalidation.
|
| 216174 |
04-Dec-2010 |
nwhitehorn |
Add an abstraction layer to the 64-bit AIM MMU's page table manipulation
logic to support modifying the page table through a hypervisor. This
uses KOBJ inheritance to provide subclasses of the base 64-bit AIM MMU
class with additional methods for page table manipulation.
Many thanks to Peter Grehan for suggesting this design and implementing
the MMU KOBJ inheritance mechanism.
|
| 215701 |
22-Nov-2010 |
dim |
After some off-list discussion, revert a number of changes to the
DPCPU_DEFINE and VNET_DEFINE macros, as these cause problems for various
people working on the affected files. A better long-term solution is
still being considered. This reversal may give some modules empty
set_pcpu or set_vnet sections, but these are harmless.
Changes reverted:
------------------------------------------------------------------------
r215318 | dim | 2010-11-14 21:40:55 +0100 (Sun, 14 Nov 2010) | 4 lines
Instead of unconditionally emitting .globl's for the __start_set_xxx and
__stop_set_xxx symbols, only emit them when the set_vnet or set_pcpu
sections are actually defined.
------------------------------------------------------------------------
r215317 | dim | 2010-11-14 21:38:11 +0100 (Sun, 14 Nov 2010) | 3 lines
Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.
------------------------------------------------------------------------
r215316 | dim | 2010-11-14 21:23:02 +0100 (Sun, 14 Nov 2010) | 2 lines
Add macros to define static instances of VNET_DEFINE and DPCPU_DEFINE.
|
| 215317 |
14-Nov-2010 |
dim |
Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.
|
| 215197 |
12-Nov-2010 |
nwhitehorn |
Partially revert r215182. There appears to be a silicon bug on the 970
that causes AP bringup to fail if some of the Cell HID-register code
is anywhere in the instruction stream. Pending a better solution, cache
performance on SMP Cell systems running without a hypervisor will be
suboptimal.
|
| 215182 |
12-Nov-2010 |
nwhitehorn |
Add CPU support code for the IBM Cell Broadband Engine.
|
| 215163 |
12-Nov-2010 |
nwhitehorn |
Remove use of a separate ofw_pmap on 32-bit CPUs. Many Open Firmware
mappings need to end up in the kernel anyway since the kernel begins
executing in OF context. Separating them adds needless complexity,
especially since the powerpc64 and mmu_oea64 code gave up on it a long
time ago.
As a side effect, the PPC ofw_machdep code is no longer AIM-specific,
so move it to powerpc/ofw.
|
| 215160 |
12-Nov-2010 |
nwhitehorn |
Remove or conditionalize some hypervisor-unfriendly instruction sequences.
|
| 215159 |
12-Nov-2010 |
nwhitehorn |
Add some platform KOBJ extensions and continue integrating PowerPC
hypervisor infrastructure support:
- Fix coexistence of multiple platform modules in the same kernel
- Allow platform modules to provide an SMP topology
- PowerPC hypervisors limit the amount of memory accessible in real mode.
Allow the platform modules to specify the maximum real-mode address,
and modify the bits of the kernel that need to allocate
real-mode-accessible buffers to respect this limits.
|
| 215158 |
12-Nov-2010 |
nwhitehorn |
Fix an error in r215067. An existing /chosen/mmu but missing translations
property just means we shouldn't add any translations, not that we should
panic.
|
| 215157 |
12-Nov-2010 |
nwhitehorn |
Centralize CPU idle routines into powerpc/cpu.c and use the same
cpu_idle_hook mechanism that x86 uses for overriding the idle routine.
This is required for supporting ilding the CPU under PowerPC hypervisors.
|
| 215121 |
11-Nov-2010 |
raj |
Fix typo in the comment.
|
| 215107 |
11-Nov-2010 |
nwhitehorn |
Add support for the IMISS, DLMISS, and DSMISS traps required to run
FreeBSD on a G2 core.
PR: powerpc/111296
Submitted by: Andrew Turner
|
| 215067 |
10-Nov-2010 |
nwhitehorn |
Make AIM early-boot code function correctly without Open Firmware.
|
| 214835 |
05-Nov-2010 |
jhb |
Adjust the order of operations in spinlock_enter() and spinlock_exit() to
work properly with single-stepping in a kernel debugger. Specifically,
these routines have always disabled interrupts before increasing the nesting
count and restored the prior state of interrupts after decreasing the nesting
count to avoid problems with a nested interrupt not disabling interrupts
when acquiring a spin lock. However, trap interrupts for single-stepping
can still occur even when interrupts are disabled. Now the saved state of
interrupts is not saved in the thread until after interrupts have been
disabled and the nesting count has been increased. Similarly, the saved
state from the thread cannot be read once the nesting count has been
decreased to zero. To fix this, use temporary variables to store interrupt
state and shuffle it between the thread's MD area and the appropriate
registers.
In cooperation with: bde
MFC after: 1 month
|
| 214749 |
03-Nov-2010 |
nwhitehorn |
Fix two mistakes on 32-bit systems. The slbmte code in syscall() is 64-bit
only, and should be protected with an ifdef, and the no-execute bit in
32-bit set_user_sr() should be set before the comparison, not after, or
it will never match.
|
| 214739 |
03-Nov-2010 |
nwhitehorn |
Clean up the user segment handling code a little more. Now that
set_user_sr() itself caches the user segment VSID, there is no need for
cpu_switch() to do it again. This change also unifies the 32 and 64-bit
code paths for kernel faults on user pages and remaps the user SLB slot
on 64-bit systems when taking a syscall to avoid some unnecessary segment
exception traps.
|
| 214617 |
01-Nov-2010 |
alc |
Implement pmap_is_prefaultable().
Reviewed by: nwhitehorn
|
| 214610 |
31-Oct-2010 |
nwhitehorn |
Add a security nit to recent copyin/out changes: map the user segment
no-execute in case of exploitable kernel bugs.
MFC after: 1 week
|
| 214607 |
31-Oct-2010 |
nwhitehorn |
Next-to-leading-order perturbation of synchronization operations for
switching the user segment register. All races should now be closed and
a minimum of pipelines flushes be required to close them.
|
| 214601 |
31-Oct-2010 |
nwhitehorn |
Add some missing parentheses so that moea_bat_mapped() actually works.
Submitted by: alc
MFC after: 3 days
|
| 214574 |
30-Oct-2010 |
nwhitehorn |
Restructure the way the copyin/copyout segment is stored to prevent a
concurrency bug. Since all SLB/SR entries were invalidated during an
exception, a decrementer exception could cause the user segment to be
invalidated during a copyin()/copyout() without a thread switch that
would cause it to be restored from the PCB, potentially causing the
operation to continue on invalid memory. This is now handled by explicit
restoration of segment 12 from the PCB on 32-bit systems and a check in
the Data Segment Exception handler on 64-bit.
While here, cause copyin()/copyout() to check whether the requested
user segment is already installed, saving some pipeline flushes, and
fix the synchronization primitives around the mtsr and slbmte
instructions to prevent accessing stale segments.
MFC after: 2 weeks
|
| 213456 |
05-Oct-2010 |
nwhitehorn |
Handle vector assist traps without a kernel panic, by setting denormalized
values to zero. A correct solution would involve emulating vector
operations on denormalized values, but this has little effect on accuracy
and is much less complicated for now.
MFC after: 2 weeks
|
| 213407 |
04-Oct-2010 |
nwhitehorn |
Follow exactly the steps in architecture manual for correctly invalidating
TLB entries instead of trying to cut corners.
|
| 213335 |
01-Oct-2010 |
nwhitehorn |
Fix pmap_page_set_memattr() behavior in the presence of fictitious pages
by just caching the mode for later use by pmap_enter(), following amd64.
While here, correct some mismerges from mmu_oea64 -> mmu_oea and clean
up some dead code found while fixing the fictitious page behavior.
|
| 213307 |
30-Sep-2010 |
nwhitehorn |
Add support for memory attributes (pmap_mapdev_attr() and friends) on
PowerPC/AIM. This is currently stubbed out on Book-E, since I have no
idea how to implement it there.
|
| 212722 |
16-Sep-2010 |
nwhitehorn |
Split the SLB mirror cache into two kinds of object, one for kernel maps
which are similar to the previous ones, and one for user maps, which
are arrays of pointers into the SLB tree. This changes makes user SLB
updates atomic, closing a window for memory corruption. While here,
rearrange the allocation functions to make context switches faster.
|
| 212715 |
16-Sep-2010 |
nwhitehorn |
Replace the SLB backing store splay tree used on 64-bit PowerPC AIM
hardware with a lockless sparse tree design. This marginally improves
the performance of PMAP and allows copyin()/copyout() to run without
acquiring locks when used on wired mappings.
Submitted by: mdf
|
| 212627 |
15-Sep-2010 |
grehan |
Introduce inheritance into the PowerPC MMU kobj interface.
include/mmuvar.h - Change the MMU_DEF macro to also create the class
definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT,
which has an extra parameter specifying the MMU class to inherit methods
from. Update the comments at the start of the header file to describe the
new macros.
booke/pmap.c
aim/mmu_oea.c
aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro
The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to
allow it to inherit the stock powerpc64 MMU methods.
Reviewed by: nwhitehorn
|
| 212597 |
14-Sep-2010 |
grehan |
Resurrect PSIM support by moving the cacheline size-detection warning
printf outside of the MMU-disabled region. A call into OpenFirmware
with the MMU off resulted in an internal PSIM assert.
|
| 212541 |
13-Sep-2010 |
mav |
Refactor timer management code with priority to one-shot operation mode.
The main goal of this is to generate timer interrupts only when there is
some work to do. When CPU is busy interrupts are generating at full rate
of hz + stathz to fullfill scheduler and timekeeping requirements. But
when CPU is idle, only minimum set of interrupts (down to 8 interrupts per
second per CPU now), needed to handle scheduled callouts is executed.
This allows significantly increase idle CPU sleep time, increasing effect
of static power-saving technologies. Also it should reduce host CPU load
on virtualized systems, when guest system is idle.
There is set of tunables, also available as writable sysctls, allowing to
control wanted event timer subsystem behavior:
kern.eventtimer.timer - allows to choose event timer hardware to use.
On x86 there is up to 4 different kinds of timers. Depending on whether
chosen timer is per-CPU, behavior of other options slightly differs.
kern.eventtimer.periodic - allows to choose periodic and one-shot
operation mode. In periodic mode, current timer hardware taken as the only
source of time for time events. This mode is quite alike to previous kernel
behavior. One-shot mode instead uses currently selected time counter
hardware to schedule all needed events one by one and program timer to
generate interrupt exactly in specified time. Default value depends of
chosen timer capabilities, but one-shot mode is preferred, until other is
forced by user or hardware.
kern.eventtimer.singlemul - in periodic mode specifies how much times
higher timer frequency should be, to not strictly alias hardclock() and
statclock() events. Default values are 2 and 4, but could be reduced to 1
if extra interrupts are unwanted.
kern.eventtimer.idletick - makes each CPU to receive every timer interrupt
independently of whether they busy or not. By default this options is
disabled. If chosen timer is per-CPU and runs in periodic mode, this option
has no effect - all interrupts are generating.
As soon as this patch modifies cpu_idle() on some platforms, I have also
refactored one on x86. Now it makes use of MONITOR/MWAIT instrunctions
(if supported) under high sleep/wakeup rate, as fast alternative to other
methods. It allows SMP scheduler to wake up sleeping CPUs much faster
without using IPI, significantly increasing performance on some highly
task-switching loads.
Tested by: many (on i386, amd64, sparc64 and powerc)
H/W donated by: Gheorghe Ardelean
Sponsored by: iXsystems, Inc.
|
| 212453 |
11-Sep-2010 |
mav |
Update PowerPC event timer code to use new event timers infrastructure.
Reviewed by: nwitehorn
Tested by: andreast
H/W donated by: Gheorghe Ardelean
|
| 212413 |
10-Sep-2010 |
avg |
bus_add_child: change type of order parameter to u_int
This reflects actual type used to store and compare child device orders.
Change is mostly done via a Coccinelle (soon to be devel/coccinelle)
semantic patch.
Verified by LINT+modules kernel builds.
Followup to: r212213
MFC after: 10 days
|
| 212363 |
09-Sep-2010 |
nwhitehorn |
Reorder statistics tracking and table lock acquisitions already in place
to avoid race conditions updating the PVO statistics.
|
| 212331 |
08-Sep-2010 |
nwhitehorn |
Fix a printf specifier on 64-bit systems.
|
| 212322 |
08-Sep-2010 |
nwhitehorn |
Fix a typo in the original import of this code from NetBSD that caused the
wrong element of the VSID bitmap array to be examined after a collision,
leading to reallocation of in-use VSIDs under some circumstances, with
attendant memory corruption. Also add an assert to check for this kind of
problem in the future.
MFC after: 4 days
|
| 212308 |
07-Sep-2010 |
nwhitehorn |
Fix an error made in r209975 related to context ID allocation for 64-bit
PowerPC CPUs running a 32-bit kernel. This bug could cause in-use VSIDs
to be allocated again to another process, causing memory space overlaps
and corruption.
Reported by: linimon
|
| 212278 |
06-Sep-2010 |
nwhitehorn |
Fix the same race condition on 32-bit AIM CPUs that was fixed for 64-bit
ones in r211967 involving VSID allocation.
|
| 212239 |
05-Sep-2010 |
mav |
Make nexus report name and compat fields as pnpinfo for devices on the
first level of hierarchy, same as done on deeper levels.
|
| 212054 |
31-Aug-2010 |
nwhitehorn |
Restructure how reset and poweroff are handled on PowerPC systems, since
the existing code was very platform specific, and broken for SMP systems
trying to reboot from KDB.
- Add a new PLATFORM_RESET() method to the platform KOBJ interface, and
migrate existing reset functions into platform modules.
- Modify the OF_reboot() routine to submit the request by hand to avoid
the IPIs involved in the regular openfirmware() routine. This fixes
reboot from KDB on SMP machines.
- Move non-KDB reset and poweroff functions on the Powermac platform
into the relevant power control drivers (cuda, pmu, smu), instead of
using them through the Open Firmware backdoor.
- Rename platform_chrp to platform_powermac since it has become
increasingly Powermac specific. When we gain support for IBM systems,
we will grow a new platform_chrp.
|
| 212053 |
31-Aug-2010 |
nwhitehorn |
Remove some code made obsolete by the powerpc64 import.
|
| 212044 |
31-Aug-2010 |
nwhitehorn |
Missed one place the SLB lock should be held in r211967.
|
| 211967 |
29-Aug-2010 |
nwhitehorn |
Avoid a race in the allocation of new segment IDs that could result in
memory corruption on heavily loaded SMP systems.
MFC after: 2 weeks
|
| 211861 |
27-Aug-2010 |
nwhitehorn |
pmap_mapdev() does not appear to actually need GIANT to be held here,
and asserting that is held breaks drm.
MFC after: 2 weeks
|
| 211515 |
19-Aug-2010 |
jhb |
Remove unused KTRACE includes.
|
| 210704 |
31-Jul-2010 |
nwhitehorn |
Improve hash coverage for kernel page table entries by modifying the kernel
ESID -> VSID map function. This makes ZFS run stably on PowerPC under
heavy loads (repeated simultaneous SVN checkouts and updates).
|
| 209975 |
13-Jul-2010 |
nwhitehorn |
MFppc64:
Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.
|
| 209950 |
12-Jul-2010 |
nwhitehorn |
Unify ABI-related bits of the Book-E and AIM machdep routines
(exec_setregs, etc.) in order to simplify the addition of 64-bit support,
and possible future extension of the Book-E code to handle hard floating
point and Altivec.
MFC after: 1 month
|
| 209853 |
09-Jul-2010 |
nwhitehorn |
The number after 2 is 3, not 4.
MFC after: 3 days
|
| 209852 |
09-Jul-2010 |
nwhitehorn |
Remove an unnecessary include of opt_psim.h, which is not present on
powerpc64.
|
| 209851 |
09-Jul-2010 |
nwhitehorn |
MFppc64:
Minor 64-bit-cleanliness upgrades and support for platform detection on
subtly-broken OF implementations like in the Mambo simulator.
|
| 209812 |
08-Jul-2010 |
nwhitehorn |
Replace the existing PowerPC busdma implementation with the one from
amd64 (with slight modifications). This provides support for bounce
buffers, which are required on systems with RAM above 4 GB.
|
| 209639 |
02-Jul-2010 |
marcel |
Remove the unneeded header <machine/intr.h>.
|
| 209613 |
30-Jun-2010 |
jhb |
Move prototypes for kern_sigtimedwait() and kern_sigprocmask() to
<sys/syscallsubr.h> where all other kern_<syscall> prototypes live.
|
| 209486 |
23-Jun-2010 |
nwhitehorn |
Configure interrupts on SMP systems to be distributed among all online
CPUs by default, and provide a functional version of BUS_BIND_INTR().
While here, fix some potential concurrency problems in the interrupt
handling code.
|
| 209369 |
20-Jun-2010 |
nwhitehorn |
Temporarily disable instruction relocation while setting up the kernel's
IBAT entry in early boot in order to prevent possible faults from races
between the instruction cache and the MMU.
PR: powerpc/148003
MFC after: 3 days
|
| 209298 |
18-Jun-2010 |
nwhitehorn |
Provide for multiple, cascaded PICs on PowerPC systems, and extend the
OFW interrupt map interface to also return the device's interrupt parent.
MFC after: 8.1-RELEASE
|
| 209114 |
12-Jun-2010 |
nwhitehorn |
Make SMP work on MPC7400-based Apple desktops like the PowerMac3,3.
|
| 209048 |
11-Jun-2010 |
alc |
Relax one of the new assertions in pmap_enter() a little. Specifically,
allow pmap_enter() to be performed on an unmanaged page that doesn't have
VPO_BUSY set. Having VPO_BUSY set really only matters for managed pages.
(See, for example, pmap_remove_write().)
|
| 208990 |
10-Jun-2010 |
alc |
Reduce the scope of the page queues lock and the number of
PG_REFERENCED changes in vm_pageout_object_deactivate_pages().
Simplify this function's inner loop using TAILQ_FOREACH(), and shorten
some of its overly long lines. Update a stale comment.
Assert that PG_REFERENCED may be cleared only if the object containing
the page is locked. Add a comment documenting this.
Assert that a caller to vm_page_requeue() holds the page queues lock,
and assert that the page is on a page queue.
Push down the page queues lock into pmap_ts_referenced() and
pmap_page_exists_quick(). (As of now, there are no longer any pmap
functions that expect to be called with the page queues lock held.)
Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever
be passed an unmanaged page. Assert this rather than returning "0"
and "FALSE" respectively.
ARM:
Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH().
Push down the page queues lock inside of pmap_clearbit(), simplifying
pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write().
Additionally, this allows for avoiding the acquisition of the page
queues lock in some cases.
PowerPC/AIM:
moea*_page_exits_quick() and moea*_page_wired_mappings() will never be
called before pmap initialization is complete. Therefore, the check
for moea_initialized can be eliminated.
Push down the page queues lock inside of moea*_clear_bit(),
simplifying moea*_clear_modify() and moea*_clear_reference().
The last parameter to moea*_clear_bit() is never used. Eliminate it.
PowerPC/BookE:
Simplify mmu_booke_page_exists_quick()'s control flow.
Reviewed by: kib@
|
| 208847 |
05-Jun-2010 |
nwhitehorn |
Correct a harmless typo introduced when copying code from mmu_oea64.
Submitted by: alc
MFC after: 8.1-RELEASE
|
| 208810 |
05-Jun-2010 |
alc |
Don't set PG_WRITEABLE in pmap_enter() unless the page is managed.
|
| 208574 |
26-May-2010 |
alc |
Push down page queues lock acquisition in pmap_enter_object() and
pmap_is_referenced(). Eliminate the corresponding page queues lock
acquisitions from vm_map_pmap_enter() and mincore(), respectively. In
mincore(), this allows some additional cases to complete without ever
acquiring the page queues lock.
Assert that the page is managed in pmap_is_referenced().
On powerpc/aim, push down the page queues lock acquisition from
moea*_is_modified() and moea*_is_referenced() into moea*_query_bit().
Again, this will allow some additional cases to complete without ever
acquiring the page queues lock.
Reorder a few statements in vm_page_dontneed() so that a race can't lead
to an old reference persisting. This scenario is described in detail by a
comment.
Correct a spelling error in vm_page_dontneed().
Assert that the object is locked in vm_page_clear_dirty(), and restrict the
page queues lock assertion to just those cases in which the page is
currently writeable.
Add object locking to vnode_pager_generic_putpages(). This was the one
and only place where vm_page_clear_dirty() was being called without the
object being locked.
Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call
to vm_page_clear_dirty().
Change vnode_pager_generic_putpages() to the modern-style of function
definition. Also, change the name of one of the parameters to follow
virtual memory system naming conventions.
Reviewed by: kib
|
| 208504 |
24-May-2010 |
alc |
Roughly half of a typical pmap_mincore() implementation is machine-
independent code. Move this code into mincore(), and eliminate the
page queues lock from pmap_mincore().
Push down the page queues lock into pmap_clear_modify(),
pmap_clear_reference(), and pmap_is_modified(). Assert that these
functions are never passed an unmanaged page.
Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m:
Contrary to what the comment says, pmap_mincore() is not simply an
optimization. Without a complete pmap_mincore() implementation,
mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED
because only the pmap can provide this information.
Eliminate the page queues lock from vfs_setdirty_locked_object(),
vm_pageout_clean(), vm_object_page_collect_flush(), and
vm_object_page_clean(). Generally speaking, these are all accesses
to the page's dirty field, which are synchronized by the containing
vm object's lock.
Reduce the scope of the page queues lock in vm_object_madvise() and
vm_page_dontneed().
Reviewed by: kib (an earlier version)
|
| 208453 |
23-May-2010 |
kib |
Reorganize syscall entry and leave handling.
Extend struct sysvec with three new elements:
sv_fetch_syscall_args - the method to fetch syscall arguments from
usermode into struct syscall_args. The structure is machine-depended
(this might be reconsidered after all architectures are converted).
sv_set_syscall_retval - the method to set a return value for usermode
from the syscall. It is a generalization of
cpu_set_syscall_retval(9) to allow ABIs to override the way to set a
return value.
sv_syscallnames - the table of syscall names.
Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding
the call to cpu_set_syscall_retval().
The new functions syscallenter(9) and syscallret(9) are provided that
use sv_*syscall* pointers and contain the common repeated code from
the syscall() implementations for the architecture-specific syscall
trap handlers.
Syscallenter() fetches arguments, calls syscall implementation from
ABI sysent table, and set up return frame. The end of syscall
bookkeeping is done by syscallret().
Take advantage of single place for MI syscall handling code and
implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and
PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the
thread is stopped at syscall entry or return point respectively. The
EXEC flag augments SCX and notifies debugger that the process address
space was changed by one of exec(2)-family syscalls.
The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are
changed to use syscallenter()/syscallret(). MIPS and arm are not
converted and use the mostly unchanged syscall() implementation.
Reviewed by: jhb, marcel, marius, nwhitehorn, stas
Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc),
stas (mips)
MFC after: 1 month
|
| 208405 |
21-May-2010 |
nwhitehorn |
Now that single-threaded access to firmware is enforced by
IPI_RENDEZVOUS, the ofw mutex is irrelevant.
|
| 208364 |
20-May-2010 |
nwhitehorn |
Fix a long-standing bug in the PowerPC OFW call function on SMP machines
where running ofwdump could cause hangs by forcing all secondary CPUs
into a busy wait with interrupts off during the call.
Following section 8.4 of the Open Firmware PowerPC processor binding,
the firmware is free to overwrite the system interrupt handlers during
OF calls, restoring the OS handlers on exit. On single CPU systems, this
process is invisible to the operating system. On multiple CPU systems,
taking any exception on a secondary CPU while an OF call is in progress
ends with that exception vectored into OF, resulting in a slow movement
of the entire system into firmware context and a machine hang.
MFC after: 3 days
|
| 208175 |
16-May-2010 |
alc |
On entry to pmap_enter(), assert that the page is busy. While I'm
here, make the style of assertion used by pmap_enter() consistent
across all architectures.
On entry to pmap_remove_write(), assert that the page is neither
unmanaged nor fictitious, since we cannot remove write access to
either kind of page.
With the push down of the page queues lock, pmap_remove_write() cannot
condition its behavior on the state of the PG_WRITEABLE flag if the
page is busy. Assert that the object containing the page is locked.
This allows us to know that the page will neither become busy nor will
PG_WRITEABLE be set on it while pmap_remove_write() is running.
Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly
do copy-on-write-based zero-copy transmit on unmanaged or fictitious
pages, so don't even try. Previously, the call to pmap_remove_write()
would have failed silently.
|
| 208172 |
16-May-2010 |
nwhitehorn |
Pull OF_quiesce() out of the MI Open Firmware layer and entirely into
PPC ofw_machdep.c, in recognition of its state as a machine specific hack.
Requested by: marius
|
| 208152 |
16-May-2010 |
nwhitehorn |
On PowerMac11,2 and (presumably) PowerMac12,1, we need to quiesce the
firmware in order to take over control of the SMU. Without doing this,
the firmware background process doing fan control will run amok as we
take over the system and crash the management chip.
This is limited to these two machines because our kernel is heavily
dependent on firmware accesses, and so quiescing firmware can cause
nasty problems.
|
| 207796 |
08-May-2010 |
alc |
Push down the page queues into vm_page_cache(), vm_page_try_to_cache(), and
vm_page_try_to_free(). Consequently, push down the page queues lock into
pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and
pmap_remove_write().
Push down the page queues lock into Xen's pmap_page_is_mapped(). (I
overlooked the Xen pmap in r207702.)
Switch to a per-processor counter for the total number of pages cached.
|
| 207410 |
30-Apr-2010 |
kmacy |
On Alan's advice, rather than do a wholesale conversion on a single
architecture from page queue lock to a hashed array of page locks
(based on a patch by Jeff Roberson), I've implemented page lock
support in the MI code and have only moved vm_page's hold_count
out from under page queue mutex to page lock. This changes
pmap_extract_and_hold on all pmaps.
Supported by: Bitgravity Inc.
Discussed with: alc, jeffr, and kib
|
| 207155 |
24-Apr-2010 |
alc |
Resurrect pmap_is_referenced() and use it in mincore(). Essentially,
pmap_ts_referenced() is not always appropriate for checking whether or
not pages have been referenced because it clears any reference bits
that it encounters. For example, in mincore(), clearing the reference
bits has two negative consequences. First, it throws off the activity
count calculations performed by the page daemon. Specifically, a page
on which mincore() has called pmap_ts_referenced() looks less active
to the page daemon than it should. Consequently, the page could be
deactivated prematurely by the page daemon. Arguably, this problem
could be fixed by having mincore() duplicate the activity count
calculation on the page. However, there is a second problem for which
that is not a solution. In order to clear a reference on a 4KB page,
it may be necessary to demote a 2/4MB page mapping. Thus, a mincore()
by one process can have the side effect of demoting a superpage
mapping within another process!
|
| 205642 |
25-Mar-2010 |
nwhitehorn |
Change the arguments of exec_setregs() so that it receives a pointer
to the image_params struct instead of several members of that struct
individually. This makes it easier to expand its arguments in the future
without touching all platforms.
Reviewed by: jhb
|
| 205506 |
23-Mar-2010 |
nwhitehorn |
Get nexus(4) out of the RTC business. The interface used by nexus(4)
in Open Firmware was Apple-specific, and we have complete coverage of Apple
system controllers, so move RTC responsibilities into the system controller
drivers. This avoids interesting problems from manipulating these devices
through Open Firmware behind the backs of their drivers.
Obtained from: NetBSD
MFC after: 2 weeks
|
| 205497 |
23-Mar-2010 |
nwhitehorn |
Open Firmware on powerpc is generally non-reetrant, so serialize all
OF calls with a mutex.
|
| 205370 |
20-Mar-2010 |
nwhitehorn |
Revisit locking in the 64-bit AIM PMAP. The PVO head for a page is
generally protected by the VM page queue mutex. Instead of extending the
table lock to cover the PVO heads, add some asserts that the page queue
mutex is in fact held. This fixes several LORs and possible deadlocks.
This also adds an optimization to moea64_kextract() useful for
direct-mapped quantities, like UMA buffers. Being able to use this from
inside UMA removes an additional LOR.
|
| 205163 |
15-Mar-2010 |
nwhitehorn |
Fix two small bugs. The PowerPC 970 does not support non-coherent memory
access, and reflects this by autonomously writing LPTE_M into PTE entries.
As such, we should not panic if LPTE_M changes by itself. While here,
fix a harmless typo in moea64_sync_icache().
|
| 204903 |
09-Mar-2010 |
nwhitehorn |
Place interrupt handling in a critical section and remove double
counting in incrementing the interrupt nesting level. This fixes a number
of bugs in which the interrupt thread could be preempted by an IPI,
indefinitely delaying acknowledgement of the interrupt to the PIC, causing
interrupt starvation and hangs.
Reported by: linimon
Reviewed by: marcel, jhb
MFC after: 1 week
|
| 204719 |
04-Mar-2010 |
nwhitehorn |
Fix an obvious lock escape and fix a typo in a comment.
|
| 204694 |
04-Mar-2010 |
nwhitehorn |
Patch some more concurrency issues here. This expands the page table
lock to cover the PVOs, and removes the scratchpage PTEs from the PVOs
entirely to avoid the system trying to be helpful and rewriting them.
|
| 204640 |
03-Mar-2010 |
joel |
The NetBSD Foundation has granted permission to remove clause 3 and 4 from
their software.
Obtained from: NetBSD
|
| 204297 |
25-Feb-2010 |
nwhitehorn |
Move the OEA64 scratchpage to the end of KVA from the beginning, and set
its PVO to map physical address 0 instead of kernelstart. This fixes a
situation in which a user process could attempt to return this address
via KVM, have it fault while being modified, and then panic the kernel
because (a) it is supposed to map a valid address and (b) it lies in the
no-fault region between VM_MIN_KERNEL_ADDRESS and virtual_avail.
While here, move msgbuf and dpcpu make into regular KVA space for
consistency with other implementations.
|
| 204296 |
25-Feb-2010 |
nwhitehorn |
Provide an implementation of pmap_dev_direct_mapped() on OEA64. This is
required in order to be able to mmap the running kernel, which is turn
required to avoid fstat returning gibberish.
|
| 204269 |
24-Feb-2010 |
nwhitehorn |
Use dcbz instead of word stores for page zeroing, providing a factor of
3-4 speedup.
|
| 204268 |
24-Feb-2010 |
nwhitehorn |
Close a race involving the OEA64 scratchpage. When the scratch page's
physical address is changed, there is a brief window during which its PTE
is invalid. Since moea64_set_scratchpage_pa() does not and cannot hold
the page table lock, it was possible for another CPU to insert a new PTE
into the scratch page's PTEG slot during this interval, corrupting both
mappings.
Solve this by creating a new flag, LPTE_LOCKED, such that
moea64_pte_insert will avoid claiming locked PTEG slots even if they
are invalid. This change also incorporates some additional paranoia
added to solve things I thought might be this bug.
Reported by: linimon
|
| 204197 |
22-Feb-2010 |
nwhitehorn |
Allow user programs to execute mfpvr instructions. Linux allows this, and
some math-related software like GMP expects to be able to use it to pick
a target appropriately.
MFC after: 1 week
|
| 204128 |
20-Feb-2010 |
nwhitehorn |
Reduce KVA pressure on OEA64 systems running in bridge mode by mapping
UMA segments at their physical addresses instead of into KVA. This emulates
the direct mapping behavior of OEA32 in an ad-hoc way. To make this work
properly required sharing the entire kernel PMAP with Open Firmware, so
ofw_pmap is transformed into a stub on 64-bit CPUs.
Also implement some more tweaks to get more mileage out of our limited
amount of KVA, principally by extending KVA into segment 16 until the
beginning of the first OFW mapping.
Reported by: linimon
|
| 204042 |
18-Feb-2010 |
nwhitehorn |
Fix a bug where pages being removed from memory entirely no longer have
PVOs, and so the modified state of the page can no longer be communicated
to the VM layer, causing pages not to be flushed to swap when needed, in
turn causing memory corruption. Also make several correctness adjustments
to I-Cache synchronization and TLB invalidation for 64-bit Book-S CPUs.
Obtained from: projects/ppc64
Discussed with: grehan
MFC after: 2 weeks
|
| 201758 |
07-Jan-2010 |
mbr |
Remove extraneous semicolons, no functional changes.
Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week
|
| 200083 |
03-Dec-2009 |
nwhitehorn |
The first argument of dcbz interprets r0 as a literal zero, not the second.
This worked before by accident.
MFC after: 1 week
|
| 199886 |
28-Nov-2009 |
nwhitehorn |
Add a CPU features framework on PowerPC and simplify CPU setup a little
more. This provides three new sysctls to user space:
hw.cpu_features - A bitmask of available CPU features
hw.floatingpoint - Whether or not there is hardware FP support
hw.altivec - Whether or not Altivec is available
PR: powerpc/139154
MFC after: 10 days
|
| 199868 |
27-Nov-2009 |
alc |
Simplify the invocation of vm_fault(). Specifically, eliminate the flag
VM_FAULT_DIRTY. The information provided by this flag can be trivially
inferred by vm_fault().
Discussed with: kib
|
| 199669 |
22-Nov-2009 |
nwhitehorn |
Garbage collect some code that was never compiled in to handle Altivec
during traps. It predates actual Altivec support and was never used.
|
| 199226 |
12-Nov-2009 |
nwhitehorn |
Provide a real fix to the too-many-translations problem when booting
from CD on 64-bit hardware to replace existing band-aids. This occurred
when the preloaded mdroot required too many mappings for the static
buffer.
Since we only use the translations buffer once, allocate a dynamic
buffer on the stack. This early in the boot process, the call chain
is quite short and we can be assured of having sufficient stack space.
Reviewed by: grehan
|
| 199135 |
10-Nov-2009 |
kib |
Extract the code that records syscall results in the frame into MD
function cpu_set_syscall_retval().
Suggested by: marcel
Reviewed by: marcel, davidxu
PowerPC, ARM, ia64 changes: marcel
Sparc64 tested and reviewed by: marius, also sunv reviewed
MIPS tested by: gonzo
MFC after: 1 month
|
| 199108 |
09-Nov-2009 |
nwhitehorn |
Spell sz correctly.
Pointed out by: jmallett
|
| 199084 |
09-Nov-2009 |
nwhitehorn |
Increase the size of the OFW translations buffer to handle G5 systems
that use many translation regions in firmware, and add bounds checking
to prevent buffer overflows in case even the new value is exceeded.
Reported by: Jacob Lambert
MFC after: 3 days
|
| 198731 |
31-Oct-2009 |
nwhitehorn |
Unbreak cpu_switch(). The register allocator in my brain is clearly
broken. Also, Altivec context switching worked before only by accident,
but should work now by design.
|
| 198725 |
31-Oct-2009 |
nwhitehorn |
Remove an unnecessary sync that crept in the last commit.
|
| 198724 |
31-Oct-2009 |
nwhitehorn |
Fix a race in casuword() exposed by csup. casuword() non-atomically read
the current value of its argument before atomically replacing it, which
could occasionally return the wrong value on an SMP system. This resulted
in user mutex operations hanging when using threaded applications.
|
| 198723 |
31-Oct-2009 |
nwhitehorn |
Loop on blocked threads when using ULE scheduler, removing an
XXX MP comment.
|
| 198722 |
31-Oct-2009 |
nwhitehorn |
Garbage collect set_user_sr(), which is declared static inline and
never called.
|
| 198588 |
29-Oct-2009 |
nwhitehorn |
Turn off Altivec data-stream prefetching before going into power-save
mode on those CPUs that need it.
|
| 198507 |
27-Oct-2009 |
kib |
In r197963, a race with thread being selected for signal delivery
while in kernel mode, and later changing signal mask to block the
signal, was fixed for sigprocmask(2) and ptread_exit(3). The same race
exists for sigreturn(2), setcontext(2) and swapcontext(2) syscalls.
Use kern_sigprocmask() instead of direct manipulation of td_sigmask to
reschedule newly blocked signals, closing the race.
Reviewed by: davidxu
Tested by: pho
MFC after: 1 month
|
| 198428 |
23-Oct-2009 |
nwhitehorn |
Remove debugging printf that snuck in here.
Pointy hat to: me
|
| 198427 |
23-Oct-2009 |
nwhitehorn |
Add some more paranoia to setting HID registers, and update the AIM
clock routines to work better with SMP. This makes SMP work fully and
stably on an Xserve G5.
Obtained from: Book-E (clock bits)
|
| 198400 |
23-Oct-2009 |
nwhitehorn |
Do not map the trap vectors into the kernel's address space. They are
only used in real mode and keeping them mapped only serves to make NULL
a valid address, which results in silent NULL pointer deferences.
Suggested by: Patrick Kerharo
Obtained from: projects/ppc64
|
| 198378 |
23-Oct-2009 |
nwhitehorn |
Add SMP support on U3-based G5 systems. This does not yet work perfectly:
at least on my Xserve, getting the decrementer and timebase on APs to tick
requires setting up a clock chip over I2C, which is not yet done.
While here, correct the 64-bit tlbie function to set the CPU to 64-bit
mode correctly.
Hardware donated by: grehan
|
| 198341 |
21-Oct-2009 |
marcel |
o Introduce vm_sync_icache() for making the I-cache coherent with
the memory or D-cache, depending on the semantics of the platform.
vm_sync_icache() is basically a wrapper around pmap_sync_icache(),
that translates the vm_map_t argumument to pmap_t.
o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc
it replaces the pmap_page_executable() function, added to solve
the I-cache problem in uiomove_fromphys().
o In proc_rwmem() call vm_sync_icache() when writing to a page that
has execute permissions. This assures that when breakpoints are
written, the I-cache will be coherent and the process will actually
hit the breakpoint.
o This also fixes the Book-E PMAP implementation that was missing
necessary locking while trying to deal with the I-cache coherency
in pmap_enter() (read: mmu_booke_enter_locked).
The key property of this change is that the I-cache is made coherent
*after* writes have been done. Doing it in the PMAP layer when adding
or changing a mapping means that the I-cache is made coherent *before*
any writes happen. The difference is key when the I-cache prefetches.
|
| 198212 |
18-Oct-2009 |
nwhitehorn |
Don't assume that physical addresses are identity mapped. This allows
the second processor on G5 systems to start. Note that SMP is still
non-functional on these systems because of IPI delivery problems.
|
| 197962 |
11-Oct-2009 |
nwhitehorn |
Correct another typo. Actually save the condition register instead
of overwriting r12 by mistake.
|
| 197961 |
11-Oct-2009 |
nwhitehorn |
Correct a typo here and actually save DSISR instead of overwriting it.
|
| 195632 |
12-Jul-2009 |
nwhitehorn |
Increase the size of the page table on 64-bit PowerPC machines as a
bandaid to prevent exhaustion of the primary and secondary hash groups
in the event of extreme stress on the PMAP layer (e.g. a forkbomb). This
wastes memory, and should be revised to properly handle PTEG spills instead.
Suggested by: grehan
Approved by: re (kensmith)
|
| 194784 |
23-Jun-2009 |
jeff |
Implement a facility for dynamic per-cpu variables.
- Modules and kernel code alike may use DPCPU_DEFINE(),
DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined
PCPU_*. Requires only one extra instruction more than PCPU_* and is
virtually the same as __thread for builtin and much faster for shared
objects. DPCPU variables can be initialized when defined.
- Modules are supported by relocating the module's per-cpu linker set
over space reserved in the kernel. Modules may fail to load if there
is insufficient space available.
- Track space available for modules with a one-off extent allocator.
Free may block for memory to allocate space for an extent.
Reviewed by: jhb, rwatson, kan, sam, grehan, marius, marcel, stas
|
| 193909 |
10-Jun-2009 |
grehan |
Get the gdb/psim emulator functioning again.
aim/machdep.c:
- the RI status register bit needs to be set when doing the mtmsrd 64-bit
instruction test
- psim doesn't implement the dcbz instruction so the run-time cacheline
test fails. Set the cachline size to 32 to avoid infinite loops in
future calls to __syncicache()
aim/platform_chrp.c:
- if after iterating through / and a name property of "cpus" still isn't
found, just search directly for '/cpus'.
- psim doesn't put a "reg" property on it's cpu nodes, so assume 0
since it is uniprocessor-only at this point
powerpc/openpic.c
- the number of CPUs reported is 1 too many on psim's openpic
Reviewed by: nwhitehorn
MFC after: 1 week (openpic part)
|
| 193156 |
31-May-2009 |
nwhitehorn |
Introduce support for cpufreq on PowerPC with the dynamic frequency
switching capabilities of the MPC7447A and MPC7448.
|
| 192323 |
18-May-2009 |
marcel |
Add cpu_flush_dcache() for use after non-DMA based I/O so that a
possible future I-cache coherency operation can succeed. On ARM
for example the L1 cache can be (is) virtually mapped, which
means that any I/O that uses temporary mappings will not see the
I-cache made coherent. On ia64 a similar behaviour has been
observed. By flushing the D-cache, execution of binaries backed
by md(4) and/or NFS work reliably.
For Book-E (powerpc), execution over NFS exhibits SIGILL once in
a while as well, though cpu_flush_dcache() hasn't been implemented
yet.
Doing an explicit D-cache flush as part of the non-DMA based I/O
read operation eliminates the need to do it as part of the
I-cache coherency operation itself and as such avoids pessimizing
the DMA-based I/O read operations for which D-cache are already
flushed/invalidated. It also allows future optimizations whereby
the bcopy() followed by the D-cache flush can be integrated in a
single operation, which could be implemented using on-chips DMA
engines, by-passing the D-cache altogether.
|
| 192109 |
14-May-2009 |
raj |
PowerPC common SMP startup and time base rework.
- make mftb() shared, rewrite in C, provide complementary mttb()
- adjust SMP startup per the above, additional comments, minor naming
changes
- eliminate redundant TB defines, other minor cosmetics
Reviewed by: marcel, nwhitehorn
Obtained from: Freescale, Semihalf
|
| 192067 |
14-May-2009 |
nwhitehorn |
Factor out platform dependent things unrelated to device drivers into a
new platform module. These are probed in early boot, and have the
responsibility of determining the layout of physical memory, determining
the CPU timebase frequency, and handling the zoo of SMP mechanisms
found on PowerPC.
Reviewed by: marcel, raj
Book-E parts by: raj
|
| 191455 |
24-Apr-2009 |
raj |
Zero PCB during early AIM PowerPC init.
When memory is not zero'ed by firmware, uninitialized PCB can have bogus
contents, which appear as a saved onfault condition, Altivec context to
restore etc. and lead to corruption/crashes. This commit fixes such issues.
Submitted by: Michal Mazur arg ! semihalf dot com
Tested by: Andreas Tobler andreast-list ! fgznet dot ch
|
| 191261 |
19-Apr-2009 |
nwhitehorn |
Fix a typo in the SRR1 comparison for program exceptions. While here,
replace magic numbers with constants to keep this from happening again.
Without this fix, some programs would occasionally get SIGTRAP instead
of SIGILL on an illegal instruction. This affected Altivec detection
in pixman, and possibly other software.
Reported by: Andreas Tobler
MFC after: 1 week
|
| 191039 |
14-Apr-2009 |
nwhitehorn |
Changing the overflow trap to use bla to branch to dbtrap in r190946 was
bogus. Revert to a branch that does not set LR. It's been a long week...
|
| 190953 |
12-Apr-2009 |
nwhitehorn |
Rework the way we get the cacheline size. Instead of having a table of
CPUs known to use 128 byte cache lines and defaulting to 32, use the dcbz
instruction to measure it. Also make dcbz behave the way you would
expect on PPC 970.
|
| 190946 |
11-Apr-2009 |
nwhitehorn |
Fix recognition of kernel-mode traps that pass through the KDB trap handler
but do not actually invoke KDB. This includes recoverable machine checks
encountered in kernel mode.
This patch causes machines with Grackle host-PCI bridges to be able to
correctly enumerate them again.
MFC after: 3 days
|
| 190750 |
05-Apr-2009 |
nwhitehorn |
Fix the build when KDB is disabled. The second instance of rfi in
trap_subr.S that is patched at runtime to rfid on 64-bit systems
is inside KDB-specific code, so don't patch it without KDB.
|
| 190704 |
04-Apr-2009 |
marcel |
Perform a dummy stwcx. when we switch contexts. The context
being switched out may hold a reservation. The stwcx. will
clear the reservation. This is architecturally recommended.
The scenario this addresses is as follows:
1. Thread 1 performs a lwarx and as such holds a reservation.
2. Thread 1 gets switched out (before doing the matching
stwcx.) and thread 2 is switched in.
3. Thread 2 performs a stwcx. to the same reservation granule.
This will succeed because the processor has the reservation
even though thread 2 didn't do the lwarx.
Note that on some processors the address given the stwcx. is
not checked. On these processors the mere condition of having
a reservation would cause the stwcx. to succeed, irrespective
of whether the addresses are the same. The dummy stwcx. is
especially important for those processors.
|
| 190681 |
04-Apr-2009 |
nwhitehorn |
Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode
provided, for example, on the PowerPC 970 (G5), as well as on related CPUs
like the POWER3 and POWER4.
This also adds support for various built-in hardware found on Apple G5
hardware (e.g. the IBM CPC925 northbridge).
Reviewed by: grehan
|
| 189675 |
11-Mar-2009 |
nwhitehorn |
Change the PVO zone for fictitious pages to the unmanaged PVO zone, to match
the unmanaged flag set in the PVO attributes. Without doing this,
pmap_remove() could try to remove fictitious pages (like those created
by mmap of physical memory) from the wrong UMA zone, causing a panic.
Reported by: Justin Hibbits
MFC after: 1 week
|
| 188951 |
23-Feb-2009 |
nwhitehorn |
Fix comment: we write the trap vector to SPRG3, not SPRG0.
|
| 188860 |
20-Feb-2009 |
nwhitehorn |
Add Altivec support for supported CPUs. This is derived from the FPU support
code, and also reducing the size of trapcode to fit inside a 32 byte handler
slot.
Reviewed by: grehan
MFC after: 2 weeks
|
| 186347 |
20-Dec-2008 |
nwhitehorn |
Modularize the Open Firmware client interface to allow run-time switching
of OFW access semantics, in order to allow future support for real-mode
OF access and flattened device frees. OF client interface modules are
implemented using KOBJ, in a similar way to the PPC PMAP modules.
Because we need Open Firmware to be available before mutexes can be used on
sparc64, changes are also included to allow KOBJ to be used very early in
the boot process by only using the mutex once we know it has been initialized.
Reviewed by: marius, grehan
|
| 184319 |
27-Oct-2008 |
marcel |
Add support for kernel profiling for both AIM and BookE.
Obtained from: Juniper Networks, Inc (BookE support).
|
| 183882 |
14-Oct-2008 |
nwhitehorn |
Convert PowerPC AIM PCI and nexus busses to standard OFW bus interface. This
simplifies certain device attachments (Kauai ATA, for instance), and makes
possible others on new hardware.
On G5 systems, there are several otherwise standard PCI devices
(Serverworks SATA) that will not allow their interrupt properties to be
written, so this information must be supplied directly from Open Firmware.
Obtained from: sparc64
|
| 183319 |
24-Sep-2008 |
nwhitehorn |
Allow the cacheline size on PowerPC to be set at runtime. This is essential for
supporting 64-bit CPUs, which often have 128-byte cache lines instead of the
standard 32.
|
| 183290 |
23-Sep-2008 |
nwhitehorn |
In preparation for PowerPC G5 support, allow PVO objects to contain page
table entries for both the 32-bit and 64-bit AIM MMUs.
|
| 183094 |
16-Sep-2008 |
marcel |
o When not making a translation cache-inhibit and guarded (PTE_I|PTE_G)
make it memory-coherency enforced (PTE_M). This is required for SMP
to work.
o Serialize tlbie operations and implement the tlbie operation in a
function called tlbie(). Hardware can end up in a live-lock if
between the tlbsync and subsequent sync on one processor another
processor executes a tlbie or tlbsync.
o Eliminate the following defines:
TLBIE, TLBSYNC, SYNC and EIEIO
Use either inline assembly statements or inline functions defined
in <machine/cpufunc.h>
|
| 183090 |
16-Sep-2008 |
marcel |
Rewrite cpudep_ap_bootstrap(). We now enable L3, L2, L1D and L1I
caches if not yet enabed. This is required for coherency and
atomic operations to work, not to mention performance. We use the
L2 and L3 cache settings of the BSP to configure the APs caches.
Can't be bad.
Program NAP and not DOZE. DOZE is present only on earlier CPUs
and the bit is reserved on the MPC7441 & MPC7451. NAP will do
bus snooping to keep caches coherent.
Program the PIR with the cpuid. This may not be necessary...
|
| 183089 |
16-Sep-2008 |
marcel |
o In decr_get_timecount() only read the low timebase register.
We're only returning a 32-bit counter.
o In decr_intr(), manually perform LICM, so that we don't test
a loop invariant condition inside a loop.
o Include <machine/smp.h>
|
| 183088 |
16-Sep-2008 |
marcel |
Set pcpup->pc_curthread and pcpup->pc_curpcb before calling
pmap_activate. While pmap_activate doesn't need either, we
do need a valid curthread if we enable KTR_PMAP.
|
| 183060 |
16-Sep-2008 |
marcel |
Remove the tracing from the AP startup. The AP is known
to start and the tracing can interfere with AP startup.
Instead, use the available space in the reset vector
for the initial stack.
|
| 183030 |
15-Sep-2008 |
marcel |
Dont worry about PSL_RI (restartable interrupt indicator) in
common PowerPC code when all we want to achieve is to enable
external interrupts. We can set PSL_RI at any time before we
allow interrupts and/or exceptions, so move it to the AIM
specific initialization and do it when we also set PSL_ME
(machine check enable).
|
| 182580 |
31-Aug-2008 |
marcel |
Trace interrupts with KTR_INTR.
|
| 182573 |
31-Aug-2008 |
marcel |
Remove redundant KTR statements.
|
| 182486 |
30-Aug-2008 |
marcel |
Whitespace fixes.
|
| 182485 |
30-Aug-2008 |
marcel |
Call powerpc_sync() instead of using an asm statement.
|
| 182483 |
30-Aug-2008 |
marcel |
Don't clear PSL_RI. Disabling external interrupts
doesn't make exceptions unrecoverable.
|
| 179533 |
04-Jun-2008 |
grehan |
Add link register to fatal trap printout to better diagnose NULL
function pointer derefs.
|
| 179254 |
23-May-2008 |
marcel |
Invalidate the TLB in pmap_cpu_bootstrap(), so that it also happens
on the APs.
|
| 179229 |
23-May-2008 |
alc |
The VM system no longer uses setPQL2(). Remove it and its helpers.
|
| 178629 |
28-Apr-2008 |
marcel |
The first argment of mtdbatu or mtibatu is part of the encoding.
It needs to be constant, so eliminate the loop and "hand-unroll".
|
| 178628 |
27-Apr-2008 |
marcel |
MFp4: SMP support
|
| 178621 |
27-Apr-2008 |
marcel |
Make sure tmpstk is aligned and make it 8KB in size -- not 8KB+16.
|
| 178471 |
25-Apr-2008 |
jeff |
- Add an integer argument to idle to indicate how likely we are to wake
from idle over the next tick.
- Add a new MD routine, cpu_wake_idle() to wakeup idle threads who are
suspended in cpu specific states. This function can fail and cause the
scheduler to fall back to another mechanism (ipi).
- Implement support for mwait in cpu_idle() on i386/amd64 machines that
support it. mwait is a higher performance way to synchronize cpus
as compared to hlt & ipis.
- Allow selecting the idle routine by name via sysctl machdep.idle. This
replaces machdep.cpu_idle_hlt. Only idle routines supported by the
current machine are permitted.
Sponsored by: Nokia
|
| 178429 |
22-Apr-2008 |
phk |
Now that all platforms use genclock, shuffle things around slightly
for better structure.
Much of this is related to <sys/clock.h>, which should really have
been called <sys/calendar.h>, but unless and until we need the name,
the repocopy can wait.
In general the kernel does not know about minutes, hours, days,
timezones, daylight savings time, leap-years and such. All that
is theoretically a matter for userland only.
Parts of kernel code does however care: badly designed filesystems
store timestamps in local time and RTC chips almost universally
track time in a YY-MM-DD HH:MM:SS format, and sometimes in local
timezone instead of UTC. For this we have <sys/clock.h>
<sys/time.h> on the other hand, deals with time_t, timeval, timespec
and so on. These know only seconds and fractions thereof.
Move inittodr() and resettodr() prototypes to <sys/time.h>.
Retain the names as it is one of the few surviving PDP/VAX references.
Move startrtclock() to <machine/clock.h> on relevant platforms, it
is a MD call between machdep.c/clock.c. Remove references to it
elsewhere.
Remove a lot of unnecessary <sys/clock.h> includes.
Move the machdep.disable_rtc_set sysctl to subr_rtc.c where it belongs.
XXX: should be kern.disable_rtc_set really, it's not MD.
|
| 178367 |
21-Apr-2008 |
marcel |
Switch to using genclock. Have nexus double as clock device for
now. While here, add a proper attach() method to nexus.
Requested by: phk
|
| 178265 |
17-Apr-2008 |
marcel |
Simplify the pmap_zero_page family of functions by making use of
the fact that we have a 1:1 mapping by virtue of the BATs.
Eliminate the now unused moea_rkva_alloc(), moea_pa_map() and
moea_pa_unmap() functions.
Pointed out by: grehan.
|
| 178261 |
16-Apr-2008 |
marcel |
Allocate a stack (with optional guard pages) for thread0 and
switch to it before calling mi_startup().
|
| 177642 |
26-Mar-2008 |
phk |
The "free-lance" timer in the i8254 is only used for the speaker
these days, so de-generalize the acquire_timer/release_timer api
to just deal with speakers.
The new (optional) MD functions are:
timer_spkr_acquire()
timer_spkr_release()
and
timer_spkr_setfreq()
the last of which configures the timer to generate a tone of a given
frequency, in Hz instead of 1/1193182th of seconds.
Drop entirely timer2 on pc98, it is not used anywhere at all.
Move sysbeep() to kern/tty_cons.c and use the timer_spkr*() if
they exist, and do nothing otherwise.
Remove prototypes and empty acquire-/release-timer() and sysbeep()
functions from the non-beeping archs.
This eliminate the need for the speaker driver to know about
i8254frequency at all. In theory this makes the speaker driver MI,
contingent on the timer_spkr_*() functions existing but the driver
does not know this yet and still attaches to the ISA bus.
Syscons is more tricky, in one function, sc_tone(), it knows the hz
and things are just fine.
In the other function, sc_bell() it seems to get the period from
the KDMKTONE ioctl in terms if 1/1193182th second, so we hardcode
the 1193182 and leave it at that. It's probably not important.
Change a few other sysbeep() uses which obviously knew that the
argument was in terms of i8254 frequency, and leave alone those
that look like people thought sysbeep() took frequency in hertz.
This eliminates the knowledge of i8254_freq from all but the actual
clock.c code and the prof_machdep.c on amd64 and i386, where I think
it would be smart to ask for help from the timecounters anyway [TBD].
|
| 177253 |
16-Mar-2008 |
rwatson |
In keeping with style(9)'s recommendations on macros, use a ';'
after each SYSINIT() macro invocation. This makes a number of
lightweight C parsers much happier with the FreeBSD kernel
source, including cflow's prcc and lxr.
MFC after: 1 month
Discussed with: imp, rink
|
| 177091 |
12-Mar-2008 |
jeff |
Remove kernel support for M:N threading.
While the KSE project was quite successful in bringing threading to
FreeBSD, the M:N approach taken by the kse library was never developed
to its full potential. Backwards compatibility will be provided via
libmap.conf for dynamically linked binaries and static binaries will
be broken.
|
| 176919 |
07-Mar-2008 |
marcel |
For AIM, have cpu_idle() set MSR_POW when the powerpc_pow_enabled
variable is set. On my Mac Mini this puts the CPU in NAP mode when
the kernel is idle and, any technical or environmental reasons
aside, avoids that I have to listen to the fan all day :-)
|
| 176742 |
02-Mar-2008 |
raj |
Unify and generalize PowerPC headers, adjust AIM code accordingly.
Rework of this area is a pre-requirement for importing e500 support (and
other PowerPC core variations in the future). Mainly the following
headers are refactored so that we can cover for low-level differences between
various machines within PowerPC architecture:
<machine/pcpu.h>
<machine/pcb.h>
<machine/kdb.h>
<machine/hid.h>
<machine/frame.h>
Areas which use the above are adjusted and cleaned up.
Credits for this rework go to marcel@
Approved by: cognet (mentor)
MFp4: e500
|
| 176617 |
27-Feb-2008 |
marcel |
Avoid hardcoding the kernel link address in the linker script.
Use KERNBASE instead. While here, move the text sections
forward to the beginning of the text segment.
|
| 176222 |
12-Feb-2008 |
marcel |
Remove SMP left-overs from NetBSD.
|
| 174898 |
25-Dec-2007 |
rwatson |
Add a new 'why' argument to kdb_enter(), and a set of constants to use
for that argument. This will allow DDB to detect the broad category of
reason why the debugger has been entered, which it can use for the
purposes of deciding which DDB script to run.
Assign approximate why values to all current consumers of the
kdb_enter() interface.
|
| 174782 |
19-Dec-2007 |
marcel |
Redefine bus_space_tag_t on PowerPC from a 32-bit integral to
a pointer to struct bus_space. The structure contains function
pointers that do the actual bus space access.
The reason for this change is that previously all bus space
accesses were little endian (i.e. had an explicit byte-swap
for multi-byte accesses), because all busses on Macs are little
endian.
The upcoming support for Book E, and in particular the E500
core, requires support for big-endian busses because all
embedded peripherals are in the native byte-order.
With this change, there's no distinction between I/O port
space and memory mapped I/O. PowerPC doesn't have I/O port
space. Busses assign tags based on the byte-order only.
For that purpose, two global structures exist (bs_be_tag and
bs_le_tag), of which the address can be taken to get a valid
tag.
Obtained from: Juniper, Semihalf
|
| 174599 |
14-Dec-2007 |
marcel |
Forced commit to record that this file was repocopied from
src/sys/powerpc/powerpc and modified for its new location.
|
| 173708 |
17-Nov-2007 |
alc |
Prevent the leakage of wired pages in the following circumstances:
First, a file is mmap(2)ed and then mlock(2)ed. Later, it is truncated.
Under "normal" circumstances, i.e., when the file is not mlock(2)ed, the
pages beyond the EOF are unmapped and freed. However, when the file is
mlock(2)ed, the pages beyond the EOF are unmapped but not freed because
they have a non-zero wire count. This can be a mistake. Specifically,
it is a mistake if the sole reason why the pages are wired is because of
wired, managed mappings. Previously, unmapping the pages destroys these
wired, managed mappings, but does not reduce the pages' wire count.
Consequently, when the file is unmapped, the pages are not unwired
because the wired mapping has been destroyed. Moreover, when the vm
object is finally destroyed, the pages are leaked because they are still
wired. The fix is to reduce the pages' wired count by the number of
wired, managed mappings destroyed. To do this, I introduce a new pmap
function pmap_page_wired_mappings() that returns the number of managed
mappings to the given physical page that are wired, and I use this
function in vm_object_page_remove().
Reviewed by: tegge
MFC after: 6 weeks
|
| 173615 |
14-Nov-2007 |
marcel |
o Rename cpu_thread_setup() to cpu_thread_alloc() to better
communicate that it relates to (is called by) thread_alloc()
o Add cpu_thread_free() which is called from thread_free()
to counter-act cpu_thread_alloc().
i386: Have cpu_thread_free() call cpu_thread_clean() to
preserve behaviour.
ia64: Have cpu_thread_free() call mtx_destroy() for the
mutex initialized in cpu_thread_alloc().
PR: ia64/118024
|
| 173601 |
14-Nov-2007 |
julian |
A bunch more files that should probably print out a thread name
instead of a process name.
|
| 173600 |
14-Nov-2007 |
julian |
generally we are interested in what thread did something as
opposed to what process. Since threads by default have teh name of the
process unless over-written with more useful information, just print the
thread name instead.
|
| 173584 |
13-Nov-2007 |
grehan |
Split decr_init() into two, with the section that reads the timebase
frequency from OpenFirmware moved out and into a routine that is called
from cpu_startup().
This allows correct reporting of the CPU clockspeed when printing out
CPU information at boot time.
Reported by: numerous
Reviewed by: marcel
MFC after: 1 day
|
| 173361 |
05-Nov-2007 |
kib |
Fix for the panic("vm_thread_new: kstack allocation failed") and
silent NULL pointer dereference in the i386 and sparc64 pmap_pinit()
when the kmem_alloc_nofault() failed to allocate address space. Both
functions now return error instead of panicing or dereferencing NULL.
As consequence, vmspace_exec() and vmspace_unshare() returns the errno
int. struct vmspace arg was added to vm_forkproc() to avoid dealing
with failed allocation when most of the fork1() job is already done.
The kernel stack for the thread is now set up in the thread_alloc(),
that itself may return NULL. Also, allocation of the first process
thread is performed in the fork1() to properly deal with stack
allocation failure. proc_linkup() is separated into proc_linkup()
called from fork1(), and proc_linkup0(), that is used to set up the
kernel process (was known as swapper).
In collaboration with: Peter Holm
Reviewed by: jhb
|
| 172887 |
23-Oct-2007 |
grehan |
Cut over to ULE on PowerPC
kern/sched_ule.c - Add __powerpc__ to the list of supported architectures
powerpc/conf/GENERIC - Swap SCHED_4BSD with SCHED_ULE
powerpc/powerpc/genassym.c - Export TD_LOCK field of thread struct
powerpc/powerpc/swtch.S - Handle new 3rd parameter to cpu_switch() by
updating the old thread's lock. Note: uniprocessor-only, will require
modification for MP support.
powerpc/powerpc/vm_machdep.c - Set 3rd param of cpu_switch to mutex of
old thread's lock, making the call a no-op.
Reviewed by: marcel, jeffr (slightly older version)
|
| 172189 |
15-Sep-2007 |
alc |
It has been observed on the mailing lists that the different categories
of pages don't sum to anywhere near the total number of pages on amd64.
This is for the most part because uma_small_alloc() pages have never been
counted as wired pages, like their kmem_malloc() brethren. They should
be. This changes fixes that.
It is no longer necessary for the page queues lock to be held to free
pages allocated by uma_small_alloc(). I removed the acquisition and
release of the page queues lock from uma_small_free() on amd64 and ia64
weeks ago. This patch updates the other architectures that have
uma_small_alloc() and uma_small_free().
Approved by: re (kensmith)
|
| 171805 |
11-Aug-2007 |
marcel |
Revamp the interrupt handling in support of INTR_FILTER. This includes:
o Revamp the PIC I/F to only abstract the PIC hardware. The
resource handling has been moved to nexus, where it belongs.
o Include EOI and MASK+EOI methods to the PIC I/F in support of
INTR_FILTER.
o With the allocation of interrupt resources and setup of
interrupt handlers in the common platform code we can delay
talking to the PIC hardware after enumeration of all devices.
Introduce a call to powerpc_intr_enable() in configure_final()
to achieve that and have powerpc_setup_intr() only program the
PIC when !cold.
o As a consequence of the above, remove all early_attach() glue
from the OpenPIC and Heathrow PIC drivers and have them
register themselves when they're found during enumeration.
o Decouple the interrupt vector from the interrupt request line.
Allocate vectors increasingly so that they can be used for
the intrcnt index as well. Extend the Heathrow PIC driver to
translate between IRQ and vector. The OpenPIC driver already
has the support for vectors in hardware.
Approved by: re (blanket)
|
| 171787 |
08-Aug-2007 |
marcel |
Re-enable external interrupts for faults, traps and syscalls.
Approved by: re (blanket)
|
| 171785 |
07-Aug-2007 |
marcel |
Eliminate <machine/interruptvar.h> as it has only a single
prototype. In the future that prototype will not be needed
at all anyway, but for now it's moved to intr_machdep.h.
Approved by: re (blanket)
|
| 171783 |
07-Aug-2007 |
marcel |
Remove redundant prototype.
Approved by: re (blanket)
|
| 171670 |
31-Jul-2007 |
marcel |
Fix backward compatibility of the "old" (i.e. FreeBSD6) lseek
syscall. It was broken when a new lseek syscall was introduced.
The problem is that we need to swap the 32-bit td_retval values
for the __syscall indirect syscall when the actual syscall has
a 32-bit return value. Hence, we need to exclude lseek(2). And
this means the "old" lseek(2) as well -- which we didn't.
Based on a patch from: grehan@
Approved by: re (rwatson)
|
| 170473 |
09-Jun-2007 |
marcel |
Add kdb_cpu_sync_icache(), intended to synchronize instruction
caches with data caches after writing to memory. This typically
is required to make breakpoints work on ia64 and powerpc. For
those architectures the function is implemented.
|
| 170363 |
06-Jun-2007 |
grehan |
Fix the compile. Band-aid until it is worked out how to use the context
switch api on ppc.
|
| 170305 |
04-Jun-2007 |
jeff |
- Change comments and asserts to reflect the removal of the global
scheduler lock.
Tested by: kris, current@
Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc.
Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)
|
| 170291 |
04-Jun-2007 |
attilio |
Rework the PCPU_* (MD) interface:
- Rename PCPU_LAZY_INC into PCPU_INC
- Add the PCPU_ADD interface which just does an add on the pcpu member
given a specific value.
Note that for most architectures PCPU_INC and PCPU_ADD are not safe.
This is a point that needs some discussions/work in the next days.
Reviewed by: alc, bde
Approved by: jeff (mentor)
|
| 170170 |
31-May-2007 |
attilio |
Revert VMCNT_* operations introduction.
Probabilly, a general approach is not the better solution here, so we should
solve the sched_lock protection problems separately.
Requested by: alc
Approved by: jeff (mentor)
|
| 170036 |
27-May-2007 |
marcel |
Don't initialize the decrementer before initclocks() is called.
Use cpu_initclocks() for that as it assures that relevant locks
have been initialized.
|
| 169667 |
18-May-2007 |
jeff |
- define and use VMCNT_{GET,SET,ADD,SUB,PTR} macros for manipulating
vmcnts. This can be used to abstract away pcpu details but also changes
to use atomics for all counters now. This means sched lock is no longer
responsible for protecting counts in the switch routines.
Contributed by: Attilio Rao <attilio@FreeBSD.org>
|
| 168885 |
20-Apr-2007 |
grehan |
Add ofw bus methods to the ppc nexus driver. This will be used in future
EFIKA platform support.
PR: 111522
Submitted by: Andrew Turner, andrew at fubar geek nz
|
| 167352 |
09-Mar-2007 |
mohans |
Over NFS, an open() call could result in multiple over-the-wire
GETATTRs being generated - one from lookup()/namei() and the other
from nfs_open() (for cto consistency). This change eliminates the
GETATTR in nfs_open() if an otw GETATTR was done from the namei()
path. Instead of extending the vop interface, we timestamp each attr
load, and use this to detect whether a GETATTR was done from namei()
for this syscall. Introduces a thread-local variable that counts the
syscalls made by the thread and uses <pid, tid, thread syscalls> as
the attrload timestamp. Thanks to jhb@ and peter@ for a discussion on
thread state that could be used as the timestamp with minimal overhead.
|
| 167289 |
07-Mar-2007 |
piso |
Update openpic to support the new bus_setup_intr() syntax.
Reviewed by: marcel
|
| 166659 |
12-Feb-2007 |
kevlo |
Remove the cast to caddr_t for sfp, they're not needed.
Reviewed by: marcel
|
| 166011 |
14-Jan-2007 |
marcel |
Propagate the CPU model to the hw.model sysctl.
|
| 165608 |
28-Dec-2006 |
marcel |
In cpu_reset(), call OF_reboot() instead of OF_exit(). The latter
doesn't do a reboot and has been observed to reset the NVRAM to its
default values.
|
| 165362 |
20-Dec-2006 |
grehan |
Remove bogus increment of re-hashed PTEG index. This snuck in with r1.12 of
pmap.c, and is potentially the cause of hangs reported on machines with a
small amount of memory. On machines with sufficient RAM, and without a lot
of processes running, this situation would probably never occur.
Testing is still incomplete, but it is obviously wrong so remove the
offending code now.
The issue of what to do when both the primary and secondary hash overflow
is still open.
Reported by: Dan Kresja at windriver dot com, via alc
|
| 165151 |
13-Dec-2006 |
marcel |
Implement OF_decode_addr(). This makes uart(4) work as a serial
console on a Xserve G4.
|
| 164936 |
06-Dec-2006 |
julian |
Threading cleanup.. part 2 of several.
Make part of John Birrell's KSE patch permanent..
Specifically, remove:
Any reference of the ksegrp structure. This feature was
never fully utilised and made things overly complicated.
All code in the scheduler that tried to make threaded programs
fair to unthreaded programs. Libpthread processes will already
do this to some extent and libthr processes already disable it.
Also:
Since this makes such a big change to the scheduler(s), take the opportunity
to rename some structures and elements that had to be moved anyhow.
This makes the code a lot more readable.
The ULE scheduler compiles again but I have no idea if it works.
The 4bsd scheduler still reqires a little cleaning and some functions that now do
ALMOST nothing will go away, but I thought I'd do that as a separate commit.
Tested by David Xu, and Dan Eischen using libthr and libpthread.
|
| 164895 |
05-Dec-2006 |
grehan |
Fix gdb issue where the i-cache was not being updated when a breakpoint
was written into a user's address space. The fix is to modify uiomove_fromphys
to sync the icache when an executable user-space page is written into.
Alan Cox suggested that there should probably be a higher-level interface
to this in the ptrace code, but agreed that this is an OK short-term solution.
Files changed:
pmap.h - declaration of pmap_page_executable()
pmap_dispatch.c - pass through the page_executable call to the mmu object
mmu_oea.c - implement the page_executable method by examining the PTE_EXEC
field in the vm_page_t
uio_machdep.c - in uiomove_fromphys(), if the op was a UIO_WRITE to user-space,
and if the page is executable, sync the icache since this is at the least
a breakpoint-write from gdb.
Reported by: marcel
Tested by: marcel, grehan on g3+g4
Discussed with: alc
MFC after: 2 weeks
|
| 164765 |
30-Nov-2006 |
grehan |
Don't use vm_page_flag_set() if installing bootstrap page-table entries
since the vm page mutex's aren't yet initialized. Fixes boot-time panic.
Reported by: Dario Freni saturnero at freesbie dot org
|
| 164229 |
12-Nov-2006 |
alc |
Make pmap_enter() responsible for setting PG_WRITEABLE instead
of its caller. (As a beneficial side-effect, a high-contention
acquisition of the page queues lock in vm_fault() is eliminated.)
|
| 163709 |
26-Oct-2006 |
jb |
Make KSE a kernel option, turned on by default in all GENERIC
kernel configs except sun4v (which doesn't process signals properly
with KSE).
Reviewed by: davidxu@
|
| 163488 |
18-Oct-2006 |
grehan |
Fix remaining compile error.
|
| 163472 |
18-Oct-2006 |
davidxu |
Attempt to fix compiling problem.
Noticed by: tinderbox
|
| 163449 |
17-Oct-2006 |
davidxu |
o Add keyword volatile for user mutex owner field.
o Fix type consistent problem by using type long for old
umtx and wait channel.
o Rename casuptr to casuword.
|
| 163021 |
05-Oct-2006 |
grehan |
Catch up with recent clock modifications:
- include <sys/clock.h> for inittodr prototype
- remove now-conflicting SECDAY definition that is in <sys/clock.h>
|
| 162961 |
02-Oct-2006 |
phk |
remove orphaned sysctl_machdep_adjkerntz()
|
| 162958 |
02-Oct-2006 |
phk |
Second part of a little cleanup in the calendar/timezone/RTC handling.
Split subr_clock.c in two parts (by repo-copy):
subr_clock.c contains generic RTC and calendaric stuff. etc.
subr_rtc.c contains the newbus'ified RTC interface.
Centralize the machdep.{adjkerntz,disable_rtc_set,wall_cmos_clock}
sysctls and associated variables into subr_clock.c. They are
not machine dependent and we have generic code that relies on being
present so they are not even optional.
|
| 162361 |
16-Sep-2006 |
rwatson |
Add audit hooks for ppc, ia64 system call paths.
Reviewed by: marcel (ia64)
Obtained from: TrustedBSD Project
MFC after: 3 days
|
| 161797 |
01-Sep-2006 |
marcel |
In cpu_set_user_tls(), properly set the thread pointer. It is 0x7000
bytes after the end of the TCB, which is itself 8 bytes.
|
| 161675 |
28-Aug-2006 |
davidxu |
Implement casuword32, compare and set user integer, thank Marcel Moolenarr
who wrote the IA64 version of casuword32.
|
| 160959 |
03-Aug-2006 |
sobomax |
Use proper trap code for the EXC_ALI traps. This fixes SIGBUS during
unaligned 64-bits load/stores.
MFC after: 2 weeks
|
| 160889 |
01-Aug-2006 |
alc |
Complete the transition from pmap_page_protect() to pmap_remove_write().
Originally, I had adopted sparc64's name, pmap_clear_write(), for the
function that is now pmap_remove_write(). However, this function is more
like pmap_remove_all() than like pmap_clear_modify() or
pmap_clear_reference(), hence, the name change.
The higher-level rationale behind this change is described in
src/sys/amd64/amd64/pmap.c revision 1.567. The short version is that I'm
trying to clean up and fix our support for execute access.
Reviewed by: marcel@ (ia64)
|
| 160801 |
28-Jul-2006 |
jhb |
Retire SYF_ARGMASK and remove both SYF_MPSAFE and SYF_ARGMASK. sy_narg is
now back to just being an argument count.
|
| 160798 |
28-Jul-2006 |
jhb |
Now that all system calls are MPSAFE, retire the SYF_MPSAFE flag used to
mark system calls as being MPSAFE:
- Stop conditionally acquiring Giant around system call invocations.
- Remove all of the 'M' prefixes from the master system call files.
- Remove support for the 'M' prefix from the script that generates the
syscall-related files from the master system call files.
- Don't explicitly set SYF_MPSAFE when registering nfssvc.
|
| 160773 |
27-Jul-2006 |
jhb |
Unify the checking for lock misbehavior in the various syscall()
implementations and adjust some of the checks while I'm here:
- Add a new check to make sure we don't return from a syscall in a critical
section.
- Add a new explicit check before userret() to make sure we don't return
with any locks held. The advantage here is that we can include the
syscall number and name in syscall() whereas that info is not available
in userret().
- Drop the mtx_assert()'s of sched_lock and Giant. They are replaced by
the more general checks just added.
MFC after: 2 weeks
|
| 160764 |
27-Jul-2006 |
jhb |
Add missing ptrace(2) system-call stops to various syscall()
implementations.
MFC after: 1 week
|
| 160714 |
26-Jul-2006 |
marcel |
o Move the prototype of mem_valid() from ofw_machdep.h to md_var.h.
This avoids that mem.c has to include ofw_machdep.h, including
all OFW related headers.
o Provide a stub for OF_decode_addr(), which is used by low-level
console drivers to obtain a tag and handle given a OFW phandle.
This is different from sparc64, where a fake bus tag needs to be
created explicitly.
|
| 160713 |
26-Jul-2006 |
marcel |
Include needed clock.h.
|
| 160235 |
10-Jul-2006 |
alc |
Add synchronization to moea_zero_page() and moea_zero_page_area().
Remove the acquisition and release of Giant from moea_zero_page_idle().
Tested by: grehan@
|
| 160068 |
01-Jul-2006 |
alc |
Eliminate the acquisition and release of Giant from moea_extract_and_hold()
and moea_protect().
Tested by: grehan@ and rink@
|
| 159928 |
25-Jun-2006 |
alc |
Synchronize accesses to the PTEG table.
Add many lock assertions.
Tested by: grehan@
|
| 159705 |
17-Jun-2006 |
rink |
Prevent 'mutex not owned' panic on boot if INVARIANTS is in the kernel. This
makes the GENERIC kernel boot on ppc.
Reviewed by: grehan
Approved by: imp (mentor)
MFC after: 1 week
dCVS: ----------------------------------------------------------------------
|
| 159627 |
15-Jun-2006 |
ups |
Remove mpte optimization from pmap_enter_quick().
There is a race with the current locking scheme and removing
it should have no measurable performance impact.
This fixes page faults leading to panics in pmap_enter_quick_locked()
on amd64/i386.
Reviewed by: alc,jhb,peter,ps
|
| 159324 |
06-Jun-2006 |
alc |
Correct a typo in the previous revision.
|
| 159303 |
05-Jun-2006 |
alc |
Introduce the function pmap_enter_object(). It maps a sequence of resident
pages from the same object. Use it in vm_map_pmap_enter() to reduce the
locking overhead of premapping objects.
Reviewed by: tegge@
|
| 158651 |
16-May-2006 |
phk |
Since DELAY() was moved, most <machine/clock.h> #includes have been
unnecessary.
|
| 158449 |
11-May-2006 |
phk |
Remove straggling reference to CPU_ macros
|
| 155455 |
08-Feb-2006 |
phk |
Simplify system time accounting for profiling.
Rename struct thread's td_sticks to td_pticks, we will need the
other name for more appropriately named use shortly. Reduce it
from uint64_t to u_int.
Clear td_pticks whenever we enter the kernel instead of recording
its value as reference for userret(). Use the absolute value of
td->pticks in userret() and eliminate third argument.
|
| 154091 |
07-Jan-2006 |
grehan |
Set the siginfo si_addr field, and also the mysterious 3rd parameter
to old-style signals, to be the DAR register for DSI miss exceptions.
This gives the address of the access rather than the instruction
address. The behaviour is now the same as on i386.
Found by: libsigsegv tests
|
| 153940 |
31-Dec-2005 |
netchild |
MI changes:
- provide an interface (macros) to the page coloring part of the VM system,
this allows to try different coloring algorithms without the need to
touch every file [1]
- make the page queue tuning values readable: sysctl vm.stats.pagequeue
- autotuning of the page coloring values based upon the cache size instead
of options in the kernel config (disabling of the page coloring as a
kernel option is still possible)
MD changes:
- detection of the cache size: only IA32 and AMD64 (untested) contains
cache size detection code, every other arch just comes with a dummy
function (this results in the use of default values like it was the
case without the autotuning of the page coloring)
- print some more info on Intel CPU's (like we do on AMD and Transmeta
CPU's)
Note to AMD owners (IA32 and AMD64): please run "sysctl vm.stats.pagequeue"
and report if the cache* values are zero (= bug in the cache detection code)
or not.
Based upon work by: Chad David <davidc@acns.ab.ca> [1]
Reviewed by: alc, arch (in 2004)
Discussed with: alc, Chad David, arch (in 2004)
|
| 153685 |
23-Dec-2005 |
grehan |
Mark the return address of the call to ast() in the generic trap
handling code so the stack trace unwinders don't start trying to
go into user-space.
Found by trying to create core dumps with a KTR_COMPILE/KTR_GEOM
kernel, which results in a stack_save() call in the ast() coredump
path - this created a panic, and then calling 'trace' in ddb resulted
in the black screen of death after printing out most of the backtrace.
|
| 153666 |
22-Dec-2005 |
jhb |
Tweak how the MD code calls the fooclock() methods some. Instead of
passing a pointer to an opaque clockframe structure and requiring the
MD code to supply CLKF_FOO() macros to extract needed values out of the
opaque structure, just pass the needed values directly. In practice this
means passing the pair (usermode, pc) to hardclock() and profclock() and
passing the boolean (usermode) to hardclock_cpu() and hardclock_process().
Other details:
- Axe clockframe and CLKF_FOO() macros on all architectures. Basically,
all the archs were taking a trapframe and converting it into a clockframe
one way or another. Now they can just extract the PC and usermode values
directly out of the trapframe and pass it to fooclock().
- Renamed hardclock_process() to hardclock_cpu() as the latter is more
accurate.
- On Alpha, we now run profclock() at hz (profhz == hz) rather than at
the slower stathz.
- On Alpha, for the TurboLaser machines that don't have an 8254
timecounter, call hardclock() directly. This removes an extra
conditional check from every clock interrupt on Alpha on the BSP.
There is probably room for even further pruning here by changing Alpha
to use the simplified timecounter we use on x86 with the lapic timer
since we don't get interrupts from the 8254 on Alpha anyway.
- On x86, clkintr() shouldn't ever be called now unless using_lapic_timer
is false, so add a KASSERT() to that affect and remove a condition
to slightly optimize the non-lapic case.
- Change prototypeof arm_handler_execute() so that it's first arg is a
trapframe pointer rather than a void pointer for clarity.
- Use KCOUNT macro in profclock() to lookup the kernel profiling bucket.
Tested on: alpha, amd64, arm, i386, ia64, sparc64
Reviewed by: bde (mostly)
|
| 152304 |
11-Nov-2005 |
grehan |
Fix compile warning: pmap_bootstrap is now declared extern in pmap.h,
remove redundant declaration.
|
| 152180 |
08-Nov-2005 |
grehan |
Name change from pmap_* to moea_* to fit into the new order of
mmu implementation.
This code handles the 32-bit 'OEA' MMU found on G2/G3/G4 PPC cores.
|
| 152179 |
08-Nov-2005 |
grehan |
Insert a layer of indirection to the pmap code, using a kobj for
the interface. This allows run-time selection of MMU code, based
on CPU-type detection, or tunable-overrides when testing new code.
Pre-requisite for G5 support.
conf/files.powerpc
- remove pmap.c
- add mmu_if.h, mmu_oea.c, pmap_dispatch.c
powerpc/include/mmuvar.h
- definitions for MMU implementations
powerpc/include/pmap.h
- remove pmap_pte_spill declaration
- add pmap_mmu_install declaration
- size the phys_avail array
- pmap_bootstrapped is now global-scope
powerpc/powerpc/machdep.c
- call kobj_machdep_init early in the boot sequence to allow
kobj usage prior to SI_SUB_LOCK
- install the OEA pmap code. This will be moved to CPU-specific
init code in the future.
powerpc/powerpc/mmu_if.m
- Kobj MMU interface definitions
powerpc/powerpc/pmap_dispatch.c
- central dispatch for pmap calls
- contains the global mmu kobj and the routine to locate the
the mmu implementation and init the kobj
|
| 151891 |
30-Oct-2005 |
grehan |
Copy SPRG0-3 registers at boot-time and restore when calling into
OpenFirmware. FreeBSD/ppc uses SPRG0 as the per-cpu data area pointer,
and SPRG1-3 as temporary registers during exception handling. There
have been a few instances where OpenFirmware does require these to
be part of it's context, such as cd-booting an eMac.
reported by: many
MFC after: 3 days
|
| 151316 |
14-Oct-2005 |
davidxu |
1. Change prototype of trapsignal and sendsig to use ksiginfo_t *, most
changes in MD code are trivial, before this change, trapsignal and
sendsig use discrete parameters, now they uses member fields of
ksiginfo_t structure. For sendsig, this change allows us to pass
POSIX realtime signal value to user code.
2. Remove cpu_thread_siginfo, it is no longer needed because we now always
generate ksiginfo_t data and feed it to libpthread.
3. Add p_sigqueue to proc structure to hold shared signals which were
blocked by all threads in the proc.
4. Add td_sigqueue to thread structure to hold all signals delivered to
thread.
5. i386 and amd64 now return POSIX standard si_code, other arches will
be fixed.
6. In this sigqueue implementation, pending signal set is kept as before,
an extra siginfo list holds additional siginfo_t data for signals.
kernel code uses psignal() still behavior as before, it won't be failed
even under memory pressure, only exception is when deleting a signal,
we should call sigqueue_delete to remove signal from sigqueue but
not SIGDELSET. Current there is no kernel code will deliver a signal
with additional data, so kernel should be as stable as before,
a ksiginfo can carry more information, for example, allow signal to
be delivered but throw away siginfo data if memory is not enough.
SIGKILL and SIGSTOP have fast path in sigqueue_add, because they can
not be caught or masked.
The sigqueue() syscall allows user code to queue a signal to target
process, if resource is unavailable, EAGAIN will be returned as
specification said.
Just before thread exits, signal queue memory will be freed by
sigqueue_flush.
Current, all signals are allowed to be queued, not only realtime signals.
Earlier patch reviewed by: jhb, deischen
Tested on: i386, amd64
|
| 149958 |
10-Sep-2005 |
grehan |
Fix boot-time hang/panic on G3 systems when modifying IBAT0 in
pmap_bootstrap by using the sync;isync big hammer to make sure
all prior operations have completed.
Reported by: Nathan Whitehorn <nathan at uchicago edu>
MFC after: 2 days
|
| 149768 |
03-Sep-2005 |
alc |
Pass a value of type vm_prot_t to pmap_enter_quick() so that it determine
whether the mapping should permit execute access.
|
| 148568 |
30-Jul-2005 |
grehan |
Temporary band-aid to fix hang when a process exec's Altivec instructions.
trap_subr.S: declare a stub for the a-unavailable trap
that does an absolute jump to the vector-assist trap.
This is due to the fact that the vec-unavail trap
doesn't start at a 256-byte boundary, so the trick of
masking the bottom 8 bits of the link register to identify
the interrupt doesn't work, so let the vec-assist
case handle Altivec-disabled for the time being.
Note that this will be fixed in the future with a much
smaller vector code-stub (< 16 bytes) that will allow
use of strange vector offsets that are also present in
4xx processors, and also allow smaller differences in
vector codepaths on the G5.
trap.c: Treat altivec-unavailable/assist process traps as SIGILL.
Not quite correct, since altivec-assist should really be a panic,
but it is fine for the moment due to the above measure.
machdep.c Install the stub code for the altivec-unavailable trap, and
the standard trap code at the altivec-assist.
Reported by: Andreas Tobler <toa at pop agri ch>
MFC after: 3 days
|
| 147889 |
10-Jul-2005 |
davidxu |
Validate if the value written into {FS,GS}.base is a canonical
address, writting non-canonical address can cause kernel a panic,
by restricting base values to 0..VM_MAXUSER_ADDRESS, ensuring
only canonical values get written to the registers.
Reviewed by: peter, Josepha Koshy < joseph.koshy at gmail dot com >
Approved by: re (scottl)
|
| 147217 |
10-Jun-2005 |
alc |
Introduce a procedure, pmap_page_init(), that initializes the
vm_page's machine-dependent fields. Use this function in
vm_pageq_add_new_page() so that the vm_page's machine-dependent and
machine-independent fields are initialized at the same time.
Remove code from pmap_init() for initializing the vm_page's
machine-dependent fields.
Remove stale comments from pmap_init().
Eliminate the Boolean variable pmap_initialized from the alpha, amd64,
i386, and ia64 pmap implementations. Its use is no longer required
because of the above changes and earlier changes that result in physical
memory that is being mapped at initialization time being mapped without
pv entries.
Tested by: cognet, kensmith, marcel
|
| 145433 |
23-Apr-2005 |
davidxu |
Change cpu_set_kse_upcall to more generic style, so we can reuse it
in other codes. Add cpu_set_user_tls, use it to tweak user register
and setup user TLS. I ever wanted to merge it into cpu_set_kse_upcall,
but since cpu_set_kse_upcall is also used by M:N threads which may
not need this feature, so I wrote a separated cpu_set_user_tls.
|
| 145343 |
20-Apr-2005 |
ps |
Don't enter the debugger if KDB_UNATTENDED is set or if
debug.debugger_on_panic=0.
MFC after: 2 weeks
|
| 144971 |
12-Apr-2005 |
jhb |
Use PCPU_LAZY_INC() for cnt.v_{intr,trap,syscalls} rather than atomic
operations in some places and simple non-per CPU math in others.
|
| 144807 |
08-Apr-2005 |
jhb |
Change an instance of md_savecrit to md_saved_msr that I missed.
|
| 144637 |
04-Apr-2005 |
jhb |
Divorce critical sections from spinlocks. Critical sections as denoted by
critical_enter() and critical_exit() are now solely a mechanism for
deferring kernel preemptions. They no longer have any affect on
interrupts. This means that standalone critical sections are now very
cheap as they are simply unlocked integer increments and decrements for the
common case.
Spin mutexes now use a separate KPI implemented in MD code: spinlock_enter()
and spinlock_exit(). This KPI is responsible for providing whatever MD
guarantees are needed to ensure that a thread holding a spin lock won't
be preempted by any other code that will try to lock the same lock. For
now all archs continue to block interrupts in a "spinlock section" as they
did formerly in all critical sections. Note that I've also taken this
opportunity to push a few things into MD code rather than MI. For example,
critical_fork_exit() no longer exists. Instead, MD code ensures that new
threads have the correct state when they are created. Also, we no longer
try to fixup the idlethreads for APs in MI code. Instead, each arch sets
the initial curthread and adjusts the state of the idle thread it borrows
in order to perform the initial context switch.
This change is largely a big NOP, but the cleaner separation it provides
will allow for more efficient alternative locking schemes in other parts
of the kernel (bare critical sections rather than per-CPU spin mutexes
for per-CPU data for example).
Reviewed by: grehan, cognet, arch@, others
Tested on: i386, alpha, sparc64, powerpc, arm, possibly more
|
| 143633 |
15-Mar-2005 |
grehan |
Include <sys/signalvar.h> for trapsignal prototype.
|
| 143234 |
07-Mar-2005 |
grehan |
Replaced previous hw.physmem extraction with des's mods to
getenv_ulong() - much simpler.
Pointed out by: des
|
| 143201 |
07-Mar-2005 |
grehan |
physmem is a much better indicator for 'real' memory on PPC than Maxmem
since there are often significant holes in the memory map due to the
kernel, loader and OFW data structures not being included: Maxmem is
the highest available, so can be misleading.
|
| 143200 |
07-Mar-2005 |
grehan |
Allow user to undersize memory with hw.physmem loader variable.
Obtained from: i386/machdep.c:getmemsize()
|
| 142882 |
01-Mar-2005 |
grehan |
Catch up with "physical memory" sysctl change.
(MFi386: rev 1.608)
|
| 142771 |
28-Feb-2005 |
grehan |
Catch the case where the idle loop is entered with interrupts disabled,
causing a hard hang.
|
| 142764 |
28-Feb-2005 |
grehan |
- switch pcpu to a struct declaration ala amd64. It may be more efficient to
cache-align this struct, but that's a topic for a far-in-the-future
commit.
- eliminate commented-out reference to a non-existent pcpu field.
|
| 142756 |
28-Feb-2005 |
grehan |
Correctly set kernelname for kern.bootfile sysctl
Noticed by: gad
Code stolen from: sparc64
|
| 142416 |
25-Feb-2005 |
grehan |
Add PVO_FAKE flag to pvo entries for PG_FICTITIOUS mappings, to
avoid trying to reverse-map a device physical address to the
vm_page array and walking into non-existent vm weeds.
found by: Xorg server exiting
|
| 141378 |
06-Feb-2005 |
njl |
Finish the job of sorting all includes and fix the build by including
malloc.h before proc.h on sparc64. Noticed by das@
Compiled on: alpha, amd64, i386, pc98, sparc64
|
| 141249 |
04-Feb-2005 |
njl |
Sort includes a little so that bus.h comes before cpu.h (for device_t).
|
| 141237 |
04-Feb-2005 |
njl |
Add an implementation of cpu_est_clockrate(9). This function estimates the
current clock frequency for the given CPU id in units of Hz.
|
| 141227 |
04-Feb-2005 |
grehan |
- add wall_cmos_clock and adjkerntz variables, required by msdosfs
- support adjkerntz sysctl to silence NTP, though it's a null
implementation at the moment.
|
| 140538 |
21-Jan-2005 |
grehan |
Fix (accidental?) lock order reversal in pmap_remove. Found when
a process that has mmap'd device mem exits.
|
| 140256 |
14-Jan-2005 |
jhb |
- Remove some OBE comments regarding cpu_exit(). cpu_exit() is no longer
the last action of kern_exit(). Instead, it is a MD callout to cleanup
per-process state during exit.
- Add notes of concern to Alpha and ia64 about the possible need to drop
fp state in cpu_thread_exit() rather than in cpu_exit() since it is
per-thread state rather than per-process.
|
| 139825 |
07-Jan-2005 |
imp |
/* -> /*- for license, minor formatting changes
|
| 139401 |
29-Dec-2004 |
grehan |
Correctly initialise the 2nd kernel segment, and don't
forget to actually install it in the segment register.
This may fix some of the weird panics seen when kernel VM
is heavily used.
|
| 139241 |
23-Dec-2004 |
alc |
Modify pmap_enter_quick() so that it expects the page queues to be locked
on entry and it assumes the responsibility for releasing the page queues
lock if it must sleep.
Remove a bogus comment from pmap_enter_quick().
Using the first change, modify vm_map_pmap_enter() so that the page queues
lock is acquired and released once, rather than each time that a page
is mapped.
|
| 138897 |
15-Dec-2004 |
alc |
In the common case, pmap_enter_quick() completes without sleeping.
In such cases, the busying of the page and the unlocking of the
containing object by vm_map_pmap_enter() and vm_fault_prefault() is
unnecessary overhead. To eliminate this overhead, this change
modifies pmap_enter_quick() so that it expects the object to be locked
on entry and it assumes the responsibility for busying the page and
unlocking the object if it must sleep. Note: alpha, amd64, i386 and
ia64 are the only implementations optimized by this change; arm,
powerpc, and sparc64 still conservatively busy the page and unlock the
object within every pmap_enter_quick() call.
Additionally, this change is the first case where we synchronize
access to the page's PG_BUSY flag and busy field using the containing
object's lock rather than the global page queues lock. (Modifications
to the page's PG_BUSY flag and busy field have asserted both locks for
several weeks, enabling an incremental transition.)
|
| 138129 |
27-Nov-2004 |
das |
Don't include sys/user.h merely for its side-effect of recursively
including other headers.
|
| 137912 |
20-Nov-2004 |
das |
U areas are going away, so don't allocate one for process 0.
Reviewed by: arch@
|
| 135172 |
13-Sep-2004 |
alc |
Lock the kernel pmap in pmap_kenter().
Tested by: gallatin@
|
| 134791 |
05-Sep-2004 |
julian |
Refactor a bunch of scheduler code to give basically the same behaviour
but with slightly cleaned up interfaces.
The KSE structure has become the same as the "per thread scheduler
private data" structure. In order to not make the diffs too great
one is #defined as the other at this time.
The KSE (or td_sched) structure is now allocated per thread and has no
allocation code of its own.
Concurrency for a KSEGRP is now kept track of via a simple pair of counters
rather than using KSE structures as tokens.
Since the KSE structure is different in each scheduler, kern_switch.c
is now included at the end of each scheduler. Nothing outside the
scheduler knows the contents of the KSE (aka td_sched) structure.
The fields in the ksegrp structure that are to do with the scheduler's
queueing mechanisms are now moved to the kg_sched structure.
(per ksegrp scheduler private data structure). In other words how the
scheduler queues and keeps track of threads is no-one's business except
the scheduler's. This should allow people to write experimental
schedulers with completely different internal structuring.
A scheduler call sched_set_concurrency(kg, N) has been added that
notifies teh scheduler that no more than N threads from that ksegrp
should be allowed to be on concurrently scheduled. This is also
used to enforce 'fainess' at this time so that a ksegrp with
10000 threads can not swamp a the run queue and force out a process
with 1 thread, since the current code will not set the concurrency above
NCPU, and both schedulers will not allow more than that many
onto the system run queue at a time. Each scheduler should eventualy develop
their own methods to do this now that they are effectively separated.
Rejig libthr's kernel interface to follow the same code paths as
linkse for scope system threads. This has slightly hurt libthr's performance
but I will work to recover as much of it as I can.
Thread exit code has been cleaned up greatly.
exit and exec code now transitions a process back to
'standard non-threaded mode' before taking the next step.
Reviewed by: scottl, peter
MFC after: 1 week
|
| 134571 |
31-Aug-2004 |
julian |
Remove an unneeded argument..
The removed argument could trivially be derived from the remaining one.
That in turn should be the same as curthread, but it is possible that curthread could be expensive to derive on some syste,s so leave it as an argument.
Having both proc and thread as an argumen tjust gives an opportunity for
them to get out sync.
MFC after: 3 days
|
| 134568 |
31-Aug-2004 |
julian |
Remove sched_free_thread() which was only used
in diagnostics. It has outlived its usefulness and has started
causing panics for people who turn on DIAGNOSTIC, in what is otherwise
good code.
MFC after: 2 days
|
| 134535 |
30-Aug-2004 |
alc |
- Introduce a lock for synchronizing access to the pvo and pteg tables.
- In pmap_enter(), only the acquisition and release of the page queues
lock needs to check the bootstrap flag.
Tested by: gallatin@
|
| 134453 |
28-Aug-2004 |
alc |
Eliminate unnecessary indirection.
|
| 134329 |
26-Aug-2004 |
alc |
Add pmap locking to many of the functions.
Many thanks to Andrew Gallatin for resolving a powerpc-specific
initialization problem in my original patch.
Tested by: gallatin@
|
| 133862 |
16-Aug-2004 |
marius |
Instead of "OpenFirmware", "openfirmware", etc. use the official spelling
"Open Firmware" from IEEE 1275 and OpenFirmware.org (no pun intended).
Ok'ed by: tmm
|
| 133855 |
16-Aug-2004 |
ssouhlal |
Add /dev/mem and /dev/kmem to powerpc.
Approved by: grehan (mentor)
|
| 133166 |
05-Aug-2004 |
grehan |
In pmap_page_protect, clear the vm page's PG_WRITEABLE flag if
downgrading to read-only. Found by triggering the KASSERT in
vm_pageout_flush().
|
| 133143 |
04-Aug-2004 |
alc |
- Push down the acquisition and release of Giant into pmap_enter_quick()
on those architectures without pmap locking.
- Eliminate the acquisition and release of Giant in vm_map_pmap_enter().
|
| 132994 |
02-Aug-2004 |
grehan |
Kernel traps were not being passed to trap_fatal in some
circumstances.
Spotted by: gallatin
|
| 132899 |
30-Jul-2004 |
alc |
- Push down the acquisition and release of Giant into pmap_protect() on
those architectures without pmap locking.
- Eliminate the acquisition and release of Giant from vm_map_protect().
(Translation: mprotect(2) runs to completion without touching Giant on
alpha, amd64, i386 and ia64.)
|
| 132836 |
29-Jul-2004 |
ssouhlal |
Implement MD parts of ptrace.
Approved by: grehan (mentor)
|
| 132683 |
27-Jul-2004 |
grehan |
Save DAR/DSISR in DDB regsave area when stack overflow detected. It's
hard to work out where the problem was without these.
|
| 132681 |
27-Jul-2004 |
grehan |
Improve boot-time debugging with DDB by extracting the ksym start/end
values from the loader.
|
| 132666 |
26-Jul-2004 |
alc |
Implement the protection check required by the pmap_extract_and_hold()
specification.
Reviewed and tested by: grehan@
|
| 132571 |
23-Jul-2004 |
grehan |
Detect kernel stack excursion into guard pages. Drop into KDB
with a wired stack if this is found.
Mostly obtained from: NetBSD
|
| 132570 |
23-Jul-2004 |
grehan |
Bring KDB stack size into line with thread stack size (4 pages).
|
| 132569 |
23-Jul-2004 |
grehan |
Allow DSI exceptions to invoke DDB.
|
| 132520 |
22-Jul-2004 |
grehan |
Update the callframe structure to leave space for the frame pointer
and saved link register as per the ABI call sequence. Update code
that uses this (fork_trampoline etc) to use the correct genassym'd
offsets.
This fixes the 'invalid LR' message when backtracing kernel
threads in DDB.
|
| 132426 |
20-Jul-2004 |
grehan |
Properly obey PPC context synchronization rules when modifying
the address translation bits of the MSR. This fixes the boot-time
panic reported by Drew Gallatin.
|
| 132220 |
15-Jul-2004 |
alc |
Push down the acquisition and release of the page queues lock into
pmap_protect() and pmap_remove(). In general, they require the lock in
order to modify a page's pv list or flags. In some cases, however,
pmap_protect() can avoid acquiring the lock.
|
| 132088 |
13-Jul-2004 |
davidxu |
Add ptrace_clear_single_step(), alpha already has it for years, the function
will be used by ptrace to clear a thread's single step state.
|
| 132075 |
12-Jul-2004 |
grehan |
Rename low-level code ddb -> db. Use KDB instead of DDB.
Fix bug in setup of stack frame where 8 bytes wasn't being
saved for the callee's frame pointer and saved LR.
|
| 132074 |
12-Jul-2004 |
grehan |
Bring into KDB new order.
|
| 132073 |
12-Jul-2004 |
grehan |
- DDB -> KDB, with kdb routines
- ddb -> db for low-level trapcode
- implement makectx. I think it only matters that the stack is setup
correctly.
- bring over ddb_trap_glue and rename to db_trap_glue
|
| 132072 |
12-Jul-2004 |
grehan |
No need for ddb option. Never a need for ipkdb option.
|
| 132011 |
12-Jul-2004 |
alc |
pmap_remove_pages() must not remove wired mappings. Since
pmap_remove_pages() is an optimization, its implementation is optional.
Discussed with: grehan
|
| 131867 |
09-Jul-2004 |
grehan |
- correctly set the return value for the copyin/out fault buffer to 1
so setfault would return correctly when a page fault was invalid
(e.g. a syscall with a bad parameter).
This caused an endless DSI loop, seen when running sendmail which
does a setlogin() call with a NULL pointer.
- introduce KTR_SYSC tracing. expose the syscallnames[] array to
make the tracing more readable.
|
| 131808 |
08-Jul-2004 |
grehan |
G4 requires isync after 256Mb ibat/dbat update, G3 requires
isync after each bat update. Otherwise, pmap_bootstrap causes
an ISI exception. A fall-out of loader BAT removal.
|
| 131698 |
06-Jul-2004 |
grehan |
- trailing white-space cleanup
- add call to thread_user_enter for P_SA processes before
trap processing ala all other arches
|
| 131658 |
05-Jul-2004 |
alc |
Correct pmap_extract()'s return type. It should be vm_paddr_t, not
vm_offset_t.
|
| 131401 |
01-Jul-2004 |
grehan |
Modify loop test when cycling through phys_avail array. It's possible
for an OpenFirmware implementation to have a single memory region
(hello PearPC).
|
| 131102 |
25-Jun-2004 |
grehan |
Catchup to now-required <sys/module.h> for PowerPC
|
| 130028 |
03-Jun-2004 |
tjr |
Remove checks for curthread == NULL - it can't happen.
|
| 130023 |
03-Jun-2004 |
tjr |
Move TDF_DEADLKTREAT into td_pflags (and rename it accordingly) to avoid
having to acquire sched_lock when manipulating it in lockmgr(), uiomove(),
and uiomove_fromphys().
Reviewed by: jhb
|
| 129750 |
26-May-2004 |
tmm |
Retire cpu_sched_exit(); it is not used any more.
|
| 129416 |
19-May-2004 |
grehan |
trap_pfault() shouldn't be acquiring Giant. Found to blow up
with MUTEX_PROFILING.
Submitted by: Suleiman Souhlal <refugee@segfaulted.com>
|
| 128395 |
18-Apr-2004 |
alc |
MFamd64
Simplify the sf_buf implementation. In short, make it a veneer
over the direct virtual-to-physical mapping.
|
| 128103 |
11-Apr-2004 |
alc |
Remove avail_end. It is not used.
|
| 127977 |
07-Apr-2004 |
imp |
Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.
Approved by: core, peter, alc, rwatson
|
| 127875 |
05-Apr-2004 |
alc |
Remove avail_start on those platforms that no longer use it. (Only amd64
does anything with it beyond simple initialization.)
|
| 127869 |
05-Apr-2004 |
alc |
Remove unused arguments from pmap_init().
|
| 127788 |
03-Apr-2004 |
alc |
In some cases, sf_buf_alloc() should sleep with pri PCATCH; in others, it
should not. Add a new parameter so that the caller can specify which is
the case.
Reported by: dillon
|
| 127618 |
30-Mar-2004 |
benno |
Replace td2 with td on the assumption that this was a typo. This should at
least unbreak the build.
Pointy hat to: peter
Not tested either by: benno
|
| 127582 |
29-Mar-2004 |
peter |
Finish tidying up a couple of leftovers from the KSTACK_PAGES stuff. Some
files still #included the opt_ file. powerpc hadn't been updated yet.
|
| 127333 |
23-Mar-2004 |
alc |
Add an implementation of uiomove_fromphys() for PowerPC. This
implementation uses the direct virtual-to-physical mapping.
Discussed with: grehan
|
| 127086 |
16-Mar-2004 |
alc |
Refactor the existing machine-dependent sf_buf_free() into a machine-
dependent function by the same name and a machine-independent function,
sf_buf_mext(). Aside from the virtue of making more of the code machine-
independent, this change also makes the interface more logical. Before,
sf_buf_free() did more than simply undo an sf_buf_alloc(); it also
unwired and if necessary freed the page. That is now the purpose of
sf_buf_mext(). Thus, sf_buf_alloc() and sf_buf_free() can now be used
as a general-purpose emphemeral map cache.
|
| 126728 |
07-Mar-2004 |
alc |
Retire pmap_pinit2(). Alpha was the last platform that used it. However,
ever since alpha/alpha/pmap.c revision 1.81 introduced the list allpmaps,
there has been no reason for having this function on Alpha. Briefly,
when pmap_growkernel() relied upon the list of all processes to find and
update the various pmaps to reflect a growth in the kernel's valid
address space, pmap_init2() served to avoid a race between pmap
initialization and pmap_growkernel(). Specifically, pmap_pinit2() was
responsible for initializing the kernel portions of the pmap and
pmap_pinit2() was called after the process structure contained a pointer
to the new pmap for use by pmap_growkernel(). Thus, an update to the
kernel's address space might be applied to the new pmap unnecessarily,
but an update would never be lost.
|
| 126478 |
02-Mar-2004 |
grehan |
Increase kernel VA from 256Mb to 512Mb by shifting the segment used
for user copyinout down to 12, and keeping segments 13/14 for
kernel VA.
It would be nice to have more available, but segments lower than
this are reserved for either memory or 1:1 mapped device i/o,
and seg 15 is OpenFirmware ROM. Also, the effort to keep OpenFirmware
available for callbacks limits the use of VA-mapped segments.
Fortunately UMA_MD_SMALL_ALLOC takes away a lot of VM pressure.
Obtained from: NetBSD
|
| 126474 |
02-Mar-2004 |
grehan |
Kernel changes for libthr (and probably libpthread).
include/ucontext.h
- remove trapframe and switch over to 'generic' description of machine
state. Include version field to help with future modifications.
Include floating point and altivec state, and hopefully align
correctly
powerpc/copyinout.c
- fill out casuptr() sync primitive, required by kern_umtx.c
powerpc/machdep.c
- shifted proc0/thread0/pcpu setup to before cninit, since
syscons -> make_dev -> devlock requires a valid curthread
- implemented get_mcontext/set_mcontext
- recast sendsig/sigreturn to use get/set_mcontext and new
ucontext struct. floating point now saved
- TODO: save/restore altivec state
powerpc/vm_machdep.c
- implemented cpu_thread_setup/cpu_set_upcall/cpu_set_upcall_kse
- eliminated trailing whitespace
Submitted by: Suleiman Souhlal <refugee@segfaulted.com>, ucontext by grehan
|
| 125708 |
11-Feb-2004 |
grehan |
Interrupt statistics, vmstat -i now works.
Submitted by: Suleiman Souhlal <refugee@segfaulted.com>
Slightly modified by: grehan
Derived from: i386
|
| 125702 |
11-Feb-2004 |
grehan |
- constify devinfo strings to eliminate compile warning
- remove trailing whitespace
|
| 125691 |
11-Feb-2004 |
grehan |
- fix compile warnings
- removed obsolete NetBSD-derived ADB conditionals
|
| 125687 |
11-Feb-2004 |
grehan |
Cleaned up param.h:
- culled long-dead #define's
- segment register defs moved to sr.h
- NPMAPS moved to pmap.h
- KERNBASE moved to vmparam.h
- removed include of <machine/cpu.h> and fixed src files that
relied on this.
Modifying segment register code no longer causes gcc rebuilds :-)
|
| 125682 |
11-Feb-2004 |
grehan |
Add sysctl hw.uma_mdpages to track how many pages have been allocated
by UMA_MD_SMALL_ALLOC
|
| 125442 |
04-Feb-2004 |
grehan |
Remove pmap_pvo_allocf zone alloc function. It was a way of
using the direct-mapping of physmem to force PTE data structures
to be physically addressable so the interrupt-time real-mode
DSI trap handler could perform PTE spills. However, the memory
may have been > 256Mb, which would have caused a BAT spill and
double-interrupt.
The new trap code no longer handles PTE spills, so the requirement
that these pages be direct-mapped no longer applies. The irony is
UMA_MD_SMALL_ALLOC will return direct mappings for these structs :-)
|
| 125441 |
04-Feb-2004 |
grehan |
Major overhaul of common trap code
- remove unused 601 and tlb exception code
- remove interrupt-time PTE spill code. The pmap code
will now take care of pinning kernel PTEs, and there
are no longer issues about physical mapping of PTE
data structures
- All segment registers are switched on kernel entry/exit,
allowing the kernel to have more virtual space and for
user virtual space to extend to 4G.
- The temporary register save area has been shifted from
unused exception vector space to the per-cpu data area.
This allows interrupts to be delivered to multiple CPUs
- ISI traps no longer spill to BAT tables. It is assumed
that all of kernel instruction memory is pinned.
- shift from 'ldmw/stmw' instructions to individual register
loads/stores when saving context. All PPC manuals indicate
this should be much faster.
- use '%r' for register names throughout.
TODO: need to test if DSI traps were the result of kernel stack
guard-page hits.
Reworked from: NetBSD
|
| 125440 |
04-Feb-2004 |
grehan |
- remove unused trap definitions
- ISI traps are now handled by the generic trap routine
- direct diagnostic traps to DDB if defined
- remove unused asngen pcpu init
|
| 125438 |
04-Feb-2004 |
grehan |
- Add definition for GET_CPUINFO, required by new trap_subr code
- garbage-collect unused defs
|
| 125434 |
04-Feb-2004 |
grehan |
Allow child devices to set the OpenFirmware device node ivar
|
| 125185 |
29-Jan-2004 |
grehan |
When UMA_MD_SMALL_ALLOC is defined, pmap_kextract will be called
for direct-mapped addresses. Assume that any address less than KVA
is one of these and return it. Also assert that an address is KVA
does have a valid mapping - callers of pmap_kextract don't check
the return value, since they assume that they have a valid virtual
address.
|
| 125184 |
29-Jan-2004 |
grehan |
Implement UMA_MD_SMALL_ALLOC, since the BAT registers allow direct
addressing of memory. Makes a substantial improvement for apps that
stress the limited amount of KVM on PPC (e.g. untarring the ports tree).
uma_machdep.c stolen from amd64/ia64.
|
| 124772 |
21-Jan-2004 |
grehan |
- Catch up with panic __LINE__/__FILE__ changes by moving panic calls
out of asm.
- remove some long-dead code from machdep.c
|
| 124771 |
21-Jan-2004 |
grehan |
A syscons implementation using the 8-bit framebuffer set up by
OpenFirmware. Not at all optimized, but provides a PC-style
user-experience.
Tested on revA imac, B&W G3, 2k iBook, and G4 eMac.
|
| 124468 |
13-Jan-2004 |
grehan |
Remove hard-coded knowledge of specific OFW devices. Use bus_generic_probe
and add_child entry point to allow devices to use the identify
method to add themselves if need be (e.g. openpic, syscons).
Export interrupt-controller-add routine for extern int cntlr drivers.
Eliminate recursive OFW device-tree walk and only iterate the
top-level ala sparc64. Allow child devices to set the device
type with write_ivars.
Step 1 of many in removing the hard-dependency on OpenFirmware.
|
| 124092 |
03-Jan-2004 |
davidxu |
Make sigaltstack as per-threaded, because per-process sigaltstack state
is useless for threaded programs, multiple threads can not share same
stack.
The alternative signal stack is private for thread, no lock is needed,
the orignal P_ALTSTACK is now moved into td_pflags and renamed to
TDP_ALTSTACK.
For single thread or Linux clone() based threaded program, there is no
semantic changed, because those programs only have one kernel thread
in every process.
Reviewed by: deischen, dfr
|
| 123929 |
28-Dec-2003 |
silby |
Track three new sendfile-related statistics:
- The number of times sendfile had to do disk I/O
- The number of times sfbuf allocation failed
- The number of times sfbuf allocation had to wait
|
| 123920 |
28-Dec-2003 |
silby |
Move the declaration of sfbufspeak and sfbufsused to mbuf.h,
and use imax instead of max, as sfbufspeak and sfbufsused
are signed.
Submitted by: bde
|
| 123884 |
27-Dec-2003 |
silby |
Track current and peak sfbuf usage, export the values via sysctl.
|
| 123560 |
16-Dec-2003 |
grehan |
Disable the per-vm_page PTE cache. This was not being invalidated
correctly, resulting in the dreaded "vm_pageout_flush: partially
invalid page" panic. The caching issue will be revisited in the
future, but opt for safety over performance in the meantime.
Tested by: gallatin
|
| 123370 |
10-Dec-2003 |
grehan |
- removed obsolete ppc_exit/ppc_boot functions
- OpenFirmware returns overlapping memory regions. Use a simple
brute force algorithm to merge these into non-overlapping
regions. This fixes bugs in reporting of available memory
and also prevents pages from being added twice in the VM system.
|
| 123354 |
09-Dec-2003 |
gallatin |
pmap_query_bit() should return false if the bit is not set.
Reviewed by: grehan
|
| 123353 |
09-Dec-2003 |
gallatin |
Use the "shut-down" and "reset-all" Forth procedures to halt and
reboot, as calling OF_exit() just hangs a mac.
FreeBSD on my G4 800Mhz mac behaves identically to OSX for halt
and reboot now.
Reviewed by: grehan (who also supplied the concept and sample code)
|
| 122821 |
16-Nov-2003 |
alc |
- Remove unnecessary synchronization from sf_buf_init(). (There is only
one active CPU when sf_buf_init() is performed.)
|
| 122780 |
16-Nov-2003 |
alc |
- Modify alpha's sf_buf implementation to use the direct virtual-to-
physical mapping.
- Move the sf_buf API to its own header file; make struct sf_buf's
definition machine dependent. In this commit, we remove an
unnecessary field from struct sf_buf on the alpha, amd64, and ia64.
Ultimately, we may eliminate struct sf_buf on those architecures
except as an opaque pointer that references a vm page.
|
| 122364 |
09-Nov-2003 |
marcel |
Change the clear_ret argument of get_mcontext() to be a flags argument.
Since all callers either passed 0 or 1 for clear_ret, define bit 0 in
the flags for use as clear_ret. Reserve bits 1, 2 and 3 for use by MI
code for possible (but unlikely) future use. The remaining bits are for
use by MD code.
This change is triggered by a need on ia64 to have another knob for
get_mcontext().
|
| 121237 |
19-Oct-2003 |
peter |
Add a stub cpu_idle() function for sparc64, alpha, powerpc. This is a
MI declared function so it should be everywhere.
|
| 120722 |
03-Oct-2003 |
alc |
Migrate pmap_prefault() into the machine-independent virtual memory layer.
A small helper function pmap_is_prefaultable() is added. This function
encapsulate the few lines of pmap_prefault() that actually vary from
machine to machine. Note: pmap_is_prefaultable() and pmap_mincore() have
much in common. Going forward, it's worth considering their merger.
|
| 120460 |
26-Sep-2003 |
grehan |
DELAY must be a routine, not a macro definition.
|
| 120336 |
22-Sep-2003 |
grehan |
Soften assert in pmap_remove_all.
Introduct pmap_extract_and_hold.
Stolen from: sparc64
|
| 119563 |
29-Aug-2003 |
alc |
Migrate the sf_buf allocator that is used by sendfile(2) and zero-copy
sockets into machine-dependent files. The rationale for this
migration is illustrated by the modified amd64 allocator. It uses the
amd64's direct map to avoid emphemeral mappings in the kernel's
address space. On an SMP, the emphemeral mappings result in an IPI
for TLB shootdown for each transmitted page. Yuck.
Maintainers of other 64-bit platforms with direct maps should be able
to use the amd64 allocator as a reference implementation.
|
| 119004 |
16-Aug-2003 |
marcel |
In vm_thread_swap{in|out}(), remove the alpha specific conditional
compilation and replace it with a call to cpu_thread_swap{in|out}().
This allows us to add similar code on ia64 without cluttering the
code even more.
|
| 118893 |
14-Aug-2003 |
grehan |
Update powerpc to use the (old thread,new thread) calling convention
for cpu_throw() and cpu_switch().
|
| 118365 |
02-Aug-2003 |
alc |
Use kmem_alloc_nofault() rather than kmem_alloc_pageable() in pmap_mapdev().
See revision 1.140 of kern/sys_pipe.c for a detailed rationale.
Submitted by: tegge
|
| 118244 |
31-Jul-2003 |
bmilekic |
Make sure that when the PV ENTRY zone is created in pmap, that it's
created not only with UMA_ZONE_VM but also with UMA_ZONE_NOFREE. In
the i386 case in particular, the pmap code would hook a special
page allocation routine that allocated from kernel_map and not kmem_map,
and so when/if the pageout daemon drained the zones, it could actually
push out slabs from the PV ENTRY zone but call UMA's default page_free,
which resulted in pages allocated from kernel_map being freed to
kmem_map; bad. kmem_free() ignores the return value of the
vm_map_delete and just returns. I'm not sure what the exact
repercussions could be, but it doesn't look good.
In the PAE case on i386, we also set-up a zone in pmap, so be
conservative for now and make that zone also ZONE_NOFREE and
ZONE_VM. Do this for the pmap zones for the other archs too,
although in some cases it may not be entirely necessarily. We'd
rather be safe than sorry at this point.
Perhaps all UMA_ZONE_VM zones should by default be also
UMA_ZONE_NOFREE?
May fix some of silby's crashes on the PV ENTRY zone.
|
| 118239 |
31-Jul-2003 |
peter |
Deal with 'options KSTACK_PAGES' being a global option.
|
| 118100 |
27-Jul-2003 |
alc |
Make pmap_pvo_allocf() callable without Giant.
|
| 117600 |
15-Jul-2003 |
davidxu |
Rename thread_siginfo to cpu_thread_siginfo.
Suggested by: jhb
|
| 117206 |
03-Jul-2003 |
alc |
Background: pmap_object_init_pt() premaps the pages of a object in
order to avoid the overhead of later page faults. In general, it
implements two cases: one for vnode-backed objects and one for
device-backed objects. Only the device-backed case is really
machine-dependent, belonging in the pmap.
This commit moves the vnode-backed case into the (relatively) new
function vm_map_pmap_enter(). On amd64 and i386, this commit only
amounts to code rearrangement. On alpha and ia64, the new machine
independent (MI) implementation of the vnode case is smaller and more
efficient than their pmap-based implementations. (The MI
implementation takes advantage of the fact that objects in -CURRENT
are ordered collections of pages.) On sparc64, pmap_object_init_pt()
hadn't (yet) been implemented.
|
| 117045 |
29-Jun-2003 |
alc |
- Export pmap_enter_quick() to the MI VM. This will permit the
implementation of a largely MI pmap_object_init_pt() for vnode-backed
objects. pmap_enter_quick() is implemented via pmap_enter() on sparc64
and powerpc.
- Correct a mismatch between pmap_object_init_pt()'s prototype and its
various implementations. (I plan to keep pmap_object_init_pt() as
the MD hook for device-backed objects on i386 and amd64.)
- Correct an error in ia64's pmap_enter_quick() and adjust its interface
to match the other versions. Discussed with: marcel
|
| 116958 |
28-Jun-2003 |
davidxu |
Add a machine depended function thread_siginfo, SA signal code
will use the function to construct a siginfo structure and use
the result to export to userland.
Reviewed by: julian
|
| 116804 |
25-Jun-2003 |
grehan |
Remove unused bootpath[] variable. It conflicted with a declaration
in the sunlabel utility, causing build problems.
|
| 116355 |
14-Jun-2003 |
alc |
Migrate the thread stack management functions from the machine-dependent
to the machine-independent parts of the VM. At the same time, this
introduces vm object locking for the non-i386 platforms.
Two details:
1. KSTACK_GUARD has been removed in favor of KSTACK_GUARD_PAGES. The
different machine-dependent implementations used various combinations
of KSTACK_GUARD and KSTACK_GUARD_PAGES. To disable guard page, set
KSTACK_GUARD_PAGES to 0.
2. Remove the (unnecessary) clearing of PG_ZERO in vm_thread_new. In
5.x, (but not 4.x,) PG_ZERO can only be set if VM_ALLOC_ZERO is passed
to vm_page_alloc() or vm_page_grab().
|
| 116328 |
14-Jun-2003 |
alc |
Move the *_new_altkstack() and *_dispose_altkstack() functions out of the
various pmap implementations into the machine-independent vm. They were
all identical.
|
| 116188 |
11-Jun-2003 |
peter |
GC unused cpu_wait() function
|
| 115858 |
04-Jun-2003 |
marcel |
Change the second (and last) argument of cpu_set_upcall(). Previously
we were passing in a void* representing the PCB of the parent thread.
Now we pass a pointer to the parent thread itself.
The prime reason for this change is to allow cpu_set_upcall() to copy
(parts of) the trapframe instead of having it done in MI code in each
caller of cpu_set_upcall(). Copying the trapframe cannot always be
done with a simply bcopy() or may not always be optimal that way. On
ia64 specifically the trapframe contains information that is specific
to an entry into the kernel and can only be used by the corresponding
exit from the kernel. A trapframe copied verbatim from another frame
is in most cases useless without some additional normalization.
Note that this change removes the assignment to td->td_frame in some
implementations of cpu_set_upcall(). The assignment is redundant.
A previous call to cpu_thread_setup() already did the exact same
assignment. An added benefit of removing the redundant assignment is
that we can now change td_pcb without nasty side-effects.
This change officially marks the ability on ia64 for 1:1 threading.
Not tested on: amd64, powerpc
Compile & boot tested on: alpha, sparc64
Functionally tested on: i386, ia64
|
| 115614 |
01-Jun-2003 |
phk |
Remove #include <sys/disklabel.h>
|
| 114983 |
13-May-2003 |
jhb |
- Merge struct procsig with struct sigacts.
- Move struct sigacts out of the u-area and malloc() it using the
M_SUBPROC malloc bucket.
- Add a small sigacts_*() API for managing sigacts structures: sigacts_alloc(),
sigacts_free(), sigacts_copy(), sigacts_share(), and sigacts_shared().
- Remove the p_sigignore, p_sigacts, and p_sigcatch macros.
- Add a mutex to struct sigacts that protects all the members of the struct.
- Add sigacts locking.
- Remove Giant from nosys(), kill(), killpg(), and kern_sigaction() now
that sigacts is locked.
- Several in-kernel functions such as psignal(), tdsignal(), trapsignal(),
and thread_stopped() are now MP safe.
Reviewed by: arch@
Approved by: re (rwatson)
|
| 114727 |
05-May-2003 |
obrien |
Things run thru the C preprocessor must use C-style comments.
|
| 114374 |
01-May-2003 |
peter |
Back out last commits. The elf64/elf32 kernel name thing was more pain
than it was worth.
|
| 114342 |
30-Apr-2003 |
peter |
Fix transcription error. Use == NULL, not != NULL. Fortunately this
was harmless.
|
| 114340 |
30-Apr-2003 |
peter |
Look for an elf32 kernel (powerpc) and elf64 kernel (sparc64) as well
as a plain "elf kernel".
|
| 114305 |
30-Apr-2003 |
jhb |
Range check the syscall number before looking it up in the syscallnames[]
array.
Submitted by: pho
|
| 113998 |
25-Apr-2003 |
deischen |
Add an argument to get_mcontext() which specified whether the
syscall return values should be cleared. The system calls
getcontext() and swapcontext() want to return 0 on success
but these contexts can be switched to at a later time so
the return values need to be cleared in the saved register
sets. Other callers of get_mcontext() would normally want
the context without clearing the return values.
Remove the i386-specific context saving from the KSE code.
get_mcontext() is not i386-specific any more.
Fix a bad pointer in the alpha get_mcontext() code. The
context was being bcopy()'d from &td->tf_frame, but tf_frame
is itself a pointer, so the thread was being copied instead.
Spotted by jake.
Glanced at by: jake
Reviewed by: bde (months ago)
|
| 113038 |
03-Apr-2003 |
obrien |
Use __FBSDID rather than rcsid[].
|
| 112898 |
01-Apr-2003 |
jeff |
- Define a new md function 'casuptr'. This atomically compares and sets
a pointer that is in user space. It will be used as the basic primitive
for a kernel supported user space lock implementation.
- Implement this function in x86's support.s
- Provide stubs that return -1 in all other architectures. Implementations
will follow along shortly.
Reviewed by: jake
|
| 112888 |
31-Mar-2003 |
jeff |
- Move p->p_sigmask to td->td_sigmask. Signal masks will be per thread with
a follow on commit to kern_sig.c
- signotify() now operates on a thread since unmasked pending signals are
stored in the thread.
- PS_NEEDSIGCHK moves to TDF_NEEDSIGCHK.
|
| 112883 |
31-Mar-2003 |
jeff |
- Change trapsignal() to accept a thread and not a proc.
- Change all consumers to pass in a thread.
Right now this does not cause any functional changes but it will be important
later when signals can be delivered to specific threads.
|
| 112429 |
20-Mar-2003 |
grehan |
Enable the FPU on first use per-thread and save state across context
switches. Not as lazy as it could be. Changing FPU state with sigcontext
still TODO.
fpu.c - convert some asm to inline C, and macroize fpu loads/stores
swtch.S - call out to save/restore fpu routines
trap.c - always call enable_fpu, since this shouldn't be called once
the FPU has been enabled for a thread
genassym.c - define for pcb fpu flag
|
| 112402 |
19-Mar-2003 |
grehan |
Add machine check handler. While generally useful, it's required when
issuing PCI config cycles on MPC106-based PowerMacs, which cause machine
checks when accessing non-existent/empty slots.
|
| 111883 |
04-Mar-2003 |
jhb |
Replace calls to WITNESS_SLEEP() and witness_list() with equivalent calls
to WITNESS_WARN().
|
| 111551 |
26-Feb-2003 |
grehan |
Register typo and incorrect 32-bit constant load in previous commit.
Resulted in AST delivery not working.
|
| 111462 |
25-Feb-2003 |
mux |
Cleanup of the d_mmap_t interface.
- Get rid of the useless atop() / pmap_phys_address() detour. The
device mmap handlers must now give back the physical address
without atop()'ing it.
- Don't borrow the physical address of the mapping in the returned
int. Now we properly pass a vm_offset_t * and expect it to be
filled by the mmap handler when the mapping was successful. The
mmap handler must now return 0 when successful, any other value
is considered as an error. Previously, returning -1 was the only
way to fail. This change thus accidentally fixes some devices
which were bogusly returning errno constants which would have been
considered as addresses by the device pager.
- Garbage collect the poorly named pmap_phys_address() now that it's
no longer used.
- Convert all the d_mmap_t consumers to the new API.
I'm still not sure wheter we need a __FreeBSD_version bump for this,
since and we didn't guarantee API/ABI stability until 5.1-RELEASE.
Discussed with: alc, phk, jake
Reviewed by: peter
Compile-tested on: LINT (i386), GENERIC (alpha and sparc64)
Runtime-tested on: i386
|
| 111155 |
20-Feb-2003 |
grehan |
Catch up to latest KSE changes
|
| 111119 |
19-Feb-2003 |
imp |
Back out M_* changes, per decision of the TRB.
Approved by: trb
|
| 111028 |
17-Feb-2003 |
jeff |
- Split the struct kse into struct upcall and struct kse. struct kse will
soon be visible only to schedulers. This greatly simplifies much the
KSE code.
Submitted by: davidxu
|
| 111024 |
17-Feb-2003 |
jeff |
- Move ke_sticks, ke_iticks, ke_uticks, ke_uu, ke_su, and ke_iu back into
the proc. These counters are only examined through calcru.
Submitted by: davidxu
Tested on: x86, alpha, UP/SMP
|
| 110389 |
05-Feb-2003 |
benno |
GC an unused variable.
|
| 110388 |
05-Feb-2003 |
benno |
Export the ns_per_tick variable through md_var.h rather than by declaring
it extern in cpu.c.
|
| 110387 |
05-Feb-2003 |
benno |
- Use cpu_setup() instead of identifycpu().
- Remove identifycpu().
|
| 110381 |
05-Feb-2003 |
benno |
Replace the inline asm in delay() with a while loop. This may not be as
efficient but it appears to actually work. Some investigation may be
required.
|
| 110380 |
05-Feb-2003 |
benno |
- Rename the "powerpc" timecounter to the "decrementer" timecounter.
- Initialise it earlier.
|
| 110296 |
03-Feb-2003 |
jake |
Split statclock into statclock and profclock, and made the method for driving
statclock based on profhz when profiling is enabled MD, since most platforms
don't use this anyway. This removes the need for statclock_process, whose
only purpose was to subdivide profhz, and gets the profiling clock running
outside of sched_lock on platforms that implement suswintr.
Also changed the interface for starting and stopping the profiling clock to
do just that, instead of changing the rate of statclock, since they can now
be separate.
Reviewed by: jhb, tmm
Tested on: i386, sparc64
|
| 110190 |
01-Feb-2003 |
julian |
Reversion of commit by Davidxu plus fixes since applied.
I'm not convinced there is anything major wrong with the patch but
them's the rules..
I am using my "David's mentor" hat to revert this as he's
offline for a while.
|
| 110172 |
01-Feb-2003 |
grehan |
- add pmap_pagedaemon_waken variable
- remove dead code and fix warnings in pmap_zero_page/zero_page_area
- implement
pmap_clear_reference
pmap_ts_referenced
pmap_page_exists_quick
pmap_remove_all
- align pmap_qenter/qremove closer with i386 code
- fix vm_page locking in pmap_new_thread (from benno)
- add new parameter to pmap_clear_bit to return original
pte value
Approved by: benno
|
| 109920 |
27-Jan-2003 |
benno |
Back out some changes that snuck in with the last commit.
Pointy hat to: benno
|
| 109919 |
27-Jan-2003 |
benno |
Flesh out bus_dmamap_sync.
|
| 109918 |
27-Jan-2003 |
benno |
Use td->td_sticks, not td->td_kse->ke_sticks.
Forgotten by: davidxu
|
| 109877 |
26-Jan-2003 |
davidxu |
Move UPCALL related data structure out of kse, introduce a new
data structure called kse_upcall to manage UPCALL. All KSE binding
and loaning code are gone.
A thread owns an upcall can collect all completed syscall contexts in
its ksegrp, turn itself into UPCALL mode, and takes those contexts back
to userland. Any thread without upcall structure has to export their
contexts and exit at user boundary.
Any thread running in user mode owns an upcall structure, when it enters
kernel, if the kse mailbox's current thread pointer is not NULL, then
when the thread is blocked in kernel, a new UPCALL thread is created and
the upcall structure is transfered to the new UPCALL thread. if the kse
mailbox's current thread pointer is NULL, then when a thread is blocked
in kernel, no UPCALL thread will be created.
Each upcall always has an owner thread. Userland can remove an upcall by
calling kse_exit, when all upcalls in ksegrp are removed, the group is
atomatically shutdown. An upcall owner thread also exits when process is
in exiting state. when an owner thread exits, the upcall it owns is also
removed.
KSE is a pure scheduler entity. it represents a virtual cpu. when a thread
is running, it always has a KSE associated with it. scheduler is free to
assign a KSE to thread according thread priority, if thread priority is changed,
KSE can be moved from one thread to another.
When a ksegrp is created, there is always N KSEs created in the group. the
N is the number of physical cpu in the current system. This makes it is
possible that even an userland UTS is single CPU safe, threads in kernel still
can execute on different cpu in parallel. Userland calls kse_create to add more
upcall structures into ksegrp to increase concurrent in userland itself, kernel
is not restricted by number of upcalls userland provides.
The code hasn't been tested under SMP by author due to lack of hardware.
Reviewed by: julian
|
| 109677 |
22-Jan-2003 |
grehan |
Remove BAT invalidation. This is done later in the boot sequence,
so isn't required here, and seems to cause problems when booting
from disk.
Approved by: benno
|
| 109623 |
21-Jan-2003 |
alfred |
Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.
|
| 109342 |
16-Jan-2003 |
dillon |
Merge all the various copies of vm_fault_quick() into a single
portable copy.
|
| 109340 |
15-Jan-2003 |
dillon |
Merge all the various copies of vmapbuf() and vunmapbuf() into a single
portable copy. Note that pmap_extract() must be used instead of
pmap_kextract().
This is precursor work to a reorganization of vmapbuf() to close remaining
user/kernel races (which can lead to a panic).
|
| 108945 |
08-Jan-2003 |
grehan |
Add page queues locking to vunmapbuf().
Obtained from: sparc64
Approved by: benno
|
| 108944 |
08-Jan-2003 |
grehan |
Sync the i-cache after copying down the interrupt code
Approved by: benno
|
| 108943 |
08-Jan-2003 |
grehan |
Be more conservative about re-enabling interrupts during trap processing
until atomic issues are fully sorted.
Approved by: benno
|
| 108942 |
08-Jan-2003 |
grehan |
Fix incorrect error returns and sign-extension.
Approved by: benno
|
| 108941 |
08-Jan-2003 |
grehan |
Fetch the initial time from the rtc OpenFirmware node. This is a short-term
measure until the rtc h/w driver is written, and it's a lot better
than having "jan 1 1970" on filesys times.
Approved by: benno
|
| 107719 |
10-Dec-2002 |
julian |
Unbreak the KSE code. Keep track of zobie threads using the Per-CPU storage
during the context switch. Rearrange thread cleanups
to avoid problems with Giant. Clean threads when freed or
when recycled.
Approved by: re (jhb)
|
| 107180 |
22-Nov-2002 |
mux |
Under certain circumstances, we were calling kmem_free() from
i386 cpu_thread_exit(). This resulted in a panic with WITNESS
since we need to hold Giant to call kmem_free(), and we weren't
helding it anymore in cpu_thread_exit(). We now do this from a
new MD function, cpu_thread_dtor(), called by thread_dtor().
Approved by: re@
Suggested by: jhb
|
| 106977 |
16-Nov-2002 |
deischen |
Add getcontext, setcontext, and swapcontext as system calls.
Previously these were libc functions but were requested to
be made into system calls for atomicity and to coalesce what
might be two entrances into the kernel (signal mask setting
and floating point trap) into one.
A few style nits and comments from bde are also included.
Tested on alpha by: gallatin
|
| 106838 |
13-Nov-2002 |
alc |
Move pmap_collect() out of the machine-dependent code, rename it
to reflect its new location, and add page queue and flag locking.
Notes: (1) alpha, i386, and ia64 had identical implementations
of pmap_collect() in terms of machine-independent interfaces;
(2) sparc64 doesn't require it; (3) powerpc had it as a TODO.
|
| 106697 |
09-Nov-2002 |
des |
Print real / avail memory in megabytes rather than kilobytes.
|
| 106605 |
07-Nov-2002 |
tmm |
Move the definitions of the hw.physmem, hw.usermem and hw.availpages
sysctls to MI code; this reduces code duplication and makes all of them
available on sparc64, and the latter two on powerpc.
The semantics by the i386 and pc98 hw.availpages is slightly changed:
previously, holes between ranges of available pages would be included,
while they are excluded now. The new behaviour should be more correct
and brings i386 in line with the other architectures.
Move physmem to vm/vm_init.c, where this variable is used in MI code.
|
| 106503 |
06-Nov-2002 |
jmallett |
Remove what was a temporary bogus assignment of bits of siginfo_t, as it does
not look like the prerequisites to fill it in properly will be in the tree
for the upcoming release, but it's mostly done, so there is no need for these
to stay around to remind us.
|
| 105950 |
25-Oct-2002 |
peter |
Split 4.x and 5.x signal handling so that we can keep 4.x signal
handling clean and functional as 5.x evolves. This allows some of the
nasty bandaids in the 5.x codepaths to be unwound.
Encapsulate 4.x signal handling under COMPAT_FREEBSD4 (there is an
anti-foot-shooting measure in place, 5.x folks need this for a while) and
finish encapsulating the older stuff under COMPAT_43. Since the ancient
stuff is required on alpha (longjmp(3) passes a 'struct osigcontext *'
to the current sigreturn(2), instead of the 'ucontext_t *' that sigreturn
is supposed to take), add a compile time check to prevent foot shooting
there too. Add uniform COMPAT_43 stubs for ia64/sparc64/powerpc.
Tested on: i386, alpha, ia64. Compiled on sparc64 (a few days ago).
Approved by: re
|
| 105611 |
21-Oct-2002 |
grehan |
Add the USER_SR segment register to pcb state. Initialize correctly,
and save/restore during a context switch.
The USER_SR could be overwritten when the current thread was switched
out with a faulting copyin/copyout.
Approved by: Benno
|
| 104435 |
04-Oct-2002 |
grehan |
Clean up ddb warnings/errors and enable in GENERIC
Approved by: benno
Motivated by: gallatin
|
| 104434 |
04-Oct-2002 |
grehan |
- fix zero-sized stack alloc from previous commit. a default is now
selected ala sparc64
- KSEIII routines implemented (taken from i386/sparc64)
Approved by: Benno
|
| 104354 |
02-Oct-2002 |
scottl |
Some kernel threads try to do significant work, and the default KSTACK_PAGES
doesn't give them enough stack to do much before blowing away the pcb.
This adds MI and MD code to allow the allocation of an alternate kstack
who's size can be speficied when calling kthread_create. Passing the
value 0 prevents the alternate kstack from being created. Note that the
ia64 MD code is missing for now, and PowerPC was only partially written
due to the pmap.c being incomplete there.
Though this patch does not modify anything to make use of the alternate
kstack, acpi and usb are good candidates.
Reviewed by: jake, peter, jhb
|
| 103609 |
19-Sep-2002 |
grehan |
- bring vm_mapbuf/unmapbuf in line with other archs
- update for recent KSE changes
Approved by: benno
|
| 103608 |
19-Sep-2002 |
grehan |
- make sure recoverable interrupts are re-enabled in the trap handler
- turn on ast() loop to enable signal delivery
Approved by: benno
|
| 103607 |
19-Sep-2002 |
grehan |
- worked around 32-bit big-endian syscall return value problem
- syscall register spills weren't copied in correctly
- removed VM_PROT_READ from the fault type on write protect faults
Approved by: benno
|
| 103604 |
19-Sep-2002 |
grehan |
- use BAT registers to map device space and physical memory
- remove test in pmap_activate that prevented vmspace sharing (v/rfork)
- always sync icache in pmap_enter until problems are sorted
- fix incorrect use of regions in pmap_kenter
- bring in pmap_release from NetBSD
- fix overwrite of bootstrap flag in pmap_pvo_enter
Approved by: benno
|
| 103602 |
19-Sep-2002 |
grehan |
Clear on-demand BAT entries to properly restore OpenFirmware's
address space
Approved by: benno
|
| 103601 |
19-Sep-2002 |
grehan |
psim device support
Approved by: benno
|
| 103600 |
19-Sep-2002 |
grehan |
- implemented sendsig/sigreturn
- sysctl for cacheline size, required by libc/rtld
- init'd more exception vectors
- fixed problem with register overwrite in exec_setregs
- removed redundant NetBSD code
Approved by: benno
|
| 103599 |
19-Sep-2002 |
grehan |
- moved intrcnt/intrnames to locore.s to fix sysctl -a panic
Approved by: benno
|
| 103367 |
15-Sep-2002 |
julian |
Allocate KSEs and KSEGRPs separatly and remove them from the proc structure.
next step is to allow > 1 to be allocated per process. This would give
multi-processor threads. (when the rest of the infrastructure is
in place)
While doing this I noticed libkvm and sys/kern/kern_proc.c:fill_kinfo_proc
are diverging more than they should.. corrective action needed soon.
|
| 103049 |
07-Sep-2002 |
peter |
Zap the implementations of the i386-aout specific cpu_coredump function.
Most of the non-i386 platforms had rather broken implementations anyway.
|
| 102666 |
31-Aug-2002 |
peter |
Take a shot at fixing up a whole stack of style and other embarresing
unforced errors that Bruce identified. I have not yet addressed all of
his concerns.
|
| 102600 |
30-Aug-2002 |
peter |
Change hw.physmem and hw.usermem to unsigned long like they used to be
in the original hardwired sysctl implementation.
The buf size calculator still overflows an integer on machines with large
KVA (eg: ia64) where the number of pages does not fit into an int. Use
'long' there.
Change Maxmem and physmem and related variables to 'long', mostly for
completeness. Machines are not likely to overflow 'int' pages in the
near term, but then again, 640K ought to be enough for anybody. This
comes for free on 32 bit machines, so why not?
|
| 102561 |
29-Aug-2002 |
jake |
Renamed poorly named setregs to exec_setregs. Moved its prototype to
imgact.h with the other exec support functions.
|
| 102399 |
25-Aug-2002 |
alc |
o Retire pmap_pageable(). It's an advisory routine that none
of our platforms implements.
|
| 101941 |
15-Aug-2002 |
rwatson |
In order to better support flexible and extensible access control,
make a series of modifications to the credential arguments relating
to file read and write operations to cliarfy which credential is
used for what:
- Change fo_read() and fo_write() to accept "active_cred" instead of
"cred", and change the semantics of consumers of fo_read() and
fo_write() to pass the active credential of the thread requesting
an operation rather than the cached file cred. The cached file
cred is still available in fo_read() and fo_write() consumers
via fp->f_cred. These changes largely in sys_generic.c.
For each implementation of fo_read() and fo_write(), update cred
usage to reflect this change and maintain current semantics:
- badfo_readwrite() unchanged
- kqueue_read/write() unchanged
pipe_read/write() now authorize MAC using active_cred rather
than td->td_ucred
- soo_read/write() unchanged
- vn_read/write() now authorize MAC using active_cred but
VOP_READ/WRITE() with fp->f_cred
Modify vn_rdwr() to accept two credential arguments instead of a
single credential: active_cred and file_cred. Use active_cred
for MAC authorization, and select a credential for use in
VOP_READ/WRITE() based on whether file_cred is NULL or not. If
file_cred is provided, authorize the VOP using that cred,
otherwise the active credential, matching current semantics.
Modify current vn_rdwr() consumers to pass a file_cred if used
in the context of a struct file, and to always pass active_cred.
When vn_rdwr() is used without a file_cred, pass NOCRED.
These changes should maintain current semantics for read/write,
but avoid a redundant passing of fp->f_cred, as well as making
it more clear what the origin of each credential is in file
descriptor read/write operations.
Follow-up commits will make similar changes to other file descriptor
operations, and modify the MAC framework to pass both credentials
to MAC policy modules so they can implement either semantic for
revocation.
Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs
|
| 101346 |
05-Aug-2002 |
alc |
o Don't set PG_MAPPED or PG_WRITEABLE when a page is mapped
using pmap_kenter() or pmap_qenter().
o Use VM_ALLOC_WIRED in pmap_new_thread().
|
| 100319 |
18-Jul-2002 |
benno |
Remove the statically allocated array that holds OpenFirmware memory mappings
during pmap_bootstrap. Instead, temporarily help ourselves to some memory
from phys_avail since we won't need it post-boostrap.
|
| 99900 |
13-Jul-2002 |
mini |
Add additional cred_free_thread() calls that I had missed the first time.
Pointed out by: jhb
|
| 99887 |
12-Jul-2002 |
jhb |
Set the thread state of the newly chosen to run thread to TDS_RUNNING in
choosethread() in MI C code instead of doing it in in assembly in all the
various cpu_switch() functions. This fixes problems on ia64 and sparc64.
Reviewed by: julian, peter, benno
Tested on: i386, alpha, sparc64
|
| 99730 |
10-Jul-2002 |
benno |
Add DDB support.
|
| 99729 |
10-Jul-2002 |
benno |
- Make sure we don't trample our metadata pointer in our initial bootstrap.
- Load metadata parameters.
|
| 99725 |
10-Jul-2002 |
benno |
Remove some diagnostic code that snuck in.
|
| 99666 |
09-Jul-2002 |
benno |
Add an implementation for pmap_zero_page_area.
|
| 99665 |
09-Jul-2002 |
benno |
Add the OF_getetheraddr function required by if_gem.
|
| 99664 |
09-Jul-2002 |
benno |
Tidy up trap vector and external interrupt setup.
|
| 99659 |
09-Jul-2002 |
benno |
Changes for KSE3.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 99652 |
09-Jul-2002 |
benno |
- Add the "compatible" property to the list that we keep in ivars.
- Add interrupt alloc/setup/teardown/dealloc support, via whichever PIC
OpenFirmware gives us.
|
| 99571 |
08-Jul-2002 |
peter |
Add a special page zero entry point intended to be called via the single
threaded VM pagezero kthread outside of Giant. For some platforms, this
is really easy since it can just use the direct mapped region. For others,
IPI sending is involved or there are other issues, so grab Giant when
needed.
We still have preemption issues to deal with, but Alan Cox has an
interesting suggestion on how to minimize the problem on x86.
Use Luigi's hack for preserving the (lack of) priority.
Turn the idle zeroing back on since it can now actually do something useful
outside of Giant in many cases.
|
| 99559 |
07-Jul-2002 |
peter |
Collect all the (now equivalent) pmap_new_proc/pmap_dispose_proc/
pmap_swapin_proc/pmap_swapout_proc functions from the MD pmap code
and use a single equivalent MI version. There are other cleanups
needed still.
While here, use the UMA zone hooks to keep a cache of preinitialized
proc structures handy, just like the thread system does. This eliminates
one dependency on 'struct proc' being persistent even after being freed.
There are some comments about things that can be factored out into
ctor/dtor functions if it is worth it. For now they are mostly just
doing statistics to get a feel of how it is working.
|
| 99557 |
07-Jul-2002 |
peter |
Update for post-kse3 pmap kthread allocation changes
|
| 99038 |
29-Jun-2002 |
benno |
Add pmap_mapdev and pmap_unmapdev.
|
| 99037 |
29-Jun-2002 |
benno |
- Initialise battable to cover I/O spaces.
- Statically size the bpvo entries to avoid conflicts between bpvo allocation
and the vm allocator.
- Shift pmap_init2 code into pmap_init.
- Add UMA_ZONE_VM flag to uma_zcreate.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 99036 |
29-Jun-2002 |
benno |
To quote Peter:
The case in cpu_switch() where there isn't a higher priority thread
(choosethread() == curthread) uses r4 as the PCB context pointer. However, the
use of r4 after the label L2 is incorrect, since it was probably trashed by
the call to choosethread, and in any case was set up to curthread at the start
of the routine.
This condition will occur when an interrupt thread schedules a netisr, which
is a lower priority thread.
Another (probably unnecessary) difference is that I was paranoid about
register trashing, so I decided to save r2 and r13 as well.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 99033 |
29-Jun-2002 |
benno |
- Use tmpstk exclusively in the init path.
- Remove redundant code.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 99032 |
29-Jun-2002 |
benno |
Many fixes to low-level trap and interrupt handling:
- Tidy up clock code. Don't repeatedly call hardclock().
- Remove intrnames, decrnest and intrcnt from locore.s
- Coalesce all trap handling into a single stub that then calls a dispatch
function.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 99030 |
29-Jun-2002 |
benno |
Convert this from mostly inline assembler to mostly C.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 98765 |
24-Jun-2002 |
jake |
Add an MD callout like cpu_exit, but which is called after sched_lock is
obtained, when all other scheduling activity is suspended. This is needed
on sparc64 to deactivate the vmspace of the exiting process on all cpus.
Otherwise if another unrelated process gets the exact same vmspace structure
allocated to it (same address), its address space will not be activated
properly. This seems to fix some spontaneous signal 11 problems with smp
on sparc64.
|
| 98727 |
24-Jun-2002 |
mini |
Remove unused diagnostic function cread_free_thread().
Approved by: alfred
|
| 98480 |
20-Jun-2002 |
peter |
Deorbit suibyte(). It was only used for split address space systems
for supporting UIO_USERISPACE (ie: it wasn't used).
|
| 98001 |
07-Jun-2002 |
jhb |
- Fixup / remove obsolete comments.
- ktrace no longer requires Giant so do ktrace syscall events before and
after acquiring and releasing Giant, respectively.
- For i386, ia32 syscalls on ia64, powerpc, and sparc64, get rid of the
goto bad hack and instead use the model on ia64 and alpha were we
skip the actual syscall invocation if error != 0. This fixes a bug
where if we the copyin() of the arguments failed for a syscall that
was not marked MP safe, we would try to release Giant when we had
not acquired it.
|
| 97399 |
28-May-2002 |
benno |
The stack is not at the top of the user struct.
|
| 97398 |
28-May-2002 |
benno |
Remove an assertion as to whether the current thread already had the FPU or
not. It may be desirable to put something similar back, but it's getting in
the way in it's current form.
|
| 97397 |
28-May-2002 |
benno |
- Move macros that represent where syscall args are kept in a trapframe from
trap.c to frame.h
- Use the macros in vm_machdep.c:cpu_fork() to set up the trap frame of the
new thread.
|
| 97395 |
28-May-2002 |
benno |
Remove the old prototype for kcopy. It's in cpu.h now.
|
| 97385 |
28-May-2002 |
benno |
Implement pmap_copy and pmap_copy_page.
|
| 97384 |
28-May-2002 |
benno |
Move the kcopy() function from trap.c to machdep.c. Add a prototype.
|
| 97347 |
27-May-2002 |
benno |
Print srr1 in printtrap()
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 97346 |
27-May-2002 |
benno |
Get the correct memory regions from OpenFirmware. We were getting the
"available" ranges, not the "physical" ranges. Clean up some of the
bootstrap code in the process.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
| 97342 |
27-May-2002 |
benno |
Use correct types in [sf]uword32.
|
| 97307 |
26-May-2002 |
dfr |
Add declarations of suword32 and suword64. Add implementations of one or
the other (or both) to all the platforms. Similar for fuword32 and
fuword64.
|
| 96938 |
19-May-2002 |
benno |
Make this more FreeBSD-ish.
Requested by: jhb
|
| 96906 |
19-May-2002 |
benno |
- Do a quick style pass.
- Correct the implementation of fix_unaligned to use a thread, not a proc.
- GC some #if 0'd stuff.
|
| 96773 |
17-May-2002 |
benno |
- Rename the _C_LABEL macro to CNAME.
- Rename the _ASM_LABEL macro to ASMNAME.
- Add the HIDENAME macro which is used in libc's syscall stuff.
|
| 96499 |
13-May-2002 |
benno |
FPU support.
Obtained from: NetBSD (portions)
|
| 96452 |
12-May-2002 |
benno |
More locking fixes.
|
| 96443 |
12-May-2002 |
benno |
Do the correct locking on processes for DSI and ISI traps.
Copied from: sparc64
|
| 96353 |
10-May-2002 |
benno |
Implement the following functions:
- pmap_addr_hint
- pmap_change_wiring
- pmap_extract
- pmap_is_modified
|
| 96352 |
10-May-2002 |
benno |
Install the system call trap handler.
|
| 96334 |
10-May-2002 |
benno |
Improve our detection of an attempted duplicate entry. We may be trying to
change the page protection bits.
|
| 96333 |
10-May-2002 |
benno |
Remove a debugging printf that escaped.
|
| 96255 |
09-May-2002 |
benno |
Update to newer trap code from NetBSD.
Obtained from: NetBSD
|
| 96253 |
09-May-2002 |
benno |
Add an assertion that we have a current pmap set before we try and return.
|
| 96252 |
09-May-2002 |
benno |
The per-cpu curpmap is now set by pmap_activate. We don't need to do it here
anymore.
|
| 96251 |
09-May-2002 |
benno |
- Add a prototype for the setfault() function.
- Remove some stray printf()s.
|
| 96250 |
09-May-2002 |
benno |
1. Better track the executable status of mappings.
2. Set a pcpu variable to the real address of the active pmap (used when
exiting from traps.
Obtained from: NetBSD (1)
|
| 95814 |
30-Apr-2002 |
phk |
Don't export timecounter structures under debug. with sysctl, they
contain no truly interesting data anymore.
|
| 95719 |
29-Apr-2002 |
benno |
Commit of stuff that's been sitting in my tree for a while.
Highlights include:
- New low-level trap code from NetBSD. The high level code still needs a lot
of work.
- Fixes for some pmap handling in thread switching.
- The kernel will now get to attempting to jump into init in user mode. There
are some pmap/trap issues which prevent it from actually getting there though.
Obtained from: NetBSD (parts)
|
| 95714 |
29-Apr-2002 |
benno |
- Add back calls to setfault that were removed when these functions were moved.
|
| 95710 |
29-Apr-2002 |
peter |
Tidy up some loose ends.
i386/ia64/alpha - catch up to sparc64/ppc:
- replace pmap_kernel() with refs to kernel_pmap
- change kernel_pmap pointer to (&kernel_pmap_store)
(this is a speedup since ld can set these at compile/link time)
all platforms (as suggested by jake):
- gc unused pmap_reference
- gc unused pmap_destroy
- gc unused struct pmap.pm_count
(we never used pm_count - we track address space sharing at the vmspace)
|
| 95564 |
27-Apr-2002 |
alc |
MFi386 1.222: Remove vm_map_growstack() and acquisition and release of Giant
around vm_fault() in trap_pfault().
|
| 95121 |
20-Apr-2002 |
benno |
Replace inline asm with it's inline function wrapper.
|
| 94839 |
16-Apr-2002 |
benno |
Correct a comment.
|
| 94838 |
16-Apr-2002 |
benno |
Implement the following functions:
- pmap_kextract
- pmap_object_init_pt
- pmap_protect
- pmap_remove_pages
I'm pretty sure pmap_remove_pages is at least somewhat bogus.
|
| 94837 |
16-Apr-2002 |
benno |
Remove some dead code.
|
| 94836 |
16-Apr-2002 |
benno |
Use mtsrin() instead of inline asm.
|
| 94835 |
16-Apr-2002 |
benno |
Change the value of PMAP_BOOTSTRAP so we don't stomp on the PTE index value.
|
| 94777 |
15-Apr-2002 |
peter |
Pass vm_page_t instead of physical addresses to pmap_zero_page[_area]()
and pmap_copy_page(). This gets rid of a couple more physical addresses
in upper layers, with the eventual aim of supporting PAE and dealing with
the physical addressing mostly within pmap. (We will need either 64 bit
physical addresses or page indexes, possibly both depending on the
circumstances. Leaving this to pmap itself gives more flexibilitly.)
Reviewed by: jake
Tested on: i386, ia64 and (I believe) sparc64. (my alpha was hosed)
|
| 94755 |
15-Apr-2002 |
benno |
Add a nexus device.
Copied from: sparc64
|
| 94753 |
15-Apr-2002 |
benno |
Turn some CTR's into CTR0's.
|
| 94275 |
09-Apr-2002 |
phk |
GC various bits and pieces of USERCONFIG from all over the place.
|
| 94151 |
07-Apr-2002 |
phk |
GC the "dumplo" variable, which is no longer used.
A lot of sys/*/*/machdep.c seems not to be.
|
| 93702 |
02-Apr-2002 |
jhb |
- Move the MI mutexes sched_lock and Giant from being declared in the
various machdep.c's to being declared in kern_mutex.c.
- Add a new function mutex_init() used to perform early initialization
needed for mutexes such as setting up thread0's contested lock list
and initializing MI mutexes. Change the various MD startup routines
to call this function instead of duplicating all the code themselves.
Tested on: alpha, i386
|
| 93452 |
30-Mar-2002 |
alc |
Use the MI vm_map_growstack() instead of the MD grow_stack() in trap(). Remove
the MD grow_stack().
|
| 93273 |
27-Mar-2002 |
jeff |
Add a new mtx_init option "MTX_DUPOK" which allows duplicate acquires of locks
with this flag. Remove the dup_list and dup_ok code from subr_witness. Now
we just check for the flag instead of doing string compares.
Also, switch the process lock, process group lock, and uma per cpu locks over
to this interface. The original mechanism did not work well for uma because
per cpu lock names are unique to each zone.
Approved by: jhb
|
| 92916 |
21-Mar-2002 |
benno |
Collect all functions for copying to and from userspace into the one file.
This allows me to reimplement [sf]u{byte,word} as separate functions and not
as calls to copy{in,out}.
|
| 92847 |
21-Mar-2002 |
jeff |
Remove references to vm_zone.h and switch over to the new uma API.
|
| 92842 |
20-Mar-2002 |
alfred |
Remove __P.
Reveiwed by: benno
|
| 92824 |
20-Mar-2002 |
jhb |
Change the way we ensure td_ucred is NULL if DIAGNOSTIC is defined.
Instead of caching the ucred reference, just go ahead and eat the
decerement and increment of the refcount. Now that Giant is pushed down
into crfree(), we no longer have to get Giant in the common case. In the
case when we are actually free'ing the ucred, we would normally free it on
the next kernel entry, so the cost there is not new, just in a different
place. This also removse td_cache_ucred from struct thread. This is
still only done #ifdef DIAGNOSTIC.
Tested on: i386, alpha
|
| 92757 |
20-Mar-2002 |
benno |
Increment pmap_pvo_count in the right place.
|
| 92654 |
19-Mar-2002 |
jeff |
This is the first part of the new kernel memory allocator. This replaces
malloc(9) and vm_zone with a slab like allocator.
Reviewed by: arch@
|
| 92521 |
17-Mar-2002 |
benno |
Changes and fixes in preparation for UMA:
- Bootstrap pvo entries are now allocated by stealing pages.
- Just return if we're pmap_enter'ing a mapping that's already there. Don't
remove it and re-enter it.
|
| 92520 |
17-Mar-2002 |
benno |
Lowercase all of the trap names.
|
| 92519 |
17-Mar-2002 |
benno |
Clean up and fix up copyin and copyout.
|
| 92067 |
11-Mar-2002 |
benno |
Correct a typo. (* that should've been &)
|
| 91803 |
07-Mar-2002 |
benno |
Install the DSI and ISI trap handlers and their appropriate locations.
|
| 91802 |
07-Mar-2002 |
benno |
Copy the "implementation" of pmap_prefault from sparc64.
|
| 91794 |
07-Mar-2002 |
benno |
Move tunable initialisation so it can get access to physmem.
|
| 91793 |
07-Mar-2002 |
benno |
Calculate physmem.
|
| 91504 |
28-Feb-2002 |
arr |
- Move a comment from being on the same line as a #ifdef to the line
following it. This should have gone in the previous commit, but
misviewed Bruce's patch.
Requested by: bde
|
| 91486 |
28-Feb-2002 |
benno |
cpu_switch now works, for kthreads at least.
|
| 91485 |
28-Feb-2002 |
benno |
Various cleanups.
|
| 91484 |
28-Feb-2002 |
benno |
- Prevent the decrementer interrupt handler from nesting.
- Catch some more cases of PSL_EE and PSL_RI getting out of sync.
|
| 91483 |
28-Feb-2002 |
benno |
- Modify pmap_activate so it only marks the pmap as active.
- Add a pmap_deactivate function.
|
| 91477 |
28-Feb-2002 |
benno |
GC an unused variable in cpu_fork().
|
| 91475 |
28-Feb-2002 |
arr |
- Fix panic() message and a couple style nits that snuck in from the
recent diagnostics commit (rev. 1.84).
|
| 91467 |
28-Feb-2002 |
benno |
Make fork work, at least for kthreads. Switching still has some issues.
|
| 91465 |
28-Feb-2002 |
benno |
- Rearrange the sequence of events in powerpc_init() somewhat.
- Catch another instance of PSL_EE being cleared without PSL_RI.
|
| 91456 |
28-Feb-2002 |
benno |
Implement the following functions:
- pmap_remove
- pmap_kremove
- pmap_qremove
|
| 91455 |
28-Feb-2002 |
benno |
Remove most of the usage of critical_enter/exit.
I put these in to match the use of spl*() in the NetBSD code I was basing this
on, but it appears to cause problems.
I'm doing this in a separate commit so as to be able to refer back if locking
becomes an issue at a later stage.
|
| 91403 |
27-Feb-2002 |
silby |
Fix a horribly suboptimal algorithm in the vm_daemon.
In order to determine what to page out, the vm_daemon checks
reference bits on all pages belonging to all processes. Unfortunately,
the algorithm used reacted badly with shared pages; each shared page
would be checked once per process sharing it; this caused an O(N^2)
growth of tlb invalidations. The algorithm has been changed so that
each page will be checked only 16 times.
Prior to this change, a fork/sleepbomb of 1300 processes could cause
the vm_daemon to take over 60 seconds to complete, effectively
freezing the system for that time period. With this change
in place, the vm_daemon completes in less than a second. Any system
with hundreds of processes sharing pages should benefit from this change.
Note that the vm_daemon is only run when the system is under extreme
memory pressure. It is likely that many people with loaded systems saw
no symptoms of this problem until they reached the point where swapping
began.
Special thanks go to dillon, peter, and Chuck Cranor, who helped me
get up to speed with vm internals.
PR: 33542, 20393
Reviewed by: dillon
MFC after: 1 week
|
| 91131 |
23-Feb-2002 |
benno |
Don't call critical_enter()/critical_exit() around calls to pmap_pvo_enter()
as it does it's own handling of critical sections.
|
| 91090 |
22-Feb-2002 |
julian |
Add some DIAGNOSTIC code.
While in userland, keep the thread's ucred reference in a shadow
field so that the usual place to store it is NULL.
If DIAGNOSTIC is not set, the thread ucred is kept valid until the next
kernel entry, at which time it is checked against the process cred
and possibly corrected. Produces a BIG speedup in
kernels with INVARIANTS set. (A previous commit corrected it
for the non INVARIANTS case already)
Reviewed by: dillon@freebsd.org
|
| 90895 |
19-Feb-2002 |
julian |
Add change to teh PPC to keep it in step with i386 and MI code
Pointy hat this direction please...
|
| 90643 |
14-Feb-2002 |
benno |
Complete rework of the PowerPC pmap and a number of other bits in the early
boot sequence.
The new pmap.c is based on NetBSD's newer pmap.c (for the mpc6xx processors)
which is 70% faster than the older code that the original pmap.c was based
on. It has also been based on the framework established by jake's initial
sparc64 pmap.c.
There is no change to how far the kernel gets (it makes it to the mountroot
prompt in psim) but the new pmap code is a lot cleaner.
Obtained from: NetBSD (pmap code)
|
| 90361 |
07-Feb-2002 |
julian |
Pre-KSE/M3 commit.
this is a low-functionality change that changes the kernel to access the main
thread of a process via the linked list of threads rather than
assuming that it is embedded in the process. It IS still embeded there
but remove all teh code that assumes that in preparation for the next commit
which will actually move it out.
Reviewed by: peter@freebsd.org, gallatin@cs.duke.edu, benno rice,
|
| 90065 |
01-Feb-2002 |
bde |
Compile osigreturn() unconditionally since it will always be needed on
some arches and the syscall table is machine-independent. It was
(bogusly) conditional on COMPAT_43, so this usually makes no difference.
ia64: in addition:
- replace the bogus cloned comment before osigreturn() by a correct one.
osigreturn() is just a stub fo ia64's.
- fix the formatting of cloned comment before sigreturn().
- fix the return code. use nosys() instead of returning ENOSYS to get
the same semantics as if the syscall is not in the syscall table.
Generating SIGSYS is actually correct here.
- fix style bugs.
powerpc: copy the cleaned up ia64 stub. This mainly fixes a bogus comment.
sparc64: copy the cleaned up the ia64 stub, since there was no stub before.
|
| 89919 |
28-Jan-2002 |
gallatin |
Simple fixes to get the powerpc kernel compiling again.
Reviewed by: mp
|
| 87702 |
11-Dec-2001 |
jhb |
Overhaul the per-CPU support a bit:
- The MI portions of struct globaldata have been consolidated into a MI
struct pcpu. The MD per-CPU data are specified via a macro defined in
machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the
interface would be cleaner (PCPU_GET(my_md_field) vs.
PCPU_GET(md.md_my_md_field)).
- All references to globaldata are changed to pcpu instead. In a UP kernel,
this data was stored as global variables which is where the original name
came from. In an SMP world this data is per-CPU and ideally private to each
CPU outside of the context of debuggers. This also included combining
machine/globaldata.h and machine/globals.h into machine/pcpu.h.
- The pointer to the thread using the FPU on i386 was renamed from
npxthread to fpcurthread to be identical with other architectures.
- Make the show pcpu ddb command MI with a MD callout to display MD
fields.
- The globaldata_register() function was renamed to pcpu_init() and now
init's MI fields of a struct pcpu in addition to registering it with
the internal array and list.
- A pcpu_destroy() function was added to remove a struct pcpu from the
internal array and list.
Tested on: alpha, i386
Reviewed by: peter, jake
|
| 87546 |
09-Dec-2001 |
dillon |
Allow maxusers to be specified as 0 in the kernel config, which will
cause the system to auto-size to between 32 and 512 depending on the
amount of memory.
MFC after: 1 week
|
| 86311 |
13-Nov-2001 |
mp |
Don't enable FP in the kernel. It is not needed when -msoft-float is used.
Reminded by: benno
|
| 86067 |
05-Nov-2001 |
mp |
Clean up the trap handling code and make it consistent with the other platforms.
Submitted by: jhb
|
| 86066 |
05-Nov-2001 |
mp |
Add enable_fpu/save_fpu for handling the floating point registers in the PCB.
Obtained from: NetBSD
|
| 85294 |
21-Oct-2001 |
des |
[partially forced commit due to pilot error in earlier commit attempt]
{set,fill}_{,fp,db}regs() fixup:
- Add dummy {set,fill}_dbregs() on architectures that don't have them.
- KSEfy the powerpc versions (struct proc -> struct thread).
- Some architectures had the prototypes in md_var.h, some in reg.h, and
some in both; for consistency, move them to reg.h on all platforms.
These functions aren't really MD (the implementation is MD, but the interface
is MI), so they should move to an MI header, but I haven't figured out which
one yet.
Run-tested on i386, build-tested on Alpha, untested on other platforms.
|
| 85201 |
19-Oct-2001 |
mp |
Fix includes based on recent changes to lock.h, mutex.h and ktr.h.
|
| 84977 |
15-Oct-2001 |
benno |
Flesh out cpu_fork() and cpu_set_fork_handler(). This is a work in progress.
|
| 84976 |
15-Oct-2001 |
benno |
- Correct the type of the argument to delay() so as to not conflict with
sys/boot/common/bootstrap.h.
- Add a prototype for fork_trampoline().
|
| 84946 |
15-Oct-2001 |
mp |
Fix typo.
|
| 84945 |
15-Oct-2001 |
mp |
Save WIP. Partial rewrite of cpu_switch() and savectx(). This makes it closer
to working but still needs some work to properly switch the full context
(such as saving the fpu registers, switch stacks, etc.). Also, remove some
dead code that was mixed in.
|
| 84921 |
14-Oct-2001 |
benno |
Implement pmap_mapdev.
|
| 84856 |
12-Oct-2001 |
mp |
Modify a virtual address check to allow use of the openfirmware callback
used by the PowerPC simulator (PSIM).
|
| 84643 |
08-Oct-2001 |
mp |
Add a call to init_param() to initialize some necessary variables.
|
| 84381 |
02-Oct-2001 |
mjacob |
Fix problem where a user buffer outside of the area being tested
will be corrupted.
PR: 29194
Obtained from: Tor.Egge@fast.no
MFC after: 2 weeks
|
| 83870 |
24-Sep-2001 |
mp |
Catch up to recent removal of curpcb from globals.h.
|
| 83730 |
20-Sep-2001 |
mp |
Add missing include file.
|
| 83683 |
20-Sep-2001 |
mp |
Use BATL/BATU macros instead of hardcoded hex constants.
|
| 83682 |
20-Sep-2001 |
mp |
Update PowerPC MD code to compile and do initial bootstrap based on
recent changes (KSE and VM requiring physmem to be setup).
Reviewed by: benno, jhb, julian
|
| 83366 |
12-Sep-2001 |
julian |
KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.
Sorry john! (your next MFC will be a doosie!)
Reviewed by: peter@freebsd.org, dillon@freebsd.org
X-MFC after: ha ha ha ha
|
| 83276 |
10-Sep-2001 |
peter |
Rip some well duplicated code out of cpu_wait() and cpu_exit() and move
it to the MI area. KSE touched cpu_wait() which had the same change
replicated five ways for each platform. Now it can just do it once.
The only MD parts seemed to be dealing with fpu state cleanup and things
like vm86 cleanup on x86. The rest was identical.
XXX: ia64 and powerpc did not have cpu_throw(), so I've put a functional
stub in place.
Reviewed by: jake, tmm, dillon
|
| 83223 |
08-Sep-2001 |
peter |
Missing part of dillon's coredump commit. cpu_coredump() was still
passing IO_NODELOCKED to vn_rdwr(), this would cause operations on the
unlocked core vnode and softupdates nastiness if an a.out binary cored.
|
| 82939 |
04-Sep-2001 |
peter |
Zap #if 0'ed map init code that got moved to the MI area.
Convert the powerpc tree to use the common code.
|
| 82938 |
04-Sep-2001 |
peter |
Nuke #if 0'ed "setredzone()" stub. We never used it, and probably
never will. I've implemented an optional redzone as part of the KSE
upage breakup.
|
| 82708 |
01-Sep-2001 |
jhb |
Axe stale mp_fixme().
|
| 82634 |
31-Aug-2001 |
peter |
Similar to changes on i386/alpha/etc pmap.c; converge on a similar
look/feel on pmap_new_proc() with some cosmetic style changes.
|
| 82025 |
21-Aug-2001 |
peter |
Make COMPAT_43 optional again. XXX we need COMPAT_FBSD3 etc for this
stuff.
|
| 81724 |
15-Aug-2001 |
jhb |
FreeBSD doesn't use a want_resched variable. Instead, the PS_NEEDRESCHED
p_sflag is managed in a MI fashion.
|
| 81493 |
10-Aug-2001 |
jhb |
- Close races with signals and other AST's being triggered while we are in
the process of exiting the kernel. The ast() function now loops as long
as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with
preemption disabled so that any further AST's that arrive via an
interrupt will be delayed until the low-level MD code returns to user
mode.
- Use u_int's to store the tick counts for profiling purposes so that we
do not need sched_lock just to read p_sticks. This also closes a
problem where the call to addupc_task() could screw up the arithmetic
due to non-atomic reads of p_sticks.
- Axe need_proftick(), aston(), astoff(), astpending(), need_resched(),
clear_resched(), and resched_wanted() in favor of direct bit operations
on p_sflag.
- Fix up locking with sched_lock some. In addupc_intr(), use sched_lock
to ensure pr_addr and pr_ticks are updated atomically with setting
PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing
PS_OWEUPC. We also do not grab the lock just to test a flag.
- Simplify the handling of Giant in ast() slightly.
Reviewed by: bde (mostly)
|
| 81265 |
08-Aug-2001 |
peter |
Zap 'ptrace(PT_READ_U, ...)' and 'ptrace(PT_WRITE_U, ...)' since they
are a really nasty interface that should have been killed long ago
when 'ptrace(PT_[SG]ETREGS' etc came along. The entity that they
operate on (struct user) will not be around much longer since it
is part-per-process and part-per-thread in a post-KSE world.
gdb does not actually use this except for the obscure 'info udot'
command which does a hexdump of as much of the child's 'struct user'
as it can get. It carries its own #defines so it doesn't break
compiles.
|
| 80431 |
27-Jul-2001 |
peter |
Make PMAP_SHPGPERPROC tunable. One shouldn't need to recompile a kernel
for this, since it is easy to run into with large systems with lots of
shared mmap space.
Obtained from: yahoo
|
| 79265 |
05-Jul-2001 |
dillon |
Move vm_page_zero_idle() from machine-dependant sections to a
machine-independant source file, vm/vm_zeroidle.c. It was exactly the
same for all platforms and updating them all was getting annoying.
|
| 79263 |
04-Jul-2001 |
dillon |
Reorg vm_page.c into vm_page.c, vm_pageq.c, and vm_contig.c (for contigmalloc).
Also removed some spl's and added some VM mutexes, but they are not actually
used yet, so this commit does not really make any operational changes
to the system.
vm_page.c relates to vm_page_t manipulation, including high level deactivation,
activation, etc... vm_pageq.c relates to finding free pages and aquiring
exclusive access to a page queue (exclusivity part not yet implemented).
And the world still builds... :-)
|
| 79224 |
04-Jul-2001 |
dillon |
With Alfred's permission, remove vm_mtx in favor of a fine-grained approach
(this commit is just the first stage). Also add various GIANT_ macros to
formalize the removal of Giant, making it easy to test in a more piecemeal
fashion. These macros will allow us to test fine-grained locks to a degree
before removing Giant, and also after, and to remove Giant in a piecemeal
fashion via sysctl's on those subsystems which the authors believe can
operate without Giant.
|
| 79123 |
03-Jul-2001 |
jhb |
Allow Giant to be recursed when a process terminates.
|
| 78983 |
29-Jun-2001 |
jhb |
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
|
| 78880 |
27-Jun-2001 |
benno |
Fix comment breakage.
|
| 78467 |
19-Jun-2001 |
benno |
More verbose version of identifycpu() which also contains many more CPU
versions/revisions.
Modified from the original patch to mark G3 and G4 processors as such.
Submitted by: Jeff Schottmiller <jeff@neoscale.com>
|
| 78342 |
16-Jun-2001 |
benno |
This commit (along with one pending in sys/dev/ofw and one in sys/conf) give
us our first minimal glimpse of PowerPC support.
With this code we can get to the "mountroot>" prompt on my Apple iMac. We
can't get any further due to lack of clock and interrupt handling, among other
things. This does however mean that pmap and VM are initialising.
We're fairly dependant on OpenFirmware at this point, but I hope to add
support for other classes of firmware at a later stage.
Reviewed by: obrien, dfr
|
| 77957 |
10-Jun-2001 |
benno |
Bring in NetBSD code used in the PowerPC port.
Reviewed by: obrien, dfr
Obtained from: NetBSD
|
| 77442 |
29-May-2001 |
jhb |
GC #if 0'd calls to releasing and acquiring the old style giant kernel
lock.
|
| 76932 |
21-May-2001 |
gallatin |
catch these files up to their i386 neighbors to make alpha boot
prior to the vm_mtx
|
| 76056 |
26-Apr-2001 |
jhb |
Initialize p_md.md_kernnest to 1 for newly fork'd processes since they
start off in the kernel.
|
| 74271 |
15-Mar-2001 |
gallatin |
remove bogus check -- for kernel threads we fork off of proc0, not curproc
This was causing panics when modules which create kthreads were loaded
after boot.
pointed out by: jake, jhb
|
| 73922 |
07-Mar-2001 |
jhb |
Use the proc lock to protect p_pptr when waking up our parent in cpu_exit()
and remove the mpfixme() message that is now fixed.
|
| 72906 |
22-Feb-2001 |
jhb |
Rename switch_trampoline() to fork_trampoline() on the alpha and ia64.
Suggested by: dfr
|
| 72746 |
20-Feb-2001 |
jhb |
- Don't call clear_resched() in userret(), instead, clear the resched flag
in mi_switch() just before calling cpu_switch() so that the first switch
after a resched request will satisfy the request.
- While I'm at it, move a few things into mi_switch() and out of
cpu_switch(), specifically set the p_oncpu and p_lastcpu members of
proc in mi_switch(), and handle the sched_lock state change across a
context switch in mi_switch().
- Since cpu_switch() no longer handles the sched_lock state change, we
have to setup an initial state for sched_lock in fork_exit() before we
release it.
|
| 72200 |
09-Feb-2001 |
bmilekic |
Change and clean the mutex lock interface.
mtx_enter(lock, type) becomes:
mtx_lock(lock) for sleep locks (MTX_DEF-initialized locks)
mtx_lock_spin(lock) for spin locks (MTX_SPIN-initialized)
similarily, for releasing a lock, we now have:
mtx_unlock(lock) for MTX_DEF and mtx_unlock_spin(lock) for MTX_SPIN.
We change the caller interface for the two different types of locks
because the semantics are entirely different for each case, and this
makes it explicitly clear and, at the same time, it rids us of the
extra `type' argument.
The enter->lock and exit->unlock change has been made with the idea
that we're "locking data" and not "entering locked code" in mind.
Further, remove all additional "flags" previously passed to the
lock acquire/release routines with the exception of two:
MTX_QUIET and MTX_NOSWITCH
The functionality of these flags is preserved and they can be passed
to the lock/unlock routines by calling the corresponding wrappers:
mtx_{lock, unlock}_flags(lock, flag(s)) and
mtx_{lock, unlock}_spin_flags(lock, flag(s)) for MTX_DEF and MTX_SPIN
locks, respectively.
Re-inline some lock acq/rel code; in the sleep lock case, we only
inline the _obtain_lock()s in order to ensure that the inlined code
fits into a cache line. In the spin lock case, we inline recursion and
actually only perform a function call if we need to spin. This change
has been made with the idea that we generally tend to avoid spin locks
and that also the spin locks that we do have and are heavily used
(i.e. sched_lock) do recurse, and therefore in an effort to reduce
function call overhead for some architectures (such as alpha), we
inline recursion for this case.
Create a new malloc type for the witness code and retire from using
the M_DEV type. The new type is called M_WITNESS and is only declared
if WITNESS is enabled.
Begin cleaning up some machdep/mutex.h code - specifically updated the
"optimized" inlined code in alpha/mutex.h and wrote MTX_LOCK_SPIN
and MTX_UNLOCK_SPIN asm macros for the i386/mutex.h as we presently
need those.
Finally, caught up to the interface changes in all sys code.
Contributors: jake, jhb, jasone (in no particular order)
|
| 71694 |
26-Jan-2001 |
jhb |
Update some comments, s0 in the pcb of a child returning from fork1() is
now passed in as a0 to fork_exit() and and s2 is passed in as a1.
|
| 71604 |
24-Jan-2001 |
jhb |
- Change fork_exit() to take a pointer to a trapframe as its 3rd argument
instead of a trapframe directly. (Requested by bde.)
- Convert the alpha switch_trampoline to call fork_exit() and use the MI
fork_return() instead of child_return().
- Axe child_return().
|
| 70317 |
23-Dec-2000 |
jake |
Protect proc.p_pptr and proc.p_children/p_sibling with the
proctree_lock.
linprocfs not locked pending response from informal maintainer.
Reviewed by: jhb, -smp@
|
| 69488 |
01-Dec-2000 |
gallatin |
acquire/release Giant in vm_page_zero_idle(), like on i386
Discused with: jhb
|
| 68900 |
19-Nov-2000 |
dfr |
Convert various calls to splhigh() to disable_intr() since splhigh() is
now a no-op.
|
| 68762 |
15-Nov-2000 |
jhb |
Don't perform an mi_switch() when we release Giant during cpu_exit(). We
are about to call cpu_switch() anyways.
Found by: witness
|
| 67551 |
25-Oct-2000 |
jhb |
- Overhaul the software interrupt code to use interrupt threads for each
type of software interrupt. Roughly, what used to be a bit in spending
now maps to a swi thread. Each thread can have multiple handlers, just
like a hardware interrupt thread.
- Instead of using a bitmask of pending interrupts, we schedule the specific
software interrupt thread to run, so spending, NSWI, and the shandlers
array are no longer needed. We can now have an arbitrary number of
software interrupt threads. When you register a software interrupt
thread via sinthand_add(), you get back a struct intrhand that you pass
to sched_swi() when you wish to schedule your swi thread to run.
- Convert the name of 'struct intrec' to 'struct intrhand' as it is a bit
more intuitive. Also, prefix all the members of struct intrhand with
'ih_'.
- Make swi_net() a MI function since there is now no point in it being
MD.
Submitted by: cp
|
| 67365 |
20-Oct-2000 |
jhb |
Catch up to moving headers:
- machine/ipl.h -> sys/ipl.h
- machine/mutex.h -> sys/mutex.h
|
| 66573 |
03-Oct-2000 |
dfr |
Clear pcb_schednest in cpu_fork() for the child process. This is
is necessary since the child's call stack only includes one recursive
hold of sched_lock.
|
| 65557 |
07-Sep-2000 |
jasone |
Major update to the way synchronization is done in the kernel. Highlights
include:
* Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The
alpha port is still in transition and currently uses both.)
* Per-CPU idle processes.
* Interrupts are run in their own separate kernel threads and can be
preempted (i386 only).
Partially contributed by: BSDi (BSD/OS)
Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
|
| 61825 |
19-Jun-2000 |
gallatin |
Support bounce buffers for ISA DMA on the alpha. This is required for the
irongate chipset (used in the UP1000) which does not support scatter/gather
DMA. We'll still use scatter gather if the core logic chipset supports it.
Reviewed by: dfr
|
| 61540 |
11-Jun-2000 |
alc |
cpu_fork(): Check "flags" before dereferencing "p2". Otherwise,
the call "vm_fork(p1, 0, flags);" early in fork1 can cause a kernel
panic.
|
| 60041 |
05-May-2000 |
phk |
Separate the struct bio related stuff out of <sys/buf.h> into
<sys/bio.h>.
<sys/bio.h> is now a prerequisite for <sys/buf.h> but it shall
not be made a nested include according to bdes teachings on the
subject of nested includes.
Diskdrivers and similar stuff below specfs::strategy() should no
longer need to include <sys/buf.> unless they need caching of data.
Still a few bogus uses of struct buf to track down.
Repocopy by: peter
|
| 58345 |
20-Mar-2000 |
phk |
Remove B_READ, B_WRITE and B_FREEBUF and replace them with a new
field in struct buf: b_iocmd. The b_iocmd is enforced to have
exactly one bit set.
B_WRITE was bogusly defined as zero giving rise to obvious coding
mistakes.
Also eliminate the redundant struct buf flag B_CALL, it can just
as efficiently be done by comparing b_iodone to NULL.
Should you get a panic or drop into the debugger, complaining about
"b_iocmd", don't continue. It is likely to write on your disk
where it should have been reading.
This change is a step in the direction towards a stackable BIO capability.
A lot of this patch were machine generated (Thanks to style(9) compliance!)
Vinum users: Greg has not had time to test this yet, be careful.
|
| 56096 |
16-Jan-2000 |
gallatin |
The kernel side of per-process unaligned access control (printing, fixing &
delivering SIGBUS). This will allow a non-superuser to control unaligned
access behaviour on a per-process basis once a userland control program
(uac) is written.
Reviewed by: obrien
Tested by: obrien
|
| 54207 |
06-Dec-1999 |
peter |
Make this compile again. (missing #include for RFPROC)
|
| 54188 |
06-Dec-1999 |
luoqi |
User ldt sharing.
|
| 53086 |
10-Nov-1999 |
dfr |
Re-organise the code which manages the owner of the FP state (fpcurproc).
The old code was spread out through the machdep code and was sloppy about
enabling and disabling the FEN bit (which controls access to the FP
register set). This caused a DIAGNOSTIC warning "DANGER WILL ROBINSON:
FEN SET IN cpu_fork!" sometimes when operating under high loads and could
conceivably lead to processes getting incorrect FP results.
The new code is much more strict about the FEN bit and makes sure that
*only* fpcurproc ever has it enabled. This also allows us to remove a
section of code from the exception_return path which might improve
performance marginally.
Reviewed by: gallatin
|
| 52647 |
30-Oct-1999 |
alc |
The core of this patch is to vm/vm_page.h. The effects are two-fold: (1) to
eliminate an extra (useless) level of indirection in half of the page
queue accesses and (2) to use a single name for each queue throughout,
instead of, e.g., "vm_page_queue_active" in some places and
"vm_page_queues[PQ_ACTIVE]" in others.
Reviewed by: dillon
|
| 52635 |
29-Oct-1999 |
phk |
useracc() the prequel:
Merge the contents (less some trivial bordering the silly comments)
of <vm/vm_prot.h> and <vm/vm_inherit.h> into <vm/vm.h>. This puts
the #defines for the vm_inherit_t and vm_prot_t types next to their
typedefs.
This paves the road for the commit to follow shortly: change
useracc() to use VM_PROT_{READ|WRITE} rather than B_{READ|WRITE}
as argument.
|
| 51474 |
20-Sep-1999 |
dillon |
Fix bug in pipe code relating to writes of mmap'd but illegal address
spaces which cross a segment boundry in the page table. pmap_kextract()
is not designed for access to the user space portion of the page
table and cannot handle the null-page-directory-entry case.
The fix is to have vm_fault_quick() return a success or failure which
is then used to avoid calling pmap_kextract().
|
| 50477 |
28-Aug-1999 |
peter |
$Id$ -> $FreeBSD$
|
| 50458 |
27-Aug-1999 |
gallatin |
Fix the child's return path from fork so that fork will return 0
in the child. This corrects a problem where linux/alpha binaries see
the child's return value of fork as the parent's pid. This happens because
linux/alpha binaries apparently check the return value directly, rather
than looking for a non-zero value in a4, as *BSD & OSF/1 do.
Reviewed by:dfr@nlsystems.com
|
| 49444 |
05-Aug-1999 |
jdp |
Sync with alc's revision 1.125 of i386/i386/vm_machdep.c. This
fixes the kernel build breakage.
|
| 48974 |
22-Jul-1999 |
alc |
Reduce the number of "magic constants" used for page coloring
by one: PQ_PRIME2 and PQ_PRIME3 are used to accomplish the same
thing at different places in the kernel. Drop PQ_PRIME3.
|
| 48391 |
01-Jul-1999 |
peter |
Slight reorganization of kernel thread/process creation. Instead of using
SYSINIT_KT() etc (which is a static, compile-time procedure), use a
NetBSD-style kthread_create() interface. kproc_start is still available
as a SYSINIT() hook. This allowed simplification of chunks of the
sysinit code in the process. This kthread_create() is our old kproc_start
internals, with the SYSINIT_KT fork hooks grafted in and tweaked to work
the same as the NetBSD one.
One thing I'd like to do shortly is get rid of nfsiod as a user initiated
process. It makes sense for the nfs client code to create them on the
fly as needed up to a user settable limit. This means that nfsiod
doesn't need to be in /sbin and is always "available". This is a fair bit
easier to do outside of the SYSINIT_KT() framework.
|
| 47869 |
10-Jun-1999 |
dt |
Replace my previous fix of saving the FP state with a much simpler one: when
we swap out fpcurproc, save its FP state.
Suggested by: bde
|
| 47840 |
08-Jun-1999 |
dt |
Keep fpcurproc locked in memory, so that we always can save the FP state
correctly.
This should fix the "pmap_changebit didn't" panic that some people see.
Reviewed by: dfr
|
| 45958 |
23-Apr-1999 |
dt |
Fixed several (not all) warnings.
|
| 45889 |
20-Apr-1999 |
dt |
Added consts to cpu_set_fork_handler prototype. (Follow i386 version.)
|
| 45821 |
19-Apr-1999 |
peter |
unifdef -DVM_STACK - it's been on for a while for x86 and was checked
and appeared to be working for the Alpha some time ago.
|
| 43758 |
08-Feb-1999 |
dillon |
Adjust idle zero-page fill hysteresis based on tests. Use 2/3 and 4/5
zero-fill levels.
Adjust comment for ozfod in vmmeter.h - this counter represents
non-optimal ( on the fly ) zero fills, not prefills.
|
| 43753 |
08-Feb-1999 |
dillon |
Add hysteresis to alpha version of vm_page_zero_idle().
|
| 43752 |
08-Feb-1999 |
dillon |
Rip out PQ_ZERO queue. PQ_ZERO functionality is now combined in with
PQ_FREE. There is little operational difference other then the kernel
being a few kilobytes smaller and the code being more readable.
* vm_page_select_free() has been *greatly* simplified.
* The PQ_ZERO page queue and supporting structures have been removed
* vm_page_zero_idle() revamped (see below)
PG_ZERO setting and clearing has been migrated from vm_page_alloc()
to vm_page_free[_zero]() and will eventually be guarenteed to remain
tracked throughout a page's life ( if it isn't already ).
When a page is freed, PG_ZERO pages are appended to the appropriate
tailq in the PQ_FREE queue while non-PG_ZERO pages are prepended.
When locating a new free page, PG_ZERO selection operates from within
vm_page_list_find() ( get page from end of queue instead of beginning
of queue ) and then only occurs in the nominal critical path case. If
the nominal case misses, both normal and zero-page allocation devolves
into the same _vm_page_list_find() select code without any specific
zero-page optimizations.
Additionally, vm_page_zero_idle() has been revamped. Hysteresis has been
added and zero-page tracking adjusted to conform with the other changes.
Currently hysteresis is set at 1/3 (lo) and 1/2 (hi) the number of free
pages. We may wish to increase both parameters as time permits. The
hysteresis is designed to avoid silly zeroing in borderline allocation/free
situations.
|
| 43209 |
26-Jan-1999 |
julian |
Mostly remove the VM_STACK OPTION.
This changes the definitions of a few items so that structures are the
same whether or not the option itself is enabled. This allows
people to enable and disable the option without recompilng the world.
As the author says:
|I ran into a problem pulling out the VM_STACK option. I was aware of this
|when I first did the work, but then forgot about it. The VM_STACK stuff
|has some code changes in the i386 branch. There need to be corresponding
|changes in the alpha branch before it can come out completely.
what is done:
|
|1) Pull the VM_STACK option out of the header files it appears in. This
|really shouldn't affect anything that executes with or without the rest
|of the VM_STACK patches. The vm_map_entry will then always have one
|extra element (avail_ssize). It just won't be used if the VM_STACK
|option is not turned on.
|
|I've also pulled the option out of vm_map.c. This shouldn't harm anything,
|since the routines that are enabled as a result are not called unless
|the VM_STACK option is enabled elsewhere.
|
|2) Add what appears to be appropriate code the the alpha branch, still
|protected behind the VM_STACK switch. I don't have an alpha machine,
|so we would need to get some testers with alpha machines to try it out.
|
|Once there is some testing, we can consider making the change permanent
|for both i386 and alpha.
|
[..]
|
|Once the alpha code is adequately tested, we can pull VM_STACK out
|everywhere.
|
Submitted by: "Richard Seaman, Jr." <dick@tar.com>
|
| 42175 |
30-Dec-1998 |
dfr |
Various changes to support OSF1 emulation:
* Move the user stack from VM_MAXUSER_ADDRESS to a place below the 32bit
boundary (needed to support 32bit OSF programs). This should also save
one pagetable per process.
* Add cvtqlsv to the set of instructions handled by the floating point
software completion code.
* Disable all floating point exceptions by default.
* A minor change to execve to allow the OSF1 image activator to support
dynamic loading.
|
| 41868 |
16-Dec-1998 |
bde |
Removed bogus casts of USRSTACK and/or the other operand in binary
expressions involving USRSTACK.
|
| 41499 |
04-Dec-1998 |
dfr |
Implement 'software completion' for floating point arithmetic. On the
alpha, operations involving non-finite numbers or denormalised numbers
or operations which should generate such numbers will cause an arithmetic
exception. For programs which follow some strict code generation rules,
the kernel trap handler can then 'complete' the operation by emulating
the faulting instruction.
To use software completion, a program must be compiled with the arguments
'-mtrap-precision=i' and '-mfp-trap-mode=su' or '-mfp-trap-mode=sui'.
Programs compiled in this way can use non-finite and denormalised numbers
at the expense of slightly less efficient code generation of floating
point instructions. Programs not compiled with these options will receive
a SIGFPE signal when non-finite or denormalised numbers are used or
generated.
Reviewed by: John Polstra <jdp@polstra.com>
|
| 40377 |
15-Oct-1998 |
dfr |
Change a bogus cast to the correct one.
|
| 37597 |
12-Jul-1998 |
dfr |
Don't bother calling pmap_emulate_reference() from cpu_fork(). It isn't
needed and it panics a DIAGNOSTIC kernel.
|
| 36865 |
10-Jun-1998 |
dfr |
Add missing copyrights. Thanks to Jason Thorpe for politely noting the
mistake...
|