#
331722 |
|
29-Mar-2018 |
eadler |
Revert r330897:
This was intended to be a non-functional change. It wasn't. The commit message was thus wrong. In addition it broke arm, and merged crypto related code.
Revert with prejudice.
This revert skips files touched in r316370 since that commit was since MFCed. This revert also skips files that require $FreeBSD$ property changes.
Thank you to those who helped me get out of this mess including but not limited to gonzo, kevans, rgrimes.
Requested by: gjb (re)
|
#
330897 |
|
14-Mar-2018 |
eadler |
Partial merge of the SPDX changes
These changes are incomplete but are making it difficult to determine what other changes can/should be merged.
No objections from: pfg
|
#
310442 |
|
22-Dec-2016 |
jhibbits |
MFC r304052:
Add missing pmap_kremove() method for book-e.
|
#
302408 |
|
07-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle. Prune svn:mergeinfo from the new branch, as nothing has been merged here.
Additional commits post-branch will follow.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
296142 |
|
27-Feb-2016 |
jhibbits |
Implement pmap_change_attr() for PowerPC (Book-E only for now)
Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access.
Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this.
Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
|
#
292680 |
|
24-Dec-2015 |
jhibbits |
Extend Book-E to support >4GB RAM
Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time.
Differential Revision: https://reviews.freebsd.org/D4345
|
#
287418 |
|
02-Sep-2015 |
jhibbits |
pmap_mapdev_attr() also takes a vm_paddr_t.
This was missed in r235936. With recent work for 36-bit paddr, this is now needed.
|
#
286296 |
|
04-Aug-2015 |
jah |
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva)
These will create and destroy a temporary, CPU-local KVA mapping of a specified page.
Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread
Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page
The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.
NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.
Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
|
#
276772 |
|
06-Jan-2015 |
markj |
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact.
PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
|
#
269728 |
|
08-Aug-2014 |
kib |
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep.
For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed.
In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
|
#
269485 |
|
03-Aug-2014 |
alc |
Retire pmap_change_wiring(). We have never used it to wire virtual pages. We continue to use pmap_enter() for that. For unwiring virtual pages, we now use pmap_unwire(), which unwires a range of virtual addresses instead of a single virtual page.
Sponsored by: EMC / Isilon Storage Division
|
#
268591 |
|
13-Jul-2014 |
alc |
Implement pmap_unwire(). See r268327 for the motivation behind this change.
|
#
255724 |
|
20-Sep-2013 |
alc |
The pmap function pmap_clear_reference() is no longer used. Remove it.
pmap_clear_reference() has had exactly one caller in the kernel for several years, more precisely, since FreeBSD 8. Now, that call no longer exists.
Approved by: re (kib) Sponsored by: EMC / Isilon Storage Division
|
#
255028 |
|
29-Aug-2013 |
alc |
Significantly reduce the cost, i.e., run time, of calls to madvise(..., MADV_DONTNEED) and madvise(..., MADV_FREE). Specifically, introduce a new pmap function, pmap_advise(), that operates on a range of virtual addresses within the specified pmap, allowing for a more efficient implementation of MADV_DONTNEED and MADV_FREE. Previously, the implementation of MADV_DONTNEED and MADV_FREE relied on per-page pmap operations, such as pmap_clear_reference(). Intuitively, the problem with this implementation is that the pmap-level locks are acquired and released and the page table traversed repeatedly, once for each resident page in the range that was specified to madvise(2). A more subtle flaw with the previous implementation is that pmap_clear_reference() would clear the reference bit on all mappings to the specified page, not just the mapping in the range specified to madvise(2).
Since our malloc(3) makes heavy use of madvise(2), this change can have a measureable impact. For example, the system time for completing a parallel "buildworld" on a 6-core amd64 machine was reduced by about 1.5% to 2.0%.
Note: This change only contains pmap_advise() implementations for a subset of our supported architectures. I will commit implementations for the remaining architectures after further testing. For now, a stub function is sufficient because of the advisory nature of pmap_advise().
Discussed with: jeff, jhb, kib Tested by: pho (i386), marcel (ia64) Sponsored by: EMC / Isilon Storage Division
|
#
248508 |
|
19-Mar-2013 |
kib |
Implement the concept of the unmapped VMIO buffers, i.e. buffers which do not map the b_pages pages into buffer_map KVA. The use of the unmapped buffers eliminate the need to perform TLB shootdown for mapping on the buffer creation and reuse, greatly reducing the amount of IPIs for shootdown on big-SMP machines and eliminating up to 25-30% of the system time on i/o intensive workloads.
The unmapped buffer should be explicitely requested by the GB_UNMAPPED flag by the consumer. For unmapped buffer, no KVA reservation is performed at all. The consumer might request unmapped buffer which does have a KVA reserve, to manually map it without recursing into buffer cache and blocking, with the GB_KVAALLOC flag.
When the mapped buffer is requested and unmapped buffer already exists, the cache performs an upgrade, possibly reusing the KVA reservation.
Unmapped buffer is translated into unmapped bio in g_vfs_strategy(). Unmapped bio carry a pointer to the vm_page_t array, offset and length instead of the data pointer. The provider which processes the bio should explicitely specify a readiness to accept unmapped bio, otherwise g_down geom thread performs the transient upgrade of the bio request by mapping the pages into the new bio_transient_map KVA submap.
The bio_transient_map submap claims up to 10% of the buffer map, and the total buffer_map + bio_transient_map KVA usage stays the same. Still, it could be manually tuned by kern.bio_transient_maxcnt tunable, in the units of the transient mappings. Eventually, the bio_transient_map could be removed after all geom classes and drivers can accept unmapped i/o requests.
Unmapped support can be turned off by the vfs.unmapped_buf_allowed tunable, disabling which makes the buffer (or cluster) creation requests to ignore GB_UNMAPPED and GB_KVAALLOC flags. Unmapped buffers are only enabled by default on the architectures where pmap_copy_page() was implemented and tested.
In the rework, filesystem metadata is not the subject to maxbufspace limit anymore. Since the metadata buffers are always mapped, the buffers still have to fit into the buffer map, which provides a reasonable (but practically unreachable) upper bound on it. The non-metadata buffer allocations, both mapped and unmapped, is accounted against maxbufspace, as before. Effectively, this means that the maxbufspace is forced on mapped and unmapped buffers separately. The pre-patch bufspace limiting code did not worked, because buffer_map fragmentation does not allow the limit to be reached.
By Jeff Roberson request, the getnewbuf() function was split into smaller single-purpose functions.
Sponsored by: The FreeBSD Foundation Discussed with: jeff (previous version) Tested by: pho, scottl (previous version), jhb, bf MFC after: 2 weeks
|
#
248280 |
|
14-Mar-2013 |
kib |
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified.
The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture.
Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement.
For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking.
Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
|
#
235936 |
|
24-May-2012 |
raj |
Fix physical address type to vm_paddr_t.
|
#
235689 |
|
20-May-2012 |
nwhitehorn |
Replace the list of PVOs owned by each PMAP with an RB tree. This simplifies range operations like pmap_remove() and pmap_protect() as well as allowing simple operations like pmap_extract() not to involve any global state. This substantially reduces lock coverages for the global table lock and improves concurrency.
|
#
227537 |
|
15-Nov-2011 |
marius |
As it turns out, r186347 actually is insufficient to avoid the use of the curthread-accessing part of mtx_{,un}lock(9) when using a r210623-style curthread implementation on sparc64, crashing the kernel in its early cycles as PCPU isn't set up, yet (and can't be set up as OFW is one of the things we need for that, which leads to a chicken-and-egg problem). What happens is that due to the fact that the idea of r210623 actually is to allow the compiler to cache invocations of curthread, it factors out obtaining curthread needed for both mtx_lock(9) and mtx_unlock(9) to before the branch based on kobj_mutex_inited when compiling the kernel without the debugging options. So change kobj_class_compile_static(9) to just never acquire kobj_mtx, effectively restricting it to its documented use, and add a kobj_init_static(9) for initializing objects using a class compiled with the former and that also avoids using mutex(9) (and malloc(9)). Also assert in both of these functions that they are used in their intended way only. While at it, inline kobj_register_method() and kobj_unregister_method() as there wasn't much point for factoring them out in the first place and so that a reader of the code has to figure out the locking for fewer functions missing a KOBJ_ASSERT. Tested on powerpc{,64} by andreast.
Reviewed by: nwhitehorn (earlier version), jhb MFC after: 3 days
|
#
213360 |
|
02-Oct-2010 |
nwhitehorn |
Fix some KTR arguments that were breaking the LINT build.
Pointy hat to: me
|
#
213307 |
|
30-Sep-2010 |
nwhitehorn |
Add support for memory attributes (pmap_mapdev_attr() and friends) on PowerPC/AIM. This is currently stubbed out on Book-E, since I have no idea how to implement it there.
|
#
208504 |
|
24-May-2010 |
alc |
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore().
Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page.
Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information.
Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock.
Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed().
Reviewed by: kib (an earlier version)
|
#
207155 |
|
24-Apr-2010 |
alc |
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
|
#
198341 |
|
21-Oct-2009 |
marcel |
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked).
The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
|
#
192067 |
|
13-May-2009 |
nwhitehorn |
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC.
Reviewed by: marcel, raj Book-E parts by: raj
|
#
190684 |
|
04-Apr-2009 |
marcel |
PowerPC, meet kernel core dumps. The support is based on a generic dumper that creates an ELF core file and uses PMAP functions to scan and iterate over memory chunks, as well as handle memory mappings used during dumping. the PMAP layer can choose to return physical memory chunks or virtual memory chunks. For minidumps, the chunks should be virtual.
The default MMU I/F implementation for the scan_md() method returns NULL. Thus, when a PMAP implementation does not implement the required methods, an empty core file is created. Here, empty means having an ELF header only.
Obtained from: Juniper Networks
|
#
190681 |
|
03-Apr-2009 |
nwhitehorn |
Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode provided, for example, on the PowerPC 970 (G5), as well as on related CPUs like the POWER3 and POWER4.
This also adds support for various built-in hardware found on Apple G5 hardware (e.g. the IBM CPC925 northbridge).
Reviewed by: grehan
|
#
186347 |
|
19-Dec-2008 |
nwhitehorn |
Modularize the Open Firmware client interface to allow run-time switching of OFW access semantics, in order to allow future support for real-mode OF access and flattened device frees. OF client interface modules are implemented using KOBJ, in a similar way to the PPC PMAP modules.
Because we need Open Firmware to be available before mutexes can be used on sparc64, changes are also included to allow KOBJ to be used very early in the boot process by only using the mutex once we know it has been initialized.
Reviewed by: marius, grehan
|
#
182571 |
|
31-Aug-2008 |
marcel |
Trace all PMAP calls using KTR_PMAP.
|
#
179081 |
|
18-May-2008 |
alc |
Retire pmap_addr_hint(). It is no longer used.
|
#
178893 |
|
09-May-2008 |
alc |
Add a stub for pmap_align_superpage() on machines that don't (yet) implement pmap-level support for superpages.
|
#
175067 |
|
03-Jan-2008 |
alc |
Add an access type parameter to pmap_enter(). It will be used to implement superpage promotion.
Correct a style error in kmem_malloc(): pmap_enter()'s last parameter is a Boolean.
|
#
173708 |
|
17-Nov-2007 |
alc |
Prevent the leakage of wired pages in the following circumstances: First, a file is mmap(2)ed and then mlock(2)ed. Later, it is truncated. Under "normal" circumstances, i.e., when the file is not mlock(2)ed, the pages beyond the EOF are unmapped and freed. However, when the file is mlock(2)ed, the pages beyond the EOF are unmapped but not freed because they have a non-zero wire count. This can be a mistake. Specifically, it is a mistake if the sole reason why the pages are wired is because of wired, managed mappings. Previously, unmapping the pages destroys these wired, managed mappings, but does not reduce the pages' wire count. Consequently, when the file is unmapped, the pages are not unwired because the wired mapping has been destroyed. Moreover, when the vm object is finally destroyed, the pages are leaked because they are still wired. The fix is to reduce the pages' wired count by the number of wired, managed mappings destroyed. To do this, I introduce a new pmap function pmap_page_wired_mappings() that returns the number of managed mappings to the given physical page that are wired, and I use this function in vm_object_page_remove().
Reviewed by: tegge MFC after: 6 weeks
|
#
173361 |
|
05-Nov-2007 |
kib |
Fix for the panic("vm_thread_new: kstack allocation failed") and silent NULL pointer dereference in the i386 and sparc64 pmap_pinit() when the kmem_alloc_nofault() failed to allocate address space. Both functions now return error instead of panicing or dereferencing NULL.
As consequence, vmspace_exec() and vmspace_unshare() returns the errno int. struct vmspace arg was added to vm_forkproc() to avoid dealing with failed allocation when most of the fork1() job is already done.
The kernel stack for the thread is now set up in the thread_alloc(), that itself may return NULL. Also, allocation of the first process thread is performed in the fork1() to properly deal with stack allocation failure. proc_linkup() is separated into proc_linkup() called from fork1(), and proc_linkup0(), that is used to set up the kernel process (was known as swapper).
In collaboration with: Peter Holm Reviewed by: jhb
|
#
164895 |
|
05-Dec-2006 |
grehan |
Fix gdb issue where the i-cache was not being updated when a breakpoint was written into a user's address space. The fix is to modify uiomove_fromphys to sync the icache when an executable user-space page is written into.
Alan Cox suggested that there should probably be a higher-level interface to this in the ptrace code, but agreed that this is an OK short-term solution.
Files changed:
pmap.h - declaration of pmap_page_executable() pmap_dispatch.c - pass through the page_executable call to the mmu object mmu_oea.c - implement the page_executable method by examining the PTE_EXEC field in the vm_page_t uio_machdep.c - in uiomove_fromphys(), if the op was a UIO_WRITE to user-space, and if the page is executable, sync the icache since this is at the least a breakpoint-write from gdb.
Reported by: marcel Tested by: marcel, grehan on g3+g4 Discussed with: alc MFC after: 2 weeks
|
#
164198 |
|
11-Nov-2006 |
alc |
Eliminate unused global variables.
|
#
160889 |
|
01-Aug-2006 |
alc |
Complete the transition from pmap_page_protect() to pmap_remove_write(). Originally, I had adopted sparc64's name, pmap_clear_write(), for the function that is now pmap_remove_write(). However, this function is more like pmap_remove_all() than like pmap_clear_modify() or pmap_clear_reference(), hence, the name change.
The higher-level rationale behind this change is described in src/sys/amd64/amd64/pmap.c revision 1.567. The short version is that I'm trying to clean up and fix our support for execute access.
Reviewed by: marcel@ (ia64)
|
#
159627 |
|
14-Jun-2006 |
ups |
Remove mpte optimization from pmap_enter_quick(). There is a race with the current locking scheme and removing it should have no measurable performance impact. This fixes page faults leading to panics in pmap_enter_quick_locked() on amd64/i386.
Reviewed by: alc,jhb,peter,ps
|
#
159323 |
|
06-Jun-2006 |
alc |
Add a stub for pmap_enter_object().
|
#
157443 |
|
03-Apr-2006 |
peter |
Remove the unused sva and eva arguments from pmap_remove_pages().
|
#
152630 |
|
20-Nov-2005 |
alc |
Eliminate pmap_init2(). It's no longer used.
|
#
152228 |
|
09-Nov-2005 |
grehan |
Apply r1.103 to correct place.
pmap.c on PowerPC is now a combo of mmu_if.m, pmap_dispatch.c and mmu_oea.c
(I forgot to delete pmap.c from CVS in last jumbo commit)
|
#
152179 |
|
08-Nov-2005 |
grehan |
Insert a layer of indirection to the pmap code, using a kobj for the interface. This allows run-time selection of MMU code, based on CPU-type detection, or tunable-overrides when testing new code.
Pre-requisite for G5 support.
conf/files.powerpc - remove pmap.c - add mmu_if.h, mmu_oea.c, pmap_dispatch.c
powerpc/include/mmuvar.h - definitions for MMU implementations
powerpc/include/pmap.h - remove pmap_pte_spill declaration - add pmap_mmu_install declaration - size the phys_avail array - pmap_bootstrapped is now global-scope
powerpc/powerpc/machdep.c - call kobj_machdep_init early in the boot sequence to allow kobj usage prior to SI_SUB_LOCK - install the OEA pmap code. This will be moved to CPU-specific init code in the future.
powerpc/powerpc/mmu_if.m - Kobj MMU interface definitions
powerpc/powerpc/pmap_dispatch.c - central dispatch for pmap calls - contains the global mmu kobj and the routine to locate the the mmu implementation and init the kobj
|