#
fee2a2fa |
|
09-Sep-2019 |
Mark Johnston <markj@FreeBSD.org> |
Change synchonization rules for vm_page reference counting. There are several mechanisms by which a vm_page reference is held, preventing the page from being freed back to the page allocator. In particular, holding the page's object lock is sufficient to prevent the page from being freed; holding the busy lock or a wiring is sufficent as well. These references are protected by the page lock, which must therefore be acquired for many per-page operations. This results in false sharing since the page locks are external to the vm_page structures themselves and each lock protects multiple structures. Transition to using an atomically updated per-page reference counter. The object's reference is counted using a flag bit in the counter. A second flag bit is used to atomically block new references via pmap_extract_and_hold() while removing managed mappings of a page. Thus, the reference count of a page is guaranteed not to increase if the page is unbusied, unmapped, and the object's write lock is held. As a consequence of this, the page lock no longer protects a page's identity; operations which move pages between objects are now synchronized solely by the objects' locks. The vm_page_wire() and vm_page_unwire() KPIs are changed. The former requires that either the object lock or the busy lock is held. The latter no longer has a return value and may free the page if it releases the last reference to that page. vm_page_unwire_noq() behaves the same as before; the caller is responsible for checking its return value and freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is introduced for use in pmap_extract_and_hold(). It fails if the page is concurrently being unmapped, typically triggering a fallback to the fault handler. vm_page_wire() no longer requires the page lock and vm_page_unwire() now internally acquires the page lock when releasing the last wiring of a page (since the page lock still protects a page's queue state). In particular, synchronization details are no longer leaked into the caller. The change excises the page lock from several frequently executed code paths. In particular, vm_object_terminate() no longer bounces between page locks as it releases an object's pages, and direct I/O and sendfile(SF_NOCACHE) completions no longer require the page lock. In these latter cases we now get linear scalability in the common scenario where different threads are operating on different files. __FreeBSD_version is bumped. The DRM ports have been updated to accomodate the KPI changes. Reviewed by: jeff (earlier version) Tested by: gallatin (earlier version), pho Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20486
|
#
eeacb3b0 |
|
08-Jul-2019 |
Mark Johnston <markj@FreeBSD.org> |
Merge the vm_page hold and wire mechanisms. The hold_count and wire_count fields of struct vm_page are separate reference counters with similar semantics. The remaining essential differences are that holds are not counted as a reference with respect to LRU, and holds have an implicit free-on-last unhold semantic whereas vm_page_unwire() callers must explicitly determine whether to free the page once the last reference to the page is released. This change removes the KPIs which directly manipulate hold_count. Functions such as vm_fault_quick_hold_pages() now return wired pages instead. Since r328977 the overhead of maintaining LRU for wired pages is lower, and in many cases vm_fault_quick_hold_pages() callers would swap holds for wirings on the returned pages anyway, so with this change we remove a number of page lock acquisitions. No functional change is intended. __FreeBSD_version is bumped. Reviewed by: alc, kib Discussed with: jeff Discussed with: jhb, np (cxgbe) Tested by: pho (previous version) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19247
|
#
01c0757b |
|
24-Sep-2017 |
Alan Cox <alc@FreeBSD.org> |
Modernize the use of vm_page_unwire(). Since r288122, vm_page_unwire() has returned TRUE when the wire count transitions to zero, eliminating the need for callers to inspect the page's wire count. MFC after: 1 week
|
#
6286dc78 |
|
17-Apr-2017 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Remove unneeded include of vm_phys.h.
|
#
a89156f5 |
|
05-Feb-2016 |
Michal Meloun <mmel@FreeBSD.org> |
ARM: Use new ARMv6 naming conventions for cache and TLB functions in all but ARMv4 specific files. Expand ARMv6 compatibility stubs in cpu-v4.h. Use physical address in L2 cache functions if ARM_L2_PIPT is defined.
|
#
780fefef |
|
10-Nov-2015 |
Svatopluk Kraus <skra@FreeBSD.org> |
Fix slots DMA memory handling. It's similar to r290553. Discussed with: gonzo Approved by: kib (mentor)
|
#
72683266 |
|
08-Nov-2015 |
Svatopluk Kraus <skra@FreeBSD.org> |
Fix pagelist bus_dmamap_t map handling. Memory for pagelist is allocated by bus_dmamem_alloc() which creates associated bus_dmamap_t map for us. When this memory is freed by bus_dmamem_free(), the map is freed as well. Thus there is no need to free it explicitly by bus_dmamap_destroy(), which leads to double freeing. Discussed with: gonzo Approved by: kib (mentor)
|
#
ff4cdb15 |
|
02-Nov-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
Fix cache issues with bulk transfers - Use pmap_quick_enter_page/pmap_quick_remove_page to bounce non-cacheline aligned head and tail fragments - Switch from static fragment size to configurable one, newer firmware passes cache line size as cache_line_size DTB parameter. With these changes both RPi and RPi2 pass functinal part of vchiq_test
|
#
a0b87461 |
|
01-Nov-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
Synchronize with latest upstream VCHI code: - Add LIB_VERSION ioctl - Add CLOSE_DELIVERED ioctl - Bump code version Upstream version: 3782f2ad42c08f4d32f64138f8be7341afc380f5
|
#
c26ee519 |
|
29-Oct-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
Fix BULK read transfer if destination buffer is not cache line-aligned. We can't use copyout because destination memory is userland address in another process but we have reference to respective page so map the page into kernel address space and copy fragments there
|
#
b5d68989 |
|
11-Feb-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
- Perform bus_dmamap_sync on pagelist structure - Wire pages of bulk transfer buffer when preparing pagelist
|
#
e6bf0bdb |
|
08-Feb-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
Remove unused variables
|
#
262f27b2 |
|
05-Feb-2015 |
Oleksandr Tymoshenko <gonzo@FreeBSD.org> |
Import VCHI driver for Broadcom's VideoCore IV GPU Differential Revision: D1753
|