#
90c6930e |
|
04-Nov-2011 |
Michael Lotz <mmlr@mlotz.ch> |
Add VM page allocation tracking similar to what happens in the slab already. Introduces "page_allocation_infos" and "page_allocations_per_caller" KDL commands. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43190 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
b9447668 |
|
10-Jul-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the vm_page initialization from vm_page.cpp:vm_page_init() to the new vm_page::Init(). * Made vm_page::wired_count private and added accessor methods. * Added VMCache::fWiredPagesCount (the number of wired pages the cache contains) and accessor methods. * Made more use of vm_page::IsMapped(). * vm_copy_on_write_area(): Added vm_page_reservation* parameter that can be used to request a special handling for wired pages. If given the wired pages are replaced by copies and the original pages are moved to the upper cache. * vm_copy_area(): - We don't need to do any wired ranges handling, if the source area is a B_SHARED_AREA, since we don't touch the area's mappings in this case. - We no longer wait for wired ranges of the concerned areas to disappear. Instead we use the new vm_copy_on_write_area() feature and just let it copy the wired pages. This fixes #6288, an issue introduced with the use of user mutexes in libroot: When executing multiple concurrent fork()s all but the first one would wait on the fork mutex, which (being a user mutex) would wire a page that the vm_copy_area() of the first fork() would wait for. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37460 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
a8ad734f |
|
14-Jun-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Introduced structures {virtual,physical}_address_restrictions, which specify restrictions for virtual/physical addresses. * vm_page_allocate_page_run(): - Fixed conversion of base/limit to array indexes. sPhysicalPageOffset was not taken into account. - Takes a physical_address_restrictions instead of base/limit and also supports alignment and boundary restrictions, now. * map_backing_store(), VM[User,Kernel]AddressSpace::InsertArea()/ ReserveAddressRange() take a virtual_address_restrictions parameter, now. They also support an alignment independent from the range size. * create_area_etc(), vm_create_anonymous_area(): Take {virtual,physical}_address_restrictions parameters, now. * Removed no longer needed B_PHYSICAL_BASE_ADDRESS. * DMAResources: - Fixed potential overflows of uint32 when initializing from device node attributes. - Fixed bounce buffer creation TODOs: By using create_area_etc() with the new restrictions parameters we can directly support physical high address, boundary, and alignment. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37131 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
147133b7 |
|
25-May-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* First run through the kernel's private parts to use phys_{addr,size}_t where appropriate. * Typedef'ed page_num_t to phys_addr_t and used it in more places in vm_page.{h,cpp}. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36937 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
fe0963a8 |
|
18-Mar-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Added commands to the page access debugging panic()s. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35904 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
bd7645a1 |
|
20-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Made vm_page::state private and added accessor methods. * Added kernel tracing for page state transitions. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
d36b9aef |
|
19-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Added vm_page::IsMapped() for convenience. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35531 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
40bb9481 |
|
03-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed useless return parameter from vm_remove_all_page_mappings(). * Added vm_clear_page_mapping_accessed_flags() and vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality of vm_test_map_activation(), vm_clear_map_flags(), and vm_remove_all_page_mappings(), thus saving lots of calls to translation map methods. The backend is the new method VMTranslationMap::ClearAccessedAndModified(). * Started to make use of the cached page queue and changed the meaning of the other non-free queues slightly: - Active queue: Contains mapped pages that have been used recently. - Inactive queue: Contains mapped pages that have not been used recently. Also contains unmapped temporary pages. - Modified queue: Contains unmapped modified pages. - Cached queue: Contains unmapped unmodified pages (LRU sorted). Unless we're actually low on memory and actively do paging, modified and cached queues only contain non-temporary pages. Cached pages are considered quasi free. They still belong to a cache, but since they are unmodified and unmapped, they can be freed immediately. And this is what vm_page_[try_]reserve_pages() do now when there are no more actually free pages at hand. Essentially this means that pages storing cached file data, unless mmap()ped, no longer are considered used and don't contribute to page pressure. Paging will not happen as long there are enough free + cached pages available. * Reimplemented the page daemon. It no longer scans all pages, but instead works the page queues. As long as the free pages situation is harmless, it only iterates through the active queue and deactivates pages that have not been used recently. When paging occurs it additionally scans the inactive queue and frees pages that have not been used recently. * Changed the page reservation/allocation interface: vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and vm_page_allocate_page() now take a vm_page_reservation structure pointer. The reservation functions initialize the structure -- currently consisting only of a count member for the number of still reserved pages. vm_page_allocate_page() decrements the count and vm_page_unreserve_pages() unreserves the remaining pages (if any). Advantages are that reservation/ unreservation mismatches cannot occur anymore, that vm_page_allocate_page() can verify that the caller has indeed a reserved page left, and that there's no unnecessary pressure on the free page pool anymore. The only disadvantage is that the vm_page_reservation object needs to be passed around a bit. * Reworked the page reservation implementation: - Got rid of sSystemReservedPages and sPageDeficit. Instead sUnreservedFreePages now actually contains the number of free pages that have not yet been reserved (it cannot become negative anymore) and the new sUnsatisfiedPageReservations contains the number of pages that are still needed for reservation. - Threads waiting for reservations do now add themselves to a waiter queue, which is ordered by descending priority (VM priority and thread priority). High priority waiters are served first when pages become available. Fixes #5328. * cache_prefetch_vnode(): Would reserve one less page than allocated later, if the size wasn't page aligned. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
e65c4002 |
|
29-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Replaced the vm_page_allocate_page*() "pageState" parameter by a more general "flags" parameter. It encodes the target state of the page -- so that the page isn't unnecessarily put in the wrong page queue first -- a flag whether the page should be cleared, and one to indicate whether the page should be marked busy. * Added page state PAGE_STATE_CACHED. Not used yet. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
72382fa6 |
|
29-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy flag. The obvious advantage is that one can still see what state a page is in and even move it between states while being marked busy. * Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which in all cases has the same effect. Introduced a vm_page_is_dummy() that can still check whether a given page is a dummy page. * vm_page_unreserve_pages(): Before adding to the system reserve make sure sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages available for allocation. steal_pages() still has the same problem and it can't be solved that easily. * map_page(): No longer changes the page state/mark the page unbusy. That's the caller's responsibility. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
6379e53e |
|
19-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
vm_page no longer points directly to its containing cache, but rather to a VMCacheRef object which points to the cache. This allows to optimize VMCache::MoveAllPages(), since it no longer needs to iterate over all pages to adjust their cache pointer. It can simple swap the cache refs of the two caches instead. Reduces the total -j8 Haiku image build time only marginally. The kernel time drops almost 10%, though. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35155 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
f082f7f0 |
|
15-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Added vm_page::accessed flag. Works analogously to vm_page::modified. * Reorganized the code for [un]mapping pages: - Added new VMTranslationMap::Unmap{Area,Page[s]}() which essentially do what vm_unmap_page[s]() did before, just in the architecture specific code, which allows for specific optimizations. UnmapArea() is for the special case that the complete area is unmapped. Particularly in case the address space is deleted, some work can be saved. Several TODOs could be slain. - Since they are only used within vm.cpp vm_map_page() and vm_unmap_page[s]() are now static and have lost their prefix (and the "preserveModified" parameter). * Added VMTranslationMap::Protect{Page,Area}(). They are just inline wrappers for Protect(). * X86VMTranslationMap::Protect(): Make sure not to accidentally clear the accessed/dirty flags. * X86VMTranslationMap::Unmap()/Protect(): Make page table skipping actually work. It was only skipping to the next page. * Adjusted the PPC code to at least compile. No measurable effect for the -j8 Haiku image build time, though the kernel time drops minimally. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35089 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
3b8c056d |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Should have been part of r35023: Introduction of the vm_page::modified flag. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35024 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
451ca8b4 |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
PAGE_TYPE_GUARD was unused and for the other two types a simple one bit flag suffices. Therefore replaced vm_page::type by vm_page::is_dummy. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35013 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
d7455de2 |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
allocate_page_run(): Use temporary lists to store the pages we're allocating. This makes appending the pages to the active queue more efficient and we don't need the vm_page::is_cleared bit anymore. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35011 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
3cd20943 |
|
06-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent access to a vm_page. It is basically an atomically accessed thread ID field in the vm_page structure, which is explicitly set by macros marking the critical sections. As a first positive effect I had to review quite a bit of code and found several issues. * Added several TODOs and comments. Some harmless ones, but also a few troublesome ones in vm.cpp regarding page unmapping. * file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous vm_page_allocate_page() return value checks. It cannot fail anymore. * Removed the heavily contended "pages" lock. We use different policies now: - sModifiedTemporaryPages is accessed atomically. - sPageDeficitLock and sFreePageCondition are protected by a new mutex. - The page queues have individual locks (mutexes). - Renamed set_page_state_nolock() to set_page_state(). Unless the caller says otherwise, it does now lock the affected pages queues itself. Also changed the return value to void -- we panic() anyway. * set_page_state(): Add free/clear pages to the beginning of their respective queues as this is more cache-friendly. * Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer in any queue. They were in the "active" queue, but there's no good reason to have them there. In case we decide to let the page daemon work the queues (like FreeBSD) they would just be in the way. * Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper function. Also fixed a bug I introduced previously: The functions must not vm_page_unreserve_pages() on success, since they remove the pages from the free/clear queue without decrementing sUnreservedFreePages. * vm_page_set_state(): Changed return type to void. The function cannot really fail and no-one was checking it anyway. * vm_page_free(), vm_page_set_state(): Added assertion: The page must not be free/clear before. This is implied by the policy that no-one is allowed to access free/clear pages without holding the respective queue's lock, which is not the case at this point. This found the bug fixed in r34912. * vm_page_requeue(): Added general assertions. panic() when requeuing of free/clear pages is requested. Same reason as above. * vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is still not correct, though. My usual -j8 Haiku build test runs another 10% faster, now. The total kernel time drops about 18%. As hoped the new locks have only a fraction of the old "pages" lock contention. Other locks lead the "most wanted list" now. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
5800e8a4 |
|
03-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the page management functionality into its own file. * Renamed page_queue to VMPageQueue and made it a proper C++ class. Use DoublyLinkedList instead of own list code. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
eb8dc1eb |
|
27-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed DEBUG_PAGE_CACHE_TRANSITIONS debugging. * Added VMCache::MovePage() and MoveAllPages() to move pages between caches. * VMAnonymousCache: - _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out pages. - _MergeSwapPages() does now also remove source cache pages that are shadowed by consumer swap pages. This allows us to call _MergeSwapPages() before _MergePagesSmallerSource(), save the swap page shadowing check there and get rid of the vm_page::merge_swap flag. This is an optimization based on the assumption that usually none or only few pages are swapped out, so we save a lot of checks. - Implemented _MergePagesSmallerConsumer() as an alternative to _MergePagesSmallerSource(). The former is used when the source cache has more pages than the consumer cache. It iterates over the consumer cache's pages, moves them to the source and finally moves all pages back to the consumer. The final move is relatively cheap (though unfortunately we still have to update all pages' vm_page::cache field), so that overall we save iterations of the main loop with the more expensive checks. The optimizations particularly improve the common fork()+exec*() situations. fork() uses CoW, which is implemented by putting two new empty caches between the to be copied area and its cache. exec*() destroys one copy of the area, its cache and thus causes merging of the other new cache with the old cache. Since this usually happens in a very short time, the old cache does still contain many pages and the new cache only few. Previously the many pages were all checked and moved individually. Now we do that for the few pages instead. A very extreme example of this situation is the Haiku image build. jam has a huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed. Since during the cache merging the cache is locked, any write access to a heap page causes jam to block until the cache merging is done. Formerly that took so long that it killed a lot of parallelism in multi-job builds. That could be observed particularly well when lots of small actions where executed (like the Link, XRes, Mimeset, SetType, SetVersion combos when building executables/libraries/add-ons). Those look dramatically better now. The overall speed improvement for a -j8 image build on my machine is only about 15%, though. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
be7328a9 |
|
07-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Moved VMCache related definitions to <vm/VMCache.h>. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34535 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
f34a1dd5 |
|
02-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Created VMArea.{h,cpp} and moved VMArea and the global area hash table (new class VMAreaHash) there. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34450 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
e50cf876 |
|
02-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the VM headers into subdirectory vm/. * Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
90c6930ebba06f6c9b220e712a53c063a542062a |
|
04-Nov-2011 |
Michael Lotz <mmlr@mlotz.ch> |
Add VM page allocation tracking similar to what happens in the slab already. Introduces "page_allocation_infos" and "page_allocations_per_caller" KDL commands. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43190 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
b9447668707741085389f650383b018d33d7d0bf |
|
10-Jul-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the vm_page initialization from vm_page.cpp:vm_page_init() to the new vm_page::Init(). * Made vm_page::wired_count private and added accessor methods. * Added VMCache::fWiredPagesCount (the number of wired pages the cache contains) and accessor methods. * Made more use of vm_page::IsMapped(). * vm_copy_on_write_area(): Added vm_page_reservation* parameter that can be used to request a special handling for wired pages. If given the wired pages are replaced by copies and the original pages are moved to the upper cache. * vm_copy_area(): - We don't need to do any wired ranges handling, if the source area is a B_SHARED_AREA, since we don't touch the area's mappings in this case. - We no longer wait for wired ranges of the concerned areas to disappear. Instead we use the new vm_copy_on_write_area() feature and just let it copy the wired pages. This fixes #6288, an issue introduced with the use of user mutexes in libroot: When executing multiple concurrent fork()s all but the first one would wait on the fork mutex, which (being a user mutex) would wire a page that the vm_copy_area() of the first fork() would wait for. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37460 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
a8ad734f1c698917badb15e1641e0f38b3e9a013 |
|
14-Jun-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Introduced structures {virtual,physical}_address_restrictions, which specify restrictions for virtual/physical addresses. * vm_page_allocate_page_run(): - Fixed conversion of base/limit to array indexes. sPhysicalPageOffset was not taken into account. - Takes a physical_address_restrictions instead of base/limit and also supports alignment and boundary restrictions, now. * map_backing_store(), VM[User,Kernel]AddressSpace::InsertArea()/ ReserveAddressRange() take a virtual_address_restrictions parameter, now. They also support an alignment independent from the range size. * create_area_etc(), vm_create_anonymous_area(): Take {virtual,physical}_address_restrictions parameters, now. * Removed no longer needed B_PHYSICAL_BASE_ADDRESS. * DMAResources: - Fixed potential overflows of uint32 when initializing from device node attributes. - Fixed bounce buffer creation TODOs: By using create_area_etc() with the new restrictions parameters we can directly support physical high address, boundary, and alignment. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37131 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
147133b76cbb1603bdbff295505f5b830cb4e688 |
|
25-May-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* First run through the kernel's private parts to use phys_{addr,size}_t where appropriate. * Typedef'ed page_num_t to phys_addr_t and used it in more places in vm_page.{h,cpp}. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36937 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
fe0963a83539b139455a49d17a0e7fe1b98090aa |
|
18-Mar-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Added commands to the page access debugging panic()s. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35904 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
bd7645a12ab594d5eb1c6d6a70c3a82a429410dd |
|
20-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Made vm_page::state private and added accessor methods. * Added kernel tracing for page state transitions. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
d36b9aef3edc3fb15cea5a29f51d83a1ac4168d1 |
|
19-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Added vm_page::IsMapped() for convenience. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35531 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
40bb94819e6c39d72ab29edc1a0dcd80b15b8b42 |
|
03-Feb-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed useless return parameter from vm_remove_all_page_mappings(). * Added vm_clear_page_mapping_accessed_flags() and vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality of vm_test_map_activation(), vm_clear_map_flags(), and vm_remove_all_page_mappings(), thus saving lots of calls to translation map methods. The backend is the new method VMTranslationMap::ClearAccessedAndModified(). * Started to make use of the cached page queue and changed the meaning of the other non-free queues slightly: - Active queue: Contains mapped pages that have been used recently. - Inactive queue: Contains mapped pages that have not been used recently. Also contains unmapped temporary pages. - Modified queue: Contains unmapped modified pages. - Cached queue: Contains unmapped unmodified pages (LRU sorted). Unless we're actually low on memory and actively do paging, modified and cached queues only contain non-temporary pages. Cached pages are considered quasi free. They still belong to a cache, but since they are unmodified and unmapped, they can be freed immediately. And this is what vm_page_[try_]reserve_pages() do now when there are no more actually free pages at hand. Essentially this means that pages storing cached file data, unless mmap()ped, no longer are considered used and don't contribute to page pressure. Paging will not happen as long there are enough free + cached pages available. * Reimplemented the page daemon. It no longer scans all pages, but instead works the page queues. As long as the free pages situation is harmless, it only iterates through the active queue and deactivates pages that have not been used recently. When paging occurs it additionally scans the inactive queue and frees pages that have not been used recently. * Changed the page reservation/allocation interface: vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and vm_page_allocate_page() now take a vm_page_reservation structure pointer. The reservation functions initialize the structure -- currently consisting only of a count member for the number of still reserved pages. vm_page_allocate_page() decrements the count and vm_page_unreserve_pages() unreserves the remaining pages (if any). Advantages are that reservation/ unreservation mismatches cannot occur anymore, that vm_page_allocate_page() can verify that the caller has indeed a reserved page left, and that there's no unnecessary pressure on the free page pool anymore. The only disadvantage is that the vm_page_reservation object needs to be passed around a bit. * Reworked the page reservation implementation: - Got rid of sSystemReservedPages and sPageDeficit. Instead sUnreservedFreePages now actually contains the number of free pages that have not yet been reserved (it cannot become negative anymore) and the new sUnsatisfiedPageReservations contains the number of pages that are still needed for reservation. - Threads waiting for reservations do now add themselves to a waiter queue, which is ordered by descending priority (VM priority and thread priority). High priority waiters are served first when pages become available. Fixes #5328. * cache_prefetch_vnode(): Would reserve one less page than allocated later, if the size wasn't page aligned. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
e65c400299386f99a251395ff2e59572705d7e49 |
|
29-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Replaced the vm_page_allocate_page*() "pageState" parameter by a more general "flags" parameter. It encodes the target state of the page -- so that the page isn't unnecessarily put in the wrong page queue first -- a flag whether the page should be cleared, and one to indicate whether the page should be marked busy. * Added page state PAGE_STATE_CACHED. Not used yet. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
72382fa6291e810be2949a70abd8f274f92dbd2c |
|
29-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy flag. The obvious advantage is that one can still see what state a page is in and even move it between states while being marked busy. * Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which in all cases has the same effect. Introduced a vm_page_is_dummy() that can still check whether a given page is a dummy page. * vm_page_unreserve_pages(): Before adding to the system reserve make sure sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages available for allocation. steal_pages() still has the same problem and it can't be solved that easily. * map_page(): No longer changes the page state/mark the page unbusy. That's the caller's responsibility. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
6379e53e2dd7021ba0e35d41c276dfe94c079596 |
|
19-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
vm_page no longer points directly to its containing cache, but rather to a VMCacheRef object which points to the cache. This allows to optimize VMCache::MoveAllPages(), since it no longer needs to iterate over all pages to adjust their cache pointer. It can simple swap the cache refs of the two caches instead. Reduces the total -j8 Haiku image build time only marginally. The kernel time drops almost 10%, though. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35155 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
f082f7f019941732f1d2b99f627fbeeeec3746af |
|
15-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Added vm_page::accessed flag. Works analogously to vm_page::modified. * Reorganized the code for [un]mapping pages: - Added new VMTranslationMap::Unmap{Area,Page[s]}() which essentially do what vm_unmap_page[s]() did before, just in the architecture specific code, which allows for specific optimizations. UnmapArea() is for the special case that the complete area is unmapped. Particularly in case the address space is deleted, some work can be saved. Several TODOs could be slain. - Since they are only used within vm.cpp vm_map_page() and vm_unmap_page[s]() are now static and have lost their prefix (and the "preserveModified" parameter). * Added VMTranslationMap::Protect{Page,Area}(). They are just inline wrappers for Protect(). * X86VMTranslationMap::Protect(): Make sure not to accidentally clear the accessed/dirty flags. * X86VMTranslationMap::Unmap()/Protect(): Make page table skipping actually work. It was only skipping to the next page. * Adjusted the PPC code to at least compile. No measurable effect for the -j8 Haiku image build time, though the kernel time drops minimally. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35089 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
3b8c056da097f1b7386f013726bb3ad13ea2ef98 |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Should have been part of r35023: Introduction of the vm_page::modified flag. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35024 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
451ca8b4b43e650a8c6cd7edd7f59640ebfee7f9 |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
PAGE_TYPE_GUARD was unused and for the other two types a simple one bit flag suffices. Therefore replaced vm_page::type by vm_page::is_dummy. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35013 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
d7455de2f603518cf6470bbca9aa19fe5a41c4ca |
|
11-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
allocate_page_run(): Use temporary lists to store the pages we're allocating. This makes appending the pages to the active queue more efficient and we don't need the vm_page::is_cleared bit anymore. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35011 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
3cd2094396dde9ca42263c535041a95d5f0d5fff |
|
06-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent access to a vm_page. It is basically an atomically accessed thread ID field in the vm_page structure, which is explicitly set by macros marking the critical sections. As a first positive effect I had to review quite a bit of code and found several issues. * Added several TODOs and comments. Some harmless ones, but also a few troublesome ones in vm.cpp regarding page unmapping. * file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous vm_page_allocate_page() return value checks. It cannot fail anymore. * Removed the heavily contended "pages" lock. We use different policies now: - sModifiedTemporaryPages is accessed atomically. - sPageDeficitLock and sFreePageCondition are protected by a new mutex. - The page queues have individual locks (mutexes). - Renamed set_page_state_nolock() to set_page_state(). Unless the caller says otherwise, it does now lock the affected pages queues itself. Also changed the return value to void -- we panic() anyway. * set_page_state(): Add free/clear pages to the beginning of their respective queues as this is more cache-friendly. * Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer in any queue. They were in the "active" queue, but there's no good reason to have them there. In case we decide to let the page daemon work the queues (like FreeBSD) they would just be in the way. * Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper function. Also fixed a bug I introduced previously: The functions must not vm_page_unreserve_pages() on success, since they remove the pages from the free/clear queue without decrementing sUnreservedFreePages. * vm_page_set_state(): Changed return type to void. The function cannot really fail and no-one was checking it anyway. * vm_page_free(), vm_page_set_state(): Added assertion: The page must not be free/clear before. This is implied by the policy that no-one is allowed to access free/clear pages without holding the respective queue's lock, which is not the case at this point. This found the bug fixed in r34912. * vm_page_requeue(): Added general assertions. panic() when requeuing of free/clear pages is requested. Same reason as above. * vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is still not correct, though. My usual -j8 Haiku build test runs another 10% faster, now. The total kernel time drops about 18%. As hoped the new locks have only a fraction of the old "pages" lock contention. Other locks lead the "most wanted list" now. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
5800e8a486993399f7f68be67099dc64bdb8a4be |
|
03-Jan-2010 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the page management functionality into its own file. * Renamed page_queue to VMPageQueue and made it a proper C++ class. Use DoublyLinkedList instead of own list code. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
eb8dc1ebfbe911a6af06efe02d003aa37687faad |
|
27-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Removed DEBUG_PAGE_CACHE_TRANSITIONS debugging. * Added VMCache::MovePage() and MoveAllPages() to move pages between caches. * VMAnonymousCache: - _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out pages. - _MergeSwapPages() does now also remove source cache pages that are shadowed by consumer swap pages. This allows us to call _MergeSwapPages() before _MergePagesSmallerSource(), save the swap page shadowing check there and get rid of the vm_page::merge_swap flag. This is an optimization based on the assumption that usually none or only few pages are swapped out, so we save a lot of checks. - Implemented _MergePagesSmallerConsumer() as an alternative to _MergePagesSmallerSource(). The former is used when the source cache has more pages than the consumer cache. It iterates over the consumer cache's pages, moves them to the source and finally moves all pages back to the consumer. The final move is relatively cheap (though unfortunately we still have to update all pages' vm_page::cache field), so that overall we save iterations of the main loop with the more expensive checks. The optimizations particularly improve the common fork()+exec*() situations. fork() uses CoW, which is implemented by putting two new empty caches between the to be copied area and its cache. exec*() destroys one copy of the area, its cache and thus causes merging of the other new cache with the old cache. Since this usually happens in a very short time, the old cache does still contain many pages and the new cache only few. Previously the many pages were all checked and moved individually. Now we do that for the few pages instead. A very extreme example of this situation is the Haiku image build. jam has a huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed. Since during the cache merging the cache is locked, any write access to a heap page causes jam to block until the cache merging is done. Formerly that took so long that it killed a lot of parallelism in multi-job builds. That could be observed particularly well when lots of small actions where executed (like the Link, XRes, Mimeset, SetType, SetVersion combos when building executables/libraries/add-ons). Those look dramatically better now. The overall speed improvement for a -j8 image build on my machine is only about 15%, though. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
be7328a9f6b3c55eec71d403a0367d80fd4ebd12 |
|
07-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Moved VMCache related definitions to <vm/VMCache.h>. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34535 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
f34a1dd5d701373687b6f3f0e6e76bd2b1ae6007 |
|
02-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Created VMArea.{h,cpp} and moved VMArea and the global area hash table (new class VMAreaHash) there. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34450 a95241bf-73f2-0310-859d-f6bbb57e9c96
|
#
e50cf8765be50a7454c9488db38b638cf90805af |
|
02-Dec-2009 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
* Moved the VM headers into subdirectory vm/. * Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace. git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96
|