kernel/vm: put pages in order on the clear page queue

possibly help with #15789:
I was testing virtual_block devices with dd, and noticed it didn't work with higher block sizes.
Indeed the driver was passing a list of single pages, very inefficient. To be remarked is that
the physical pages were reversed in memory, causing get_memory_map to provide a map of single pages.
What happens: dd fills up the buffer from 0 until the block size, causing soft faults for still unmapped
pages. Each fault maps a cleared physical page out of the clear page queue. This means that some page runs
are actually put reversed in the clear page queue, which this commit fixes.

Change-Id: I36395849c7f13086b44a1b284cc0c858d8c29f75
Reviewed-on: https://review.haiku-os.org/c/haiku/+/7081
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
Reviewed-by: Axel Dörfler <axeld@pinc-software.de>

kernel/vm_page: add vm_page table dump debugger command

Change-Id: I493228736f90f5e248f1d4fa3e354beb186546cb
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4308
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

fix build after MethodDeleter interface change

Change-Id: Iadb9e37772fd6588636085944c1455249ac0f926
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3457
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

kernel/vm: Make the page_scrubber wait on a condition.

On an idle system with 1GB RAM and the 100ms timeout, it takes multiple
minutes for all the pages in the system to get cleared after boot, but once
they do, the page scrubber will then remain idle for seconds to even
minutes at a time, so this is clearly worth it.

The "free pages condition" was unused before this commit, so I have
repurposed it (and unpublished it.)

Change-Id: I7034677a1e51c97c2baf11b772db3a31c0e1adfa
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1699
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

kernel: Whitespace cleanup only.

PVS V611: Use delete [] for memory allocated by new [].

Change-Id: I712350ae7f742aaa8e0ea58186265fac4bbac73e
Reviewed-on: https://review.haiku-os.org/c/1627
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Add vm_page_free_etc()

It additionally gets a vm_page_reservation* argument. If not NULL, the
page count of the reservation is incremented for the freed page.

kernel: Fix missing else in dump_page_queue debugger command.

The entered page queue address would always be overwritten. Pointed
out by CID 610504.

kernel/vm: Fix unsigned/signed comparison warning

kernel, libroot: Add more memory info in system_info

system_info now contains all information previously available only
through __get_system_info_etc(B_MEMORY_INFO, ...).

Fix various 64 bit related warnings

Mostly printf() format strings and signed-unsigned comparisons.
Fixes the x86_64 build.

vm: allocate wrappers on the heap with small stack array fallback

see comments on #10169. Thanks Ingo for reviewing!

Remove remaining unnecessary 'volatile' qualifiers

kernel: Add thread_unblock() and use it where possible

kernel: Remove possibility to yield to all threads

Kernel support for yielding to all (including lower priority) threads
has been removed. POSIX sched_yield() remains unchanged.

If a thread really needs to yield to everyone it can reduce its priority
to the lowest possible and then yield (it will then need to manually
return to its prvious priority upon continuing).

dump_page_queue(): fix output

* Determine the cache type per page instead of printing the first page's
cache type for all pages.
* Use vm_cache_type_to_string().

vm_page_allocate_page_run(): fix debug output

vm_page_allocate_page_run: use a mask to enforce the boundary

* use a mask to enforce the boundary as suggested by Ingo. Thanks!

vm_page_allocate_page_run: fix previous commit

* remove superfluous codes
* when aligning, sPhysicalPageOffset would be substracted twice
+alpha4

Signed-off-by: Ingo Weinhold <ingo_weinhold at gmx dot de>

vm_page_allocate_page_run: fix for aligned page allocations

* don't enforce a zero boundary or a zero alignment
* when going to the next range, takes alignment into account.
It could previously just be enforced again through alignment and loop infinite.
* it should help with some FreeBSD based drivers

Fixed tracing printf formats in VM code.

Logging calculations was done outside the #def TRACE_VM_PAGE, but never printed.

Now done inside #def as well as this triggered unused variables warnings.

More 64-bit compilation/safety fixes.

More 'GCC4.6 variable set but not used' fixes.

Fix harmless oversight marking pages with the wrong state.

While the log of hrev35726 says that unusable page ranges are supposed
to be marked with PAGE_STATE_UNUSED and allocated ones with
PAGE_STATE_WIRED, both actually marked with PAGE_STATE_UNUSED.

Tiny optimization by skipping a no-op iteration.

We initialize the physicalPagesEnd from physical_memory_range[0] so
re-evaluating that range is a no-op.

Fix the LIMIT_AVAILABLE_MEMORY debug option.

When limiting the available memory by reducing the page count it may not
be enough to just limit sNumPages. Depending on the physical memory map
non existing pages between ranges (sNonExistingPages) would still be
added up and later subtracted from the sNumPages, resulting in a wrong
max page count. Also due to the fixed removal of non existing page
ranges the actually available memory would usually not be the amount
set via LIMIT_AVAILABLE_MEMORY.

Instead we now calculate the available memory when going through the
physical memory ranges and limit/exit as soon as we've reached the
desired amount of available memory (also ignoring further non-existing
pages).

Ensure the sanity of the stats returned, make the TODO a Note.

* Ensure that we don't underflow the used_pages count and that used
+ cached pages don't overflow max_pages. As there is no locking the
values may change while we read them so that such situations could
arise.
* Make the TODO about the missing locking into a Note explaining the
above, as it is not really worth adding locking here. The stats are
only informational.

Fix the computation of used memory and add a TODO.

* The altered used pages calculation of hrev43168 wasn't correct, as
the inactive page queue may (validly) contain mapped pages as well.
Those would then get counted twice (as they are included in
gMappedPagesCount already).
Instead we calculate the used pages from the total page count, minus
everything we account for otherwise. Doing it this way is possible
without introducing any additional counters, as all the counts to
subtract are already present (as opposed to some of the ones that
would be needed for adding the counts up). Fixes #8109.
* Added TODO regarding the problem of not locking any of the counters
which runs the risk of them getting modified while we haven't yet read
all of them.

Add VM page allocation tracking similar to what happens in the slab already.
Introduces "page_allocation_infos" and "page_allocations_per_caller" KDL
commands.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43190 a95241bf-73f2-0310-859d-f6bbb57e9c96

Move some of the trace entries around so that we know the page(s) they concern
and add that info to the trace entries.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43187 a95241bf-73f2-0310-859d-f6bbb57e9c96

Add optional stack traces to {Allocate|Free}Page[Run] tracing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43185 a95241bf-73f2-0310-859d-f6bbb57e9c96

Indent cleanup only, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43184 a95241bf-73f2-0310-859d-f6bbb57e9c96

When calculating the number of used pages for a get_system_info() request,
also include inactive pages. Fixes #7714.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43168 a95241bf-73f2-0310-859d-f6bbb57e9c96

Remove superflous test done a few lines above already.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43014 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged the signals branch into trunk, with these changes:
* The team and thread kernel structures have been renamed to Team and Thread
respectively and moved into the new BKernel namespace.
* Several (kernel add-on) sources have been converted from C to C++ since
private kernel headers are included that are no longer C compatible.

Changes after merging:
* Fixed gcc 2 build (warnings mainly in the scary firewire bus manager).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40196 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a bug that would cause allocate_page_run() to be called with an out of
bounds index - the system would overwrite memory then and eventually KDL.
This could best be reproduced with overlays after a while.
* Added a TODO comment that explains why free_cached_pages() might fail even
though the page is actually free now.
* Added an explanation of how the sFreePageQueuesLock is to be used, thanks to
Ingo for explaining it to me in the first place :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38895 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer(): Fixed the pagesSinceLastSuccessfulWrite computation. Only the
skipped pages were added, not the failed ones.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37682 a95241bf-73f2-0310-859d-f6bbb57e9c96

* PageWriteWrapper::Done(): Returns whether the page was successfully written,
now.
* PageWriterRun::Go(): Returns the number of pages that could not be written.
* page_writer()/next_modified_page():
- Don't use a marker page anymore. A visited page is requeued at the tail.
This also makes the functions that schedule pages work a bit better (they
queue the pages at the head of the queue).
- Have an eye on pages that are busy or failed to write. If we ran through
the whole modified queue without writing a single page, sleep for a short
time. This mitigates the busyness the page writer falls into when there
are enough modified pages to make it run all the time but none that can be
written successfully.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37681 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer(): Added TODO about potential deadlock in case of loop file
systems/devices.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37502 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the vm_page initialization from vm_page.cpp:vm_page_init() to the new
vm_page::Init().
* Made vm_page::wired_count private and added accessor methods.
* Added VMCache::fWiredPagesCount (the number of wired pages the cache
contains) and accessor methods.
* Made more use of vm_page::IsMapped().
* vm_copy_on_write_area(): Added vm_page_reservation* parameter that can be
used to request a special handling for wired pages. If given the wired pages
are replaced by copies and the original pages are moved to the upper cache.
* vm_copy_area():
- We don't need to do any wired ranges handling, if the source area is a
B_SHARED_AREA, since we don't touch the area's mappings in this case.
- We no longer wait for wired ranges of the concerned areas to disappear.
Instead we use the new vm_copy_on_write_area() feature and just let it
copy the wired pages. This fixes #6288, an issue introduced with the use
of user mutexes in libroot: When executing multiple concurrent fork()s all
but the first one would wait on the fork mutex, which (being a user mutex)
would wire a page that the vm_copy_area() of the first fork() would wait
for.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37460 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed warnings with TRACE_VM_PAGE enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37411 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added vm_page_max_address() which returns the greatest address of accessible
physical memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37230 a95241bf-73f2-0310-859d-f6bbb57e9c96

Report ignored memory pages seperately, thanks to bonefish for the patch review. Closes ticket #6178

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37175 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced structures {virtual,physical}_address_restrictions, which specify
restrictions for virtual/physical addresses.
* vm_page_allocate_page_run():
- Fixed conversion of base/limit to array indexes. sPhysicalPageOffset was not
taken into account.
- Takes a physical_address_restrictions instead of base/limit and also
supports alignment and boundary restrictions, now.
* map_backing_store(), VM[User,Kernel]AddressSpace::InsertArea()/
ReserveAddressRange() take a virtual_address_restrictions parameter, now. They
also support an alignment independent from the range size.
* create_area_etc(), vm_create_anonymous_area(): Take
{virtual,physical}_address_restrictions parameters, now.
* Removed no longer needed B_PHYSICAL_BASE_ADDRESS.
* DMAResources:
- Fixed potential overflows of uint32 when initializing from device node
attributes.
- Fixed bounce buffer creation TODOs: By using create_area_etc() with the
new restrictions parameters we can directly support physical high address,
boundary, and alignment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37131 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_get_stats(): Forgot to added sIgnoredPages to system_info::max_pages
in r37117 as well, which was really the main point of introducing it.
Improves #6124 (reported memory lower than installed).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37127 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_page_init_num_pages(): Sum up the number of pages between the usable
physical memory ranges.
* vm_page_num_pages(), vm_page_get_stats(): Report the actually existing
memory, count memory ignored by the boot loader as used.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37117 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_allocate_page_run(): Added parameter "limit", specifying the upper
physical address limit for the page run to allocate.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37086 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_allocate_early(): Replace "bool blockAlign" parameter by a more flexible
"addr_t aligmnent".
* X86PagingMethod32Bit::PhysicalPageSlotPool::InitInitial(),
generic_vm_physical_page_mapper_init(): Use vm_allocate_early()'s alignment
feature instead of aligning by hand.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37070 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed more address types related issues. Mostly printf() or comparison
warnings, but also some oversights from earlier changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37000 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced type generic_io_vec, which is similar to iovec, but uses types
that are wide enough for both virtual and physical addresses.
* DMABuffer, IORequest, IOScheduler,... and code using them: Use
generic_io_vec and generic_{addr,size}_t where necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36997 a95241bf-73f2-0310-859d-f6bbb57e9c96

* First run through the kernel's private parts to use phys_{addr,size}_t
where appropriate.
* Typedef'ed page_num_t to phys_addr_t and used it in more places in
vm_page.{h,cpp}.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36937 a95241bf-73f2-0310-859d-f6bbb57e9c96

Missing line breaks in dprintf().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36785 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_requeue(): Disabled DEBUG_PAGE_ACCESS_CHECK(). Since r36155
idle_scan_active_pages() cannot satisfy this assertion anymore. Added TODO
for a better fix which can wait until after the release.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36649 a95241bf-73f2-0310-859d-f6bbb57e9c96

mark_page_range_in_use(): When the given range lies partially outside the
array at least work with the part intersecting with the array. Log those
quasi error cases.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36633 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The "page_stats" command does now also print the longest contiguous runs of
free respective free and cached pages.
* Removed the unused vm_page_allocate_page_run_no_base().
* vm_page_allocate_page_run() (and allocate_page_run()):
- Use vm_page_reserve_pages() instead of vm_page_try_reserve_pages(), i.e.
wait until the reservation succeeds.
- Now we iterates two times through the pages to find a suitable page run. In
the first iteration it only looks for free/clear pages, in the second
iteration it also considers cached pages. This increases the chance of the
function to succeed, when a lot of caching is going on.
This reduces the amount of memory required to use the IOCache when booting
off the anyboot Live CD to around 160 MB in qemu. It also seems to work with
128 MB, but the syslog indicates that some memory allocations fail, which
is not exactly inspiring confidence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36489 a95241bf-73f2-0310-859d-f6bbb57e9c96

full_scan_inactive_pages(): Don't call DEBUG_PAGE_ACCESS_START() before
checking whether the page is busy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36155 a95241bf-73f2-0310-859d-f6bbb57e9c96

The "page" and the "cache" kernel debugger commands set a few temporary
variables, now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35903 a95241bf-73f2-0310-859d-f6bbb57e9c96

Boot loader (x86 mmu.cpp):
* Made the page table allocation more flexible. Got rid of sMaxVirtualAddress
and added new virtual_end address to the architecture specific kernel args.
* Increased the virtual space we reserve for the kernel to 16 MB. That
should suffice for quite a while. The previous 2 MB were too tight when
building the kernel with debug info.
* mmu_init(): The way we were translating the BIOS' extended memory map to
our physical ranges arrays was broken. Small gaps between usable memory
ranges would be ignored and instead marked allocated. This worked fine for
the boot loader and during the early kernel initialization, but after the
VM has been fully set up it frees all physical ranges that have not been
claimed otherwise. So those ranges could be entered into the free pages
list and would be used later. This could possibly cause all kinds of weird
problems, probably including ACPI issues. Now we add only the actually
usable ranges to our list.

Kernel:
* vm_page_init(): The pages of the ranges between the usable physical memory
ranges are now marked PAGE_STATE_UNUSED, the allocated ranges
PAGE_STATE_WIRED.
* unmap_and_free_physical_pages(): Don't free pages marked as unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35726 a95241bf-73f2-0310-859d-f6bbb57e9c96

idle_scan_active_pages(): vm_page_requeue() requires the caller to mark the
page accessed for the access debugging feature.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35542 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Also print the page for the page related asserts.
* Used vm_page::IsMapped() to simplify things.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35541 a95241bf-73f2-0310-859d-f6bbb57e9c96

free_cached_page(): Removed incorrect asserts. The same ones appear a few
lines later anyway, after the cache has been locked and it has been verified
that the page is still a candidate. Fixes #5432.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35539 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Made vm_page::state private and added accessor methods.
* Added kernel tracing for page state transitions.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added assert in free_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35533 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added some more asserts for cached pages.
* full_scan_inactive_pages(): Fixed syntactical glitch (missing "else").
Affected only the active paging mode and was relatively harmless. The worst
case would be that an inactive page would be moved to the cached queue
although its usage_count hadn't dropped to 0 yet, thus freeing it before
pages that deserved it more.
* move_page_to_active_or_inactive_queue():
- Was ignoring the page's modified flag, thus potentially moving a modified
page to the cached queue. That could happen only in rare cases though,
like when the page was still mmap()ped while being written and modified and
unmapped before being done.
- No longer move the page to the inactive queue, even if its usage count is 0.
In idle mode the page daemon doesn't look at inactive pages, so the page's
stats wouldn't be updated anymore.
- Renamed to move_page_to_appropriate_queue().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35499 a95241bf-73f2-0310-859d-f6bbb57e9c96

* page_writer(): Missing continue in the temporary skipping case.
* vm_page_write_modified_page_range(): Need to DEBUG_PAGE_ACCESS_END() a bit
later.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35487 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer():
* Use the same criterion when to write back temporary pages as the page daemon.
* Move wired and temporary pages that shall not be written back to the active or
inactive queue so they don't get stuck in the modified queue and potentially
cause the page writer to run permanently without actually making progress
(#5382).

page writing implementation:
* If writing back a temporary page failed, move it back to the inactive or
active queue, so it doesn't get stuck in the modified queue. If active paging
continues, it might find its way back to the modified queue in the next
iteration, but that improves the situation a bit at least. Particularly with
the port heap pages not really being swappable ATM.
* Never dequeue pages from the modified queue. We mark them busy, so the page
writer will skip them anyway. This allows others to play with the page to some
extend (at least allowing to move it between the queues). This fixes #5404.
* Removed PageWriteWrapper::ClearModifiedFlag(). We clear the modified flag in
PageWriteWrapper::SetTo(), now, so that the page writer doesn't need to do
that explicitly either.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35485 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed PageWriteWrapper::CheckRemoveFromShrunkenCache() and moved removing
the page to Done().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35474 a95241bf-73f2-0310-859d-f6bbb57e9c96

Reenabled interrupts before notifying the page writer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35399 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the commented out scanning of the inactive queue when no paging is
going on. I only wanted to have it in the repository in case we decide at a
later point that it is a good idea after all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35395 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
of vm_test_map_activation(), vm_clear_map_flags(), and
vm_remove_all_page_mappings(), thus saving lots of calls to translation map
methods. The backend is the new method
VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
other non-free queues slightly:
- Active queue: Contains mapped pages that have been used recently.
- Inactive queue: Contains mapped pages that have not been used recently. Also
contains unmapped temporary pages.
- Modified queue: Contains unmapped modified pages.
- Cached queue: Contains unmapped unmodified pages (LRU sorted).
Unless we're actually low on memory and actively do paging, modified and
cached queues only contain non-temporary pages. Cached pages are considered
quasi free. They still belong to a cache, but since they are unmodified and
unmapped, they can be freed immediately. And this is what
vm_page_[try_]reserve_pages() do now when there are no more actually free
pages at hand. Essentially this means that pages storing cached file data,
unless mmap()ped, no longer are considered used and don't contribute to page
pressure. Paging will not happen as long there are enough free + cached pages
available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
the page queues. As long as the free pages situation is harmless, it only
iterates through the active queue and deactivates pages that have not been
used recently. When paging occurs it additionally scans the inactive queue and
frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
vm_page_allocate_page() now take a vm_page_reservation structure pointer.
The reservation functions initialize the structure -- currently consisting
only of a count member for the number of still reserved pages.
vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
unreserves the remaining pages (if any). Advantages are that reservation/
unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
can verify that the caller has indeed a reserved page left, and that there's
no unnecessary pressure on the free page pool anymore. The only disadvantage
is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
- Got rid of sSystemReservedPages and sPageDeficit. Instead
sUnreservedFreePages now actually contains the number of free pages that
have not yet been reserved (it cannot become negative anymore) and the new
sUnsatisfiedPageReservations contains the number of pages that are still
needed for reservation.
- Threads waiting for reservations do now add themselves to a waiter queue,
which is ordered by descending priority (VM priority and thread priority).
High priority waiters are served first when pages become available.
Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
the size wasn't page aligned.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the vm_page_allocate_page*() "pageState" parameter by a more
general "flags" parameter. It encodes the target state of the page -- so
that the page isn't unnecessarily put in the wrong page queue first -- a
flag whether the page should be cleared, and one to indicate whether the
page should be marked busy.
* Added page state PAGE_STATE_CACHED. Not used yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy
flag. The obvious advantage is that one can still see what state a page is in
and even move it between states while being marked busy.
* Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which
in all cases has the same effect. Introduced a vm_page_is_dummy() that can
still check whether a given page is a dummy page.
* vm_page_unreserve_pages(): Before adding to the system reserve make sure
sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages
available for allocation. steal_pages() still has the same problem and it
can't be solved that easily.
* map_page(): No longer changes the page state/mark the page unbusy. That's the
caller's responsibility.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved the clarity of some loops in the page reservation functions. No
functional change (other than avoiding no-ops like subtracting 0).
* vm_page_try_reserve_pages(): Moved the kernel tracing calls from the top of
the function to the points where the reservation already succeeded.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35322 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced {malloc,memalign,free}_etc() which take an additional "flags"
argument. They replace the previous special-purpose allocation functions
(malloc_nogrow(), vip_io_request_malloc()).
* Moved the I/O VIP heap to heap.cpp accordingly.
* Added quite a bit of passing around of allocation flags in the VM,
particularly in the VM*AddressSpace classes.
* Fixed IOBuffer::GetNextVirtualVec(): It was ignoring the VIP flag and always
allocated on the normal heap.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35316 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The system now holds back a small reserve of committable memory and pages. The
memory and page reservation functions have a new "priority" parameter that
indicates how deep the function may tap into that reserve. The currently
existing priority levels are "user", "system", and "VIP". The idea is that
user programs should never be able to cause a state that gets the kernel into
trouble due to heavy battling for memory. The "VIP" level (not really used
yet) is intended for allocations that are required to free memory eventually
(in the page writer). More levels are thinkable in the future, like "user real
time" or "user system server".
* Added "priority" parameters to several VMCache methods.
* Replaced the map_backing_store() "unmapAddressRange" parameter by a "flags"
parameter.
* Added area creation flag CREATE_AREA_PRIORITY_VIP and slab allocator flag
CACHE_PRIORITY_VIP indicating the importance of the request.
* Changed most code to pass the right priorities/flags.

These changes already significantly improve the behavior in low memory
situations. I've tested a bit with 64 MB (virtual) RAM and, while not
particularly fast and responsive, the system remains at least usable under high
memory pressure.
As a side effect the slab allocator can now be used as general memory allocator.
Not done by default yet, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35295 a95241bf-73f2-0310-859d-f6bbb57e9c96

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page no longer points directly to its containing cache, but rather to a
VMCacheRef object which points to the cache. This allows to optimize
VMCache::MoveAllPages(), since it no longer needs to iterate over all pages
to adjust their cache pointer. It can simple swap the cache refs of the two
caches instead.

Reduces the total -j8 Haiku image build time only marginally. The kernel time
drops almost 10%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35155 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page::accessed flag. Works analogously to vm_page::modified.
* Reorganized the code for [un]mapping pages:
- Added new VMTranslationMap::Unmap{Area,Page[s]}() which essentially do what
vm_unmap_page[s]() did before, just in the architecture specific code, which
allows for specific optimizations. UnmapArea() is for the special case that
the complete area is unmapped. Particularly in case the address space is
deleted, some work can be saved. Several TODOs could be slain.
- Since they are only used within vm.cpp vm_map_page() and vm_unmap_page[s]()
are now static and have lost their prefix (and the "preserveModified"
parameter).
* Added VMTranslationMap::Protect{Page,Area}(). They are just inline wrappers
for Protect().
* X86VMTranslationMap::Protect(): Make sure not to accidentally clear the
accessed/dirty flags.
* X86VMTranslationMap::Unmap()/Protect(): Make page table skipping actually
work. It was only skipping to the next page.
* Adjusted the PPC code to at least compile.

No measurable effect for the -j8 Haiku image build time, though the kernel time
drops minimally.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35089 a95241bf-73f2-0310-859d-f6bbb57e9c96

Refactored vm_translation_map:
* Pulled the physical page mapping functions out of vm_translation_map into
a new interface VMPhysicalPageMapper.
* Renamed vm_translation_map to VMTranslationMap and made it a proper C++
class. The functions in the operations vector have become methods.
* Added class GenericVMPhysicalPageMapper implementing VMPhysicalPageMapper
as far as possible (without actually writing new code).
* Adjusted the x86 and the PPC specifics accordingly (untested for the
latter). For the other architectures the build is, I'm afraid, seriously
broken.

The next steps will modify and extend the VMTranslationMap interface, so that
it will be possible to fix the bugs in vm_unmap_page[s]() and employ
architecture specific optimizations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35066 a95241bf-73f2-0310-859d-f6bbb57e9c96

Introduced vm_page::modified flag, which is used by vm_unmap_page[s]() to
preserve the dirty flag of the mapping without having to potentially move the
page to the modified queue. This lifts the (ignored) requirement that the
pages to be unmapped must not be busy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35023 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed two instances of debug code I missed to adjust in r35004. Fixes #5242.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35020 a95241bf-73f2-0310-859d-f6bbb57e9c96

PAGE_TYPE_GUARD was unused and for the other two types a simple one bit flag
suffices. Therefore replaced vm_page::type by vm_page::is_dummy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35013 a95241bf-73f2-0310-859d-f6bbb57e9c96

allocate_page_run(): Use temporary lists to store the pages we're allocating.
This makes appending the pages to the active queue more efficient and we
don't need the vm_page::is_cleared bit anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35011 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the page queue mutexes by spinlocks. The critical sections are very
short and quite hot, so mutexes just cause more overhead due to frequent
rescheduling than waiting for the spinlocks does. The free and clear queues
are additionally protected by a R/W lock, which is mostly read-locked, save
for rare cases like allocating page runs.

The total -j8 Haiku image build speedup is marginal. The kernel time drops
about 8%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35004 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_schedule_write_page_range(): The debug code was a bit too possessive.
We can only access the page, if it is not busy. Fixes #5228.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34980 a95241bf-73f2-0310-859d-f6bbb57e9c96

"page" debugger command:
* Various smaller fixes.
* Used add_debugger_command_etc() and added more verbose usage message.
* Added option "-m" which iterates through all address spaces and finds out
which virtual pages are mapped to the page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34979 a95241bf-73f2-0310-859d-f6bbb57e9c96

"find_page" didn't search the inactive queue.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34972 a95241bf-73f2-0310-859d-f6bbb57e9c96

Don't announce page access before being sure that the page isn't busy. Fixes
the erroneously trigger panic() in #5216.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34958 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96

steal_pages(): Recheck sUnreservedFreePages after incrementing sPageDeficit.
Since the former is no longer guarded by any lock, there's a race condition
with vm_page_unreserve_pages() which would cause us to wait longer than
necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34898 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the sReservedPages counter tracking the page reservations by
sUnreservedFreePages which tracks the difference between free/clear and
reserved pages. Access to it uses atomic operations which allows the three
page (un)reservation to avoid locking in most cases, thus reducing contention
of the "pages" lock.

In the -j8 Haiku image build that decreases the contention of the "pages"
lock to about one third of the previous value. As a positive side effect the
VMCache lock contention drops about the same factor. The total build speedup
is about 20%, the total kernel time drops about 20%.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34888 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed duplicate condition.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the page management functionality into its own file.
* Renamed page_queue to VMPageQueue and made it a proper C++ class. Use
DoublyLinkedList instead of own list code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96

* agp_gart(): Use vm_page_[un]reserve_pages().
* Removed unused vm_page_allocate_pages().
* Removed now unused (always true) "reserved" parameter from
vm_page_allocate_page().
* Removed unused (always false) "stealActive" parameter from steal_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34836 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed DEBUG_PAGE_CACHE_TRANSITIONS debugging.
* Added VMCache::MovePage() and MoveAllPages() to move pages between caches.
* VMAnonymousCache:
- _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out
pages.
- _MergeSwapPages() does now also remove source cache pages that are
shadowed by consumer swap pages. This allows us to call _MergeSwapPages()
before _MergePagesSmallerSource(), save the swap page shadowing check
there and get rid of the vm_page::merge_swap flag. This is an
optimization based on the assumption that usually none or only few pages
are swapped out, so we save a lot of checks.
- Implemented _MergePagesSmallerConsumer() as an alternative to
_MergePagesSmallerSource(). The former is used when the source cache has
more pages than the consumer cache. It iterates over the consumer cache's
pages, moves them to the source and finally moves all pages back to the
consumer. The final move is relatively cheap (though unfortunately we
still have to update all pages' vm_page::cache field), so that overall we
save iterations of the main loop with the more expensive checks.

The optimizations particularly improve the common fork()+exec*() situations.
fork() uses CoW, which is implemented by putting two new empty caches between
the to be copied area and its cache. exec*() destroys one copy of the area,
its cache and thus causes merging of the other new cache with the old cache.
Since this usually happens in a very short time, the old cache does still
contain many pages and the new cache only few. Previously the many pages were
all checked and moved individually. Now we do that for the few pages instead.

A very extreme example of this situation is the Haiku image build. jam has a
huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed.
Since during the cache merging the cache is locked, any write access to a
heap page causes jam to block until the cache merging is done. Formerly that
took so long that it killed a lot of parallelism in multi-job builds. That
could be observed particularly well when lots of small actions where executed
(like the Link, XRes, Mimeset, SetType, SetVersion combos when building
executables/libraries/add-ons). Those look dramatically better now.
The overall speed improvement for a -j8 image build on my machine is only
about 15%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a simple mechanism to wait for events to VMCache. WaitForPageEvents()
waits for certain events on a given page, NotifyPageEvents() wakes up
waiting threads respectively.
* Used the new feature instead of condition variables for waiting on busy
pages. We save publishing and unpublishing of a condition variable whenever
a page is marked busy. There's only something to do, if there's at least
one thread waiting in the list of the respective cache. The general
assumption is that this is only rarely the case and even if it happens,
there should be only very few threads.
* Added an apparently missing notification in cache_io(). At least I didn't
see the reason for it not being there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34537 a95241bf-73f2-0310-859d-f6bbb57e9c96

Created VMArea.{h,cpp} and moved VMArea and the global area hash table (new
class VMAreaHash) there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34450 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved VMAddressSpace definition to vm_address_space.h.
* "Classified" VMAddressSpace, i.e. turned the vm_address_space_*() functions
into methods, made all attributes (but "areas") private, and added
accessors.
* Also turned the vm.cpp functions vm_area_lookup() and
remove_area_from_address_space() into VMAddressSpace methods. The rest of
the area management functionality will follow soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34447 a95241bf-73f2-0310-859d-f6bbb57e9c96

Finished renaming of vm_cache to VMCache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34424 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed vm_address_space to VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34422 a95241bf-73f2-0310-859d-f6bbb57e9c96

* There is no need for the sPageLock to be a spinlock - a mutex should do. This
should improve the kernel latencies, as things like
vm_page_allocate_page_run() is very expensive.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33807 a95241bf-73f2-0310-859d-f6bbb57e9c96

Add a vm_page_allocate_page_run_no_base. It bases its search on the pages found
in the free and/or clear queue. This performs better in the case where only few
pages are free/clear but performs worse in the case where there are a lot of
usable pages. It's not used anywhere but it might come in handy one time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33527 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implement combining scattered physical pages using seperate iovecs in
PageWriteTransfer. This makes the transfer accept virtually contiguous pages,
where the offset is contiguous on either end of the current transfer, but where
the pages aren't physically contiguous. It will then add seperate iovecs for
these pages (32 at max right now). This reduces the number of IO requests
generated and allows for optimizations down the IO path (like in the physical to
virtual mapping case for example).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33526 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Rework page writing to combine page writes where possible. For now the pages
are required to be physically contiguos, which should be reworked to put them
into seperate iovecs. Still this manages to combine a great deal of page
writes into larger bursts already. Reduces the amount of IO requests being
scheduled (and greatly benefits media where page wise writes are slow when
they are accessed through a non-IOScheduler path, i.e. USB mass storage until
that is properly implemented).
* Abstracted per page page writing tasks into a PageWriteWrapper class.
* Abstracted per transfer page writing tasks into PageWriteTransfer class which
formerly was the PageWriterCallback.
* Use both classes from the PageWriterRun and from
vm_page_write_modified_page_range to remove code duplication.
* Adjusted synchronous VMAnonymousCache::Write() to cope correctly with larger
iovecs and more than one iovec. It assumed that there was exactly one page per
vector previously.
* Introduced MaxPagesPerWrite() and MaxPagesPerAsyncWrite() to VMCache to allow
a cache to specify restricitions. VMAnonymousCache does restrict the max pages
to 1 for WriteAsync right now as I didn't feel like reworking that one to cope
with non single page writes just yet.
* Pulled out PageWriteTransfer methods for better readability.
* Some typo fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33507 a95241bf-73f2-0310-859d-f6bbb57e9c96

* When reserving pages and there aren't yet enough free pages, only steal as
many pages as are actually missing, not the full count.
* Take into account that free_page_queue_count() can be less than sReservedPages
(when some of the reserved pages have been allocated already) in
vm_page_num_unused_pages(). Before it could return negative and therefore
wrapped numbers.
* Simplify the page scrubber loop by continuing early. Also avoids a needless
interrupt spin lock acquisition when there's nothing to do.
* Some minor coding style cleanup.
* Fix a typo.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32980 a95241bf-73f2-0310-859d-f6bbb57e9c96

panic() when a page that is still mapped is freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30606 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TODO about a problematic use of vm_page_allocate_page() in combination
with vm_cache_acquire_locked_page_cache().
* Added new function vm_page_num_unused_pages() which returns the pages that are
actually completely free and unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30514 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduces VMCache::CanWritePage() returning whether the given cache can
theoretically write the given page.
* page writer: Fixed the incorrect check whether a temporary page can be
written by using the new CanWritePage().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30281 a95241bf-73f2-0310-859d-f6bbb57e9c96

page writer:
* When writing a page failed it is not a good idea to re-enqueue it at the tail
of the modified queue, since that is definitely behind the page writer's
marker and the page would be picked up again before reaching the end of the
queue. If that happened with more than 256 pages, the page writer would keep
picking up only those non-writable pages and make no more progress.
* When selecting pages also skip temporary pages, if there's no more swap space
available, since trying to write those pages would most likely fail anyway
(triggering the first problem).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30230 a95241bf-73f2-0310-859d-f6bbb57e9c96

* PageWriterRun::Go(), vm_page_write_modified_page_range(): When writing the
page failed since the cache has been shrunk, we need not only remove the
page from the cache, we also need to remove all of its area mappings and
free it. Not removing the area mappings might have been the cause of #3110,
not freeing it would cause it to be leaked for good.
* vm_page_write_modified_page_range(): When writing failed for another reason
and the page wasn't in the modified queue before, we would lose the info
in which queue it was before and setting the page state to modified would
assume the active queue. This could potentially screw up our page queue
structures.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29992 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed src/system/kernel/device_manager/io_requests.{h,cpp} to
IORequest.{h,cpp}.
* Introduced public <io_requests.h> header. Currently it only declares the
single function BFS uses.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29446 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96

Dynamically adjust the page writer's I/O priority depending on how many
pages have to be written and how tight the memory situation is.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28198 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix ifdef, should be DEBUG_PAGE_QUEUE there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28009 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the vm_get_physical_page() "flags"
PHYSICAL_PAGE_{NO,CAN}_WAIT into an actual flag
PHYSICAL_PAGE_DONT_WAIT.
* Pass the flags through to the chunk mapper callback.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27979 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added kernel tracing for the page daemon and the page writer.
* Added some commented out debug output in vm.cpp.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27971 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed warnings with tracing enabled and made some very noisy dprintfs trace-only
as they were flooding the syslog.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27935 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved several VM related debug settings to kernel_debug_config.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27876 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a (private for now) get_system_info_etc() call, that can retrieve
various system information.
* Implemented retrieving some VM stats via this call.
* The VM now maintains a page fault counter, and sets system_info::page_faults
accordingly.
* Added a (pretty simple) "vmstat" command line app.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27597 a95241bf-73f2-0310-859d-f6bbb57e9c96

Print some more interesting timing info for the page writer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27245 a95241bf-73f2-0310-859d-f6bbb57e9c96

steal_pages() was leaking a cache reference since r26572, i.e caches
locked there would be leaked eventually.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27228 a95241bf-73f2-0310-859d-f6bbb57e9c96

Increased the number of pages the page writer tries to write per run.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27217 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The LIMIT_AVAILABLE_MEMORY macro can be defined to limit the amount of
RAM Haiku uses (for debugging purposes).
* "page_stats" prints some more infos now.
* page_writer():
- Moved the low_resource_state() invocation out of the inner loop.
Reduces lock contention on the sLowResourceLock recursive lock.
- Additional debug output: Every 1024 written pages the page writer
prints a message.
* steal_pages(): Addressed the TODO: When there should be pages to
steal, but we can't get them ATM, we now timeout on the free pages
condition variable instead of snoozing unconditionally, so that we
wake up earlier when someone frees pages in the meantime.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27181 a95241bf-73f2-0310-859d-f6bbb57e9c96

Introduced vm_page_try_reserve_pages(), which fails when not enough
pages are free ATM.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27116 a95241bf-73f2-0310-859d-f6bbb57e9c96

* VMCache::Write(): Added "uint32 flags" argument which is supposed to
be passed on to the IORequest. Most relevantly physical pages can now
be written directly by passing B_PHYSICAL_IO_REQUEST.
* Added VMCache::WriteAsync() which is supposed to write pages
asynchronously. The base class version version falls back to the
synchronous Write(). Only VMVnodeCache implements WriteAsync() ATM,
VMAnonymousCache (swap support) still has to be adjusted accordingly.
* write_page() doesn't need to map the page anymore as it can write the
physical page directly.
* Modified the page writer to write pages asynchronously. This shouldn't
have any noticeable effect yet. It will though as soon as the I/O
scheduler reorders I/O operations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27056 a95241bf-73f2-0310-859d-f6bbb57e9c96

"page_stats" does now also print the number of mapped pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26875 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the incrementing/decrementing of vm_page::wired_count into
dedicated functions.
* Introduced gMappedPagesCount variable which counts the total number of
physical pages that are mapped.
* Added vm_page_get_stats() which fills in the memory related part of
the system_info structure. Used and cached pages are computed
differently, now. The "available" (== not committed) memory is no
longer used for the computation as it doesn't say anything about the
actually used/free pages (with swap support enabled it is even
less meaningful, since we first commit swap space when possible).
We do also consider the memory used by the block cache as cached
pages, now. All in all these changes should fix the memory statistics
reported by get_system_info(), IOW bug #2574.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26837 a95241bf-73f2-0310-859d-f6bbb57e9c96

The page queues no longer count added dummy pages. This fixes a problem
in steal_pages() which uses the number of pages in the inactive queue as
return criterion. It would thus return, if only marker pages were in the
queue and no page could be stolen. This led to a busy loop in
vm_page_allocate_page(). The whole system would become unusable when the
thread in question was the heap grower, since it would starve everyone
else due to its high priority.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26710 a95241bf-73f2-0310-859d-f6bbb57e9c96

Reverted most of r26641.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26658 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Zhao Shuai with some modifications by myself:
* Renamed sModifiedTemporaryPages to sModifiedNoSwapPages to better
express what this variable is about.
* Changed tracking of sModifiedNoSwapPages. It really counts
non-swappable pages only, now (if swap support is enabled).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26641 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Zhao Shuai with changes by myself:
* Init swap support in main().
* Added "bool swappable" parameter to
VMCacheFactory::CreateAnonymousCache(). A cache supporting swapping
is created when true. Adjusted invocations accordingly.
* The page writer does now write non-locked swappable pages (when
memory is low).
* Fixed header guard of VMAnonymousNoSwapCache.h.
* Swap support is compiled conditionally, controlled by the
ENABLE_SWAP_SUPPORT in src/system/kernel/vm/VMAnonymousCache.h. It is
disabled ATM. Since no swap files are added, it wouldn't have much
effect anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26625 a95241bf-73f2-0310-859d-f6bbb57e9c96

* bfs_fsync() was the only place which could cause the
fs_vnode_ops::write_pages() to be called with fsReenter = true. Since
this is no longer the case, the argument has become superfluous. For
read_pages() it always was. Removed the argument from the functions
and all functions that propagated it.
* Some whitespace at the end of lines was removed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26579 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added B_PHYSICAL_BASE_ADDRESS address specification for anonymous areas.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26456 a95241bf-73f2-0310-859d-f6bbb57e9c96

Remove a few unused variables.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26249 a95241bf-73f2-0310-859d-f6bbb57e9c96

Spotted this while reading through page code. The usage_count would be set for
wrong pages when allocating a page run unless the run started at page 0 as the
start offset was missing.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26245 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the global cache pages hash table by an IteratableSplayTree
per cache.
* Changed the strategy vm_cache_acquire_page_cache_ref() uses to ensure
that the cache isn't deleted while trying to get a reference. Instead
of the global cache pages hash table lock, it holds the global cache
list lock now. We acquire + release this lock in delete_cache() after
removing all pages and just before deleting the object.
* Some small optimizations using the property that the cache's pages are
ordered, now (vm_cache_resize(), vm_page_write_modified_page_range(),
vm_page_schedule_write_page_range()).
* Replaced some code counting a cache's pages by simply using
vm_cache::page_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26160 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fix cache reference leak


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26110 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_cache::virtual_size is not always a multiple of B_PAGE_SIZE for files.
In these cases, the last partial page would have never been written back.
* This fixes bug #2282, and eventually others.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25747 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page_write_modified_page_range(), which is similar to
vm_page_write_modified_pages(), save that it only writes pages in the
given range.
* Added vm_page_schedule_write_page_range() which schedules all modified
pages in the given cache's range for writing by the page writer.
* Added _kern_sync_memory() syscall and the msync() POSIX function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25620 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed old mutex implementation and renamed cutex to mutex.
* Trivial adjustments of code using mutexes. Mostly removing the
mutex_init() return value check.
* Added mutex_lock_threads_locked(), which is called with the threads
spinlock being held. The spinlock is released while waiting, of
course. This function is useful in cases where the existence of the
mutex object is ensured by holding the threads spinlock.
* Changed the two instances in the VFS code where an IO context of
another team needs to be locked to use mutex_lock_threads_locked().
Before it required a semaphore-based mutex implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25283 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the vm_cache mutex by a cutex. This should save quite a few
semaphores.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25277 a95241bf-73f2-0310-859d-f6bbb57e9c96

* steal_page() called remove_page_from_queue() without holding the sPageLock.
This fixes bug #1900 for real.
* Rearranged find_page_candidate() a bit, removed duplicate code, added a panic
in case the marker state is invalid.
* Some cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25255 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The page writer was calling remove_page_marker() without holding the
sPageLock. This could easily mess up the page queue.
* Now, remove_page_marker() gets the lock itself. This fixes bug #1900.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25250 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed macros that enable tracing for individual components from
defined/undefined to numeric values (0 for undefined). This allows for
trace levels.
* Set SYSCALL_TRACING_IGNORE_KTRACE_OUTPUT default to 1, since this is
what one usually wants.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25213 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed vm_get_available_memory() to vm_available_memory() to fit better
into our usual naming scheme.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24745 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed the page allocation tracking history to kernel tracing instead.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24614 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed a quasi-livelock in steal_pages() as proposed in ticket #1929.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24605 a95241bf-73f2-0310-859d-f6bbb57e9c96

Applied patch by Rene Gollent:
* Add a cached_pages field to the system_info structure, and change the
meaning of the used_pages field to not include cached pages.
* Provide the needed info using the new calls vm_get_available_memory(),
and vm_page_num_available_pages().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24571 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed race condition in the page writer: The state of the page we have
picked might have changed while we were locking its cache. Might fix
#1931.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24430 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added a boolean "force" parameter to thread_yield(). When true, the
function has the old behavior. When false, it just calls the scheduler
without any priority adjustment or other stuff.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23906 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a deadlock (and resolved TODO): we need to make all pages unbusy again
before releasing our cache reference. Otherwise removing a vnode (triggered
by releasing the cache in our thread) could need pages we still own.
* Put the caches and pages into a union to save stack space; they are not
needed at the same time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23580 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added TODO comment about a possible deadlock Marcus just found :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23450 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed (or rather, worked around) a deadlock in the VM: when a file was
resized but still had dirty pages to be written back,
vm_cache_resize() (which is called with the inode lock being held)
deadlocked with the page writer.
* Now, I reintroduced busy_writing: it'll be set by everything that
writes back pages (vm_page_write_modified(), and the page writer),
and will be checked for in vm_cache_resize() - other functions are not
affected for now, AFAICT.
* vm_cache_resize() will clear that flag, and the writer will check it
again after it wrote back the page (which will fail when it's outside
the file bounds), and if it's cleared, it will get rid of the page
(if the file has been resized again in the mean time, writing it will
succeed then, and we'll keep the page around).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23334 a95241bf-73f2-0310-859d-f6bbb57e9c96

Minor refactoring.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23319 a95241bf-73f2-0310-859d-f6bbb57e9c96

* My last change to vm_page.cpp made an existing bug much more likely to
appear: when freeing a modified page, it wouldn't have a cache
anymore, but set_page_state_nolock() depended on it.
* To work around this, I added a vm_page_free() function, which the
caches that free modified pages have to call (but others may, too).
It will correctly maintain the sModifiedTemporaryPages counter in case
the cache has already been removed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23318 a95241bf-73f2-0310-859d-f6bbb57e9c96

The modified temporary page counter could go out of sync pretty easily.
This could cause the page writer to run endlessly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23309 a95241bf-73f2-0310-859d-f6bbb57e9c96

If there are more than 4 MB dirty cache pages, the page writer won't stop anymore
writing back pages when there is enough free memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23086 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added the sModifiedTemporaryPages counter to the output of the "page_stats" KDL
command.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22856 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Made page_writer() use a marker page as well, so that it won't try to get
the same pages over and over.
* Increased the priority of the page writer a bit, so that it is higher than
the one of the page daemon.
* Added a sModifiedTemporaryPages counter to let the page_writer decide how
many pages are there to write back (temporary pages would go to the swap file
and are only written back when memory is low).
* In case there are more than 1024 modified (non-temporary) pages around, the
page writer will constantly write out pages, not only when being pushed.
* The page writer now temporarily always leave out temporary pages (as long as
we don't have a swap file).
* Shuffled functions around a bit.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22852 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added opt-in debugging feature: Recent allocation-related operations
on pages are recorded in a history and can be printed via the
"page_allocations" command.
* Fixed a problem in the page scrubber. It temporarily removed pages
from the free list, which could have been reserved by someone else.
When actually allocating the reserved pages, that someone could find
free and clear lists empty and would therefore rightfully panic
("Had reserved page, but there is none!").


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22841 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If steal_pages() stole more than originally asked for, it might have lost those pages in the
non-reserve case.
* vm_page_allocate_page() could also lose a stolen page in case more free pages were available
after steal_pages() was called.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22834 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked stealing pages: the page thief thread is gone now,
vm_page_reserve_pages() and vm_page_allocate_page() will now steal pages from
the inactive queue as needed.
* We currently never steal active pages anymore, but this might need to be
revised later (therefore, the page scanner never waits anymore, but uses
mutex_trylock() to lock a cache).
* The page scanner and writer now both run at normal priority - let's see how
that will work out.
* Introduced an inactive queue.
* Instead of shuffling pages around in the queue (and therefore destroying LRU)
the page stealing mechanism now uses a marker page to be able to release the
page lock without losing its position in the queue.
* The page writer now always grabs the whole release count of the semaphore, so
that there won't be a huge backlog to catch up with.
* vm_page_num_free_pages() now also includes the inactive queue as well as the
reserved pages (they are no longer regarded as free pages).
* Added a insert_page_after() function that inserts a page after another one,
needed by the marker code.
* clear_page() now gets a vm_page instead of a physical address which simplified
some code.
* Removed superfluous initialization of the queues (if those aren't zeroed on
start, we would have serious problems, anyway).
* Removed old and unimplemented dump_free_page_table() ("free_pages") KDL
command.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22506 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_set_area_protection() would remap the whole area instead of just the page
it intended to. That resulted in more writable pages where you wouldn't want
them (ie. allowing the area to change pages in lower caches).
* We were losing modified pages: vm_unmap_pages() sometimes has to preserve
the modified flag (eg. when called from page fault).
* Both of these were responsible that stealing active pages would crash
applications - even if less likely, this could also have happened when
stealing inactive pages. Therefore, I've activated stealing active pages
again.
* The page writer now pushes the pages of busy vnodes to the end of the queue,
so that it won't pick them up again too soon (the vnode destruction would
be in the process of writing those pages back, anyway).
* The page thief now triggers the page writer to run once it has to steal
active pages. This might be a bit too aggressive, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22495 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed the page_thief(): it never noticed it could not get a page.
* The page_thief() will no longer steal any pages in B_LOW_MEMORY_NOTE state,
only in the more serious cases.
* I've disabled stealing active pages for now again; there seem to be some
problems with it (either with how we do it, or with other stuff).
* vm_page_schedule_write_page() now always releases the page writer semaphore,
resulting in many more written pages - this isn't optimal as long as there is
no I/O scheduler, but before it was much too rare when there are many dirty
pages.
* Customized the thread priorities a bit to make the page scanner/thief/writer
experience a bit nicer with our current scheduler.
* vm_page_reserve_pages() would return too early, it did not test if really
enough pages are free.
* Under certain circumstances, the wakeup from vm_page_reserve_pages() did not
work - we now always notify in vm_page_unreserve_pages() to work around that
problem.
* Checked if the page reservations are done when needed and are always balanced
for the whole kernel.
* vm_page_allocate_page() now panics if it can't deliver a reserved page.
* vm_page_allocate_page_run() can no longer steal reserved pages.
* With all of those changes, I could finally copy a file in emulation, ie. the
Luposian bug should finally be fixed, even though the system might still not
be perfectly stable under low memory. Will do some more testing.
* write_page() no longer prints something on failure.
* Dumping a whole page queue will now also write the cache type of each page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22486 a95241bf-73f2-0310-859d-f6bbb57e9c96

Actually forgot this nail: the file cache will now reserve the pages it will
allocate with the vm_cache locked - this is necessary to be able to steal pages
from itself (large files...).
The system doesn't actually lock up anymore, but it still renders itself unusable;
obviously the page thief does not work correctly, yet. The rest of the experience
is created by our current scheduler (the page thief runs and runs, but it doesn't
free any pages anymore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22466 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a acquire_unreferenced_ref() to vm_store
* his has to be used by the page writer to make sure the vnode is still valid.
* This should have been the final nail on the Luposian bug - I haven't tested
it yet, but we'll certainly see :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22462 a95241bf-73f2-0310-859d-f6bbb57e9c96

The page thief can now also steal active pages under pressure.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22456 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Mapping a page might actually need memory - since we usually have locks that
interfere with the page thief, we always need to have reserved a page for
this upfront. I introduced a function to the vm_translation_map layer that
estimates how much pages a mapping might need at maximum. All functions that
map a page now call this and reserve the needed pages upfront.
It might not be a nice solution, but it works.
* The page thief could run into a panic when trying to call vm_cache_release_ref()
on a non-existing (NULL) cache.
* Also, it will now ignore wired active pages.
* There is still a race condition between the page writer and the vnode
destruction - writing a page back needs a valid vnode, but that might just
have been deleted.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22455 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added some docs.
* Removed dead code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22439 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_remove_all_page_mappings() now returns an accumulation of the flags of
the unmapped page.
* This is needed by everyone who calls this to make sure modifications to a
page aren't ignored. Namely, the page scanner and the page thief were
affected.
* Cleaned up locking the page's cache a bit in page_thief(); there is now
a helper class that takes care of everything.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22438 a95241bf-73f2-0310-859d-f6bbb57e9c96

* fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added a TODO what we need to do with stolen active pages - for now, we don't
do anything with them, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22413 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decoupled the page_thief() from the low memory handler chain. This is
necessary because low memory handlers might need to wait for other services
while we need to make sure the page thief can always steal pages.
* Made the page thief more aggressive when memory is critical: added a third
run where it will also steal a small amount of active pages to keep things
going.
* The page thief did not maintain the cache's reference correctly in all
situations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22382 a95241bf-73f2-0310-859d-f6bbb57e9c96

* I just saw the page writer passing a free page to write_page().
Apparently not everyone checks the page state for PAGE_STATE_BUSY.
vm_page_write_modified_pages() does now at least.
* Fixed typos in comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22349 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+axeld:
* We now have a page writer that takes some pages from the modified queue
and writes it back every few seconds. It can be triggered by the page
scanner to do that more often, though. That mechanism can be greatly
improved once we have our I/O scheduler working.
* Removed vm_page_write_modified_page() again - it was all "eaten up" by
the page writer.
* Reworked vm_page_write_modified_pages() a bit: it now uses
vm_test_map_modification() and vm_clear_map_flags() instead of the
iterating over all areas which wouldn't even work correctly.
The code is much simpler now, too.
* You usually put something to the tail of a queue, and remove the contents
from the head, not vice versa - changed queue implementation to reflect this.
* Additionally, there is now a enqueue_page_to_head() if you actually want the
opposite.
* vm_page_requeue() allows you to move a page in a queue to the head or tail.
* Replaced vm_clear_map_activation() with vm_clear_map_flags() which allows
you to clear other flags than PAGE_ACCESSED.
* The page scanner dumps now some arguments with each run.
* Removed the old disabled pageout_daemon() from NewOS.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22348 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed the automatic release of the "modified_pages_available" semaphore
in enqueue_page() - it doesn't really belong there.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22328 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed vm_page_write_modified() to vm_page_write_modifed_pages() and
introduced a new vm_page_write_modified_page().
* Resolved a TODO: vm_page_write_modified_pages() did not mark a to be
written page busy but unlocked its cache which could let someone else
steal that page in the mean time.
* Moved the logic when to move a page to the active or inactive queue to
a new function move_page_to_active_or_inactive_queue().
* Moved page_state_to_string() to vm_page(); it's now also used by the
"page" and "page_queue" KDL commands.
* Made the output of the "page_queue list" command more useful.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22323 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Since the page scanner and thief can work more effectively when no vm_caches
are locked, there is now a vm_page_reserve_pages() call to ensure upfront that
there is a page for me when I need it, and may have locked some caches.
* The vm_soft_fault() routine now makes use of that feature.
* vm_page_allocate_page() now resets the vm_page::usage_count, so that the file
cache does not need to do this in read_chunk_into_cache() and
write_chunk_to_cache().
* In cache_io() however, it need to update the usage_count - and it does that
now. Since non-mapped caches don't have mappings, the page scanner will punish
the cache pages stronger than other pages which is accidently just what we
want.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22319 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+axeld:
* Removed the vm_cache/vm_store ref_count duality that besides being a bit ugly
also created the page dameon cache retrieval problem: now, only areas (and
cache consumers) retrieve a reference to the store (and therefore, the vnode).
The page daemon doesn't need to care about this at all anymore, and the pseudo
references of the vm_cache could be removed again.
* Rearranged deletion of vnodes such that its ID can be reused directly after
fs_remove_vnode() has been called.
* vm_page_allocate_page() no longer panics when it runs out of pages, but just
waits for new pages to become available using the new sFreeCondition condition
variable - to make sure this happens in an acceptable time frame, it'll
trigger a run of the low memory handlers.
* Implemented a page_thief() that steals inactive pages from caches and puts
them into the free queue. It runs as a low memory handler.
* The file cache now sets the usage count on the pages it inserts into the
cache (needs some rework though, cache_io() doesn't do it yet).
* Instead of panicking, the kernel will currently dead lock in low memory
situations, since BFS does a bit too much in bfs_release_vnode().
* Some minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22315 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed unused vm_page::busy_{reading|writing} fields.
* Fixed vm_page_allocate_page_run(): it did not take the pageState into account,
and would therefore return uninitialized memory (ie. B_CONTIGUOUS areas would
contain garbage).
Now, it stores if a page is cleared in a new vm_page::is_cleared field.
* Some cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22306 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed page queues according to our style guide.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22155 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Use condition variables when waiting for busy pages or busy caches.
* Removed a few instances where the page state was set busy directly after
allocating it. This is a no-op, since a page is always busy after
allocation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21875 a95241bf-73f2-0310-859d-f6bbb57e9c96

Add vm_page_free_etc()

It additionally gets a vm_page_reservation* argument. If not NULL, the
page count of the reservation is incremented for the freed page.

kernel: Fix missing else in dump_page_queue debugger command.

The entered page queue address would always be overwritten. Pointed
out by CID 610504.

kernel/vm: Fix unsigned/signed comparison warning

kernel, libroot: Add more memory info in system_info

system_info now contains all information previously available only
through __get_system_info_etc(B_MEMORY_INFO, ...).

Fix various 64 bit related warnings

Mostly printf() format strings and signed-unsigned comparisons.
Fixes the x86_64 build.

vm: allocate wrappers on the heap with small stack array fallback

see comments on #10169. Thanks Ingo for reviewing!

Remove remaining unnecessary 'volatile' qualifiers

kernel: Add thread_unblock() and use it where possible

kernel: Remove possibility to yield to all threads

Kernel support for yielding to all (including lower priority) threads
has been removed. POSIX sched_yield() remains unchanged.

If a thread really needs to yield to everyone it can reduce its priority
to the lowest possible and then yield (it will then need to manually
return to its prvious priority upon continuing).

dump_page_queue(): fix output

* Determine the cache type per page instead of printing the first page's
cache type for all pages.
* Use vm_cache_type_to_string().

vm_page_allocate_page_run(): fix debug output

vm_page_allocate_page_run: use a mask to enforce the boundary

* use a mask to enforce the boundary as suggested by Ingo. Thanks!

vm_page_allocate_page_run: fix previous commit

* remove superfluous codes
* when aligning, sPhysicalPageOffset would be substracted twice
+alpha4

Signed-off-by: Ingo Weinhold <ingo_weinhold at gmx dot de>

vm_page_allocate_page_run: fix for aligned page allocations

* don't enforce a zero boundary or a zero alignment
* when going to the next range, takes alignment into account.
It could previously just be enforced again through alignment and loop infinite.
* it should help with some FreeBSD based drivers

Fixed tracing printf formats in VM code.

Logging calculations was done outside the #def TRACE_VM_PAGE, but never printed.

Now done inside #def as well as this triggered unused variables warnings.

More 64-bit compilation/safety fixes.

More 'GCC4.6 variable set but not used' fixes.

Fix harmless oversight marking pages with the wrong state.

While the log of hrev35726 says that unusable page ranges are supposed
to be marked with PAGE_STATE_UNUSED and allocated ones with
PAGE_STATE_WIRED, both actually marked with PAGE_STATE_UNUSED.

Tiny optimization by skipping a no-op iteration.

We initialize the physicalPagesEnd from physical_memory_range[0] so
re-evaluating that range is a no-op.

Fix the LIMIT_AVAILABLE_MEMORY debug option.

When limiting the available memory by reducing the page count it may not
be enough to just limit sNumPages. Depending on the physical memory map
non existing pages between ranges (sNonExistingPages) would still be
added up and later subtracted from the sNumPages, resulting in a wrong
max page count. Also due to the fixed removal of non existing page
ranges the actually available memory would usually not be the amount
set via LIMIT_AVAILABLE_MEMORY.

Instead we now calculate the available memory when going through the
physical memory ranges and limit/exit as soon as we've reached the
desired amount of available memory (also ignoring further non-existing
pages).

Ensure the sanity of the stats returned, make the TODO a Note.

* Ensure that we don't underflow the used_pages count and that used
+ cached pages don't overflow max_pages. As there is no locking the
values may change while we read them so that such situations could
arise.
* Make the TODO about the missing locking into a Note explaining the
above, as it is not really worth adding locking here. The stats are
only informational.

Fix the computation of used memory and add a TODO.

* The altered used pages calculation of hrev43168 wasn't correct, as
the inactive page queue may (validly) contain mapped pages as well.
Those would then get counted twice (as they are included in
gMappedPagesCount already).
Instead we calculate the used pages from the total page count, minus
everything we account for otherwise. Doing it this way is possible
without introducing any additional counters, as all the counts to
subtract are already present (as opposed to some of the ones that
would be needed for adding the counts up). Fixes #8109.
* Added TODO regarding the problem of not locking any of the counters
which runs the risk of them getting modified while we haven't yet read
all of them.

Add VM page allocation tracking similar to what happens in the slab already.
Introduces "page_allocation_infos" and "page_allocations_per_caller" KDL
commands.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43190 a95241bf-73f2-0310-859d-f6bbb57e9c96

Move some of the trace entries around so that we know the page(s) they concern
and add that info to the trace entries.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43187 a95241bf-73f2-0310-859d-f6bbb57e9c96

Add optional stack traces to {Allocate|Free}Page[Run] tracing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43185 a95241bf-73f2-0310-859d-f6bbb57e9c96

Indent cleanup only, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43184 a95241bf-73f2-0310-859d-f6bbb57e9c96

When calculating the number of used pages for a get_system_info() request,
also include inactive pages. Fixes #7714.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43168 a95241bf-73f2-0310-859d-f6bbb57e9c96

Remove superflous test done a few lines above already.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43014 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged the signals branch into trunk, with these changes:
* The team and thread kernel structures have been renamed to Team and Thread
respectively and moved into the new BKernel namespace.
* Several (kernel add-on) sources have been converted from C to C++ since
private kernel headers are included that are no longer C compatible.

Changes after merging:
* Fixed gcc 2 build (warnings mainly in the scary firewire bus manager).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40196 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a bug that would cause allocate_page_run() to be called with an out of
bounds index - the system would overwrite memory then and eventually KDL.
This could best be reproduced with overlays after a while.
* Added a TODO comment that explains why free_cached_pages() might fail even
though the page is actually free now.
* Added an explanation of how the sFreePageQueuesLock is to be used, thanks to
Ingo for explaining it to me in the first place :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38895 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer(): Fixed the pagesSinceLastSuccessfulWrite computation. Only the
skipped pages were added, not the failed ones.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37682 a95241bf-73f2-0310-859d-f6bbb57e9c96

* PageWriteWrapper::Done(): Returns whether the page was successfully written,
now.
* PageWriterRun::Go(): Returns the number of pages that could not be written.
* page_writer()/next_modified_page():
- Don't use a marker page anymore. A visited page is requeued at the tail.
This also makes the functions that schedule pages work a bit better (they
queue the pages at the head of the queue).
- Have an eye on pages that are busy or failed to write. If we ran through
the whole modified queue without writing a single page, sleep for a short
time. This mitigates the busyness the page writer falls into when there
are enough modified pages to make it run all the time but none that can be
written successfully.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37681 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer(): Added TODO about potential deadlock in case of loop file
systems/devices.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37502 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the vm_page initialization from vm_page.cpp:vm_page_init() to the new
vm_page::Init().
* Made vm_page::wired_count private and added accessor methods.
* Added VMCache::fWiredPagesCount (the number of wired pages the cache
contains) and accessor methods.
* Made more use of vm_page::IsMapped().
* vm_copy_on_write_area(): Added vm_page_reservation* parameter that can be
used to request a special handling for wired pages. If given the wired pages
are replaced by copies and the original pages are moved to the upper cache.
* vm_copy_area():
- We don't need to do any wired ranges handling, if the source area is a
B_SHARED_AREA, since we don't touch the area's mappings in this case.
- We no longer wait for wired ranges of the concerned areas to disappear.
Instead we use the new vm_copy_on_write_area() feature and just let it
copy the wired pages. This fixes #6288, an issue introduced with the use
of user mutexes in libroot: When executing multiple concurrent fork()s all
but the first one would wait on the fork mutex, which (being a user mutex)
would wire a page that the vm_copy_area() of the first fork() would wait
for.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37460 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed warnings with TRACE_VM_PAGE enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37411 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added vm_page_max_address() which returns the greatest address of accessible
physical memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37230 a95241bf-73f2-0310-859d-f6bbb57e9c96

Report ignored memory pages seperately, thanks to bonefish for the patch review. Closes ticket #6178

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37175 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced structures {virtual,physical}_address_restrictions, which specify
restrictions for virtual/physical addresses.
* vm_page_allocate_page_run():
- Fixed conversion of base/limit to array indexes. sPhysicalPageOffset was not
taken into account.
- Takes a physical_address_restrictions instead of base/limit and also
supports alignment and boundary restrictions, now.
* map_backing_store(), VM[User,Kernel]AddressSpace::InsertArea()/
ReserveAddressRange() take a virtual_address_restrictions parameter, now. They
also support an alignment independent from the range size.
* create_area_etc(), vm_create_anonymous_area(): Take
{virtual,physical}_address_restrictions parameters, now.
* Removed no longer needed B_PHYSICAL_BASE_ADDRESS.
* DMAResources:
- Fixed potential overflows of uint32 when initializing from device node
attributes.
- Fixed bounce buffer creation TODOs: By using create_area_etc() with the
new restrictions parameters we can directly support physical high address,
boundary, and alignment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37131 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_get_stats(): Forgot to added sIgnoredPages to system_info::max_pages
in r37117 as well, which was really the main point of introducing it.
Improves #6124 (reported memory lower than installed).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37127 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_page_init_num_pages(): Sum up the number of pages between the usable
physical memory ranges.
* vm_page_num_pages(), vm_page_get_stats(): Report the actually existing
memory, count memory ignored by the boot loader as used.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37117 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_allocate_page_run(): Added parameter "limit", specifying the upper
physical address limit for the page run to allocate.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37086 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_allocate_early(): Replace "bool blockAlign" parameter by a more flexible
"addr_t aligmnent".
* X86PagingMethod32Bit::PhysicalPageSlotPool::InitInitial(),
generic_vm_physical_page_mapper_init(): Use vm_allocate_early()'s alignment
feature instead of aligning by hand.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37070 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed more address types related issues. Mostly printf() or comparison
warnings, but also some oversights from earlier changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37000 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced type generic_io_vec, which is similar to iovec, but uses types
that are wide enough for both virtual and physical addresses.
* DMABuffer, IORequest, IOScheduler,... and code using them: Use
generic_io_vec and generic_{addr,size}_t where necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36997 a95241bf-73f2-0310-859d-f6bbb57e9c96

* First run through the kernel's private parts to use phys_{addr,size}_t
where appropriate.
* Typedef'ed page_num_t to phys_addr_t and used it in more places in
vm_page.{h,cpp}.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36937 a95241bf-73f2-0310-859d-f6bbb57e9c96

Missing line breaks in dprintf().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36785 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_requeue(): Disabled DEBUG_PAGE_ACCESS_CHECK(). Since r36155
idle_scan_active_pages() cannot satisfy this assertion anymore. Added TODO
for a better fix which can wait until after the release.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36649 a95241bf-73f2-0310-859d-f6bbb57e9c96

mark_page_range_in_use(): When the given range lies partially outside the
array at least work with the part intersecting with the array. Log those
quasi error cases.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36633 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The "page_stats" command does now also print the longest contiguous runs of
free respective free and cached pages.
* Removed the unused vm_page_allocate_page_run_no_base().
* vm_page_allocate_page_run() (and allocate_page_run()):
- Use vm_page_reserve_pages() instead of vm_page_try_reserve_pages(), i.e.
wait until the reservation succeeds.
- Now we iterates two times through the pages to find a suitable page run. In
the first iteration it only looks for free/clear pages, in the second
iteration it also considers cached pages. This increases the chance of the
function to succeed, when a lot of caching is going on.
This reduces the amount of memory required to use the IOCache when booting
off the anyboot Live CD to around 160 MB in qemu. It also seems to work with
128 MB, but the syslog indicates that some memory allocations fail, which
is not exactly inspiring confidence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36489 a95241bf-73f2-0310-859d-f6bbb57e9c96

full_scan_inactive_pages(): Don't call DEBUG_PAGE_ACCESS_START() before
checking whether the page is busy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36155 a95241bf-73f2-0310-859d-f6bbb57e9c96

The "page" and the "cache" kernel debugger commands set a few temporary
variables, now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35903 a95241bf-73f2-0310-859d-f6bbb57e9c96

Boot loader (x86 mmu.cpp):
* Made the page table allocation more flexible. Got rid of sMaxVirtualAddress
and added new virtual_end address to the architecture specific kernel args.
* Increased the virtual space we reserve for the kernel to 16 MB. That
should suffice for quite a while. The previous 2 MB were too tight when
building the kernel with debug info.
* mmu_init(): The way we were translating the BIOS' extended memory map to
our physical ranges arrays was broken. Small gaps between usable memory
ranges would be ignored and instead marked allocated. This worked fine for
the boot loader and during the early kernel initialization, but after the
VM has been fully set up it frees all physical ranges that have not been
claimed otherwise. So those ranges could be entered into the free pages
list and would be used later. This could possibly cause all kinds of weird
problems, probably including ACPI issues. Now we add only the actually
usable ranges to our list.

Kernel:
* vm_page_init(): The pages of the ranges between the usable physical memory
ranges are now marked PAGE_STATE_UNUSED, the allocated ranges
PAGE_STATE_WIRED.
* unmap_and_free_physical_pages(): Don't free pages marked as unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35726 a95241bf-73f2-0310-859d-f6bbb57e9c96

idle_scan_active_pages(): vm_page_requeue() requires the caller to mark the
page accessed for the access debugging feature.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35542 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Also print the page for the page related asserts.
* Used vm_page::IsMapped() to simplify things.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35541 a95241bf-73f2-0310-859d-f6bbb57e9c96

free_cached_page(): Removed incorrect asserts. The same ones appear a few
lines later anyway, after the cache has been locked and it has been verified
that the page is still a candidate. Fixes #5432.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35539 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Made vm_page::state private and added accessor methods.
* Added kernel tracing for page state transitions.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added assert in free_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35533 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added some more asserts for cached pages.
* full_scan_inactive_pages(): Fixed syntactical glitch (missing "else").
Affected only the active paging mode and was relatively harmless. The worst
case would be that an inactive page would be moved to the cached queue
although its usage_count hadn't dropped to 0 yet, thus freeing it before
pages that deserved it more.
* move_page_to_active_or_inactive_queue():
- Was ignoring the page's modified flag, thus potentially moving a modified
page to the cached queue. That could happen only in rare cases though,
like when the page was still mmap()ped while being written and modified and
unmapped before being done.
- No longer move the page to the inactive queue, even if its usage count is 0.
In idle mode the page daemon doesn't look at inactive pages, so the page's
stats wouldn't be updated anymore.
- Renamed to move_page_to_appropriate_queue().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35499 a95241bf-73f2-0310-859d-f6bbb57e9c96

* page_writer(): Missing continue in the temporary skipping case.
* vm_page_write_modified_page_range(): Need to DEBUG_PAGE_ACCESS_END() a bit
later.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35487 a95241bf-73f2-0310-859d-f6bbb57e9c96

page_writer():
* Use the same criterion when to write back temporary pages as the page daemon.
* Move wired and temporary pages that shall not be written back to the active or
inactive queue so they don't get stuck in the modified queue and potentially
cause the page writer to run permanently without actually making progress
(#5382).

page writing implementation:
* If writing back a temporary page failed, move it back to the inactive or
active queue, so it doesn't get stuck in the modified queue. If active paging
continues, it might find its way back to the modified queue in the next
iteration, but that improves the situation a bit at least. Particularly with
the port heap pages not really being swappable ATM.
* Never dequeue pages from the modified queue. We mark them busy, so the page
writer will skip them anyway. This allows others to play with the page to some
extend (at least allowing to move it between the queues). This fixes #5404.
* Removed PageWriteWrapper::ClearModifiedFlag(). We clear the modified flag in
PageWriteWrapper::SetTo(), now, so that the page writer doesn't need to do
that explicitly either.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35485 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed PageWriteWrapper::CheckRemoveFromShrunkenCache() and moved removing
the page to Done().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35474 a95241bf-73f2-0310-859d-f6bbb57e9c96

Reenabled interrupts before notifying the page writer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35399 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the commented out scanning of the inactive queue when no paging is
going on. I only wanted to have it in the repository in case we decide at a
later point that it is a good idea after all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35395 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
of vm_test_map_activation(), vm_clear_map_flags(), and
vm_remove_all_page_mappings(), thus saving lots of calls to translation map
methods. The backend is the new method
VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
other non-free queues slightly:
- Active queue: Contains mapped pages that have been used recently.
- Inactive queue: Contains mapped pages that have not been used recently. Also
contains unmapped temporary pages.
- Modified queue: Contains unmapped modified pages.
- Cached queue: Contains unmapped unmodified pages (LRU sorted).
Unless we're actually low on memory and actively do paging, modified and
cached queues only contain non-temporary pages. Cached pages are considered
quasi free. They still belong to a cache, but since they are unmodified and
unmapped, they can be freed immediately. And this is what
vm_page_[try_]reserve_pages() do now when there are no more actually free
pages at hand. Essentially this means that pages storing cached file data,
unless mmap()ped, no longer are considered used and don't contribute to page
pressure. Paging will not happen as long there are enough free + cached pages
available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
the page queues. As long as the free pages situation is harmless, it only
iterates through the active queue and deactivates pages that have not been
used recently. When paging occurs it additionally scans the inactive queue and
frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
vm_page_allocate_page() now take a vm_page_reservation structure pointer.
The reservation functions initialize the structure -- currently consisting
only of a count member for the number of still reserved pages.
vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
unreserves the remaining pages (if any). Advantages are that reservation/
unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
can verify that the caller has indeed a reserved page left, and that there's
no unnecessary pressure on the free page pool anymore. The only disadvantage
is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
- Got rid of sSystemReservedPages and sPageDeficit. Instead
sUnreservedFreePages now actually contains the number of free pages that
have not yet been reserved (it cannot become negative anymore) and the new
sUnsatisfiedPageReservations contains the number of pages that are still
needed for reservation.
- Threads waiting for reservations do now add themselves to a waiter queue,
which is ordered by descending priority (VM priority and thread priority).
High priority waiters are served first when pages become available.
Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
the size wasn't page aligned.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the vm_page_allocate_page*() "pageState" parameter by a more
general "flags" parameter. It encodes the target state of the page -- so
that the page isn't unnecessarily put in the wrong page queue first -- a
flag whether the page should be cleared, and one to indicate whether the
page should be marked busy.
* Added page state PAGE_STATE_CACHED. Not used yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy
flag. The obvious advantage is that one can still see what state a page is in
and even move it between states while being marked busy.
* Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which
in all cases has the same effect. Introduced a vm_page_is_dummy() that can
still check whether a given page is a dummy page.
* vm_page_unreserve_pages(): Before adding to the system reserve make sure
sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages
available for allocation. steal_pages() still has the same problem and it
can't be solved that easily.
* map_page(): No longer changes the page state/mark the page unbusy. That's the
caller's responsibility.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved the clarity of some loops in the page reservation functions. No
functional change (other than avoiding no-ops like subtracting 0).
* vm_page_try_reserve_pages(): Moved the kernel tracing calls from the top of
the function to the points where the reservation already succeeded.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35322 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced {malloc,memalign,free}_etc() which take an additional "flags"
argument. They replace the previous special-purpose allocation functions
(malloc_nogrow(), vip_io_request_malloc()).
* Moved the I/O VIP heap to heap.cpp accordingly.
* Added quite a bit of passing around of allocation flags in the VM,
particularly in the VM*AddressSpace classes.
* Fixed IOBuffer::GetNextVirtualVec(): It was ignoring the VIP flag and always
allocated on the normal heap.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35316 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The system now holds back a small reserve of committable memory and pages. The
memory and page reservation functions have a new "priority" parameter that
indicates how deep the function may tap into that reserve. The currently
existing priority levels are "user", "system", and "VIP". The idea is that
user programs should never be able to cause a state that gets the kernel into
trouble due to heavy battling for memory. The "VIP" level (not really used
yet) is intended for allocations that are required to free memory eventually
(in the page writer). More levels are thinkable in the future, like "user real
time" or "user system server".
* Added "priority" parameters to several VMCache methods.
* Replaced the map_backing_store() "unmapAddressRange" parameter by a "flags"
parameter.
* Added area creation flag CREATE_AREA_PRIORITY_VIP and slab allocator flag
CACHE_PRIORITY_VIP indicating the importance of the request.
* Changed most code to pass the right priorities/flags.

These changes already significantly improve the behavior in low memory
situations. I've tested a bit with 64 MB (virtual) RAM and, while not
particularly fast and responsive, the system remains at least usable under high
memory pressure.
As a side effect the slab allocator can now be used as general memory allocator.
Not done by default yet, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35295 a95241bf-73f2-0310-859d-f6bbb57e9c96

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page no longer points directly to its containing cache, but rather to a
VMCacheRef object which points to the cache. This allows to optimize
VMCache::MoveAllPages(), since it no longer needs to iterate over all pages
to adjust their cache pointer. It can simple swap the cache refs of the two
caches instead.

Reduces the total -j8 Haiku image build time only marginally. The kernel time
drops almost 10%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35155 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page::accessed flag. Works analogously to vm_page::modified.
* Reorganized the code for [un]mapping pages:
- Added new VMTranslationMap::Unmap{Area,Page[s]}() which essentially do what
vm_unmap_page[s]() did before, just in the architecture specific code, which
allows for specific optimizations. UnmapArea() is for the special case that
the complete area is unmapped. Particularly in case the address space is
deleted, some work can be saved. Several TODOs could be slain.
- Since they are only used within vm.cpp vm_map_page() and vm_unmap_page[s]()
are now static and have lost their prefix (and the "preserveModified"
parameter).
* Added VMTranslationMap::Protect{Page,Area}(). They are just inline wrappers
for Protect().
* X86VMTranslationMap::Protect(): Make sure not to accidentally clear the
accessed/dirty flags.
* X86VMTranslationMap::Unmap()/Protect(): Make page table skipping actually
work. It was only skipping to the next page.
* Adjusted the PPC code to at least compile.

No measurable effect for the -j8 Haiku image build time, though the kernel time
drops minimally.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35089 a95241bf-73f2-0310-859d-f6bbb57e9c96

Refactored vm_translation_map:
* Pulled the physical page mapping functions out of vm_translation_map into
a new interface VMPhysicalPageMapper.
* Renamed vm_translation_map to VMTranslationMap and made it a proper C++
class. The functions in the operations vector have become methods.
* Added class GenericVMPhysicalPageMapper implementing VMPhysicalPageMapper
as far as possible (without actually writing new code).
* Adjusted the x86 and the PPC specifics accordingly (untested for the
latter). For the other architectures the build is, I'm afraid, seriously
broken.

The next steps will modify and extend the VMTranslationMap interface, so that
it will be possible to fix the bugs in vm_unmap_page[s]() and employ
architecture specific optimizations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35066 a95241bf-73f2-0310-859d-f6bbb57e9c96

Introduced vm_page::modified flag, which is used by vm_unmap_page[s]() to
preserve the dirty flag of the mapping without having to potentially move the
page to the modified queue. This lifts the (ignored) requirement that the
pages to be unmapped must not be busy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35023 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed two instances of debug code I missed to adjust in r35004. Fixes #5242.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35020 a95241bf-73f2-0310-859d-f6bbb57e9c96

PAGE_TYPE_GUARD was unused and for the other two types a simple one bit flag
suffices. Therefore replaced vm_page::type by vm_page::is_dummy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35013 a95241bf-73f2-0310-859d-f6bbb57e9c96

allocate_page_run(): Use temporary lists to store the pages we're allocating.
This makes appending the pages to the active queue more efficient and we
don't need the vm_page::is_cleared bit anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35011 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the page queue mutexes by spinlocks. The critical sections are very
short and quite hot, so mutexes just cause more overhead due to frequent
rescheduling than waiting for the spinlocks does. The free and clear queues
are additionally protected by a R/W lock, which is mostly read-locked, save
for rare cases like allocating page runs.

The total -j8 Haiku image build speedup is marginal. The kernel time drops
about 8%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35004 a95241bf-73f2-0310-859d-f6bbb57e9c96

vm_page_schedule_write_page_range(): The debug code was a bit too possessive.
We can only access the page, if it is not busy. Fixes #5228.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34980 a95241bf-73f2-0310-859d-f6bbb57e9c96

"page" debugger command:
* Various smaller fixes.
* Used add_debugger_command_etc() and added more verbose usage message.
* Added option "-m" which iterates through all address spaces and finds out
which virtual pages are mapped to the page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34979 a95241bf-73f2-0310-859d-f6bbb57e9c96

"find_page" didn't search the inactive queue.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34972 a95241bf-73f2-0310-859d-f6bbb57e9c96

Don't announce page access before being sure that the page isn't busy. Fixes
the erroneously trigger panic() in #5216.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34958 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96

steal_pages(): Recheck sUnreservedFreePages after incrementing sPageDeficit.
Since the former is no longer guarded by any lock, there's a race condition
with vm_page_unreserve_pages() which would cause us to wait longer than
necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34898 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the sReservedPages counter tracking the page reservations by
sUnreservedFreePages which tracks the difference between free/clear and
reserved pages. Access to it uses atomic operations which allows the three
page (un)reservation to avoid locking in most cases, thus reducing contention
of the "pages" lock.

In the -j8 Haiku image build that decreases the contention of the "pages"
lock to about one third of the previous value. As a positive side effect the
VMCache lock contention drops about the same factor. The total build speedup
is about 20%, the total kernel time drops about 20%.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34888 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed duplicate condition.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the page management functionality into its own file.
* Renamed page_queue to VMPageQueue and made it a proper C++ class. Use
DoublyLinkedList instead of own list code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96

* agp_gart(): Use vm_page_[un]reserve_pages().
* Removed unused vm_page_allocate_pages().
* Removed now unused (always true) "reserved" parameter from
vm_page_allocate_page().
* Removed unused (always false) "stealActive" parameter from steal_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34836 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed DEBUG_PAGE_CACHE_TRANSITIONS debugging.
* Added VMCache::MovePage() and MoveAllPages() to move pages between caches.
* VMAnonymousCache:
- _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out
pages.
- _MergeSwapPages() does now also remove source cache pages that are
shadowed by consumer swap pages. This allows us to call _MergeSwapPages()
before _MergePagesSmallerSource(), save the swap page shadowing check
there and get rid of the vm_page::merge_swap flag. This is an
optimization based on the assumption that usually none or only few pages
are swapped out, so we save a lot of checks.
- Implemented _MergePagesSmallerConsumer() as an alternative to
_MergePagesSmallerSource(). The former is used when the source cache has
more pages than the consumer cache. It iterates over the consumer cache's
pages, moves them to the source and finally moves all pages back to the
consumer. The final move is relatively cheap (though unfortunately we
still have to update all pages' vm_page::cache field), so that overall we
save iterations of the main loop with the more expensive checks.

The optimizations particularly improve the common fork()+exec*() situations.
fork() uses CoW, which is implemented by putting two new empty caches between
the to be copied area and its cache. exec*() destroys one copy of the area,
its cache and thus causes merging of the other new cache with the old cache.
Since this usually happens in a very short time, the old cache does still
contain many pages and the new cache only few. Previously the many pages were
all checked and moved individually. Now we do that for the few pages instead.

A very extreme example of this situation is the Haiku image build. jam has a
huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed.
Since during the cache merging the cache is locked, any write access to a
heap page causes jam to block until the cache merging is done. Formerly that
took so long that it killed a lot of parallelism in multi-job builds. That
could be observed particularly well when lots of small actions where executed
(like the Link, XRes, Mimeset, SetType, SetVersion combos when building
executables/libraries/add-ons). Those look dramatically better now.
The overall speed improvement for a -j8 image build on my machine is only
about 15%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a simple mechanism to wait for events to VMCache. WaitForPageEvents()
waits for certain events on a given page, NotifyPageEvents() wakes up
waiting threads respectively.
* Used the new feature instead of condition variables for waiting on busy
pages. We save publishing and unpublishing of a condition variable whenever
a page is marked busy. There's only something to do, if there's at least
one thread waiting in the list of the respective cache. The general
assumption is that this is only rarely the case and even if it happens,
there should be only very few threads.
* Added an apparently missing notification in cache_io(). At least I didn't
see the reason for it not being there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34537 a95241bf-73f2-0310-859d-f6bbb57e9c96

Created VMArea.{h,cpp} and moved VMArea and the global area hash table (new
class VMAreaHash) there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34450 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved VMAddressSpace definition to vm_address_space.h.
* "Classified" VMAddressSpace, i.e. turned the vm_address_space_*() functions
into methods, made all attributes (but "areas") private, and added
accessors.
* Also turned the vm.cpp functions vm_area_lookup() and
remove_area_from_address_space() into VMAddressSpace methods. The rest of
the area management functionality will follow soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34447 a95241bf-73f2-0310-859d-f6bbb57e9c96

Finished renaming of vm_cache to VMCache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34424 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed vm_address_space to VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34422 a95241bf-73f2-0310-859d-f6bbb57e9c96

* There is no need for the sPageLock to be a spinlock - a mutex should do. This
should improve the kernel latencies, as things like
vm_page_allocate_page_run() is very expensive.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33807 a95241bf-73f2-0310-859d-f6bbb57e9c96

Add a vm_page_allocate_page_run_no_base. It bases its search on the pages found
in the free and/or clear queue. This performs better in the case where only few
pages are free/clear but performs worse in the case where there are a lot of
usable pages. It's not used anywhere but it might come in handy one time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33527 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implement combining scattered physical pages using seperate iovecs in
PageWriteTransfer. This makes the transfer accept virtually contiguous pages,
where the offset is contiguous on either end of the current transfer, but where
the pages aren't physically contiguous. It will then add seperate iovecs for
these pages (32 at max right now). This reduces the number of IO requests
generated and allows for optimizations down the IO path (like in the physical to
virtual mapping case for example).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33526 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Rework page writing to combine page writes where possible. For now the pages
are required to be physically contiguos, which should be reworked to put them
into seperate iovecs. Still this manages to combine a great deal of page
writes into larger bursts already. Reduces the amount of IO requests being
scheduled (and greatly benefits media where page wise writes are slow when
they are accessed through a non-IOScheduler path, i.e. USB mass storage until
that is properly implemented).
* Abstracted per page page writing tasks into a PageWriteWrapper class.
* Abstracted per transfer page writing tasks into PageWriteTransfer class which
formerly was the PageWriterCallback.
* Use both classes from the PageWriterRun and from
vm_page_write_modified_page_range to remove code duplication.
* Adjusted synchronous VMAnonymousCache::Write() to cope correctly with larger
iovecs and more than one iovec. It assumed that there was exactly one page per
vector previously.
* Introduced MaxPagesPerWrite() and MaxPagesPerAsyncWrite() to VMCache to allow
a cache to specify restricitions. VMAnonymousCache does restrict the max pages
to 1 for WriteAsync right now as I didn't feel like reworking that one to cope
with non single page writes just yet.
* Pulled out PageWriteTransfer methods for better readability.
* Some typo fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33507 a95241bf-73f2-0310-859d-f6bbb57e9c96

* When reserving pages and there aren't yet enough free pages, only steal as
many pages as are actually missing, not the full count.
* Take into account that free_page_queue_count() can be less than sReservedPages
(when some of the reserved pages have been allocated already) in
vm_page_num_unused_pages(). Before it could return negative and therefore
wrapped numbers.
* Simplify the page scrubber loop by continuing early. Also avoids a needless
interrupt spin lock acquisition when there's nothing to do.
* Some minor coding style cleanup.
* Fix a typo.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32980 a95241bf-73f2-0310-859d-f6bbb57e9c96

panic() when a page that is still mapped is freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30606 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TODO about a problematic use of vm_page_allocate_page() in combination
with vm_cache_acquire_locked_page_cache().
* Added new function vm_page_num_unused_pages() which returns the pages that are
actually completely free and unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30514 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduces VMCache::CanWritePage() returning whether the given cache can
theoretically write the given page.
* page writer: Fixed the incorrect check whether a temporary page can be
written by using the new CanWritePage().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30281 a95241bf-73f2-0310-859d-f6bbb57e9c96

page writer:
* When writing a page failed it is not a good idea to re-enqueue it at the tail
of the modified queue, since that is definitely behind the page writer's
marker and the page would be picked up again before reaching the end of the
queue. If that happened with more than 256 pages, the page writer would keep
picking up only those non-writable pages and make no more progress.
* When selecting pages also skip temporary pages, if there's no more swap space
available, since trying to write those pages would most likely fail anyway
(triggering the first problem).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30230 a95241bf-73f2-0310-859d-f6bbb57e9c96

* PageWriterRun::Go(), vm_page_write_modified_page_range(): When writing the
page failed since the cache has been shrunk, we need not only remove the
page from the cache, we also need to remove all of its area mappings and
free it. Not removing the area mappings might have been the cause of #3110,
not freeing it would cause it to be leaked for good.
* vm_page_write_modified_page_range(): When writing failed for another reason
and the page wasn't in the modified queue before, we would lose the info
in which queue it was before and setting the page state to modified would
assume the active queue. This could potentially screw up our page queue
structures.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29992 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed src/system/kernel/device_manager/io_requests.{h,cpp} to
IORequest.{h,cpp}.
* Introduced public <io_requests.h> header. Currently it only declares the
single function BFS uses.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29446 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96

Dynamically adjust the page writer's I/O priority depending on how many
pages have to be written and how tight the memory situation is.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28198 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix ifdef, should be DEBUG_PAGE_QUEUE there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28009 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the vm_get_physical_page() "flags"
PHYSICAL_PAGE_{NO,CAN}_WAIT into an actual flag
PHYSICAL_PAGE_DONT_WAIT.
* Pass the flags through to the chunk mapper callback.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27979 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added kernel tracing for the page daemon and the page writer.
* Added some commented out debug output in vm.cpp.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27971 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed warnings with tracing enabled and made some very noisy dprintfs trace-only
as they were flooding the syslog.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27935 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved several VM related debug settings to kernel_debug_config.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27876 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a (private for now) get_system_info_etc() call, that can retrieve
various system information.
* Implemented retrieving some VM stats via this call.
* The VM now maintains a page fault counter, and sets system_info::page_faults
accordingly.
* Added a (pretty simple) "vmstat" command line app.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27597 a95241bf-73f2-0310-859d-f6bbb57e9c96

Print some more interesting timing info for the page writer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27245 a95241bf-73f2-0310-859d-f6bbb57e9c96

steal_pages() was leaking a cache reference since r26572, i.e caches
locked there would be leaked eventually.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27228 a95241bf-73f2-0310-859d-f6bbb57e9c96

Increased the number of pages the page writer tries to write per run.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27217 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The LIMIT_AVAILABLE_MEMORY macro can be defined to limit the amount of
RAM Haiku uses (for debugging purposes).
* "page_stats" prints some more infos now.
* page_writer():
- Moved the low_resource_state() invocation out of the inner loop.
Reduces lock contention on the sLowResourceLock recursive lock.
- Additional debug output: Every 1024 written pages the page writer
prints a message.
* steal_pages(): Addressed the TODO: When there should be pages to
steal, but we can't get them ATM, we now timeout on the free pages
condition variable instead of snoozing unconditionally, so that we
wake up earlier when someone frees pages in the meantime.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27181 a95241bf-73f2-0310-859d-f6bbb57e9c96

Introduced vm_page_try_reserve_pages(), which fails when not enough
pages are free ATM.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27116 a95241bf-73f2-0310-859d-f6bbb57e9c96

* VMCache::Write(): Added "uint32 flags" argument which is supposed to
be passed on to the IORequest. Most relevantly physical pages can now
be written directly by passing B_PHYSICAL_IO_REQUEST.
* Added VMCache::WriteAsync() which is supposed to write pages
asynchronously. The base class version version falls back to the
synchronous Write(). Only VMVnodeCache implements WriteAsync() ATM,
VMAnonymousCache (swap support) still has to be adjusted accordingly.
* write_page() doesn't need to map the page anymore as it can write the
physical page directly.
* Modified the page writer to write pages asynchronously. This shouldn't
have any noticeable effect yet. It will though as soon as the I/O
scheduler reorders I/O operations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27056 a95241bf-73f2-0310-859d-f6bbb57e9c96

"page_stats" does now also print the number of mapped pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26875 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the incrementing/decrementing of vm_page::wired_count into
dedicated functions.
* Introduced gMappedPagesCount variable which counts the total number of
physical pages that are mapped.
* Added vm_page_get_stats() which fills in the memory related part of
the system_info structure. Used and cached pages are computed
differently, now. The "available" (== not committed) memory is no
longer used for the computation as it doesn't say anything about the
actually used/free pages (with swap support enabled it is even
less meaningful, since we first commit swap space when possible).
We do also consider the memory used by the block cache as cached
pages, now. All in all these changes should fix the memory statistics
reported by get_system_info(), IOW bug #2574.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26837 a95241bf-73f2-0310-859d-f6bbb57e9c96

The page queues no longer count added dummy pages. This fixes a problem
in steal_pages() which uses the number of pages in the inactive queue as
return criterion. It would thus return, if only marker pages were in the
queue and no page could be stolen. This led to a busy loop in
vm_page_allocate_page(). The whole system would become unusable when the
thread in question was the heap grower, since it would starve everyone
else due to its high priority.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26710 a95241bf-73f2-0310-859d-f6bbb57e9c96

Reverted most of r26641.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26658 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Zhao Shuai with some modifications by myself:
* Renamed sModifiedTemporaryPages to sModifiedNoSwapPages to better
express what this variable is about.
* Changed tracking of sModifiedNoSwapPages. It really counts
non-swappable pages only, now (if swap support is enabled).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26641 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Zhao Shuai with changes by myself:
* Init swap support in main().
* Added "bool swappable" parameter to
VMCacheFactory::CreateAnonymousCache(). A cache supporting swapping
is created when true. Adjusted invocations accordingly.
* The page writer does now write non-locked swappable pages (when
memory is low).
* Fixed header guard of VMAnonymousNoSwapCache.h.
* Swap support is compiled conditionally, controlled by the
ENABLE_SWAP_SUPPORT in src/system/kernel/vm/VMAnonymousCache.h. It is
disabled ATM. Since no swap files are added, it wouldn't have much
effect anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26625 a95241bf-73f2-0310-859d-f6bbb57e9c96

* bfs_fsync() was the only place which could cause the
fs_vnode_ops::write_pages() to be called with fsReenter = true. Since
this is no longer the case, the argument has become superfluous. For
read_pages() it always was. Removed the argument from the functions
and all functions that propagated it.
* Some whitespace at the end of lines was removed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26579 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added B_PHYSICAL_BASE_ADDRESS address specification for anonymous areas.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26456 a95241bf-73f2-0310-859d-f6bbb57e9c96

Remove a few unused variables.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26249 a95241bf-73f2-0310-859d-f6bbb57e9c96

Spotted this while reading through page code. The usage_count would be set for
wrong pages when allocating a page run unless the run started at page 0 as the
start offset was missing.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26245 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced the global cache pages hash table by an IteratableSplayTree
per cache.
* Changed the strategy vm_cache_acquire_page_cache_ref() uses to ensure
that the cache isn't deleted while trying to get a reference. Instead
of the global cache pages hash table lock, it holds the global cache
list lock now. We acquire + release this lock in delete_cache() after
removing all pages and just before deleting the object.
* Some small optimizations using the property that the cache's pages are
ordered, now (vm_cache_resize(), vm_page_write_modified_page_range(),
vm_page_schedule_write_page_range()).
* Replaced some code counting a cache's pages by simply using
vm_cache::page_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26160 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fix cache reference leak


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26110 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vm_cache::virtual_size is not always a multiple of B_PAGE_SIZE for files.
In these cases, the last partial page would have never been written back.
* This fixes bug #2282, and eventually others.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25747 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page_write_modified_page_range(), which is similar to
vm_page_write_modified_pages(), save that it only writes pages in the
given range.
* Added vm_page_schedule_write_page_range() which schedules all modified
pages in the given cache's range for writing by the page writer.
* Added _kern_sync_memory() syscall and the msync() POSIX function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25620 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed old mutex implementation and renamed cutex to mutex.
* Trivial adjustments of code using mutexes. Mostly removing the
mutex_init() return value check.
* Added mutex_lock_threads_locked(), which is called with the threads
spinlock being held. The spinlock is released while waiting, of
course. This function is useful in cases where the existence of the
mutex object is ensured by holding the threads spinlock.
* Changed the two instances in the VFS code where an IO context of
another team needs to be locked to use mutex_lock_threads_locked().
Before it required a semaphore-based mutex implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25283 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced the vm_cache mutex by a cutex. This should save quite a few
semaphores.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25277 a95241bf-73f2-0310-859d-f6bbb57e9c96

* steal_page() called remove_page_from_queue() without holding the sPageLock.
This fixes bug #1900 for real.
* Rearranged find_page_candidate() a bit, removed duplicate code, added a panic
in case the marker state is invalid.
* Some cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25255 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The page writer was calling remove_page_marker() without holding the
sPageLock. This could easily mess up the page queue.
* Now, remove_page_marker() gets the lock itself. This fixes bug #1900.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25250 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed macros that enable tracing for individual components from
defined/undefined to numeric values (0 for undefined). This allows for
trace levels.
* Set SYSCALL_TRACING_IGNORE_KTRACE_OUTPUT default to 1, since this is
what one usually wants.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25213 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed vm_get_available_memory() to vm_available_memory() to fit better
into our usual naming scheme.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24745 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed the page allocation tracking history to kernel tracing instead.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24614 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed a quasi-livelock in steal_pages() as proposed in ticket #1929.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24605 a95241bf-73f2-0310-859d-f6bbb57e9c96

Applied patch by Rene Gollent:
* Add a cached_pages field to the system_info structure, and change the
meaning of the used_pages field to not include cached pages.
* Provide the needed info using the new calls vm_get_available_memory(),
and vm_page_num_available_pages().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24571 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed race condition in the page writer: The state of the page we have
picked might have changed while we were locking its cache. Might fix
#1931.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24430 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added a boolean "force" parameter to thread_yield(). When true, the
function has the old behavior. When false, it just calls the scheduler
without any priority adjustment or other stuff.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23906 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a deadlock (and resolved TODO): we need to make all pages unbusy again
before releasing our cache reference. Otherwise removing a vnode (triggered
by releasing the cache in our thread) could need pages we still own.
* Put the caches and pages into a union to save stack space; they are not
needed at the same time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23580 a95241bf-73f2-0310-859d-f6bbb57e9c96