kernel & add-ons: Adjustments to use DoublyLinkedList::InsertBefore.

The Insert(before, element) function has been marked deprecated since
2010, but still had many usage and kept accumulating more. It's long
past time we got rid of all them and actually deprecated the function
itself.

Insert(before, element) just calls InsertBefore, so no functional change.

kernel/cache: Set parent_data outside the if-block.

Reduces code duplication while retaining clarity.
Thanks axeld for the review.

kernel/block_cache: Fix memory leak in sub-transaction handling.

This code has been broken like this since 2012, so I am skeptical
that it is the cause of the more recently observed #17463.
It is however possible that more recent changes caused things
to get noticeably worse for some reason.

This may, however, resolve some of the other KDLs, e.g. #12847.

kernel/block_cache: Adjust logic in cache_detach_sub_transaction.

This way it should be more clear that parent_data is being modified
in tandem with original_data.

No functional change intended.

block_cache: use printf when not built in kernel mode

this is used by userlandfs

Change-Id: I2f7971be819c36fcab78eca2d18b45bb0b1a5fad
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3277
Reviewed-by: Rene Gollent <rene@gollent.com>

block_cache: Change signature of *_etc() functions

* This allows file systems to retrieve the actual error code on a
failure, and report it to the user.
* All affected file systems have been adjusted to the API change.
This is a binary incompatible change.

Change-Id: Id73392aaf9c6cb7d643ff9adcb8bf80f3037874c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2913
Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

block_cache: Compute the timeout dynamically based on write speeds.

We now record how long it takes to write a block (on average), and then
utilize this information to reduce the timeout write thread's timeout
(to 2 * block_count * average_block_time, so we don't completely
congest the drive.) Remove the "TODO" about the I/O scheduler;
this new logic will be just fine even under an I/O scheduler.

Note that this change goes both ways: while faster writes mean more
writes and quicker, slower writes will increase the timeout before
we do another one also. This then also guards against queueing
another write while one is already in progress, which was
not handled before.

Tested in KVM. Even on a SATA-backed spinning HDD, this reduces
the timeout to around *200ms* on average (!!), so a 10x improvement.
On a ramdisk, it reduces the timeout to *10-30ms* (!!!) on average,
so a 100-200x improvement, so this change will benefit everyone
but SSDs especially.

Since BFS inode and journal writes always go through the block_cache,
this very dramatically improves inode-related write performance.
The "stop and start" stutters when emptying or moving items to Trash
seem totally gone, among a lot of other things.

Change-Id: I41f46a6432ce1f50f896a853abdfe22dde0ba327

block_cache: Use an object_cache for the cache_notifications.

cache_listeners are (following packagefs changes) the second-most
allocated object during the boot process, with 20799 on a
*minimum* image. Since they are used so extensively in BFS I/O,
making them object_cached for both performance and memory reasons
seems to make a lot of sense.

Change-Id: I6ef6c3e811c0c4189fea45ee7920131796c9e7c8
Reviewed-on: https://review.haiku-os.org/c/1577
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: Stephan Aßmus <superstippi@gmx.de>

block_cache: Free up blocks more aggressively on low memory conditions.

Only keep a single block when there is a "critical" low resource state.
Also don't bother doing anything if there are no unused blocks.

Change-Id: Ibfcbd8cb0beb1446083ca83030ea8e81c59a9628
Reviewed-on: https://review.haiku-os.org/c/1576
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

block_cache: Implement cache_has_block_in_transaction function that will check the existence of block in one specific transaction.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>

kernel: Use anonymous namespaces to avoid type collisions.

The anonymous namespace makes type definitions local to the translation
unit (like static does for objects). For pretty much any type not shared
across multiple files this is what one wants to happen (and might
erroneously expect to happen automatically).

This commit solves some actual collisions that were present:

* The VFS and the rootfs both used an incompatible VnodeHash struct for
their BOpenHashTable.
* XSI semaphores and message queues both used queued_thread, Ipc and
IpcHashTableDefinition.

For release builds these did not cause problems as the types were fully
inlined. Debug builds would crash at boot however because parts of a
BOpenHashTable<VnodeHash> from the rootfs meant to operate on struct
rootfs_vnode would be applied to one from the VFS expecting struct
vnode.

As such collisions are violations of the one definition rule, the code
is at fault and unfortunatley the compiler isn't required to diagnose
such problems across translation units (which isn't actually trivial).
This can lead to subtle and hard to debug problems and it's therefore
best to avoid leaking types into the global namespace whenever possible.

BlockWriter: Do not panic on failing read/writes

These were here for debugging purposes, as often it is a sign of
inconsistencies. However, for USB disks this is a normal occurence
when someone janks out of the device without unmounting first.

Make sure we log these cases though, as it still helps debugging.

Fix sponsered by http://www.izcorp.com

block cache: convert to BOpenHashTable.

Fix typo.

cache_abort_sub_transaction() did not maintain num_blocks.

* When a block was only used in a sub-transaction, it was thrown away,
but the transaction::num_blocks field was not decremented.
* This caused transactions never considered finished which eventually
led to bug #8942. This does not explain the disk corruption occurring
in #8969, though.

Fixed the new issue in #8910 from r44585.

* The ASSERT() I introduced in r44585 was incorrect: when the sub transaction
used block_cache_get_empty() to get the block, there is no original_data for
a reason.
* Added a test case that reproduces this situation.
* The block must be moved to the unused list in this situation, though, or else
it might contain invalid data. Since the block can only be allocated in the
current transaction, this should not be a problem, though, AFAICT.

The original_data could be freed late.

* In cache_abort_sub_transaction(), the original_data can already be freed
when the block is being removed from the transaction.
* block_cache::_GetUnusedBlock() no longer frees original/parent data - it
now requires them to be freed already (it makes no sense to have them still
around at this point).
* AFAICT the previous version did not have any negative consequences besides
freeing the original data late.

Minor block cache refactoring.

* Extracted a write_blocks_in_previous_transaction() function out of
cache_end_transaction(), and cache_detach_sub_transaction().

Fixed broken block list introduced in hrev44357.

* cache_abort_sub_transaction() was setting the transaction_next pointer to
NULL in order to remove a block from a transaction -- however, it forgot to
actually remove it from the transaction's block list.
* Minor restructuring.

Fixed broken discard handling in cache_start_sub_transaction().

* This actually resolves a TODO.

Cleanup, no functional change intended.

cache_abort_[sub_]transaction() did not work correctly.

* cache_abort_transaction() left the block dirty which was causing bug
#8123 as well.
* cache_abort_sub_transaction() did, in addition to not clearing the dirty
flag, not reset the block's transaction member either if the block was
not part of the parent transaction.

Added a bit of documentation, minor cleanup.

* Documented the cached_block::transaction, and previous_transaction
members.

An unused dirty block would put again into the unused list.

* This could cause bug #8123. While this fixes the crash, one underlying
problem is still there, as dirty blocks should never leave a transaction.

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.

Change instances of wether in comments to whether. No functional change intended.

Added intermediate (addr_t) casts before the cast of the parse_expression()
result (uint64) to pointer (CID 4723-4726).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40121 a95241bf-73f2-0310-859d-f6bbb57e9c96

* We need to reset the blockNumber, or else we'll discard the wrong blocks
which has rather desastrous consequences - thanks to Stippi for the heads
up :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38242 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed CID 1447 which was an actual bug which completely voided the whole
discard mechanism eventually causing the conflicts between the file cache
and the block cache it intended to fix.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38232 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Thanks to Janito I had a closer look to cache_detach_sub_transaction(), and it
turned out it's pretty much broken. Not only did it potentially leak memory,
it would also potentially replace the original data with a NULL pointer,
making the current transaction non revertable. The code should now be much
clearer. This fixes bug #6378.
* Changed a few "if (... != NULL)" into ASSERTs, since the code should bail out
earlier in these cases already.
* Added a TODO comment to cache_start_sub_transaction() about its broken discard
handling. This can cause FS corruptions in case the parent transaction is ever
going to be aborted (which shouldn't happen in real life, though).
* Added a bit more and better comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37899 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_end_transaction(): Allow the hook parameter to be NULL.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37461 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced block_cache::unused_block_count, which counts the elements of
block_cache::unused_blocks.
* block_cache::Allocate(): No longer removes unused blocks when in a low
resource state. That just removed too many blocks too quickly, when the
cache was actively used for writing, seriously affecting performance.
* block_cache::_LowMemoryHandler(): Compute the number of unused blocks to
remove depending on the total unused block number. This way we cull huge
block caches with lots of old blocks much quicker.

Treats part of #5816.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37170 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache::NewBlock(): Allocate cached_block::current_data only when the
block was freshly allocated. A block returned by _GetUnusedBlock() already
has current_data and we would leak it before.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35918 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If get_cached_block() failed reading a block, it would leave a deleted object
in the unused list if it got the block via block_cache::_GetUnusedBlock().
* block_cache::_GetUnusedBlock() leaked the compare data block if the
BLOCK_CACHE_DEBUG_CHANGED feature had been enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35799 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_notifier_and_writer() will now flush 1000 blocks when reaching the soft
memory limit.
* Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35735 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced cached_block::accessed (the number of accesses) with last_accessed
(the time of the last access), as what we really want is a frequency/last
access time scoring, and accessed alone is useless for that.
* put_cached_block() no longer frees any unused blocks.
* The low memory handler will now only lock the cache if there is something
to do. Also, it did not take address space warnings into account.
* Even when memory is critical, we don't free all unused blocks anymore - if
the number of blocks we free now (10000) is not sufficient to get out of the
critical condition, chances are good that we will be called again :-)
* block_notifier_and_writer() now tries to make sure that the total block cache
memory consumption grows not much larger than half of the available RAM.
* This should all help to limit the block cache usage a bit better. Hopefully,
a checkfs run will no longer run out of memory here (couldn't test yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35733 a95241bf-73f2-0310-859d-f6bbb57e9c96

Re-lock the cache directly after the read. Otherwise the error case would
remove the block from an unlocked cache and the cache->lock would be unlocked
twice when the calling function unlocked. It panics in that case anyway, but
this should make it continuable and is more correct.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35648 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The low resource handler now empties the cache depot's magazines; before,
they were never freed unless the cache was destroyed (I just wondered why
my system would bury >1G in the magazines).
* Made the magazine capacity variable per cache, ie. for larger objects, it's
not a good idea to have 64*CPU buffers lying around in the worst case.
* Furthermore, the create_object_cache_etc()/object_depot_init() now have
arguments for the magazine capacity as well as the maximum number of full
unused magazines.
* By default, you might want to initialize both to zero, as then some hopefully
usable defaults are computed. Otherwise (the only current example is the
vm_page_mapping cache) you can just put in the values you'd want there.
The page mapping cache uses larger values, as its objects are usually
allocated and deleted in larger chunks.
* Beware, though, I couldn't test these changes yet as Qemu didn't like to run
today. I'll test these changes on another machine now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35601 a95241bf-73f2-0310-859d-f6bbb57e9c96

This should finally nail #5415:
* Since the same block can be in up to two transactions, it's very well possible
that one shouldn't write all transactions in a single run.
* Forgot to pass on the iterator from BlockWriter::Add(transaction) to
Add(block).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35565 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_sync_transaction() now puts all blocks into the BlockWriter, and
therefore does not unlock anymore while iterating over the transactions.
This gave other threads the opportunity to finish a transaction, causing
bug #5412. Also, the BlockWriter will now always close transactions on its
own, and you need to pass the transaction hash iterator to Add().
* Also, transactions that contain blocks that are currently written back will
be ignored by the block writer, as well as cache_sync_transaction(). This
fixes bug #5415.
* Improved error handling if BlockWriter fails to write back blocks. Most
notably, they are no longer left busy_writing, and the functions calling
it do proper error reporting (besides block_cache_discard() that does not
return any erro code; I've added a TODO note there for now).
* The BlockWriter now starts with a larger array once it has to allocate one.
* One can now limit the number of blocks that go into a BlockWriter. This is
used by the block writer thread, that shouldn't always write back everything
every two seconds.
* Also, the fixed array is larger now (leaving enough space such that the
block writer/notifier does not need to allocate anything).
* And finally, if allocating the array fails, the BlockWriter falls back to the
synchronous write back used previously. IOW it will never write back less
blocks than you ask for.
* Added static BlockWriter::WriteBlock() method replacing write_cached_block().
* Forgot to rename block_cache::busy_count to busy_reading_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35510 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Made cache_end_transaction(), cache_detach_sub_transaction(), and
block_cache_discard() use the BlockWriter directly as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35496 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved KDL command output (ie. take busy_writing into account, too).
* Added new KDL command "cached_block" that dumps a cached block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35493 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_sync_transaction() used an earlier signature of the BlockWriter
constructor, causing bug #5412.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35489 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a BlockWriter class that now performs writing back blocks.
* Renamed the "busy" stuff to "busy_reading", and added a "busy_writing"
concept.
* This now allows reading a block (and other blocks), while blocks are written
back. This should speed all operations needing to write back blocks, like
unzipping or compiling.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35464 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Use the is_writing flag to determine whether or not we may write back a block
that does not have a transaction.
* This should fix #5340.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35390 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved handling of busy blocks into separate functions.
* Added flags to avoid notifying the busy condition variable unnecessarily.
* get_writable_cached_block(): Unlock the cache while memcpy()ing/memset()ing
the block's data. The idea is to reduce lock contention. Less effective
than I hoped, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35328 a95241bf-73f2-0310-859d-f6bbb57e9c96

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If a block allocation fails, remove an unused one and try again - this might
improve the reliability as long as our slab implementation is a PITA.
* Removed an assertion that will no longer work (due to the DoublyLinkedList
changes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34877 a95241bf-73f2-0310-859d-f6bbb57e9c96

* While reading in a block, the block is now marked busy, and the cache
unlocked, allowing for more parallel access.
* Writing is still done synchronously, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34611 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The slab and the block cache now both also react on address space shortages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34310 a95241bf-73f2-0310-859d-f6bbb57e9c96

* NewBlock(), and Allocate() will now try to reuse existing memory in low
resource situations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33892 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed wrong check


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32326 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added stack traced to the Abort tracing output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31870 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31869 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Enhanced cache_blocks_in_main_transaction() to also return the correct number
of blocks in case there is no sub-transaction at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* has_sub_transaction was never initialized, no clue why this didn't come up
earlier.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31845 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decoupled block_cache_used_memory() from the sCachesLock - this should fix the
UI freezes (ActivityMonitor and ProcessController both use get_system_info() a
lot), although this is only the symptom of another problem.
* The downside is that the block cache usage information isn't as up to date as
it was previously - it's updated by the block write/notifier thread now (worst
case every 2 seconds).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31812 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a number of problems of the "block_cache_data" command.
* Also added stack traces to each BlockData entry.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31763 a95241bf-73f2-0310-859d-f6bbb57e9c96

When building the block cache for UserlandFS's haiku emulation lib, don't include tracing since that's unavailable outside of the kernel. Fixes ticket #4092. Axel or Ingo, please review.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31525 a95241bf-73f2-0310-859d-f6bbb57e9c96

* various fixes identified by gcc4, provided by Joe Prostko


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30902 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an insane block cache tracing mode that will also log the whole block
data. This is available when BLOCK_CACHE_BLOCK_TRACING is 2 or greater.
* Completely untested as of now, though. Will do so soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30104 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added RemoveTail() method.
* Renamed DoublyLinkedList::Size() to Count(), since it actually counts the
items (ie. O(n)).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29979 a95241bf-73f2-0310-859d-f6bbb57e9c96

gcc 4 fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29602 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the superfluous 'extern "C"'s.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29334 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Brought the fs_shell block_cache implementation up to date, and incorporate
the fixes applied to the kernel version of it (most notably a correctly
working block_cache_discard(), and cache_detach_sub_transaction()).
Also switched to the new notification functions, even though it still works
synchronously in the fs_shell.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28617 a95241bf-73f2-0310-859d-f6bbb57e9c96

* wait_for_notifications() must make sure that the order of the notifications
in the cache is the same as in the condition variable. It now uses the
low-contention sCachesLock to accomplish this.
* Also added an ASSERT to make sure the notification is no longer part of the
list at this point.
* Improved KDL command output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28516 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction(), and cache_abort_sub_transaction() now support
discarded blocks correctly as well.
* cache_detach_sub_transaction() left cached_block::original_data unchanged even
if the parent data was to become current (in case the sub transaction didn't
change the block yet). This could cause outdated blocks to be written back.
* cache_detach_sub_transaction() also set cached_block::previous_transaction
for all blocks, not just the ones with a previous transaction. This could
cause blocks to be written twice for no reason.
* cache_start_sub_transaction() did not change the num_blocks count for
discarded blocks.
* block_cache_discard() now panics if the block was already changed in the
current transaction.
* Improved test application, added more tests, revealing the above bugs in
cache_detach_sub_transaction().
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28514 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a potential race condition: when wait_for_notifications() is called
from the block notifier, the cache could be deleted before we have the chance
to lock it. We now lock the sCachesLock, and see if this cache is still valid.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28512 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The first functions now handle the "discard" flag correctly (namely
cache_end_transaction(), and cache_start_sub_transaction()).
* Further work on the test application, it's now actually usable, first test
passes.
* dump_block() did erroneously print 'B' for the dirty flag; now both dirty and
discard have the 'D' (3rd and 5th column).
* block_cache::LowMemoryHandler() is now private (and got an underscore prefix).
* Minor cleanup, shuffled some methods around.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28508 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleanup, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28497 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96

* I accidently broke get_next_locked_block_cache() in r27074; it would always
ignore every other cache (starting from the first).
* The consequence of this was that no blocks were written back automatically
for those caches, and their transactions were never idle, causing bug #2781.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27831 a95241bf-73f2-0310-859d-f6bbb57e9c96

Created a central place for putting kernel debug enabling macros.
Currently it only contains KDEBUG and the block cache debugging macros.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27816 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the way how a cache is destructed: now, it is removed from the
global cache list before locking it. This allows to get rid of the
block_cache::deleting field, as well as simplifies some code.
* This also fixes a possible deadlock I recently introduced (on destruction,
the locking order was wrong).
* Now uses an anonymous condition variable instead.
* Moved the block_cache initialization code into a dedicated method that will
now also fail in case the low resource handler couldn't be registered (as
pointed out by Salvatore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27074 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Only remove the block cache from the list, and its low resource handler when
this is actually needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27060 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed bug #2631 based on a patch by Salvatore: the block cache was put into
a global list before it was fully initialized.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27058 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_used_memory() that returns the memory allocated by all
block caches for the block cache buffers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26836 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96

axeld + bonefish + mmlr:
* Added trace entry for block reads from disk.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26534 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added block level tracing to the block cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26521 a95241bf-73f2-0310-859d-f6bbb57e9c96

Missed that file in the commit for r25812.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25816 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed tracing output prefix from "cache" to "block cache".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25714 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache_delete(): only lock the cache mutex and avoid MutexLocker. the mutex is destroyed in the destructor.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25467 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache_delete(): unlock the cache mutex before deleting the cache


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25463 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced benaphores with mutexes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25378 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Only enable block/transaction tracing when the macro is 1.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25247 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Factored a delete_notification() that is now used in three places.
* remove_transaction_listeners() was checking the events_pending field without
holding the sNotificatonLock - that should have been harmless (as we're in the
middle of deleting the transaction), but it now looks better.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25130 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_sync[_etc]() now also waits for all notifications to be handled.
This fixes a potential problem where a volume would be left marked dirty on
a clean reboot as reported by Vasilis; no harm would have been done to the
data on the volume, though (it would just replay a log entry that it didn't
need to).
* Unified the naming of the notifier/writer thread: renamed the thread to "block
notifier/writer", and the variable holding the thread ID to
sNotifierWriterThread.
* Added a few more doxygen descriptions to the private functions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24870 a95241bf-73f2-0310-859d-f6bbb57e9c96

* notify_transaction_listeners() would always remove all listeners when it
was asked to. Unfortunately, it also removed the TRANSACTION_WRITTEN
listeners (that were added before notifying other listeners), so that BFS
could never free up space in the log. Now, it will only remove the listeners
it may remove. This fixes bug #2024.
* Rearranged functions a bit, moved notification/listener related private
functions together.
* Added tracing entry for written transactions as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24853 a95241bf-73f2-0310-859d-f6bbb57e9c96

The wait_for_notifications() function now detects if it has been run from
within the notifier/writer thread, and will then flush the notifications
directly. This should fix #2008 again.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24806 a95241bf-73f2-0310-859d-f6bbb57e9c96

* This should fix a deadlock as reported by bga.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24793 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If BFS's Journal::_WriteTransactionToLog() noticed there wasn't enough free
space left for the new log entry, it did call cache_sync_transaction(), and
then just assumed the space would be ready. But since the transaction could
have been written before that call by the block writer, and since the
_TransactionWritten() hook is now called asynchronously,
cache_sync_transaction() actually has to flush all pending TRANSACTION_WRITTEN
notifications before returning to the caller.
* To implement this, block_cache now publishs a condition variable, and
wait_for_notifications() adds a fake notification that signals that one.
Since the notifications are handled in FIFO order, this guarantees that
the previous TRANSACTION_WRITTEN hook is done.
* notify_transaction_listeners() could accidently delete notifications that
still had pending signals. Now, it will defer the deletion to the notification
thread instead in that case. This should fix bug #2008.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24792 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed the "transaction_changed" leftover member.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24769 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decoupled notifications from the block cache functions; they are now
called without having the cache's lock held.
* The only downside to this approach is that ending a transaction now needs
additional memory, and might therefore fail - that could be fixed by
allocating that memory upfront with the start of the new transaction,
though.
* Therefore, I reverted the cache lock to a benaphore, as the recursive
lock is no longer needed.
* The block writer thread is now a block writer and notifier thread: it will
wait for up to 2 seconds on the sEventSemaphore, and if nothing comes in,
it will continue with its previously sole job.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24768 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_end_transaction() doesn't remove the transaction, so that the
check in notify_transaction_listeners() didn't hold, although the
cache_hooks were deleted anyway. I've introduced a listener_change
counter to detect changes to the listener list while iterating, since
any change makes the iteration unsafe. Should fix bug #2008 for real.
Nevertheless I thing the transaction listener concept needs revision.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24761 a95241bf-73f2-0310-859d-f6bbb57e9c96

* A notification can cause a transaction to be deleted - in this case, the
block_writer() as well as notify_transaction_listeners() must update their
data or quit. This fixes newly introduced bug #2008.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24749 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_private.h is no longer needed, moved its contents into
block_cache.cpp.
* Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24739 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* write_cached_block() now moves a block to the unused list when it's
not referenced and clean (is no longer part of a transaction).
This finally gets memory consumption to a moderate level; even
unpacking a large archive will no longer eat all memory.
* The "block_cache" KDL command will now print the number of unused
blocks as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24606 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The block_cache KDL command now sets some useful variables.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24589 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The "block_cache" command checked the wrong variable to see if a block
could be found or not, eventually resulting in a read fault.
* It now also uses parse_expression() for its second argument, since it
now returns an uint64 in Haiku.
* Tracing output cut off the label too early.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24579 a95241bf-73f2-0310-859d-f6bbb57e9c96

cached_block::Compare() was ignoring the upper 32 bit of the block
number. Was a problem only for partitions > 2^32 * block size (4TB
for 1KB blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24393 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed compare_blocks() function; it actually gets cached_block**.
* Added block_writer thread that continuously writes back blocks for all current
block caches.
* A block cache now maintains the number of dirty blocks if it doesn't use
transactions. That knowledge could also be used in block_cache_sync()...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23698 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The block cache now sorts the blocks before synchronizing a transaction.
* This should speed up writing a transaction considerably.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23694 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a new function hash_insert_grow() that grows the hash table when needed.
* Removed the public hash_grow() function again (at least for now, it's only
private).
* Removed the newSize argument from hash_grow(); it will compute the new size
automatically.
* The block cache is now using hash_insert_grow() instead of hash_insert()
which should make hash lookups much faster with some 10 thousand blocks,
also increased the initial table size from 32 to 1024...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23692 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed TraceEntry::Dump() to take a TraceOutput& and removed
AbstractTraceEntry::AddDump(char*, size_t).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23655 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved tracing macro definitions into separate tracing_config.h header.
So now there's a central place to enable tracing in general and for
individual components.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23645 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added debugger command "transaction" and added a new "-t" option to
"block_cache" to dump info about transactions.
* Improved tracing.
* Now uses the new add_debugger_command_etc().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23644 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

Gracefully ignore NULL pointers passed to block_cache::Free(). Fixes
panic when sync'ing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23575 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced my block allocator code in favour of Hugo's slab allocator
(so we can't allocate more physical pages than virtual address space
anymore, but so what?).
* Used the new CACHE_LARGE_SLAB flag as a temporary work-around; else
the slab would easily create several thousands of areas, which our
area code (and kernel heap) can't really handle that well (gets
awfully slow).
* Block caches with the same size could share the same slab, but we
don't do that yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23567 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved block allocation.
* Made the block_cache KDL command dump a bit more useful info (number of
referenced and dirty blocks).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23540 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added tracing support for transactions.
* improved the "block_cache" KDL command: it can now also dump blocks,
added support for the "--help" argument.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23532 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* bfs_free_cookie() accidently reverted a transaction after having updated
the inode's internal last modified and size copies, causing in inconsistent
data. This fixes #1643.
* The block being replayed are now dumped to the debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23050 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a part of block_cache_set_dirty() so that it can gracefully exit
in case there is nothing to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22318 a95241bf-73f2-0310-859d-f6bbb57e9c96

More informative output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21669 a95241bf-73f2-0310-859d-f6bbb57e9c96

Putting an unused block will now trigger a panic.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21481 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added some temporary means to ease debugging block caches - at least the list
of all caches should be removed at a later point.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21461 a95241bf-73f2-0310-859d-f6bbb57e9c96

Even more correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20856 a95241bf-73f2-0310-859d-f6bbb57e9c96

More correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20855 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed broken iteration in block_cache::RemoveUnusedBlocks(). The loop
would start with the first block in the unused blocks list, but then
continue with the blocks that share the same hash table slot, thus
freeing potentially used blocks. Could theoretically have caused
BFS to see and write incorrect meta/administrative data on certain
occasions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20459 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleanup:
* NewBlock()/FreeBlock() are now symmetrical in that the former no longer inserts
the block into the hash table.
* delete_transaction() also no longer removes the transaction from the hash table.
* cache_transaction_sync() now uses the new hash_remove_current() function.
* minor other cleanup (like line breaks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20374 a95241bf-73f2-0310-859d-f6bbb57e9c96

get_cached_block(): Remove the newly allocated block from the
hashtable in case of a read error (NewBlock() also adds the block,
but FreeBlock() only frees it).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20365 a95241bf-73f2-0310-859d-f6bbb57e9c96

added missing nothrow parameter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96

- moved dumpBlock() into debug.c, renamed it to dump_block() and added it
to kernel-exports.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18268 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If there are no free ranges left, the block cache will now reuse older
blocks - this is not enough, though as it would also need to ask other
volumes to free ranges.
* Increased the number of blocks to free in case of low memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16663 a95241bf-73f2-0310-859d-f6bbb57e9c96

changed get_cached_block() to no longer use a reference argument
modified to panic when an invalid block is requested (to find fs errors)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16537 a95241bf-73f2-0310-859d-f6bbb57e9c96

This fixes the work-around made in r16105, and some more:
* cached_block::parent_data can be NULL in a sub transaction in case the
block wasn't part of the parent transaction (but not in low memory
situations). cache_abort_sub_transaction() and cache_detach_sub_transaction()
didn't account for this, though, ie. the block data could end up
corrupted.
* Renamed cached_block::original in original_data.
* Renamed cached_block::data in current_data.
* Added some comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16108 a95241bf-73f2-0310-859d-f6bbb57e9c96

fixed some traces
cache_abort_sub_transaction now doesn't crash if parent_cache is NULL (happens in low memory situations)
Axel, please review and fix if needed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16105 a95241bf-73f2-0310-859d-f6bbb57e9c96

Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16042 a95241bf-73f2-0310-859d-f6bbb57e9c96

Deleted ranges were never removed from the ranges hash... this could have finally
nailed down bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15791 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added some debug helper. Hopefully they give more insight to Stephan's problem (bug #77/#78).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15767 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_abort_transaction() could let cached_block::parent_data have the same value
as cached_block::data - which led to a crash as block_cache::FreeBlock() tried to
free both later.
Since neither cached_block::parent_data nor cached_block::original are supposed
to be != NULL in block_cache::FreeBlock(), they are no longer freed, but the system
panics if one of them is not NULL.
This should fix bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15749 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_range::Delete() did not remove the block_range from the free list,
and thus, it could be used again which led to a crash.
Changed the free ranges list from a singly linked list to a doubly linked
list so that not all free ranges have to be searched for the one to be
freed anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15696 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleaned up locking and fixed a related bug: put_cached_block() could have been called without holding the cache lock.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15695 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added debug output to vm_low_memory.cpp.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15568 a95241bf-73f2-0310-859d-f6bbb57e9c96

The block_cache::LowMemoryHandler() must check if it could acquire the cache's lock;
it might have been deleted in the mean time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15565 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_delete() now deletes the cache with with its lock held.
* the link were not initialized in cached_block, as its constructor were never called
(was using malloc/free instead of new/delete).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15564 a95241bf-73f2-0310-859d-f6bbb57e9c96

Forget to remove debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15563 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now also flushes dirty blocks correctly (cannot happen if transactions are used,
ie. never happens with BFS).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15562 a95241bf-73f2-0310-859d-f6bbb57e9c96

The block cache now registers and uses a low memory handler that will flush
unused blocks when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15561 a95241bf-73f2-0310-859d-f6bbb57e9c96

Blocks are now put into an unused list when they aren't used.
When memory gets low, the blocks on that list are also freed - but right now,
that only happens when a block is put back (not directly on demand, ie. via
a low memory handler).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15552 a95241bf-73f2-0310-859d-f6bbb57e9c96

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96

First go at an implementation of sub transactions - code is not yet tested, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14429 a95241bf-73f2-0310-859d-f6bbb57e9c96

Updated a few more hash functions to be able to deal with negative numbers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14067 a95241bf-73f2-0310-859d-f6bbb57e9c96

Turned off debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14044 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache::Free() now accepts NULL pointers (it was already used this way before...).
Unmounting BFS volumes should now work as expected.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14034 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the definition of the BlockAddressPool class into block_allocator.cpp, as it's
not used outside of it.
Added tracking for the last transaction in the block cache; that way it can test
if the last transaction has been closed before opening a new one.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13022 a95241bf-73f2-0310-859d-f6bbb57e9c96

Some minor cleanup: removed unneeded includes, updated introducing comment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12932 a95241bf-73f2-0310-859d-f6bbb57e9c96

Work in progress of the new block allocator for the block cache. It works, but
currently needs even more memory than the old one 8-)
Not cleaned up at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12921 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_abort_transaction() freed the wrong block, and thus causing a file system
corruption even better than during its absence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12903 a95241bf-73f2-0310-859d-f6bbb57e9c96

Oops, forgot to implement cache_abort_transaction(); every failing transaction
could have destroyed the BFS integrity... (just happened to me, that's how I
noticed it)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12867 a95241bf-73f2-0310-859d-f6bbb57e9c96

First step into a new block allocation strategy for the block_cache
(right now, it's still malloc/free, just encapsulated in an allocator class).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12662 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96

BlockWriter: Do not panic on failing read/writes

These were here for debugging purposes, as often it is a sign of
inconsistencies. However, for USB disks this is a normal occurence
when someone janks out of the device without unmounting first.

Make sure we log these cases though, as it still helps debugging.

Fix sponsered by http://www.izcorp.com

block cache: convert to BOpenHashTable.

Fix typo.

cache_abort_sub_transaction() did not maintain num_blocks.

* When a block was only used in a sub-transaction, it was thrown away,
but the transaction::num_blocks field was not decremented.
* This caused transactions never considered finished which eventually
led to bug #8942. This does not explain the disk corruption occurring
in #8969, though.

Fixed the new issue in #8910 from r44585.

* The ASSERT() I introduced in r44585 was incorrect: when the sub transaction
used block_cache_get_empty() to get the block, there is no original_data for
a reason.
* Added a test case that reproduces this situation.
* The block must be moved to the unused list in this situation, though, or else
it might contain invalid data. Since the block can only be allocated in the
current transaction, this should not be a problem, though, AFAICT.

The original_data could be freed late.

* In cache_abort_sub_transaction(), the original_data can already be freed
when the block is being removed from the transaction.
* block_cache::_GetUnusedBlock() no longer frees original/parent data - it
now requires them to be freed already (it makes no sense to have them still
around at this point).
* AFAICT the previous version did not have any negative consequences besides
freeing the original data late.

Minor block cache refactoring.

* Extracted a write_blocks_in_previous_transaction() function out of
cache_end_transaction(), and cache_detach_sub_transaction().

Fixed broken block list introduced in hrev44357.

* cache_abort_sub_transaction() was setting the transaction_next pointer to
NULL in order to remove a block from a transaction -- however, it forgot to
actually remove it from the transaction's block list.
* Minor restructuring.

Fixed broken discard handling in cache_start_sub_transaction().

* This actually resolves a TODO.

Cleanup, no functional change intended.

cache_abort_[sub_]transaction() did not work correctly.

* cache_abort_transaction() left the block dirty which was causing bug
#8123 as well.
* cache_abort_sub_transaction() did, in addition to not clearing the dirty
flag, not reset the block's transaction member either if the block was
not part of the parent transaction.

Added a bit of documentation, minor cleanup.

* Documented the cached_block::transaction, and previous_transaction
members.

An unused dirty block would put again into the unused list.

* This could cause bug #8123. While this fixes the crash, one underlying
problem is still there, as dirty blocks should never leave a transaction.

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.

Change instances of wether in comments to whether. No functional change intended.

Added intermediate (addr_t) casts before the cast of the parse_expression()
result (uint64) to pointer (CID 4723-4726).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40121 a95241bf-73f2-0310-859d-f6bbb57e9c96

* We need to reset the blockNumber, or else we'll discard the wrong blocks
which has rather desastrous consequences - thanks to Stippi for the heads
up :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38242 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed CID 1447 which was an actual bug which completely voided the whole
discard mechanism eventually causing the conflicts between the file cache
and the block cache it intended to fix.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38232 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Thanks to Janito I had a closer look to cache_detach_sub_transaction(), and it
turned out it's pretty much broken. Not only did it potentially leak memory,
it would also potentially replace the original data with a NULL pointer,
making the current transaction non revertable. The code should now be much
clearer. This fixes bug #6378.
* Changed a few "if (... != NULL)" into ASSERTs, since the code should bail out
earlier in these cases already.
* Added a TODO comment to cache_start_sub_transaction() about its broken discard
handling. This can cause FS corruptions in case the parent transaction is ever
going to be aborted (which shouldn't happen in real life, though).
* Added a bit more and better comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37899 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_end_transaction(): Allow the hook parameter to be NULL.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37461 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced block_cache::unused_block_count, which counts the elements of
block_cache::unused_blocks.
* block_cache::Allocate(): No longer removes unused blocks when in a low
resource state. That just removed too many blocks too quickly, when the
cache was actively used for writing, seriously affecting performance.
* block_cache::_LowMemoryHandler(): Compute the number of unused blocks to
remove depending on the total unused block number. This way we cull huge
block caches with lots of old blocks much quicker.

Treats part of #5816.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37170 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache::NewBlock(): Allocate cached_block::current_data only when the
block was freshly allocated. A block returned by _GetUnusedBlock() already
has current_data and we would leak it before.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35918 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If get_cached_block() failed reading a block, it would leave a deleted object
in the unused list if it got the block via block_cache::_GetUnusedBlock().
* block_cache::_GetUnusedBlock() leaked the compare data block if the
BLOCK_CACHE_DEBUG_CHANGED feature had been enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35799 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_notifier_and_writer() will now flush 1000 blocks when reaching the soft
memory limit.
* Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35735 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced cached_block::accessed (the number of accesses) with last_accessed
(the time of the last access), as what we really want is a frequency/last
access time scoring, and accessed alone is useless for that.
* put_cached_block() no longer frees any unused blocks.
* The low memory handler will now only lock the cache if there is something
to do. Also, it did not take address space warnings into account.
* Even when memory is critical, we don't free all unused blocks anymore - if
the number of blocks we free now (10000) is not sufficient to get out of the
critical condition, chances are good that we will be called again :-)
* block_notifier_and_writer() now tries to make sure that the total block cache
memory consumption grows not much larger than half of the available RAM.
* This should all help to limit the block cache usage a bit better. Hopefully,
a checkfs run will no longer run out of memory here (couldn't test yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35733 a95241bf-73f2-0310-859d-f6bbb57e9c96

Re-lock the cache directly after the read. Otherwise the error case would
remove the block from an unlocked cache and the cache->lock would be unlocked
twice when the calling function unlocked. It panics in that case anyway, but
this should make it continuable and is more correct.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35648 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The low resource handler now empties the cache depot's magazines; before,
they were never freed unless the cache was destroyed (I just wondered why
my system would bury >1G in the magazines).
* Made the magazine capacity variable per cache, ie. for larger objects, it's
not a good idea to have 64*CPU buffers lying around in the worst case.
* Furthermore, the create_object_cache_etc()/object_depot_init() now have
arguments for the magazine capacity as well as the maximum number of full
unused magazines.
* By default, you might want to initialize both to zero, as then some hopefully
usable defaults are computed. Otherwise (the only current example is the
vm_page_mapping cache) you can just put in the values you'd want there.
The page mapping cache uses larger values, as its objects are usually
allocated and deleted in larger chunks.
* Beware, though, I couldn't test these changes yet as Qemu didn't like to run
today. I'll test these changes on another machine now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35601 a95241bf-73f2-0310-859d-f6bbb57e9c96

This should finally nail #5415:
* Since the same block can be in up to two transactions, it's very well possible
that one shouldn't write all transactions in a single run.
* Forgot to pass on the iterator from BlockWriter::Add(transaction) to
Add(block).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35565 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_sync_transaction() now puts all blocks into the BlockWriter, and
therefore does not unlock anymore while iterating over the transactions.
This gave other threads the opportunity to finish a transaction, causing
bug #5412. Also, the BlockWriter will now always close transactions on its
own, and you need to pass the transaction hash iterator to Add().
* Also, transactions that contain blocks that are currently written back will
be ignored by the block writer, as well as cache_sync_transaction(). This
fixes bug #5415.
* Improved error handling if BlockWriter fails to write back blocks. Most
notably, they are no longer left busy_writing, and the functions calling
it do proper error reporting (besides block_cache_discard() that does not
return any erro code; I've added a TODO note there for now).
* The BlockWriter now starts with a larger array once it has to allocate one.
* One can now limit the number of blocks that go into a BlockWriter. This is
used by the block writer thread, that shouldn't always write back everything
every two seconds.
* Also, the fixed array is larger now (leaving enough space such that the
block writer/notifier does not need to allocate anything).
* And finally, if allocating the array fails, the BlockWriter falls back to the
synchronous write back used previously. IOW it will never write back less
blocks than you ask for.
* Added static BlockWriter::WriteBlock() method replacing write_cached_block().
* Forgot to rename block_cache::busy_count to busy_reading_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35510 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Made cache_end_transaction(), cache_detach_sub_transaction(), and
block_cache_discard() use the BlockWriter directly as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35496 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved KDL command output (ie. take busy_writing into account, too).
* Added new KDL command "cached_block" that dumps a cached block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35493 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_sync_transaction() used an earlier signature of the BlockWriter
constructor, causing bug #5412.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35489 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a BlockWriter class that now performs writing back blocks.
* Renamed the "busy" stuff to "busy_reading", and added a "busy_writing"
concept.
* This now allows reading a block (and other blocks), while blocks are written
back. This should speed all operations needing to write back blocks, like
unzipping or compiling.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35464 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Use the is_writing flag to determine whether or not we may write back a block
that does not have a transaction.
* This should fix #5340.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35390 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved handling of busy blocks into separate functions.
* Added flags to avoid notifying the busy condition variable unnecessarily.
* get_writable_cached_block(): Unlock the cache while memcpy()ing/memset()ing
the block's data. The idea is to reduce lock contention. Less effective
than I hoped, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35328 a95241bf-73f2-0310-859d-f6bbb57e9c96

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If a block allocation fails, remove an unused one and try again - this might
improve the reliability as long as our slab implementation is a PITA.
* Removed an assertion that will no longer work (due to the DoublyLinkedList
changes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34877 a95241bf-73f2-0310-859d-f6bbb57e9c96

* While reading in a block, the block is now marked busy, and the cache
unlocked, allowing for more parallel access.
* Writing is still done synchronously, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34611 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The slab and the block cache now both also react on address space shortages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34310 a95241bf-73f2-0310-859d-f6bbb57e9c96

* NewBlock(), and Allocate() will now try to reuse existing memory in low
resource situations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33892 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed wrong check


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32326 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added stack traced to the Abort tracing output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31870 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31869 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Enhanced cache_blocks_in_main_transaction() to also return the correct number
of blocks in case there is no sub-transaction at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* has_sub_transaction was never initialized, no clue why this didn't come up
earlier.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31845 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decoupled block_cache_used_memory() from the sCachesLock - this should fix the
UI freezes (ActivityMonitor and ProcessController both use get_system_info() a
lot), although this is only the symptom of another problem.
* The downside is that the block cache usage information isn't as up to date as
it was previously - it's updated by the block write/notifier thread now (worst
case every 2 seconds).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31812 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a number of problems of the "block_cache_data" command.
* Also added stack traces to each BlockData entry.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31763 a95241bf-73f2-0310-859d-f6bbb57e9c96

When building the block cache for UserlandFS's haiku emulation lib, don't include tracing since that's unavailable outside of the kernel. Fixes ticket #4092. Axel or Ingo, please review.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31525 a95241bf-73f2-0310-859d-f6bbb57e9c96

* various fixes identified by gcc4, provided by Joe Prostko


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30902 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an insane block cache tracing mode that will also log the whole block
data. This is available when BLOCK_CACHE_BLOCK_TRACING is 2 or greater.
* Completely untested as of now, though. Will do so soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30104 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added RemoveTail() method.
* Renamed DoublyLinkedList::Size() to Count(), since it actually counts the
items (ie. O(n)).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29979 a95241bf-73f2-0310-859d-f6bbb57e9c96

gcc 4 fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29602 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the superfluous 'extern "C"'s.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29334 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Brought the fs_shell block_cache implementation up to date, and incorporate
the fixes applied to the kernel version of it (most notably a correctly
working block_cache_discard(), and cache_detach_sub_transaction()).
Also switched to the new notification functions, even though it still works
synchronously in the fs_shell.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28617 a95241bf-73f2-0310-859d-f6bbb57e9c96

* wait_for_notifications() must make sure that the order of the notifications
in the cache is the same as in the condition variable. It now uses the
low-contention sCachesLock to accomplish this.
* Also added an ASSERT to make sure the notification is no longer part of the
list at this point.
* Improved KDL command output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28516 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction(), and cache_abort_sub_transaction() now support
discarded blocks correctly as well.
* cache_detach_sub_transaction() left cached_block::original_data unchanged even
if the parent data was to become current (in case the sub transaction didn't
change the block yet). This could cause outdated blocks to be written back.
* cache_detach_sub_transaction() also set cached_block::previous_transaction
for all blocks, not just the ones with a previous transaction. This could
cause blocks to be written twice for no reason.
* cache_start_sub_transaction() did not change the num_blocks count for
discarded blocks.
* block_cache_discard() now panics if the block was already changed in the
current transaction.
* Improved test application, added more tests, revealing the above bugs in
cache_detach_sub_transaction().
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28514 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a potential race condition: when wait_for_notifications() is called
from the block notifier, the cache could be deleted before we have the chance
to lock it. We now lock the sCachesLock, and see if this cache is still valid.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28512 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The first functions now handle the "discard" flag correctly (namely
cache_end_transaction(), and cache_start_sub_transaction()).
* Further work on the test application, it's now actually usable, first test
passes.
* dump_block() did erroneously print 'B' for the dirty flag; now both dirty and
discard have the 'D' (3rd and 5th column).
* block_cache::LowMemoryHandler() is now private (and got an underscore prefix).
* Minor cleanup, shuffled some methods around.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28508 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleanup, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28497 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96

* I accidently broke get_next_locked_block_cache() in r27074; it would always
ignore every other cache (starting from the first).
* The consequence of this was that no blocks were written back automatically
for those caches, and their transactions were never idle, causing bug #2781.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27831 a95241bf-73f2-0310-859d-f6bbb57e9c96

Created a central place for putting kernel debug enabling macros.
Currently it only contains KDEBUG and the block cache debugging macros.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27816 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the way how a cache is destructed: now, it is removed from the
global cache list before locking it. This allows to get rid of the
block_cache::deleting field, as well as simplifies some code.
* This also fixes a possible deadlock I recently introduced (on destruction,
the locking order was wrong).
* Now uses an anonymous condition variable instead.
* Moved the block_cache initialization code into a dedicated method that will
now also fail in case the low resource handler couldn't be registered (as
pointed out by Salvatore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27074 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Only remove the block cache from the list, and its low resource handler when
this is actually needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27060 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed bug #2631 based on a patch by Salvatore: the block cache was put into
a global list before it was fully initialized.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27058 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_used_memory() that returns the memory allocated by all
block caches for the block cache buffers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26836 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96

axeld + bonefish + mmlr:
* Added trace entry for block reads from disk.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26534 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added block level tracing to the block cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26521 a95241bf-73f2-0310-859d-f6bbb57e9c96

Missed that file in the commit for r25812.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25816 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed tracing output prefix from "cache" to "block cache".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25714 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache_delete(): only lock the cache mutex and avoid MutexLocker. the mutex is destroyed in the destructor.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25467 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache_delete(): unlock the cache mutex before deleting the cache


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25463 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced benaphores with mutexes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25378 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Only enable block/transaction tracing when the macro is 1.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25247 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Factored a delete_notification() that is now used in three places.
* remove_transaction_listeners() was checking the events_pending field without
holding the sNotificatonLock - that should have been harmless (as we're in the
middle of deleting the transaction), but it now looks better.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25130 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_sync[_etc]() now also waits for all notifications to be handled.
This fixes a potential problem where a volume would be left marked dirty on
a clean reboot as reported by Vasilis; no harm would have been done to the
data on the volume, though (it would just replay a log entry that it didn't
need to).
* Unified the naming of the notifier/writer thread: renamed the thread to "block
notifier/writer", and the variable holding the thread ID to
sNotifierWriterThread.
* Added a few more doxygen descriptions to the private functions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24870 a95241bf-73f2-0310-859d-f6bbb57e9c96

* notify_transaction_listeners() would always remove all listeners when it
was asked to. Unfortunately, it also removed the TRANSACTION_WRITTEN
listeners (that were added before notifying other listeners), so that BFS
could never free up space in the log. Now, it will only remove the listeners
it may remove. This fixes bug #2024.
* Rearranged functions a bit, moved notification/listener related private
functions together.
* Added tracing entry for written transactions as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24853 a95241bf-73f2-0310-859d-f6bbb57e9c96

The wait_for_notifications() function now detects if it has been run from
within the notifier/writer thread, and will then flush the notifications
directly. This should fix #2008 again.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24806 a95241bf-73f2-0310-859d-f6bbb57e9c96

* This should fix a deadlock as reported by bga.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24793 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If BFS's Journal::_WriteTransactionToLog() noticed there wasn't enough free
space left for the new log entry, it did call cache_sync_transaction(), and
then just assumed the space would be ready. But since the transaction could
have been written before that call by the block writer, and since the
_TransactionWritten() hook is now called asynchronously,
cache_sync_transaction() actually has to flush all pending TRANSACTION_WRITTEN
notifications before returning to the caller.
* To implement this, block_cache now publishs a condition variable, and
wait_for_notifications() adds a fake notification that signals that one.
Since the notifications are handled in FIFO order, this guarantees that
the previous TRANSACTION_WRITTEN hook is done.
* notify_transaction_listeners() could accidently delete notifications that
still had pending signals. Now, it will defer the deletion to the notification
thread instead in that case. This should fix bug #2008.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24792 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Removed the "transaction_changed" leftover member.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24769 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decoupled notifications from the block cache functions; they are now
called without having the cache's lock held.
* The only downside to this approach is that ending a transaction now needs
additional memory, and might therefore fail - that could be fixed by
allocating that memory upfront with the start of the new transaction,
though.
* Therefore, I reverted the cache lock to a benaphore, as the recursive
lock is no longer needed.
* The block writer thread is now a block writer and notifier thread: it will
wait for up to 2 seconds on the sEventSemaphore, and if nothing comes in,
it will continue with its previously sole job.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24768 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_end_transaction() doesn't remove the transaction, so that the
check in notify_transaction_listeners() didn't hold, although the
cache_hooks were deleted anyway. I've introduced a listener_change
counter to detect changes to the listener list while iterating, since
any change makes the iteration unsafe. Should fix bug #2008 for real.
Nevertheless I thing the transaction listener concept needs revision.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24761 a95241bf-73f2-0310-859d-f6bbb57e9c96

* A notification can cause a transaction to be deleted - in this case, the
block_writer() as well as notify_transaction_listeners() must update their
data or quit. This fixes newly introduced bug #2008.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24749 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_private.h is no longer needed, moved its contents into
block_cache.cpp.
* Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24739 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* write_cached_block() now moves a block to the unused list when it's
not referenced and clean (is no longer part of a transaction).
This finally gets memory consumption to a moderate level; even
unpacking a large archive will no longer eat all memory.
* The "block_cache" KDL command will now print the number of unused
blocks as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24606 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The block_cache KDL command now sets some useful variables.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24589 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The "block_cache" command checked the wrong variable to see if a block
could be found or not, eventually resulting in a read fault.
* It now also uses parse_expression() for its second argument, since it
now returns an uint64 in Haiku.
* Tracing output cut off the label too early.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24579 a95241bf-73f2-0310-859d-f6bbb57e9c96

cached_block::Compare() was ignoring the upper 32 bit of the block
number. Was a problem only for partitions > 2^32 * block size (4TB
for 1KB blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24393 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed compare_blocks() function; it actually gets cached_block**.
* Added block_writer thread that continuously writes back blocks for all current
block caches.
* A block cache now maintains the number of dirty blocks if it doesn't use
transactions. That knowledge could also be used in block_cache_sync()...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23698 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The block cache now sorts the blocks before synchronizing a transaction.
* This should speed up writing a transaction considerably.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23694 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a new function hash_insert_grow() that grows the hash table when needed.
* Removed the public hash_grow() function again (at least for now, it's only
private).
* Removed the newSize argument from hash_grow(); it will compute the new size
automatically.
* The block cache is now using hash_insert_grow() instead of hash_insert()
which should make hash lookups much faster with some 10 thousand blocks,
also increased the initial table size from 32 to 1024...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23692 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changed TraceEntry::Dump() to take a TraceOutput& and removed
AbstractTraceEntry::AddDump(char*, size_t).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23655 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved tracing macro definitions into separate tracing_config.h header.
So now there's a central place to enable tracing in general and for
individual components.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23645 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added debugger command "transaction" and added a new "-t" option to
"block_cache" to dump info about transactions.
* Improved tracing.
* Now uses the new add_debugger_command_etc().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23644 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

Gracefully ignore NULL pointers passed to block_cache::Free(). Fixes
panic when sync'ing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23575 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced my block allocator code in favour of Hugo's slab allocator
(so we can't allocate more physical pages than virtual address space
anymore, but so what?).
* Used the new CACHE_LARGE_SLAB flag as a temporary work-around; else
the slab would easily create several thousands of areas, which our
area code (and kernel heap) can't really handle that well (gets
awfully slow).
* Block caches with the same size could share the same slab, but we
don't do that yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23567 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Improved block allocation.
* Made the block_cache KDL command dump a bit more useful info (number of
referenced and dirty blocks).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23540 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added tracing support for transactions.
* improved the "block_cache" KDL command: it can now also dump blocks,
added support for the "--help" argument.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23532 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* bfs_free_cookie() accidently reverted a transaction after having updated
the inode's internal last modified and size copies, causing in inconsistent
data. This fixes #1643.
* The block being replayed are now dumped to the debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23050 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a part of block_cache_set_dirty() so that it can gracefully exit
in case there is nothing to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22318 a95241bf-73f2-0310-859d-f6bbb57e9c96

More informative output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21669 a95241bf-73f2-0310-859d-f6bbb57e9c96

Putting an unused block will now trigger a panic.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21481 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added some temporary means to ease debugging block caches - at least the list
of all caches should be removed at a later point.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21461 a95241bf-73f2-0310-859d-f6bbb57e9c96

Even more correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20856 a95241bf-73f2-0310-859d-f6bbb57e9c96

More correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20855 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fixed broken iteration in block_cache::RemoveUnusedBlocks(). The loop
would start with the first block in the unused blocks list, but then
continue with the blocks that share the same hash table slot, thus
freeing potentially used blocks. Could theoretically have caused
BFS to see and write incorrect meta/administrative data on certain
occasions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20459 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleanup:
* NewBlock()/FreeBlock() are now symmetrical in that the former no longer inserts
the block into the hash table.
* delete_transaction() also no longer removes the transaction from the hash table.
* cache_transaction_sync() now uses the new hash_remove_current() function.
* minor other cleanup (like line breaks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20374 a95241bf-73f2-0310-859d-f6bbb57e9c96

get_cached_block(): Remove the newly allocated block from the
hashtable in case of a read error (NewBlock() also adds the block,
but FreeBlock() only frees it).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20365 a95241bf-73f2-0310-859d-f6bbb57e9c96

added missing nothrow parameter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96

- moved dumpBlock() into debug.c, renamed it to dump_block() and added it
to kernel-exports.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18268 a95241bf-73f2-0310-859d-f6bbb57e9c96

* If there are no free ranges left, the block cache will now reuse older
blocks - this is not enough, though as it would also need to ask other
volumes to free ranges.
* Increased the number of blocks to free in case of low memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16663 a95241bf-73f2-0310-859d-f6bbb57e9c96

changed get_cached_block() to no longer use a reference argument
modified to panic when an invalid block is requested (to find fs errors)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16537 a95241bf-73f2-0310-859d-f6bbb57e9c96

This fixes the work-around made in r16105, and some more:
* cached_block::parent_data can be NULL in a sub transaction in case the
block wasn't part of the parent transaction (but not in low memory
situations). cache_abort_sub_transaction() and cache_detach_sub_transaction()
didn't account for this, though, ie. the block data could end up
corrupted.
* Renamed cached_block::original in original_data.
* Renamed cached_block::data in current_data.
* Added some comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16108 a95241bf-73f2-0310-859d-f6bbb57e9c96

fixed some traces
cache_abort_sub_transaction now doesn't crash if parent_cache is NULL (happens in low memory situations)
Axel, please review and fix if needed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16105 a95241bf-73f2-0310-859d-f6bbb57e9c96

Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16042 a95241bf-73f2-0310-859d-f6bbb57e9c96

Deleted ranges were never removed from the ranges hash... this could have finally
nailed down bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15791 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added some debug helper. Hopefully they give more insight to Stephan's problem (bug #77/#78).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15767 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_abort_transaction() could let cached_block::parent_data have the same value
as cached_block::data - which led to a crash as block_cache::FreeBlock() tried to
free both later.
Since neither cached_block::parent_data nor cached_block::original are supposed
to be != NULL in block_cache::FreeBlock(), they are no longer freed, but the system
panics if one of them is not NULL.
This should fix bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15749 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_range::Delete() did not remove the block_range from the free list,
and thus, it could be used again which led to a crash.
Changed the free ranges list from a singly linked list to a doubly linked
list so that not all free ranges have to be searched for the one to be
freed anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15696 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleaned up locking and fixed a related bug: put_cached_block() could have been called without holding the cache lock.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15695 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added debug output to vm_low_memory.cpp.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15568 a95241bf-73f2-0310-859d-f6bbb57e9c96

The block_cache::LowMemoryHandler() must check if it could acquire the cache's lock;
it might have been deleted in the mean time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15565 a95241bf-73f2-0310-859d-f6bbb57e9c96

* block_cache_delete() now deletes the cache with with its lock held.
* the link were not initialized in cached_block, as its constructor were never called
(was using malloc/free instead of new/delete).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15564 a95241bf-73f2-0310-859d-f6bbb57e9c96

Forget to remove debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15563 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now also flushes dirty blocks correctly (cannot happen if transactions are used,
ie. never happens with BFS).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15562 a95241bf-73f2-0310-859d-f6bbb57e9c96

The block cache now registers and uses a low memory handler that will flush
unused blocks when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15561 a95241bf-73f2-0310-859d-f6bbb57e9c96

Blocks are now put into an unused list when they aren't used.
When memory gets low, the blocks on that list are also freed - but right now,
that only happens when a block is put back (not directly on demand, ie. via
a low memory handler).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15552 a95241bf-73f2-0310-859d-f6bbb57e9c96

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96

First go at an implementation of sub transactions - code is not yet tested, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14429 a95241bf-73f2-0310-859d-f6bbb57e9c96

Updated a few more hash functions to be able to deal with negative numbers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14067 a95241bf-73f2-0310-859d-f6bbb57e9c96

Turned off debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14044 a95241bf-73f2-0310-859d-f6bbb57e9c96

block_cache::Free() now accepts NULL pointers (it was already used this way before...).
Unmounting BFS volumes should now work as expected.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14034 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the definition of the BlockAddressPool class into block_allocator.cpp, as it's
not used outside of it.
Added tracking for the last transaction in the block cache; that way it can test
if the last transaction has been closed before opening a new one.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13022 a95241bf-73f2-0310-859d-f6bbb57e9c96

Some minor cleanup: removed unneeded includes, updated introducing comment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12932 a95241bf-73f2-0310-859d-f6bbb57e9c96

Work in progress of the new block allocator for the block cache. It works, but
currently needs even more memory than the old one 8-)
Not cleaned up at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12921 a95241bf-73f2-0310-859d-f6bbb57e9c96

cache_abort_transaction() freed the wrong block, and thus causing a file system
corruption even better than during its absence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12903 a95241bf-73f2-0310-859d-f6bbb57e9c96

Oops, forgot to implement cache_abort_transaction(); every failing transaction
could have destroyed the BFS integrity... (just happened to me, that's how I
noticed it)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12867 a95241bf-73f2-0310-859d-f6bbb57e9c96

First step into a new block allocation strategy for the block_cache
(right now, it's still malloc/free, just encapsulated in an allocator class).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12662 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96