History log of /haiku/src/system/kernel/cache/block_cache.cpp
Revision Date Author Comments
# 93845aec 11-Jun-2020 Axel Dörfler <axeld@pinc-software.de>

block_cache: Change signature of *_etc() functions

* This allows file systems to retrieve the actual error code on a
failure, and report it to the user.
* All affected file systems have been adjusted to the API change.
This is a binary incompatible change.

Change-Id: Id73392aaf9c6cb7d643ff9adcb8bf80f3037874c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2913
Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>


# 6d336fda 10-Jul-2019 Augustin Cavalier <waddlesplash@gmail.com>

block_cache: Compute the timeout dynamically based on write speeds.

We now record how long it takes to write a block (on average), and then
utilize this information to reduce the timeout write thread's timeout
(to 2 * block_count * average_block_time, so we don't completely
congest the drive.) Remove the "TODO" about the I/O scheduler;
this new logic will be just fine even under an I/O scheduler.

Note that this change goes both ways: while faster writes mean more
writes and quicker, slower writes will increase the timeout before
we do another one also. This then also guards against queueing
another write while one is already in progress, which was
not handled before.

Tested in KVM. Even on a SATA-backed spinning HDD, this reduces
the timeout to around *200ms* on average (!!), so a 10x improvement.
On a ramdisk, it reduces the timeout to *10-30ms* (!!!) on average,
so a 100-200x improvement, so this change will benefit everyone
but SSDs especially.

Since BFS inode and journal writes always go through the block_cache,
this very dramatically improves inode-related write performance.
The "stop and start" stutters when emptying or moving items to Trash
seem totally gone, among a lot of other things.

Change-Id: I41f46a6432ce1f50f896a853abdfe22dde0ba327


# 0e6ece91 10-Jul-2019 Augustin Cavalier <waddlesplash@gmail.com>

block_cache: Use an object_cache for the cache_notifications.

cache_listeners are (following packagefs changes) the second-most
allocated object during the boot process, with 20799 on a
*minimum* image. Since they are used so extensively in BFS I/O,
making them object_cached for both performance and memory reasons
seems to make a lot of sense.

Change-Id: I6ef6c3e811c0c4189fea45ee7920131796c9e7c8
Reviewed-on: https://review.haiku-os.org/c/1577
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: Stephan Aßmus <superstippi@gmx.de>


# 7a2f7448 10-Jul-2019 Augustin Cavalier <waddlesplash@gmail.com>

block_cache: Free up blocks more aggressively on low memory conditions.

Only keep a single block when there is a "critical" low resource state.
Also don't bother doing anything if there are no unused blocks.

Change-Id: Ibfcbd8cb0beb1446083ca83030ea8e81c59a9628
Reviewed-on: https://review.haiku-os.org/c/1576
Reviewed-by: waddlesplash <waddlesplash@gmail.com>


# 1750cd1e 23-Aug-2017 hyche <cvghy116@gmail.com>

block_cache: Implement cache_has_block_in_transaction function that will check the existence of block in one specific transaction.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>


# c73d1301 08-Nov-2015 Michael Lotz <mmlr@mlotz.ch>

kernel: Use anonymous namespaces to avoid type collisions.

The anonymous namespace makes type definitions local to the translation
unit (like static does for objects). For pretty much any type not shared
across multiple files this is what one wants to happen (and might
erroneously expect to happen automatically).

This commit solves some actual collisions that were present:

* The VFS and the rootfs both used an incompatible VnodeHash struct for
their BOpenHashTable.
* XSI semaphores and message queues both used queued_thread, Ipc and
IpcHashTableDefinition.

For release builds these did not cause problems as the types were fully
inlined. Debug builds would crash at boot however because parts of a
BOpenHashTable<VnodeHash> from the rootfs meant to operate on struct
rootfs_vnode would be applied to one from the VFS expecting struct
vnode.

As such collisions are violations of the one definition rule, the code
is at fault and unfortunatley the compiler isn't required to diagnose
such problems across translation units (which isn't actually trivial).
This can lead to subtle and hard to debug problems and it's therefore
best to avoid leaking types into the global namespace whenever possible.


# efdcada6 30-Mar-2015 Ithamar R. Adema <ithamar@upgrade-android.com>

BlockWriter: Do not panic on failing read/writes

These were here for debugging purposes, as often it is a sign of
inconsistencies. However, for USB disks this is a normal occurence
when someone janks out of the device without unmounting first.

Make sure we log these cases though, as it still helps debugging.

Fix sponsered by http://www.izcorp.com


# 9d1c3b8d 13-Jan-2015 Adrien Destugues <pulkomandy@gmail.com>

block cache: convert to BOpenHashTable.


# 963d68a6 14-Nov-2012 Michael Lotz <mmlr@mlotz.ch>

Fix typo.


# 8c3a938f 19-Sep-2012 Axel Dörfler <axeld@pinc-software.de>

cache_abort_sub_transaction() did not maintain num_blocks.

* When a block was only used in a sub-transaction, it was thrown away,
but the transaction::num_blocks field was not decremented.
* This caused transactions never considered finished which eventually
led to bug #8942. This does not explain the disk corruption occurring
in #8969, though.


# 098967ce 03-Sep-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed the new issue in #8910 from r44585.

* The ASSERT() I introduced in r44585 was incorrect: when the sub transaction
used block_cache_get_empty() to get the block, there is no original_data for
a reason.
* Added a test case that reproduces this situation.
* The block must be moved to the unused list in this situation, though, or else
it might contain invalid data. Since the block can only be allocated in the
current transaction, this should not be a problem, though, AFAICT.


# 713945ce 27-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

The original_data could be freed late.

* In cache_abort_sub_transaction(), the original_data can already be freed
when the block is being removed from the transaction.
* block_cache::_GetUnusedBlock() no longer frees original/parent data - it
now requires them to be freed already (it makes no sense to have them still
around at this point).
* AFAICT the previous version did not have any negative consequences besides
freeing the original data late.


# 9089ab06 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Minor block cache refactoring.

* Extracted a write_blocks_in_previous_transaction() function out of
cache_end_transaction(), and cache_detach_sub_transaction().


# dd9d6105 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed broken block list introduced in hrev44357.

* cache_abort_sub_transaction() was setting the transaction_next pointer to
NULL in order to remove a block from a transaction -- however, it forgot to
actually remove it from the transaction's block list.
* Minor restructuring.


# 153d8953 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed broken discard handling in cache_start_sub_transaction().

* This actually resolves a TODO.


# 8f9dac69 27-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Cleanup, no functional change intended.


# 752f5c97 19-Jul-2012 Axel Dörfler <axeld@pinc-software.de>

cache_abort_[sub_]transaction() did not work correctly.

* cache_abort_transaction() left the block dirty which was causing bug
#8123 as well.
* cache_abort_sub_transaction() did, in addition to not clearing the dirty
flag, not reset the block's transaction member either if the block was
not part of the parent transaction.


# 0579a695 19-Jul-2012 Axel Dörfler <axeld@pinc-software.de>

Added a bit of documentation, minor cleanup.

* Documented the cached_block::transaction, and previous_transaction
members.


# 8959a4e0 19-Jul-2012 Andreas Henriksson <sausageboy@gmail.com>

An unused dirty block would put again into the unused list.

* This could cause bug #8123. While this fixes the crash, one underlying
problem is still there, as dirty blocks should never leave a transaction.


# 0e88a887 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# 0e35d5d2 12-Dec-2011 John Scipione <jscipione@gmail.com>

Change instances of wether in comments to whether. No functional change intended.


# 44b9a93e 04-Jan-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Added intermediate (addr_t) casts before the cast of the parse_expression()
result (uint64) to pointer (CID 4723-4726).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40121 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4af30335 18-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* We need to reset the blockNumber, or else we'll discard the wrong blocks
which has rather desastrous consequences - thanks to Stippi for the heads
up :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38242 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c141a31f 17-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* Fixed CID 1447 which was an actual bug which completely voided the whole
discard mechanism eventually causing the conflicts between the file cache
and the block cache it intended to fix.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38232 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3426ce14 04-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* Thanks to Janito I had a closer look to cache_detach_sub_transaction(), and it
turned out it's pretty much broken. Not only did it potentially leak memory,
it would also potentially replace the original data with a NULL pointer,
making the current transaction non revertable. The code should now be much
clearer. This fixes bug #6378.
* Changed a few "if (... != NULL)" into ASSERTs, since the code should bail out
earlier in these cases already.
* Added a TODO comment to cache_start_sub_transaction() about its broken discard
handling. This can cause FS corruptions in case the parent transaction is ever
going to be aborted (which shouldn't happen in real life, though).
* Added a bit more and better comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37899 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bb486a72 10-Jul-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_end_transaction(): Allow the hook parameter to be NULL.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1cafaeca 18-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Introduced block_cache::unused_block_count, which counts the elements of
block_cache::unused_blocks.
* block_cache::Allocate(): No longer removes unused blocks when in a low
resource state. That just removed too many blocks too quickly, when the
cache was actively used for writing, seriously affecting performance.
* block_cache::_LowMemoryHandler(): Compute the number of unused blocks to
remove depending on the total unused block number. This way we cull huge
block caches with lots of old blocks much quicker.

Treats part of #5816.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37170 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d3166e46 29-Apr-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c561bf79 19-Mar-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

block_cache::NewBlock(): Allocate cached_block::current_data only when the
block was freshly allocated. A block returned by _GetUnusedBlock() already
has current_data and we would leak it before.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35918 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 47f23fb5 10-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* If get_cached_block() failed reading a block, it would leave a deleted object
in the unused list if it got the block via block_cache::_GetUnusedBlock().
* block_cache::_GetUnusedBlock() leaked the compare data block if the
BLOCK_CACHE_DEBUG_CHANGED feature had been enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35799 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6a7f219c 03-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* block_notifier_and_writer() will now flush 1000 blocks when reaching the soft
memory limit.
* Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35735 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0181fe4b 03-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* Replaced cached_block::accessed (the number of accesses) with last_accessed
(the time of the last access), as what we really want is a frequency/last
access time scoring, and accessed alone is useless for that.
* put_cached_block() no longer frees any unused blocks.
* The low memory handler will now only lock the cache if there is something
to do. Also, it did not take address space warnings into account.
* Even when memory is critical, we don't free all unused blocks anymore - if
the number of blocks we free now (10000) is not sufficient to get out of the
critical condition, chances are good that we will be called again :-)
* block_notifier_and_writer() now tries to make sure that the total block cache
memory consumption grows not much larger than half of the available RAM.
* This should all help to limit the block cache usage a bit better. Hopefully,
a checkfs run will no longer run out of memory here (couldn't test yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35733 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a282be40 27-Feb-2010 Michael Lotz <mmlr@mlotz.ch>

Re-lock the cache directly after the read. Otherwise the error case would
remove the block from an unlocked cache and the cache->lock would be unlocked
twice when the calling function unlocked. It panics in that case anyway, but
this should make it continuable and is more correct.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35648 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ff59ce68 24-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* The low resource handler now empties the cache depot's magazines; before,
they were never freed unless the cache was destroyed (I just wondered why
my system would bury >1G in the magazines).
* Made the magazine capacity variable per cache, ie. for larger objects, it's
not a good idea to have 64*CPU buffers lying around in the worst case.
* Furthermore, the create_object_cache_etc()/object_depot_init() now have
arguments for the magazine capacity as well as the maximum number of full
unused magazines.
* By default, you might want to initialize both to zero, as then some hopefully
usable defaults are computed. Otherwise (the only current example is the
vm_page_mapping cache) you can just put in the values you'd want there.
The page mapping cache uses larger values, as its objects are usually
allocated and deleted in larger chunks.
* Beware, though, I couldn't test these changes yet as Qemu didn't like to run
today. I'll test these changes on another machine now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35601 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9d071e27 21-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

This should finally nail #5415:
* Since the same block can be in up to two transactions, it's very well possible
that one shouldn't write all transactions in a single run.
* Forgot to pass on the iterator from BlockWriter::Add(transaction) to
Add(block).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35565 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c5e52ecb 17-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* cache_sync_transaction() now puts all blocks into the BlockWriter, and
therefore does not unlock anymore while iterating over the transactions.
This gave other threads the opportunity to finish a transaction, causing
bug #5412. Also, the BlockWriter will now always close transactions on its
own, and you need to pass the transaction hash iterator to Add().
* Also, transactions that contain blocks that are currently written back will
be ignored by the block writer, as well as cache_sync_transaction(). This
fixes bug #5415.
* Improved error handling if BlockWriter fails to write back blocks. Most
notably, they are no longer left busy_writing, and the functions calling
it do proper error reporting (besides block_cache_discard() that does not
return any erro code; I've added a TODO note there for now).
* The BlockWriter now starts with a larger array once it has to allocate one.
* One can now limit the number of blocks that go into a BlockWriter. This is
used by the block writer thread, that shouldn't always write back everything
every two seconds.
* Also, the fixed array is larger now (leaving enough space such that the
block writer/notifier does not need to allocate anything).
* And finally, if allocating the array fails, the BlockWriter falls back to the
synchronous write back used previously. IOW it will never write back less
blocks than you ask for.
* Added static BlockWriter::WriteBlock() method replacing write_cached_block().
* Forgot to rename block_cache::busy_count to busy_reading_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35510 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 66a1a16c 16-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Made cache_end_transaction(), cache_detach_sub_transaction(), and
block_cache_discard() use the BlockWriter directly as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35496 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 319a2dee 15-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Improved KDL command output (ie. take busy_writing into account, too).
* Added new KDL command "cached_block" that dumps a cached block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35493 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 751e92fd 16-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* cache_sync_transaction() used an earlier signature of the BlockWriter
constructor, causing bug #5412.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35489 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4ea83177 15-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Implemented a BlockWriter class that now performs writing back blocks.
* Renamed the "busy" stuff to "busy_reading", and added a "busy_writing"
concept.
* This now allows reading a block (and other blocks), while blocks are written
back. This should speed all operations needing to write back blocks, like
unzipping or compiling.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35464 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 22991e1d 03-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Use the is_writing flag to determine whether or not we may write back a block
that does not have a transaction.
* This should fix #5340.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35390 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e00b4c19 28-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved handling of busy blocks into separate functions.
* Added flags to avoid notifying the busy condition variable unnecessarily.
* get_writable_cached_block(): Unlock the cache while memcpy()ing/memset()ing
the block's data. The idea is to reduce lock contention. Less effective
than I hoped, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35328 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 86c794e5 21-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e7385fa6 03-Jan-2010 Axel Dörfler <axeld@pinc-software.de>

* If a block allocation fails, remove an unused one and try again - this might
improve the reliability as long as our slab implementation is a PITA.
* Removed an assertion that will no longer work (due to the DoublyLinkedList
changes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34877 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 33c82a30 10-Dec-2009 Axel Dörfler <axeld@pinc-software.de>

* While reading in a block, the block is now marked busy, and the cache
unlocked, allowing for more parallel access.
* Writing is still done synchronously, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34611 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1db31d4d 27-Nov-2009 Axel Dörfler <axeld@pinc-software.de>

* The slab and the block cache now both also react on address space shortages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34310 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a10ac91f 05-Nov-2009 Axel Dörfler <axeld@pinc-software.de>

* NewBlock(), and Allocate() will now try to reuse existing memory in low
resource situations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33892 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 818ef0e5 13-Aug-2009 Artur Wyszynski <aljen-mlists@o2.pl>

Fixed wrong check


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32326 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0a5bfde2 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Added stack traced to the Abort tracing output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 61efcc32 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31869 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 71d20343 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Enhanced cache_blocks_in_main_transaction() to also return the correct number
of blocks in case there is no sub-transaction at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31868 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 483acc43 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* has_sub_transaction was never initialized, no clue why this didn't come up
earlier.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31845 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eb26e782 27-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Decoupled block_cache_used_memory() from the sCachesLock - this should fix the
UI freezes (ActivityMonitor and ProcessController both use get_system_info() a
lot), although this is only the symptom of another problem.
* The downside is that the block cache usage information isn't as up to date as
it was previously - it's updated by the block write/notifier thread now (worst
case every 2 seconds).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31812 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 813d9285 25-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Fixed a number of problems of the "block_cache_data" command.
* Also added stack traces to each BlockData entry.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31763 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 008ec58d 11-Jul-2009 Rene Gollent <anevilyak@gmail.com>

When building the block cache for UserlandFS's haiku emulation lib, don't include tracing since that's unavailable outside of the kernel. Fixes ticket #4092. Axel or Ingo, please review.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31525 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 591bc3f2 28-May-2009 Jérôme Duval <korli@users.berlios.de>

* various fixes identified by gcc4, provided by Joe Prostko


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30902 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d1cad268 10-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Added an insane block cache tracing mode that will also log the whole block
data. This is available when BLOCK_CACHE_BLOCK_TRACING is 2 or greater.
* Completely untested as of now, though. Will do so soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30104 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fd0803f3 06-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Added RemoveTail() method.
* Renamed DoublyLinkedList::Size() to Count(), since it actually counts the
items (ie. O(n)).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29979 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 98092e98 18-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

gcc 4 fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29602 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9bb78ee2 26-Feb-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Removed the superfluous 'extern "C"'s.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29334 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c33667d4 01-Feb-2009 Michael Lotz <mmlr@mlotz.ch>

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8f1cb8fc 11-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Brought the fs_shell block_cache implementation up to date, and incorporate
the fixes applied to the kernel version of it (most notably a correctly
working block_cache_discard(), and cache_detach_sub_transaction()).
Also switched to the new notification functions, even though it still works
synchronously in the fs_shell.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28617 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fbc2cd83 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* wait_for_notifications() must make sure that the order of the notifications
in the cache is the same as in the condition variable. It now uses the
low-contention sCachesLock to accomplish this.
* Also added an ASSERT to make sure the notification is no longer part of the
list at this point.
* Improved KDL command output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28516 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a26045b7 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* cache_detach_sub_transaction(), and cache_abort_sub_transaction() now support
discarded blocks correctly as well.
* cache_detach_sub_transaction() left cached_block::original_data unchanged even
if the parent data was to become current (in case the sub transaction didn't
change the block yet). This could cause outdated blocks to be written back.
* cache_detach_sub_transaction() also set cached_block::previous_transaction
for all blocks, not just the ones with a previous transaction. This could
cause blocks to be written twice for no reason.
* cache_start_sub_transaction() did not change the num_blocks count for
discarded blocks.
* block_cache_discard() now panics if the block was already changed in the
current transaction.
* Improved test application, added more tests, revealing the above bugs in
cache_detach_sub_transaction().
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28514 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ddd20af4 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed a potential race condition: when wait_for_notifications() is called
from the block notifier, the cache could be deleted before we have the chance
to lock it. We now lock the sCachesLock, and see if this cache is still valid.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28512 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 96e19c19 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* The first functions now handle the "discard" flag correctly (namely
cache_end_transaction(), and cache_start_sub_transaction()).
* Further work on the test application, it's now actually usable, first test
passes.
* dump_block() did erroneously print 'B' for the dirty flag; now both dirty and
discard have the 'D' (3rd and 5th column).
* block_cache::LowMemoryHandler() is now private (and got an underscore prefix).
* Minor cleanup, shuffled some methods around.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28508 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f69f65ec 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Cleanup, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28497 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5b812019 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 59dbd26f 20-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 56ab6f07 01-Oct-2008 Axel Dörfler <axeld@pinc-software.de>

* I accidently broke get_next_locked_block_cache() in r27074; it would always
ignore every other cache (starting from the first).
* The consequence of this was that no blocks were written back automatically
for those caches, and their transactions were never idle, causing bug #2781.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27831 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f8bdc244 01-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Created a central place for putting kernel debug enabling macros.
Currently it only contains KDEBUG and the block cache debugging macros.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27816 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4bfeb6f3 20-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Changed the way how a cache is destructed: now, it is removed from the
global cache list before locking it. This allows to get rid of the
block_cache::deleting field, as well as simplifies some code.
* This also fixes a possible deadlock I recently introduced (on destruction,
the locking order was wrong).
* Now uses an anonymous condition variable instead.
* Moved the block_cache initialization code into a dedicated method that will
now also fail in case the low resource handler couldn't be registered (as
pointed out by Salvatore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27074 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a1f34e7e 19-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Only remove the block cache from the list, and its low resource handler when
this is actually needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27060 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1d136d5a 19-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed bug #2631 based on a patch by Salvatore: the block cache was put into
a global list before it was fully initialized.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27058 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c79e66ab 05-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Added block_cache_used_memory() that returns the memory allocated by all
block caches for the block cache buffers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26836 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2f14f5ee 20-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

axeld + bonefish + mmlr:
* Added trace entry for block reads from disk.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26534 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d066dea 19-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added block level tracing to the block cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26521 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 45b95965 05-Jun-2008 Michael Lotz <mmlr@mlotz.ch>

Missed that file in the commit for r25812.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25816 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 91312c91 29-May-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Changed tracing output prefix from "cache" to "block cache".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25714 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 036cf4dd 11-May-2008 Jérôme Duval <korli@users.berlios.de>

block_cache_delete(): only lock the cache mutex and avoid MutexLocker. the mutex is destroyed in the destructor.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25467 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c1d860fb 12-May-2008 Jérôme Duval <korli@users.berlios.de>

block_cache_delete(): unlock the cache mutex before deleting the cache


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25463 a95241bf-73f2-0310-859d-f6bbb57e9c96


# adf376c9 08-May-2008 Axel Dörfler <axeld@pinc-software.de>

Replaced benaphores with mutexes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25378 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 76824051 29-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Only enable block/transaction tracing when the macro is 1.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25247 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2909d9dc 24-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Factored a delete_notification() that is now used in three places.
* remove_transaction_listeners() was checking the events_pending field without
holding the sNotificatonLock - that should have been harmless (as we're in the
middle of deleting the transaction), but it now looks better.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25130 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6cef245e 22-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 36de598e 09-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* block_cache_sync[_etc]() now also waits for all notifications to be handled.
This fixes a potential problem where a volume would be left marked dirty on
a clean reboot as reported by Vasilis; no harm would have been done to the
data on the volume, though (it would just replay a log entry that it didn't
need to).
* Unified the naming of the notifier/writer thread: renamed the thread to "block
notifier/writer", and the variable holding the thread ID to
sNotifierWriterThread.
* Added a few more doxygen descriptions to the private functions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 391fdf1e 07-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* notify_transaction_listeners() would always remove all listeners when it
was asked to. Unfortunately, it also removed the TRANSACTION_WRITTEN
listeners (that were added before notifying other listeners), so that BFS
could never free up space in the log. Now, it will only remove the listeners
it may remove. This fixes bug #2024.
* Rearranged functions a bit, moved notification/listener related private
functions together.
* Added tracing entry for written transactions as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24853 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 98b7d71b 05-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

The wait_for_notifications() function now detects if it has been run from
within the notifier/writer thread, and will then flush the notifications
directly. This should fix #2008 again.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24806 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7686d00c 04-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* This should fix a deadlock as reported by bga.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24793 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 807d3aa8 04-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* If BFS's Journal::_WriteTransactionToLog() noticed there wasn't enough free
space left for the new log entry, it did call cache_sync_transaction(), and
then just assumed the space would be ready. But since the transaction could
have been written before that call by the block writer, and since the
_TransactionWritten() hook is now called asynchronously,
cache_sync_transaction() actually has to flush all pending TRANSACTION_WRITTEN
notifications before returning to the caller.
* To implement this, block_cache now publishs a condition variable, and
wait_for_notifications() adds a fake notification that signals that one.
Since the notifications are handled in FIFO order, this guarantees that
the previous TRANSACTION_WRITTEN hook is done.
* notify_transaction_listeners() could accidently delete notifications that
still had pending signals. Now, it will defer the deletion to the notification
thread instead in that case. This should fix bug #2008.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24792 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9a872286 03-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Removed the "transaction_changed" leftover member.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24769 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 816957bd 03-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Decoupled notifications from the block cache functions; they are now
called without having the cache's lock held.
* The only downside to this approach is that ending a transaction now needs
additional memory, and might therefore fail - that could be fixed by
allocating that memory upfront with the start of the new transaction,
though.
* Therefore, I reverted the cache lock to a benaphore, as the recursive
lock is no longer needed.
* The block writer thread is now a block writer and notifier thread: it will
wait for up to 2 seconds on the sEventSemaphore, and if nothing comes in,
it will continue with its previously sole job.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24768 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a5140cad 02-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_end_transaction() doesn't remove the transaction, so that the
check in notify_transaction_listeners() didn't hold, although the
cache_hooks were deleted anyway. I've introduced a listener_change
counter to detect changes to the listener list while iterating, since
any change makes the iteration unsafe. Should fix bug #2008 for real.
Nevertheless I thing the transaction listener concept needs revision.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24761 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 38c4c334 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* A notification can cause a transaction to be deleted - in this case, the
block_writer() as well as notify_transaction_listeners() must update their
data or quit. This fixes newly introduced bug #2008.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24749 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0aee496b 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* block_cache_private.h is no longer needed, moved its contents into
block_cache.cpp.
* Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24739 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 16d8ff2d 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 14438ad5 27-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* write_cached_block() now moves a block to the unused list when it's
not referenced and clean (is no longer part of a transaction).
This finally gets memory consumption to a moderate level; even
unpacking a large archive will no longer eat all memory.
* The "block_cache" KDL command will now print the number of unused
blocks as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24606 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3bb1e6c2 26-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* The block_cache KDL command now sets some useful variables.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24589 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cd31b326 25-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* The "block_cache" command checked the wrong variable to see if a block
could be found or not, eventually resulting in a read fault.
* It now also uses parse_expression() for its second argument, since it
now returns an uint64 in Haiku.
* Tracing output cut off the label too early.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24579 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fae99f15 14-Mar-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

cached_block::Compare() was ignoring the upper 32 bit of the block
number. Was a problem only for partitions > 2^32 * block size (4TB
for 1KB blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24393 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6eabbacd 22-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed compare_blocks() function; it actually gets cached_block**.
* Added block_writer thread that continuously writes back blocks for all current
block caches.
* A block cache now maintains the number of dirty blocks if it doesn't use
transactions. That knowledge could also be used in block_cache_sync()...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23698 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5af13da1 21-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* The block cache now sorts the blocks before synchronizing a transaction.
* This should speed up writing a transaction considerably.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23694 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4a67038e 21-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a new function hash_insert_grow() that grows the hash table when needed.
* Removed the public hash_grow() function again (at least for now, it's only
private).
* Removed the newSize argument from hash_grow(); it will compute the new size
automatically.
* The block cache is now using hash_insert_grow() instead of hash_insert()
which should make hash lookups much faster with some 10 thousand blocks,
also increased the initial table size from 32 to 1024...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23692 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f7a5d9c5 19-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Changed TraceEntry::Dump() to take a TraceOutput& and removed
AbstractTraceEntry::AddDump(char*, size_t).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23655 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ab23dfa7 19-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Moved tracing macro definitions into separate tracing_config.h header.
So now there's a central place to enable tracing in general and for
individual components.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23645 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5f7a62cf 19-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added debugger command "transaction" and added a new "-t" option to
"block_cache" to dump info about transactions.
* Improved tracing.
* Now uses the new add_debugger_command_etc().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23644 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5d0afa4e 18-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e6134992 16-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Gracefully ignore NULL pointers passed to block_cache::Free(). Fixes
panic when sync'ing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23575 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a96e7cce 16-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Replaced my block allocator code in favour of Hugo's slab allocator
(so we can't allocate more physical pages than virtual address space
anymore, but so what?).
* Used the new CACHE_LARGE_SLAB flag as a temporary work-around; else
the slab would easily create several thousands of areas, which our
area code (and kernel heap) can't really handle that well (gets
awfully slow).
* Block caches with the same size could share the same slab, but we
don't do that yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23567 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8264ebf6 15-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Improved block allocation.
* Made the block_cache KDL command dump a bit more useful info (number of
referenced and dirty blocks).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23540 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 502c4640 15-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added tracing support for transactions.
* improved the "block_cache" KDL command: it can now also dump blocks,
added support for the "--help" argument.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23532 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4fc4f2c8 13-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4afaeabb 02-Dec-2007 Axel Dörfler <axeld@pinc-software.de>

* bfs_free_cookie() accidently reverted a transaction after having updated
the inode's internal last modified and size copies, causing in inconsistent
data. This fixes #1643.
* The block being replayed are now dumped to the debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23050 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a25e129 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

Implemented a part of block_cache_set_dirty() so that it can gracefully exit
in case there is nothing to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22318 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fec9ab5d 19-Jul-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

More informative output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21669 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 84916160 20-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Putting an unused block will now trigger a panic.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21481 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5de78b2c 18-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Added some temporary means to ease debugging block caches - at least the list
of all caches should be removed at a later point.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5b1cb74b 27-Apr-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Even more correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20856 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 93ed4531 27-Apr-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

More correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20855 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c03270b 24-Apr-2007 Axel Dörfler <axeld@pinc-software.de>

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3e414ec3 28-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Fixed broken iteration in block_cache::RemoveUnusedBlocks(). The loop
would start with the first block in the unused blocks list, but then
continue with the blocks that share the same hash table slot, thus
freeing potentially used blocks. Could theoretically have caused
BFS to see and write incorrect meta/administrative data on certain
occasions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20459 a95241bf-73f2-0310-859d-f6bbb57e9c96


# de4145db 11-Mar-2007 Axel Dörfler <axeld@pinc-software.de>

Cleanup:
* NewBlock()/FreeBlock() are now symmetrical in that the former no longer inserts
the block into the hash table.
* delete_transaction() also no longer removes the transaction from the hash table.
* cache_transaction_sync() now uses the new hash_remove_current() function.
* minor other cleanup (like line breaks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20374 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e1555e1f 11-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

get_cached_block(): Remove the newly allocated block from the
hashtable in case of a read error (NewBlock() also adds the block,
but FreeBlock() only frees it).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20365 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 82029bda 16-Sep-2006 Marcus Overhagen <marcusoverhagen@gmail.com>

added missing nothrow parameter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18868 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9b906ccf 30-Aug-2006 Axel Dörfler <axeld@pinc-software.de>

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b66ae2c9 25-Jul-2006 Oliver Tappe <zooey@hirschkaefer.de>

- moved dumpBlock() into debug.c, renamed it to dump_block() and added it
to kernel-exports.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18268 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 17372b76 08-Mar-2006 Axel Dörfler <axeld@pinc-software.de>

* If there are no free ranges left, the block cache will now reuse older
blocks - this is not enough, though as it would also need to ask other
volumes to free ranges.
* Increased the number of blocks to free in case of low memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16663 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 89d76e50 27-Feb-2006 Marcus Overhagen <marcusoverhagen@gmail.com>

changed get_cached_block() to no longer use a reference argument
modified to panic when an invalid block is requested (to find fs errors)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16537 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1bb74eb1 26-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

This fixes the work-around made in r16105, and some more:
* cached_block::parent_data can be NULL in a sub transaction in case the
block wasn't part of the parent transaction (but not in low memory
situations). cache_abort_sub_transaction() and cache_detach_sub_transaction()
didn't account for this, though, ie. the block data could end up
corrupted.
* Renamed cached_block::original in original_data.
* Renamed cached_block::data in current_data.
* Added some comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16108 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2e5ac352 26-Jan-2006 Jérôme Duval <korli@users.berlios.de>

fixed some traces
cache_abort_sub_transaction now doesn't crash if parent_cache is NULL (happens in low memory situations)
Axel, please review and fix if needed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16105 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 627e4043 23-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16042 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 37675ea5 02-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

Deleted ranges were never removed from the ranges hash... this could have finally
nailed down bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15791 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e075b5a3 31-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Added some debug helper. Hopefully they give more insight to Stephan's problem (bug #77/#78).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15767 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 140dd512 30-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

cache_abort_transaction() could let cached_block::parent_data have the same value
as cached_block::data - which led to a crash as block_cache::FreeBlock() tried to
free both later.
Since neither cached_block::parent_data nor cached_block::original are supposed
to be != NULL in block_cache::FreeBlock(), they are no longer freed, but the system
panics if one of them is not NULL.
This should fix bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15749 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 529bf1c6 28-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

block_range::Delete() did not remove the block_range from the free list,
and thus, it could be used again which led to a crash.
Changed the free ranges list from a singly linked list to a doubly linked
list so that not all free ranges have to be searched for the one to be
freed anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15696 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3dcaf063 28-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Cleaned up locking and fixed a related bug: put_cached_block() could have been called without holding the cache lock.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15695 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 89f5f72a 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* Added debug output to vm_low_memory.cpp.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15568 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4c0f46e2 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

The block_cache::LowMemoryHandler() must check if it could acquire the cache's lock;
it might have been deleted in the mean time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15565 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 182f88dd 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* block_cache_delete() now deletes the cache with with its lock held.
* the link were not initialized in cached_block, as its constructor were never called
(was using malloc/free instead of new/delete).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15564 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 005108a2 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Forget to remove debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15563 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 37bd67ed 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Now also flushes dirty blocks correctly (cannot happen if transactions are used,
ie. never happens with BFS).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15562 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 41e866f3 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

The block cache now registers and uses a low memory handler that will flush
unused blocks when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15561 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ebdb9dfe 15-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Blocks are now put into an unused list when they aren't used.
When memory gets low, the blocks on that list are also freed - but right now,
that only happens when a block is put back (not directly on demand, ie. via
a low memory handler).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15552 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1c88c851 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ca7c5a9e 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 94752292 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

First go at an implementation of sub transactions - code is not yet tested, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14429 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e432cef2 25-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Updated a few more hash functions to be able to deal with negative numbers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14067 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a4462f5 21-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14044 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 30616a32 21-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

block_cache::Free() now accepts NULL pointers (it was already used this way before...).
Unmounting BFS volumes should now work as expected.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14034 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2a1492f0 08-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Moved the definition of the BlockAddressPool class into block_allocator.cpp, as it's
not used outside of it.
Added tracking for the last transaction in the block cache; that way it can test
if the last transaction has been closed before opening a new one.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13022 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 19b77242 02-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Some minor cleanup: removed unneeded includes, updated introducing comment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12932 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 317121e1 01-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Work in progress of the new block allocator for the block cache. It works, but
currently needs even more memory than the old one 8-)
Not cleaned up at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12921 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f4e5c341 30-May-2005 Axel Dörfler <axeld@pinc-software.de>

cache_abort_transaction() freed the wrong block, and thus causing a file system
corruption even better than during its absence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12903 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b3ef4fb4 28-May-2005 Axel Dörfler <axeld@pinc-software.de>

Oops, forgot to implement cache_abort_transaction(); every failing transaction
could have destroyed the BFS integrity... (just happened to me, that's how I
noticed it)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12867 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d5b04e50 13-May-2005 Axel Dörfler <axeld@pinc-software.de>

First step into a new block allocation strategy for the block_cache
(right now, it's still malloc/free, just encapsulated in an allocator class).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12662 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2d690920 13-Apr-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96


# efdcada6f30bd510641bfda1b15ab374bc22c04a 30-Mar-2015 Ithamar R. Adema <ithamar@upgrade-android.com>

BlockWriter: Do not panic on failing read/writes

These were here for debugging purposes, as often it is a sign of
inconsistencies. However, for USB disks this is a normal occurence
when someone janks out of the device without unmounting first.

Make sure we log these cases though, as it still helps debugging.

Fix sponsered by http://www.izcorp.com


# 9d1c3b8d4b93abd805a1cba96134d090200f04b2 13-Jan-2015 Adrien Destugues <pulkomandy@gmail.com>

block cache: convert to BOpenHashTable.


# 963d68a6321c389e781a47de091fe90f1435036e 14-Nov-2012 Michael Lotz <mmlr@mlotz.ch>

Fix typo.


# 8c3a938fc386a9733145ce251d538a29ecdf2987 19-Sep-2012 Axel Dörfler <axeld@pinc-software.de>

cache_abort_sub_transaction() did not maintain num_blocks.

* When a block was only used in a sub-transaction, it was thrown away,
but the transaction::num_blocks field was not decremented.
* This caused transactions never considered finished which eventually
led to bug #8942. This does not explain the disk corruption occurring
in #8969, though.


# 098967cee129b9116b468224257234c8ef1c9707 03-Sep-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed the new issue in #8910 from r44585.

* The ASSERT() I introduced in r44585 was incorrect: when the sub transaction
used block_cache_get_empty() to get the block, there is no original_data for
a reason.
* Added a test case that reproduces this situation.
* The block must be moved to the unused list in this situation, though, or else
it might contain invalid data. Since the block can only be allocated in the
current transaction, this should not be a problem, though, AFAICT.


# 713945cecb4e18a400b5dc92797a6defe3f8a4bd 27-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

The original_data could be freed late.

* In cache_abort_sub_transaction(), the original_data can already be freed
when the block is being removed from the transaction.
* block_cache::_GetUnusedBlock() no longer frees original/parent data - it
now requires them to be freed already (it makes no sense to have them still
around at this point).
* AFAICT the previous version did not have any negative consequences besides
freeing the original data late.


# 9089ab06bdf75de4116252a549f25b8d723c0267 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Minor block cache refactoring.

* Extracted a write_blocks_in_previous_transaction() function out of
cache_end_transaction(), and cache_detach_sub_transaction().


# dd9d610500dfb12b68dace7ce2dd3c0b96818e3a 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed broken block list introduced in hrev44357.

* cache_abort_sub_transaction() was setting the transaction_next pointer to
NULL in order to remove a block from a transaction -- however, it forgot to
actually remove it from the transaction's block list.
* Minor restructuring.


# 153d895337628f163c496ebc6a1ccd1630640139 26-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Fixed broken discard handling in cache_start_sub_transaction().

* This actually resolves a TODO.


# 8f9dac699d22df33062dceecd6e2445dfd1b1cc1 27-Aug-2012 Axel Dörfler <axeld@pinc-software.de>

Cleanup, no functional change intended.


# 752f5c972f1a874cc7f93787e1a3bce6396c891e 19-Jul-2012 Axel Dörfler <axeld@pinc-software.de>

cache_abort_[sub_]transaction() did not work correctly.

* cache_abort_transaction() left the block dirty which was causing bug
#8123 as well.
* cache_abort_sub_transaction() did, in addition to not clearing the dirty
flag, not reset the block's transaction member either if the block was
not part of the parent transaction.


# 0579a695648c072e5c6decb6d35bf2ac5168a37f 19-Jul-2012 Axel Dörfler <axeld@pinc-software.de>

Added a bit of documentation, minor cleanup.

* Documented the cached_block::transaction, and previous_transaction
members.


# 8959a4e0c37873f56f44ea1743902de0ec0e44d4 19-Jul-2012 Andreas Henriksson <sausageboy@gmail.com>

An unused dirty block would put again into the unused list.

* This could cause bug #8123. While this fixes the crash, one underlying
problem is still there, as dirty blocks should never leave a transaction.


# 0e88a887b4a9ecaaf1062078d9ca9bfca78fcf3a 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# 0e35d5d2e5ef3d288e056d60ef1b16dc399eaa0c 12-Dec-2011 John Scipione <jscipione@gmail.com>

Change instances of wether in comments to whether. No functional change intended.


# 44b9a93e467a1bd97aefeefc90acb90439317481 04-Jan-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Added intermediate (addr_t) casts before the cast of the parse_expression()
result (uint64) to pointer (CID 4723-4726).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40121 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4af303359b577f5c7e16e92423566aa6afb9812f 18-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* We need to reset the blockNumber, or else we'll discard the wrong blocks
which has rather desastrous consequences - thanks to Stippi for the heads
up :-)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38242 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c141a31f83dd7e7e47856d26b801e1532e8d70f2 17-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* Fixed CID 1447 which was an actual bug which completely voided the whole
discard mechanism eventually causing the conflicts between the file cache
and the block cache it intended to fix.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38232 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3426ce140439483c07d632d956065c4eba08f8fd 04-Aug-2010 Axel Dörfler <axeld@pinc-software.de>

* Thanks to Janito I had a closer look to cache_detach_sub_transaction(), and it
turned out it's pretty much broken. Not only did it potentially leak memory,
it would also potentially replace the original data with a NULL pointer,
making the current transaction non revertable. The code should now be much
clearer. This fixes bug #6378.
* Changed a few "if (... != NULL)" into ASSERTs, since the code should bail out
earlier in these cases already.
* Added a TODO comment to cache_start_sub_transaction() about its broken discard
handling. This can cause FS corruptions in case the parent transaction is ever
going to be aborted (which shouldn't happen in real life, though).
* Added a bit more and better comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37899 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bb486a72231e820d81b386c3d99670a778c4dd06 10-Jul-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_end_transaction(): Allow the hook parameter to be NULL.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1cafaecaf595342ef972c727904eaeb1454bd511 18-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Introduced block_cache::unused_block_count, which counts the elements of
block_cache::unused_blocks.
* block_cache::Allocate(): No longer removes unused blocks when in a low
resource state. That just removed too many blocks too quickly, when the
cache was actively used for writing, seriously affecting performance.
* block_cache::_LowMemoryHandler(): Compute the number of unused blocks to
remove depending on the total unused block number. This way we cull huge
block caches with lots of old blocks much quicker.

Treats part of #5816.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37170 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d3166e469cd337a6910f1cb1515b74f73d36466f 29-Apr-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Replaced no longer necessary send_signal_etc() work-arounds for
resume_thread().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36531 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c561bf792b15cfa12f5166c0094d1c0d62d53f90 19-Mar-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

block_cache::NewBlock(): Allocate cached_block::current_data only when the
block was freshly allocated. A block returned by _GetUnusedBlock() already
has current_data and we would leak it before.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35918 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 47f23fb5c19d8c7a2d2adfa7ea48e8a1d3557c80 10-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* If get_cached_block() failed reading a block, it would leave a deleted object
in the unused list if it got the block via block_cache::_GetUnusedBlock().
* block_cache::_GetUnusedBlock() leaked the compare data block if the
BLOCK_CACHE_DEBUG_CHANGED feature had been enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35799 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6a7f219c8d45d3ef0137d92b87b219e3482102a5 03-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* block_notifier_and_writer() will now flush 1000 blocks when reaching the soft
memory limit.
* Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35735 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0181fe4b21c974a3e357c2209944923c111ccf9d 03-Mar-2010 Axel Dörfler <axeld@pinc-software.de>

* Replaced cached_block::accessed (the number of accesses) with last_accessed
(the time of the last access), as what we really want is a frequency/last
access time scoring, and accessed alone is useless for that.
* put_cached_block() no longer frees any unused blocks.
* The low memory handler will now only lock the cache if there is something
to do. Also, it did not take address space warnings into account.
* Even when memory is critical, we don't free all unused blocks anymore - if
the number of blocks we free now (10000) is not sufficient to get out of the
critical condition, chances are good that we will be called again :-)
* block_notifier_and_writer() now tries to make sure that the total block cache
memory consumption grows not much larger than half of the available RAM.
* This should all help to limit the block cache usage a bit better. Hopefully,
a checkfs run will no longer run out of memory here (couldn't test yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35733 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a282be40e4394c1ee7d59f3a19aa5de15fcc0a5e 27-Feb-2010 Michael Lotz <mmlr@mlotz.ch>

Re-lock the cache directly after the read. Otherwise the error case would
remove the block from an unlocked cache and the cache->lock would be unlocked
twice when the calling function unlocked. It panics in that case anyway, but
this should make it continuable and is more correct.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35648 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ff59ce680df5d2032ea5a11c666688269225f033 24-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* The low resource handler now empties the cache depot's magazines; before,
they were never freed unless the cache was destroyed (I just wondered why
my system would bury >1G in the magazines).
* Made the magazine capacity variable per cache, ie. for larger objects, it's
not a good idea to have 64*CPU buffers lying around in the worst case.
* Furthermore, the create_object_cache_etc()/object_depot_init() now have
arguments for the magazine capacity as well as the maximum number of full
unused magazines.
* By default, you might want to initialize both to zero, as then some hopefully
usable defaults are computed. Otherwise (the only current example is the
vm_page_mapping cache) you can just put in the values you'd want there.
The page mapping cache uses larger values, as its objects are usually
allocated and deleted in larger chunks.
* Beware, though, I couldn't test these changes yet as Qemu didn't like to run
today. I'll test these changes on another machine now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35601 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9d071e27f1a9836a33a560fb07795069dc503f45 21-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

This should finally nail #5415:
* Since the same block can be in up to two transactions, it's very well possible
that one shouldn't write all transactions in a single run.
* Forgot to pass on the iterator from BlockWriter::Add(transaction) to
Add(block).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35565 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c5e52ecb785a4d9427fe227d74303324b15c6eb0 17-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* cache_sync_transaction() now puts all blocks into the BlockWriter, and
therefore does not unlock anymore while iterating over the transactions.
This gave other threads the opportunity to finish a transaction, causing
bug #5412. Also, the BlockWriter will now always close transactions on its
own, and you need to pass the transaction hash iterator to Add().
* Also, transactions that contain blocks that are currently written back will
be ignored by the block writer, as well as cache_sync_transaction(). This
fixes bug #5415.
* Improved error handling if BlockWriter fails to write back blocks. Most
notably, they are no longer left busy_writing, and the functions calling
it do proper error reporting (besides block_cache_discard() that does not
return any erro code; I've added a TODO note there for now).
* The BlockWriter now starts with a larger array once it has to allocate one.
* One can now limit the number of blocks that go into a BlockWriter. This is
used by the block writer thread, that shouldn't always write back everything
every two seconds.
* Also, the fixed array is larger now (leaving enough space such that the
block writer/notifier does not need to allocate anything).
* And finally, if allocating the array fails, the BlockWriter falls back to the
synchronous write back used previously. IOW it will never write back less
blocks than you ask for.
* Added static BlockWriter::WriteBlock() method replacing write_cached_block().
* Forgot to rename block_cache::busy_count to busy_reading_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35510 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 66a1a16cbc6821cd9786fe6bd2de6a651bd3521f 16-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Made cache_end_transaction(), cache_detach_sub_transaction(), and
block_cache_discard() use the BlockWriter directly as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35496 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 319a2dee90b95e483b2abb3da0e7cf8cf98e32d8 15-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Improved KDL command output (ie. take busy_writing into account, too).
* Added new KDL command "cached_block" that dumps a cached block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35493 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 751e92fde17d937536dc51bf1d8d43f2837defa5 16-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* cache_sync_transaction() used an earlier signature of the BlockWriter
constructor, causing bug #5412.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35489 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4ea83177ffa7c69d8a4430480b7155c873609ab2 15-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Implemented a BlockWriter class that now performs writing back blocks.
* Renamed the "busy" stuff to "busy_reading", and added a "busy_writing"
concept.
* This now allows reading a block (and other blocks), while blocks are written
back. This should speed all operations needing to write back blocks, like
unzipping or compiling.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35464 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 22991e1d536d3972f50e168c7476d3b6fb7b3151 03-Feb-2010 Axel Dörfler <axeld@pinc-software.de>

* Use the is_writing flag to determine whether or not we may write back a block
that does not have a transaction.
* This should fix #5340.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35390 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e00b4c19219e1313d823378ba56798940db9b47d 28-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved handling of busy blocks into separate functions.
* Added flags to avoid notifying the busy condition variable unnecessarily.
* get_writable_cached_block(): Unlock the cache while memcpy()ing/memset()ing
the block's data. The idea is to reduce lock contention. Less effective
than I hoped, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35328 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 86c794e5c10f1b2d99d672d424a8637639c703dd 21-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
We allocate 8 MB areas whose pages we allocate and map when needed. An area is
divided into equally-sized chunks which form the basic units of allocation. We
have areas with three possible chunk sizes (small, medium, large), which is
basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
cache and object depot functions. E.g. object_depot_store() potentially wants
to allocate memory for a magazine. But also in pure freeing functions it
might eventually become useful to have those flags, since they could end up
deleting an area, which might not be allowable in all situations. We should
introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
areas, maintains a hash table for lookup, and maps chunks to object caches,
we can quickly find out which object cache a to be freed allocation belongs
to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
when really necessary, i.e. when the object cache for the respective
allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e7385fa68ef1818324cd935a6b4131824fc95d17 03-Jan-2010 Axel Dörfler <axeld@pinc-software.de>

* If a block allocation fails, remove an unused one and try again - this might
improve the reliability as long as our slab implementation is a PITA.
* Removed an assertion that will no longer work (due to the DoublyLinkedList
changes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34877 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 33c82a30c06cfd7ad3b1d932e2e95cee240afbeb 10-Dec-2009 Axel Dörfler <axeld@pinc-software.de>

* While reading in a block, the block is now marked busy, and the cache
unlocked, allowing for more parallel access.
* Writing is still done synchronously, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34611 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1db31d4dbddb1f0af6c41f8f12da95f1b4439d67 27-Nov-2009 Axel Dörfler <axeld@pinc-software.de>

* The slab and the block cache now both also react on address space shortages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34310 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a10ac91f6d88800d3ce5abaeec1e40aa73c9c792 05-Nov-2009 Axel Dörfler <axeld@pinc-software.de>

* NewBlock(), and Allocate() will now try to reuse existing memory in low
resource situations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33892 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 818ef0e5010e37fb97311bb5673c294f4842e2cb 13-Aug-2009 Artur Wyszynski <aljen-mlists@o2.pl>

Fixed wrong check


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32326 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0a5bfde2254f95f840ddacfed5179edd6b8baa41 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Added stack traced to the Abort tracing output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 61efcc328d10b157a300db5a41bcdd4a825d564d 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31869 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 71d2034366bcc99bd33829a2c00892638117e19b 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Enhanced cache_blocks_in_main_transaction() to also return the correct number
of blocks in case there is no sub-transaction at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31868 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 483acc43829c6f420b083a5d7f3c13bf4319a3cc 28-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* has_sub_transaction was never initialized, no clue why this didn't come up
earlier.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31845 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eb26e782f332b3a3e957f5868bd5d9af03d7f478 27-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Decoupled block_cache_used_memory() from the sCachesLock - this should fix the
UI freezes (ActivityMonitor and ProcessController both use get_system_info() a
lot), although this is only the symptom of another problem.
* The downside is that the block cache usage information isn't as up to date as
it was previously - it's updated by the block write/notifier thread now (worst
case every 2 seconds).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31812 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 813d9285bfc506c5aeb287af7412a8e81cccf3fe 25-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Fixed a number of problems of the "block_cache_data" command.
* Also added stack traces to each BlockData entry.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31763 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 008ec58d9fe2d794a0cfdd4fa3628c51d1ff6a3d 11-Jul-2009 Rene Gollent <anevilyak@gmail.com>

When building the block cache for UserlandFS's haiku emulation lib, don't include tracing since that's unavailable outside of the kernel. Fixes ticket #4092. Axel or Ingo, please review.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31525 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 591bc3f2d99168a9b2a9bdcbda20c47e97a4ff80 28-May-2009 Jérôme Duval <korli@users.berlios.de>

* various fixes identified by gcc4, provided by Joe Prostko


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30902 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d1cad268540b8d4ddfcf22f85daeba8ffcedadd5 10-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Added an insane block cache tracing mode that will also log the whole block
data. This is available when BLOCK_CACHE_BLOCK_TRACING is 2 or greater.
* Completely untested as of now, though. Will do so soon.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30104 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fd0803f3003a667986c914254b4735254cb4db4e 06-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Added RemoveTail() method.
* Renamed DoublyLinkedList::Size() to Count(), since it actually counts the
items (ie. O(n)).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29979 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 98092e9867e46f572bde18ad17188779d2b3d1d1 18-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

gcc 4 fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29602 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9bb78ee2966ebaddf44d75977e1d3dc9029586d8 26-Feb-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Removed the superfluous 'extern "C"'s.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29334 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c33667d400856680a8e0122300861eda77d1847a 01-Feb-2009 Michael Lotz <mmlr@mlotz.ch>

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8f1cb8fca251ebdc5dd36bfee9f3cc09880fe142 11-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Brought the fs_shell block_cache implementation up to date, and incorporate
the fixes applied to the kernel version of it (most notably a correctly
working block_cache_discard(), and cache_detach_sub_transaction()).
Also switched to the new notification functions, even though it still works
synchronously in the fs_shell.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28617 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fbc2cd83dc8357aa039a3a9ccf52cef89851b53d 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* wait_for_notifications() must make sure that the order of the notifications
in the cache is the same as in the condition variable. It now uses the
low-contention sCachesLock to accomplish this.
* Also added an ASSERT to make sure the notification is no longer part of the
list at this point.
* Improved KDL command output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28516 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a26045b7a60b033e8d39a8fdaba0204dd91d9363 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* cache_detach_sub_transaction(), and cache_abort_sub_transaction() now support
discarded blocks correctly as well.
* cache_detach_sub_transaction() left cached_block::original_data unchanged even
if the parent data was to become current (in case the sub transaction didn't
change the block yet). This could cause outdated blocks to be written back.
* cache_detach_sub_transaction() also set cached_block::previous_transaction
for all blocks, not just the ones with a previous transaction. This could
cause blocks to be written twice for no reason.
* cache_start_sub_transaction() did not change the num_blocks count for
discarded blocks.
* block_cache_discard() now panics if the block was already changed in the
current transaction.
* Improved test application, added more tests, revealing the above bugs in
cache_detach_sub_transaction().
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28514 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ddd20af422537594bbfa86b0b5b9b0c486249083 05-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed a potential race condition: when wait_for_notifications() is called
from the block notifier, the cache could be deleted before we have the chance
to lock it. We now lock the sCachesLock, and see if this cache is still valid.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28512 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 96e19c19fc7bb2abd80384657151b1d53f73fb44 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* The first functions now handle the "discard" flag correctly (namely
cache_end_transaction(), and cache_start_sub_transaction()).
* Further work on the test application, it's now actually usable, first test
passes.
* dump_block() did erroneously print 'B' for the dirty flag; now both dirty and
discard have the 'D' (3rd and 5th column).
* block_cache::LowMemoryHandler() is now private (and got an underscore prefix).
* Minor cleanup, shuffled some methods around.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28508 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f69f65ec960793d00c63b9edb79da9d295e81c7a 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Cleanup, no functional change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28497 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5b812019b48c620b3cdc4e15f59fe3f34b431a43 04-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 59dbd26f5f41a6c1272f6cac9c8cda4b19b79097 20-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 56ab6f077f05e9ce770401699705609adac7e036 01-Oct-2008 Axel Dörfler <axeld@pinc-software.de>

* I accidently broke get_next_locked_block_cache() in r27074; it would always
ignore every other cache (starting from the first).
* The consequence of this was that no blocks were written back automatically
for those caches, and their transactions were never idle, causing bug #2781.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27831 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f8bdc2443da0549194191804a80479ee7fd4531a 01-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Created a central place for putting kernel debug enabling macros.
Currently it only contains KDEBUG and the block cache debugging macros.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27816 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4bfeb6f37ab1a47a540efa4227686ed45985639b 20-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Changed the way how a cache is destructed: now, it is removed from the
global cache list before locking it. This allows to get rid of the
block_cache::deleting field, as well as simplifies some code.
* This also fixes a possible deadlock I recently introduced (on destruction,
the locking order was wrong).
* Now uses an anonymous condition variable instead.
* Moved the block_cache initialization code into a dedicated method that will
now also fail in case the low resource handler couldn't be registered (as
pointed out by Salvatore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27074 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a1f34e7e89d783b993e02ceda31ea11c18614b3b 19-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Only remove the block cache from the list, and its low resource handler when
this is actually needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27060 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1d136d5a76951b5b9a07617db2cbc37c799a3245 19-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed bug #2631 based on a patch by Salvatore: the block cache was put into
a global list before it was fully initialized.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27058 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c79e66abb7246bb9c992bd5ca44bb33ade013f82 05-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Added block_cache_used_memory() that returns the memory allocated by all
block caches for the block cache buffers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26836 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639708df9b4e2cc847b38d510149d19ec78 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2f14f5ee995380c9619c9c9c1a4385fd36750fd3 20-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

axeld + bonefish + mmlr:
* Added trace entry for block reads from disk.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26534 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d066dea36860490a670e4120e08966ae83406df 19-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added block level tracing to the block cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26521 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 45b959656ff8fecee52b5f7855c4052747646891 05-Jun-2008 Michael Lotz <mmlr@mlotz.ch>

Missed that file in the commit for r25812.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25816 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 91312c915dc6f9c2b08e5ae4b473d7283b537080 29-May-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Changed tracing output prefix from "cache" to "block cache".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25714 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 036cf4dd20cbfbd11631dcc62f75a02e709f53b6 11-May-2008 Jérôme Duval <korli@users.berlios.de>

block_cache_delete(): only lock the cache mutex and avoid MutexLocker. the mutex is destroyed in the destructor.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25467 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c1d860fb1fbff0fc2b500dd689cd5bbce676b0ca 12-May-2008 Jérôme Duval <korli@users.berlios.de>

block_cache_delete(): unlock the cache mutex before deleting the cache


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25463 a95241bf-73f2-0310-859d-f6bbb57e9c96


# adf376c941e3af8381aa890be77a2c6f66795639 08-May-2008 Axel Dörfler <axeld@pinc-software.de>

Replaced benaphores with mutexes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25378 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 76824051fd26cf11162c039ccc9e6629e3c906b4 29-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Only enable block/transaction tracing when the macro is 1.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25247 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2909d9dc269d3d7a8d55b814ed09743c98575747 24-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Factored a delete_notification() that is now used in three places.
* remove_transaction_listeners() was checking the events_pending field without
holding the sNotificatonLock - that should have been harmless (as we're in the
middle of deleting the transaction), but it now looks better.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25130 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6cef245eca821584f07f5a13558f51ec586852e8 22-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 36de598e0fb969d1d11824fb337c85006d0ead81 09-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* block_cache_sync[_etc]() now also waits for all notifications to be handled.
This fixes a potential problem where a volume would be left marked dirty on
a clean reboot as reported by Vasilis; no harm would have been done to the
data on the volume, though (it would just replay a log entry that it didn't
need to).
* Unified the naming of the notifier/writer thread: renamed the thread to "block
notifier/writer", and the variable holding the thread ID to
sNotifierWriterThread.
* Added a few more doxygen descriptions to the private functions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 391fdf1e99fd8bc0f5d3440d63f663ec4a3db5de 07-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* notify_transaction_listeners() would always remove all listeners when it
was asked to. Unfortunately, it also removed the TRANSACTION_WRITTEN
listeners (that were added before notifying other listeners), so that BFS
could never free up space in the log. Now, it will only remove the listeners
it may remove. This fixes bug #2024.
* Rearranged functions a bit, moved notification/listener related private
functions together.
* Added tracing entry for written transactions as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24853 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 98b7d71b7cd2205532e3da8d45c0a9d49ce7cfba 05-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

The wait_for_notifications() function now detects if it has been run from
within the notifier/writer thread, and will then flush the notifications
directly. This should fix #2008 again.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24806 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7686d00c6f69e4e069f55f541fb0dc4af82c2928 04-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* This should fix a deadlock as reported by bga.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24793 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 807d3aa8e370f76f30ee805d7f7f3c2f558b7aa2 04-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* If BFS's Journal::_WriteTransactionToLog() noticed there wasn't enough free
space left for the new log entry, it did call cache_sync_transaction(), and
then just assumed the space would be ready. But since the transaction could
have been written before that call by the block writer, and since the
_TransactionWritten() hook is now called asynchronously,
cache_sync_transaction() actually has to flush all pending TRANSACTION_WRITTEN
notifications before returning to the caller.
* To implement this, block_cache now publishs a condition variable, and
wait_for_notifications() adds a fake notification that signals that one.
Since the notifications are handled in FIFO order, this guarantees that
the previous TRANSACTION_WRITTEN hook is done.
* notify_transaction_listeners() could accidently delete notifications that
still had pending signals. Now, it will defer the deletion to the notification
thread instead in that case. This should fix bug #2008.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24792 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9a8722864238d3d76ad5966ec3a617a072211d58 03-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Removed the "transaction_changed" leftover member.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24769 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 816957bd749c0a8a2403ac689fc1fce1cbeafcf5 03-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Decoupled notifications from the block cache functions; they are now
called without having the cache's lock held.
* The only downside to this approach is that ending a transaction now needs
additional memory, and might therefore fail - that could be fixed by
allocating that memory upfront with the start of the new transaction,
though.
* Therefore, I reverted the cache lock to a benaphore, as the recursive
lock is no longer needed.
* The block writer thread is now a block writer and notifier thread: it will
wait for up to 2 seconds on the sEventSemaphore, and if nothing comes in,
it will continue with its previously sole job.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24768 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a5140cad6af9654d9f1c5d98685b642d576736bc 02-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_end_transaction() doesn't remove the transaction, so that the
check in notify_transaction_listeners() didn't hold, although the
cache_hooks were deleted anyway. I've introduced a listener_change
counter to detect changes to the listener list while iterating, since
any change makes the iteration unsafe. Should fix bug #2008 for real.
Nevertheless I thing the transaction listener concept needs revision.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24761 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 38c4c334879facb3737f5a8ae275a006d1a10fef 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* A notification can cause a transaction to be deleted - in this case, the
block_writer() as well as notify_transaction_listeners() must update their
data or quit. This fixes newly introduced bug #2008.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24749 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0aee496bd04d1084dab539c6bec45d476e9efa88 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* block_cache_private.h is no longer needed, moved its contents into
block_cache.cpp.
* Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24739 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 16d8ff2dad9a1753b5fccd655d2746702078285a 02-Apr-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 14438ad555dd193fffe0dfbef669778af2358a9b 27-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* write_cached_block() now moves a block to the unused list when it's
not referenced and clean (is no longer part of a transaction).
This finally gets memory consumption to a moderate level; even
unpacking a large archive will no longer eat all memory.
* The "block_cache" KDL command will now print the number of unused
blocks as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24606 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3bb1e6c264bfe2e87679d9ca13213a3aec1a25ff 26-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* The block_cache KDL command now sets some useful variables.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24589 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cd31b326a37bcab424c5f6b4bc187a5f0ace07a4 25-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* The "block_cache" command checked the wrong variable to see if a block
could be found or not, eventually resulting in a read fault.
* It now also uses parse_expression() for its second argument, since it
now returns an uint64 in Haiku.
* Tracing output cut off the label too early.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24579 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fae99f156ea9dafc4808c0b4a572535a65a89ce0 14-Mar-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

cached_block::Compare() was ignoring the upper 32 bit of the block
number. Was a problem only for partitions > 2^32 * block size (4TB
for 1KB blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24393 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6eabbacd7065eb177a232c59eff3dc5f0e59f480 22-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed compare_blocks() function; it actually gets cached_block**.
* Added block_writer thread that continuously writes back blocks for all current
block caches.
* A block cache now maintains the number of dirty blocks if it doesn't use
transactions. That knowledge could also be used in block_cache_sync()...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23698 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5af13da19d6957cf659ad0b863bb8dfc1706380a 21-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* The block cache now sorts the blocks before synchronizing a transaction.
* This should speed up writing a transaction considerably.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23694 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4a67038e563385e8fffd2d0ea76f200c47c772aa 21-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a new function hash_insert_grow() that grows the hash table when needed.
* Removed the public hash_grow() function again (at least for now, it's only
private).
* Removed the newSize argument from hash_grow(); it will compute the new size
automatically.
* The block cache is now using hash_insert_grow() instead of hash_insert()
which should make hash lookups much faster with some 10 thousand blocks,
also increased the initial table size from 32 to 1024...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23692 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f7a5d9c5831397de59dc37ad27f5b4cd17a35db1 19-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Changed TraceEntry::Dump() to take a TraceOutput& and removed
AbstractTraceEntry::AddDump(char*, size_t).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23655 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ab23dfa7f7621e7ca1e1bc8d99f0a1a55bc299ad 19-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Moved tracing macro definitions into separate tracing_config.h header.
So now there's a central place to enable tracing in general and for
individual components.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23645 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5f7a62cfdd3b26479e66e198945a3d144a11a698 19-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added debugger command "transaction" and added a new "-t" option to
"block_cache" to dump info about transactions.
* Improved tracing.
* Now uses the new add_debugger_command_etc().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23644 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5d0afa4e4e49f1537865d29b620e957fbcf97823 18-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e613499224741d426f9e4181738359aa8809dab3 16-Jan-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Gracefully ignore NULL pointers passed to block_cache::Free(). Fixes
panic when sync'ing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23575 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a96e7cce98aafea4f027cc3499e288b3545b8d00 16-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Replaced my block allocator code in favour of Hugo's slab allocator
(so we can't allocate more physical pages than virtual address space
anymore, but so what?).
* Used the new CACHE_LARGE_SLAB flag as a temporary work-around; else
the slab would easily create several thousands of areas, which our
area code (and kernel heap) can't really handle that well (gets
awfully slow).
* Block caches with the same size could share the same slab, but we
don't do that yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23567 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8264ebf6b756ff9a017a2aa6256550252ebd97d4 15-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Improved block allocation.
* Made the block_cache KDL command dump a bit more useful info (number of
referenced and dirty blocks).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23540 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 502c4640814adb9d62244d6b8b1a7968746cf50d 15-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added tracing support for transactions.
* improved the "block_cache" KDL command: it can now also dump blocks,
added support for the "--help" argument.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23532 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4fc4f2c8aeefd8548dc95f51af64beabc07a4ec5 13-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4afaeabbd858d959a57593f0ad52e4c481709495 02-Dec-2007 Axel Dörfler <axeld@pinc-software.de>

* bfs_free_cookie() accidently reverted a transaction after having updated
the inode's internal last modified and size copies, causing in inconsistent
data. This fixes #1643.
* The block being replayed are now dumped to the debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23050 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a25e1293b0d8522f389c4ca01e474327128a243 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

Implemented a part of block_cache_set_dirty() so that it can gracefully exit
in case there is nothing to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22318 a95241bf-73f2-0310-859d-f6bbb57e9c96


# fec9ab5d63ad7ff01d24db35917ce6c1e0413072 19-Jul-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

More informative output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21669 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 849161602978d6749194b080566fa42548874908 20-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Putting an unused block will now trigger a panic.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21481 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5de78b2cdbe376a79d6d8708616595eeab306138 18-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Added some temporary means to ease debugging block caches - at least the list
of all caches should be removed at a later point.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5b1cb74b86fe685ccb4c30804c677e73fb1a731f 27-Apr-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Even more correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20856 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 93ed453113c95548e9f5a1d2cb8e945ad5e80d71 27-Apr-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

More correct return value.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20855 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c03270bea6a91f10dd57d80df860cf307693231 24-Apr-2007 Axel Dörfler <axeld@pinc-software.de>

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3e414ec314690aac0d8b71f27f8681c8e4a90d50 28-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Fixed broken iteration in block_cache::RemoveUnusedBlocks(). The loop
would start with the first block in the unused blocks list, but then
continue with the blocks that share the same hash table slot, thus
freeing potentially used blocks. Could theoretically have caused
BFS to see and write incorrect meta/administrative data on certain
occasions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20459 a95241bf-73f2-0310-859d-f6bbb57e9c96


# de4145dbb63b53a8aa353b885b42e39b429bd3da 11-Mar-2007 Axel Dörfler <axeld@pinc-software.de>

Cleanup:
* NewBlock()/FreeBlock() are now symmetrical in that the former no longer inserts
the block into the hash table.
* delete_transaction() also no longer removes the transaction from the hash table.
* cache_transaction_sync() now uses the new hash_remove_current() function.
* minor other cleanup (like line breaks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20374 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e1555e1fdf435cda7477ae521b0953b12c623304 11-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

get_cached_block(): Remove the newly allocated block from the
hashtable in case of a read error (NewBlock() also adds the block,
but FreeBlock() only frees it).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20365 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 82029bdae870b0e257a54929116ddfb6da9efdca 16-Sep-2006 Marcus Overhagen <marcusoverhagen@gmail.com>

added missing nothrow parameter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18868 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9b906ccf04dba976f4267598ecf755841a866dd6 30-Aug-2006 Axel Dörfler <axeld@pinc-software.de>

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b66ae2c9c85031c9a2d91152af849e4975a3327b 25-Jul-2006 Oliver Tappe <zooey@hirschkaefer.de>

- moved dumpBlock() into debug.c, renamed it to dump_block() and added it
to kernel-exports.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18268 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 17372b761ec81de8a1846ec24da32dc118ea9584 08-Mar-2006 Axel Dörfler <axeld@pinc-software.de>

* If there are no free ranges left, the block cache will now reuse older
blocks - this is not enough, though as it would also need to ask other
volumes to free ranges.
* Increased the number of blocks to free in case of low memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16663 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 89d76e508d4810ca489915988c9640d8c88c17fa 27-Feb-2006 Marcus Overhagen <marcusoverhagen@gmail.com>

changed get_cached_block() to no longer use a reference argument
modified to panic when an invalid block is requested (to find fs errors)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16537 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1bb74eb185a131065851b1f6edbc4bc666fd15ef 26-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

This fixes the work-around made in r16105, and some more:
* cached_block::parent_data can be NULL in a sub transaction in case the
block wasn't part of the parent transaction (but not in low memory
situations). cache_abort_sub_transaction() and cache_detach_sub_transaction()
didn't account for this, though, ie. the block data could end up
corrupted.
* Renamed cached_block::original in original_data.
* Renamed cached_block::data in current_data.
* Added some comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16108 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2e5ac35292603c1bc9cb5e91a7acd9a0f54584d2 26-Jan-2006 Jérôme Duval <korli@users.berlios.de>

fixed some traces
cache_abort_sub_transaction now doesn't crash if parent_cache is NULL (happens in low memory situations)
Axel, please review and fix if needed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16105 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 627e4043a80073e553023e4e4b67f14f9dd511b6 23-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

Improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16042 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 37675ea5808670f3137d00b1131a9c8841b44f60 02-Jan-2006 Axel Dörfler <axeld@pinc-software.de>

Deleted ranges were never removed from the ranges hash... this could have finally
nailed down bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15791 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e075b5a32e8f8f83c3756b7b1fae058137b54d83 31-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Added some debug helper. Hopefully they give more insight to Stephan's problem (bug #77/#78).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15767 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 140dd512935a49cbf6d00db16341ecc2f8ed011f 30-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

cache_abort_transaction() could let cached_block::parent_data have the same value
as cached_block::data - which led to a crash as block_cache::FreeBlock() tried to
free both later.
Since neither cached_block::parent_data nor cached_block::original are supposed
to be != NULL in block_cache::FreeBlock(), they are no longer freed, but the system
panics if one of them is not NULL.
This should fix bug #77.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15749 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 529bf1c6d767f08bf5f84d80b132119441a67085 28-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

block_range::Delete() did not remove the block_range from the free list,
and thus, it could be used again which led to a crash.
Changed the free ranges list from a singly linked list to a doubly linked
list so that not all free ranges have to be searched for the one to be
freed anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15696 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3dcaf063c20b1f1008286e8e351fb54286e5deaa 28-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Cleaned up locking and fixed a related bug: put_cached_block() could have been called without holding the cache lock.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15695 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 89f5f72a150519ca74bb3a82a40cefc2954a6acf 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* Added debug output to vm_low_memory.cpp.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15568 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4c0f46e2a3a5a767317600e9ddf01042d21e6481 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

The block_cache::LowMemoryHandler() must check if it could acquire the cache's lock;
it might have been deleted in the mean time.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15565 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 182f88dda1eb7a10964f09863deebe13b68241da 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* block_cache_delete() now deletes the cache with with its lock held.
* the link were not initialized in cached_block, as its constructor were never called
(was using malloc/free instead of new/delete).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15564 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 005108a26235cd4a865a17bec484043ea0176d69 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Forget to remove debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15563 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 37bd67edb804460caf762801cf5efed5f38031f5 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Now also flushes dirty blocks correctly (cannot happen if transactions are used,
ie. never happens with BFS).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15562 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 41e866f3ae9f335c32dc7eb3843bcc8572e42f6d 16-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

The block cache now registers and uses a low memory handler that will flush
unused blocks when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15561 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ebdb9dfebfb9db7d46625c73ba8ca35185a3f1aa 15-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

Blocks are now put into an unused list when they aren't used.
When memory gets low, the blocks on that list are also freed - but right now,
that only happens when a block is put back (not directly on demand, ie. via
a low memory handler).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15552 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1c88c851adbc9991aa81284493335993d4a99504 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ca7c5a9e131cb2b1623f2bab9e3bfab65d9561cf 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 94752292dd938ad21a70dd74da2de788333aa34f 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

First go at an implementation of sub transactions - code is not yet tested, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14429 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e432cef2962c92aafd74bf07778786d661969dfc 25-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Updated a few more hash functions to be able to deal with negative numbers.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14067 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a4462f51c3eb6a5151fc75ffac7981aeeb18973 21-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14044 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 30616a3243d33ba2adc23f3fee59619ad10c3b51 21-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

block_cache::Free() now accepts NULL pointers (it was already used this way before...).
Unmounting BFS volumes should now work as expected.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14034 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2a1492f0fa731fb7858a3e81f9d85e51bacb2c7d 08-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Moved the definition of the BlockAddressPool class into block_allocator.cpp, as it's
not used outside of it.
Added tracking for the last transaction in the block cache; that way it can test
if the last transaction has been closed before opening a new one.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13022 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 19b77242acf774ecfe6d89c9d957599a29d1965e 02-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Some minor cleanup: removed unneeded includes, updated introducing comment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12932 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 317121e102fd959d4df579226afc549b598fe264 01-Jun-2005 Axel Dörfler <axeld@pinc-software.de>

Work in progress of the new block allocator for the block cache. It works, but
currently needs even more memory than the old one 8-)
Not cleaned up at all.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12921 a95241bf-73f2-0310-859d-f6bbb57e9c96


# f4e5c34143550174c8b36c5509e40b4350d10b42 30-May-2005 Axel Dörfler <axeld@pinc-software.de>

cache_abort_transaction() freed the wrong block, and thus causing a file system
corruption even better than during its absence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12903 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b3ef4fb41a62f2f65676f0cde7cec2ba811993e1 28-May-2005 Axel Dörfler <axeld@pinc-software.de>

Oops, forgot to implement cache_abort_transaction(); every failing transaction
could have destroyed the BFS integrity... (just happened to me, that's how I
noticed it)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12867 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d5b04e50f8eca76ce5e6690dc8873861fa978de0 13-May-2005 Axel Dörfler <axeld@pinc-software.de>

First step into a new block allocation strategy for the block_cache
(right now, it's still malloc/free, just encapsulated in an allocator class).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12662 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2d690920ac4d0cd27eb3c118fb2b0862615869e0 13-Apr-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96