block_cache: Change signature of *_etc() functions

* This allows file systems to retrieve the actual error code on a
failure, and report it to the user.
* All affected file systems have been adjusted to the API change.
This is a binary incompatible change.

Change-Id: Id73392aaf9c6cb7d643ff9adcb8bf80f3037874c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2913
Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

block_cache: Implement cache_has_block_in_transaction function that will check the existence of block in one specific transaction.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>

EntryCache: Add entry_cache_add_missing() for negative caching.

It provides a way for filesystems to cache a lookup failure and
therefore prevents repeated lookups of missing entries. This is a
common scenario for example in command lookup and compiling, where
each directory in PATH or each include directory is searched for the
given entry.

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a simple directory entry cache to speed up path resolution.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27366 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96

Looks like some bloke forgot to commit *some* files that should have been part
of r24768 (block cache notification mechanism rewrite).
Thanks for the note, Vasilis!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24772 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96

Beginnings of a new, better portable FS shell with Haiku FS interface.
Doesn't do anything ATM, but already provides the required system
interface (VFS, caches, POSIX functions).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20859 a95241bf-73f2-0310-859d-f6bbb57e9c96

EntryCache: Add entry_cache_add_missing() for negative caching.

It provides a way for filesystems to cache a lookup failure and
therefore prevents repeated lookups of missing entries. This is a
common scenario for example in command lookup and compiling, where
each directory in PATH or each include directory is searched for the
given entry.

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a simple directory entry cache to speed up path resolution.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27366 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96

Looks like some bloke forgot to commit *some* files that should have been part
of r24768 (block cache notification mechanism rewrite).
Thanks for the note, Vasilis!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24772 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96

Beginnings of a new, better portable FS shell with Haiku FS interface.
Doesn't do anything ATM, but already provides the required system
interface (VFS, caches, POSIX functions).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20859 a95241bf-73f2-0310-859d-f6bbb57e9c96