block_cache: Change signature of *_etc() functions

* This allows file systems to retrieve the actual error code on a
failure, and report it to the user.
* All affected file systems have been adjusted to the API change.
This is a binary incompatible change.

Change-Id: Id73392aaf9c6cb7d643ff9adcb8bf80f3037874c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2913
Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

block_cache: Implement cache_has_block_in_transaction function that will check the existence of block in one specific transaction.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>

EntryCache: Add entry_cache_add_missing() for negative caching.

It provides a way for filesystems to cache a lookup failure and
therefore prevents repeated lookups of missing entries. This is a
common scenario for example in command lookup and compiling, where
each directory in PATH or each include directory is searched for the
given entry.

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a simple directory entry cache to speed up path resolution.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27366 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96

Looks like some bloke forgot to commit *some* files that should have been part
of r24768 (block cache notification mechanism rewrite).
Thanks for the note, Vasilis!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24772 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96

Obviously forgot to commit these; they were part of the mount_id/vnode_id to dev_t/ino_t
conversion.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21568 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reformatted to honor 80 chars/line limit.
* Renamed a few parameters of the FS module hooks:
- *file in the attribute functions to *vnode
- v to vnode
- I could barely restrain myself from renaming the "_*" parameters. I
understand this marks return parameters, but I'd prefer a nicer prefix
or suffix (that doesn't makes you think this is a private/internal
identifier) like "out", "ret", "return", "result", or something similar.

I'd also like to propose renaming {read,write}_link() to
{read,write}_symlink(). Er, and do we need write_link() at all?


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20226 a95241bf-73f2-0310-859d-f6bbb57e9c96

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added new but empty function file_cache_invalidate_file_map().
This is needed in case the on-disk representation of a file changes (due to reorganization/defragmentation).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13660 a95241bf-73f2-0310-859d-f6bbb57e9c96

Some changes to the transaction API: the prefix is now only cache_*().
Added API for sub transactions (not yet implemented).
Added file_cache_sync() function.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@10047 a95241bf-73f2-0310-859d-f6bbb57e9c96

Minor API changes add additions.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@9935 a95241bf-73f2-0310-859d-f6bbb57e9c96

First draft of the new block cache API with transaction support.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@9893 a95241bf-73f2-0310-859d-f6bbb57e9c96

The last parameter of file_cache_create() is a file descriptor, not an open mode.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@8868 a95241bf-73f2-0310-859d-f6bbb57e9c96

Preliminary version of the file caching API - does not yet support journaling and such.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@8843 a95241bf-73f2-0310-859d-f6bbb57e9c96

EntryCache: Add entry_cache_add_missing() for negative caching.

It provides a way for filesystems to cache a lookup failure and
therefore prevents repeated lookups of missing entries. This is a
common scenario for example in command lookup and compiling, where
each directory in PATH or each include directory is searched for the
given entry.

* Added function block_cache_discard() that flushes blocks from the block cache,
discarding their changes. This functionality currently only works correctly
when no transactions are used.
* Started test application for the block cache, doesn't do anything yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28496 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented a simple directory entry cache to speed up path resolution.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27366 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96

Looks like some bloke forgot to commit *some* files that should have been part
of r24768 (block cache notification mechanism rewrite).
Thanks for the note, Vasilis!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24772 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a TRANSACTION_IDLE notification that is sent when the transaction
hasn't been used for more than 2 seconds.
* Replaced the block_cache::lock benaphore with a recursive lock, so that
you can call cache functions from within the notification listeners.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24737 a95241bf-73f2-0310-859d-f6bbb57e9c96

* cache_detach_sub_transaction() didn't really work: it did not put all needed
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a transaction listener mechanism to be notified when a
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96

Obviously forgot to commit these; they were part of the mount_id/vnode_id to dev_t/ino_t
conversion.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21568 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added block_cache_sync_etc() that allows you to sync single blocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20811 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reformatted to honor 80 chars/line limit.
* Renamed a few parameters of the FS module hooks:
- *file in the attribute functions to *vnode
- v to vnode
- I could barely restrain myself from renaming the "_*" parameters. I
understand this marks return parameters, but I'd prefer a nicer prefix
or suffix (that doesn't makes you think this is a private/internal
identifier) like "out", "ret", "return", "result", or something similar.

I'd also like to propose renaming {read,write}_link() to
{read,write}_symlink(). Er, and do we need write_link() at all?


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20226 a95241bf-73f2-0310-859d-f6bbb57e9c96

* You can now create the block cache in read-only mode (using an additional
parameter during construction).
* Doing so will now result in a kernel panic whenever your file system tries to
write to a block.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18719 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented cache_detach_sub_transaction() - not tested yet, though.
Added and implemented new functions cache_blocks_in_[sub_]transaction().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14431 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added new but empty function file_cache_invalidate_file_map().
This is needed in case the on-disk representation of a file changes (due to reorganization/defragmentation).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13660 a95241bf-73f2-0310-859d-f6bbb57e9c96

Some changes to the transaction API: the prefix is now only cache_*().
Added API for sub transactions (not yet implemented).
Added file_cache_sync() function.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@10047 a95241bf-73f2-0310-859d-f6bbb57e9c96

Minor API changes add additions.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@9935 a95241bf-73f2-0310-859d-f6bbb57e9c96

First draft of the new block cache API with transaction support.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@9893 a95241bf-73f2-0310-859d-f6bbb57e9c96

The last parameter of file_cache_create() is a file descriptor, not an open mode.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@8868 a95241bf-73f2-0310-859d-f6bbb57e9c96

Preliminary version of the file caching API - does not yet support journaling and such.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@8843 a95241bf-73f2-0310-859d-f6bbb57e9c96