Searched +hist:3 +hist:d268eda (Results 1 - 22 of 22) sorted by relevance

/haiku/src/add-ons/kernel/file_systems/fat/
H A Ddosfs.hdiff 3f1684d9 Wed Jul 29 09:40:28 MDT 2009 Axel Dörfler <axeld@pinc-software.de> * Removed the superfluous "magic" stuff - if structures are broken, then please
let it crash!
* Made some headers self-contained.
* Always lock, regardless of "reenter" (we're using a recursive lock, so this
doesn't matter at all).
* Some cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31913 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3f1684d92cbcc9b678a06b88fba1743d3094efea Wed Jul 29 09:40:28 MDT 2009 Axel Dörfler <axeld@pinc-software.de> * Removed the superfluous "magic" stuff - if structures are broken, then please
let it crash!
* Made some headers self-contained.
* Always lock, regardless of "reenter" (we're using a recursive lock, so this
doesn't matter at all).
* Some cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31913 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/headers/private/fs_shell/
H A Dfssh_fs_cache.hdiff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dfssh_fs_interface.hdiff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cf43c26516b695e35b7433196a530f7541c4ca8 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dfssh_api_wrapper.hdiff 3e292446 Fri Jul 08 07:37:46 MDT 2011 Ingo Weinhold <ingo_weinhold@gmx.de> Move B_MIME_STRING_TYPE to <TypeConstants.h>
diff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e29244600395a0cea91cfc0a0323016fc0f3253 Fri Jul 08 07:37:46 MDT 2011 Ingo Weinhold <ingo_weinhold@gmx.de> Move B_MIME_STRING_TYPE to <TypeConstants.h>
diff 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/tools/fs_shell/
H A Dfile_cache.cppdiff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A DJamfilediff 3e617040 Sun Feb 17 06:42:27 MST 2008 Ingo Weinhold <ingo_weinhold@gmx.de> axeld + bonefish:
Added parameters --start-offset and --end-offset to restrict the
access of the file system to only that part of the given file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23970 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e617040425cd87f7af90804151ee8cb1f0ab8e9 Sun Feb 17 06:42:27 MST 2008 Ingo Weinhold <ingo_weinhold@gmx.de> axeld + bonefish:
Added parameters --start-offset and --end-offset to restrict the
access of the file system to only that part of the given file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23970 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dvfs.cppdiff 3f06b49a Sun Apr 16 14:58:19 MDT 2023 mbrumbelow <mbrumbelow@mac.com> fs_shell: cleaned up of a few unused variables

Change-Id: I58e91b634d8d32247c2f37a2518d4234397e8a6f
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6378
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Fredrik Holmqvist <fredrik.holmqvist@gmail.com>
Reviewed-by: Adrien Destugues <pulkomandy@pulkomandy.tk>
diff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff ea39c637 Sat Apr 05 21:13:11 MDT 2008 Ingo Weinhold <ingo_weinhold@gmx.de> Might help gcc 2.95.3 to resolve its perceived overloading of the
function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24824 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff e23824d7 Mon May 07 07:41:12 MDT 2007 Ingo Weinhold <ingo_weinhold@gmx.de> Surprising how much gcc 4 compilers on Linux differ. Another attempt to
make both gcc 2.95.3 and the plethora of gcc 4s happy: Typedefed
struct fssh_stat to fssh_struct_stat, thus hopefully avoiding spurious
errors concerning clashes with the function or regarding namespacing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21055 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 4b32170e Mon Apr 30 11:09:47 MDT 2007 Ingo Weinhold <ingo_weinhold@gmx.de> Work-around for an obscure gcc 2.95.3 bug: It complained about ambiguity
of fssh_stat struct and function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20932 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cf43c26516b695e35b7433196a530f7541c4ca8 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff ea39c6377e4d5cb97295e4c2374456ded3f6dea0 Sat Apr 05 21:13:11 MDT 2008 Ingo Weinhold <ingo_weinhold@gmx.de> Might help gcc 2.95.3 to resolve its perceived overloading of the
function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24824 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/add-ons/kernel/file_systems/bfs/
H A DDebug.hdiff 3d43a905 Sun Aug 30 15:58:47 MDT 2020 John Scipione <jscipione@gmail.com> bfs: Enable logging on DEBUG build

Fix DEBUG build errors by using B_PRI macros in BlockAllocator.cpp,
CheckVisitor.cpp, Inode.cpp, Journal.cpp, and kernel_interface.cpp

Add optional type parameter from BlockAllocator::IsValidBlockRun()
and actually PRINT it, type parameter is already optional on
BlockAllocator::CheckBlockRun().

Remove a couple of casts as we are printing the expected type, int32.

Cast mode_t to (unsigned int) and use %u because 32-bit Haiku didn't
like B_PRIu32.

Verified to compile on both regular and DEBUG builds on both 32-bit
and 64-bit.

Change-Id: I8bb39afd400768b7f69d36384974f0b91b3ef48c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3184
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: John Scipione <jscipione@gmail.com>
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A DInode.hdiff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A DInode.cppdiff 3d43a905 Sun Aug 30 15:58:47 MDT 2020 John Scipione <jscipione@gmail.com> bfs: Enable logging on DEBUG build

Fix DEBUG build errors by using B_PRI macros in BlockAllocator.cpp,
CheckVisitor.cpp, Inode.cpp, Journal.cpp, and kernel_interface.cpp

Add optional type parameter from BlockAllocator::IsValidBlockRun()
and actually PRINT it, type parameter is already optional on
BlockAllocator::CheckBlockRun().

Remove a couple of casts as we are printing the expected type, int32.

Cast mode_t to (unsigned int) and use %u because 32-bit Haiku didn't
like B_PRIu32.

Verified to compile on both regular and DEBUG builds on both 32-bit
and 64-bit.

Change-Id: I8bb39afd400768b7f69d36384974f0b91b3ef48c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3184
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: John Scipione <jscipione@gmail.com>
diff 3cd66dda Tue Jul 31 03:12:33 MDT 2012 Alex Smith <alex@alex-smith.me.uk> Fix CID 712419, 712420, 712421, 712422: Unintentional sign extension to 64 bits.
diff 3c50a646 Sun Jan 24 06:05:19 MST 2010 Axel Dörfler <axeld@pinc-software.de> * Inode::Create() should also better check if creating the file cache succeeded.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35273 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3eacb520 Tue Jan 27 02:54:47 MST 2004 Axel Dörfler <axeld@pinc-software.de> The InodeAllocator will now remove the INODE_NOT_READY flag to unblock
any pending bfs_read_vnode() calls when it is destructed and the inode
goes away.
Fixed some endian related issues.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@6350 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cd66dda5e623130a5ef7560da6851d421fd591e Tue Jul 31 03:12:33 MDT 2012 Alex Smith <alex@alex-smith.me.uk> Fix CID 712419, 712420, 712421, 712422: Unintentional sign extension to 64 bits.
diff 3c50a646979739e9151d14bf5d60df1be013fd02 Sun Jan 24 06:05:19 MST 2010 Axel Dörfler <axeld@pinc-software.de> * Inode::Create() should also better check if creating the file cache succeeded.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35273 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3eacb520e4990f6cd6c0f9dea5456efe59732305 Tue Jan 27 02:54:47 MST 2004 Axel Dörfler <axeld@pinc-software.de> The InodeAllocator will now remove the INODE_NOT_READY flag to unblock
any pending bfs_read_vnode() calls when it is destructed and the inode
goes away.
Fixed some endian related issues.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@6350 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dkernel_interface.cppdiff 3d43a905 Sun Aug 30 15:58:47 MDT 2020 John Scipione <jscipione@gmail.com> bfs: Enable logging on DEBUG build

Fix DEBUG build errors by using B_PRI macros in BlockAllocator.cpp,
CheckVisitor.cpp, Inode.cpp, Journal.cpp, and kernel_interface.cpp

Add optional type parameter from BlockAllocator::IsValidBlockRun()
and actually PRINT it, type parameter is already optional on
BlockAllocator::CheckBlockRun().

Remove a couple of casts as we are printing the expected type, int32.

Cast mode_t to (unsigned int) and use %u because 32-bit Haiku didn't
like B_PRIu32.

Verified to compile on both regular and DEBUG builds on both 32-bit
and 64-bit.

Change-Id: I8bb39afd400768b7f69d36384974f0b91b3ef48c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3184
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: John Scipione <jscipione@gmail.com>
diff 3cd66dda Tue Jul 31 03:12:33 MDT 2012 Alex Smith <alex@alex-smith.me.uk> Fix CID 712419, 712420, 712421, 712422: Unintentional sign extension to 64 bits.
diff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3daf03d3 Tue Apr 08 15:14:21 MDT 2008 Ingo Weinhold <ingo_weinhold@gmx.de> I broke the index dir access when changing the FS interface.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24868 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 5b25332a Sun Feb 01 18:50:00 MST 2004 Axel Dörfler <axeld@pinc-software.de> This should fix the recently introduced locking problems:
- locking policy is now: 1) the volume lock, 2) the inode lock, 3) start
the transaction (the journal lock) - you don't need all locks, but you
have to follow this order when you need more than one of them.
- in many cases, this makes our inode read/write lock useless. I am really
looking forward to the OpenBeOS VFS layer where we can remove many of
these locks again.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@6464 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b7911f1 Fri Jan 10 04:08:51 MST 2003 Axel Dörfler <axeld@pinc-software.de> We now check in bfs_rename() if the new name is the same as the old name
in the same directory - in this case, we will return B_OK (as nothing has
to be done to fulfill the request), instead of B_BAD_VALUE which would
be thrown later in the process.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@2400 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff ccc09203 Tue Sep 10 19:03:35 MDT 2002 Axel Dörfler <axeld@pinc-software.de> Added power management functions to support Dano (needed to boot Dano using
our file system):
- added bfs_wake_vnode(), and bfs_suspend_vnode()
- bumped B_CUR_FS_API_VERSION to 3 as needed for Dano

It should now be able to boot Dano with our BFS, although I haven't tested
it. The information was gained using some degree of reverse engineering...
It won't be part of the BFS implementation for OpenBeOS (that won't run on R5
anymore) :-)


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@1010 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/headers/os/drivers/
H A Dfs_cache.hdiff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dfs_interface.hdiff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cf43c26516b695e35b7433196a530f7541c4ca8 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/system/kernel/cache/
H A Dfile_map.cppdiff 01762bd5 Sat Dec 03 15:30:16 MST 2011 Michael Lotz <mmlr@mlotz.ch> Fix possibly harmful use of stale pointer in edge case.

The call to _MakeSpace() may move the extent data from the indirect
array (kept in a heap allocation) to the direct one kept inside the
class. In that case the lastExtent pointer would become stale and
further use of it would've lead to suboptimal extents in the best case
to reading/writing at the wrong point in files and possibly corruption
of another allocation in the worst (both unlikely though).

To mitigate that we now re-initialize the pointer to the correct location
if we hit the cache limit.

Also made the use of the start variable more understandable. Instaed of
decrementing it (possibly wrapping) when an extent wasn't going to be
used and later adding the vector index again, just increment whenever
we actually move to the next extent.

For bad things to happen a few conditions needed to come together though:
1. There needed to be multiple vectors that could be combined with the
existing last extent.
2. There first needed to be more extents than the cache limit and that
number then had to decrease below the cache limit again.
3. The memory needed to stay intact after being freed up until after the
evaluation (or similar enough data had to be written to it).

At least the last one was guaranteed to not be true anymore since we
re-introduced overwritting freed memory with 0xdeadbeef in the slab,
therefore nastily hiding this. I'm not sure that the first condition is
ever met either (probably the vectors are combined beforehand so that
there never are multiple adjacent ones) at least for the normal use case
(the page writer writing back pages). I was at least unable to reproduce
an actual file corruption in my testing.

Just the out of bounds access to the stale pointer happened rather easily
though and is now at least fixed.
diff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 01762bd57ff51c04b93cee907275d8e885d38519 Sat Dec 03 15:30:16 MST 2011 Michael Lotz <mmlr@mlotz.ch> Fix possibly harmful use of stale pointer in edge case.

The call to _MakeSpace() may move the extent data from the indirect
array (kept in a heap allocation) to the direct one kept inside the
class. In that case the lastExtent pointer would become stale and
further use of it would've lead to suboptimal extents in the best case
to reading/writing at the wrong point in files and possibly corruption
of another allocation in the worst (both unlikely though).

To mitigate that we now re-initialize the pointer to the correct location
if we hit the cache limit.

Also made the use of the start variable more understandable. Instaed of
decrementing it (possibly wrapping) when an extent wasn't going to be
used and later adding the vector index again, just increment whenever
we actually move to the next extent.

For bad things to happen a few conditions needed to come together though:
1. There needed to be multiple vectors that could be combined with the
existing last extent.
2. There first needed to be more extents than the cache limit and that
number then had to decrease below the cache limit again.
3. The memory needed to stay intact after being freed up until after the
evaluation (or similar enough data had to be written to it).

At least the last one was guaranteed to not be true anymore since we
re-introduced overwritting freed memory with 0xdeadbeef in the slab,
therefore nastily hiding this. I'm not sure that the first condition is
ever met either (probably the vectors are combined beforehand so that
there never are multiple adjacent ones) at least for the normal use case
(the page writer writing back pages). I was at least unable to reproduce
an actual file corruption in my testing.

Just the out of bounds access to the stale pointer happened rather easily
though and is now at least fixed.
diff 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dfile_cache.cppdiff 3cd20943 Wed Jan 06 19:37:05 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff d5ad7629 Thu Apr 30 09:53:41 MDT 2009 Axel Dörfler <axeld@pinc-software.de> Fixed several problems of the prefetching code:
* Did claim to have reserved pages when calling vm_page_allocate_page(), but
didn't have any (copy&paste bug). We cannot use it without reserved pages,
as we need to call vm_page_allocate_page() with a cache locked.
* No longer use low_resource_state() to determine whether to precache or not,
but use the new vm_page_num_used_pages() instead.
* Also don't (try to) precache when the cache already has more than 2/3 of its
pages to safe some unnecessary work.
* The size to precache was limited to the file size incorrectly.
* When precaching failed, the cache reference was not released.
* The precaching started one page too late, causing bug #3835.
* Reenabled precaching.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30515 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 47c40a10 Sun Oct 19 18:06:09 MDT 2008 Ingo Weinhold <ingo_weinhold@gmx.de> * Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c1a3047 Mon Aug 04 02:28:17 MDT 2008 Axel Dörfler <axeld@pinc-software.de> * Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff c6573329 Fri Oct 26 03:15:33 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * The file cache now remembers the last 3 accesses to be able to detect
sequential access.
* This is now used to let the cache free some pages (or schedule them to be
written back) before vm_reserve_pages() is called, but only if that cache
is not mapped, and there is a memory shortage.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22731 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cd2094396dde9ca42263c535041a95d5f0d5fff Wed Jan 06 19:37:05 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff d5ad762913429ae34478222a0c87199be0709439 Thu Apr 30 09:53:41 MDT 2009 Axel Dörfler <axeld@pinc-software.de> Fixed several problems of the prefetching code:
* Did claim to have reserved pages when calling vm_page_allocate_page(), but
didn't have any (copy&paste bug). We cannot use it without reserved pages,
as we need to call vm_page_allocate_page() with a cache locked.
* No longer use low_resource_state() to determine whether to precache or not,
but use the new vm_page_num_used_pages() instead.
* Also don't (try to) precache when the cache already has more than 2/3 of its
pages to safe some unnecessary work.
* The size to precache was limited to the file size incorrectly.
* When precaching failed, the cache reference was not released.
* The precaching started one page too late, causing bug #3835.
* Reenabled precaching.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30515 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/add-ons/kernel/file_systems/iso9660/
H A Dkernel_interface.cppdiff 3e015617 Fri Feb 13 17:49:41 MST 2009 Michael Lotz <mmlr@mlotz.ch> Request an overlay for the root vnode as well so attributes are available on the fs root as well.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29197 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b723f79 Tue Feb 28 10:30:47 MST 2006 Jérôme Duval <korli@users.berlios.de> added fs_get_vnode_name
cleanup of Jamfile
clean license


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16549 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e01561729cd577cd821d45f56d6f5457a60756d Fri Feb 13 17:49:41 MST 2009 Michael Lotz <mmlr@mlotz.ch> Request an overlay for the root vnode as well so attributes are available on the fs root as well.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29197 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b723f79140b51ec56913b1d7b8ccaa3a59b26f4 Tue Feb 28 10:30:47 MST 2006 Jérôme Duval <korli@users.berlios.de> added fs_get_vnode_name
cleanup of Jamfile
clean license


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16549 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/system/kernel/fs/
H A Drootfs.cppdiff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cf43c26516b695e35b7433196a530f7541c4ca8 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
H A Dvfs.cppdiff b4b1561b Fri Oct 28 20:57:45 MDT 2022 Augustin Cavalier <waddlesplash@gmail.com> kernel/vfs: Do not wait for removed vnodes to become unbusy.

Most of the time, that is harmless and will just cause a slight delay
before the vnode is removed and we return NULL instead of finding it.
However, in rare circumstances involving renames, we can wind up in
a deadlock with the thread that is trying to remove the vnode, and
would have to wait all the way to the timeout (a full ten seconds!).

The only vnodes not about to disappear from the table that can be both
"removed" and "busy" seem to be special vnodes like pipes, which will
be in an "unpublished" state while they are initially "busy" which we
can check for, in case something wants to wait for them.

The "dirconc" test readily triggered a pathological case of this behavior.
Before this commit, it ran for over 15 minutes before I killed it (and
it was not close to done at that point, either.) After this change,
it completes successfully in around 3 minutes or so on my test VM.

Thanks to mjg@freebsd.org for pointing out this testcase and its
misbehavior on Haiku!

Change-Id: Id1accf0aaf0724e1aec927a437d3a2ac1596cd98
diff 3ac5d980 Mon Jun 01 15:39:36 MDT 2020 Michael Lotz <mmlr@mlotz.ch> kernel/vfs: Fix array size off by one in _user_open_query.

The BStackOrHeapArray was allocated with one byte less than user_strlcpy
would write to, corrupting random heap memory. The error was introduced
as part of hrev53864.

Fixes #16175.

Change-Id: I80e1a1bb104183545fbd2a2702cdf3dec089908d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2860
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
diff 3f77fdd1 Mon Jan 21 22:47:07 MST 2019 Augustin Cavalier <waddlesplash@gmail.com> kernel/vfs: Fix -Werror=maybe-uninitialized.
diff 3cc71079 Mon Jun 20 16:43:30 MDT 2011 Ingo Weinhold <ingo_weinhold@gmx.de> Replaced fs_mount::covers_vnode by Vnode::covers

Introduce a Vnode::covers field. It is currently only used for the root
node of an fs_mount, replacing fs_mount::covers_vnode.
diff 3caec287 Fri Jan 01 13:20:11 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * Resolved TODO in free_vnode(): There was a race condition between vnode
destruction and VMVnodeCache::AcquireUnreferencedStoreRef(). Solved by
adding a flag to VMVnodeCache and letting AcquireUnreferencedStoreRef()
fail, if set.
* Added TODO regarding replacing the snooze() waiting for busy vnodes.
* get_vnode(): Unlock sVnodeMutex while calling the put_vnode() hook on
error.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34841 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 8ae594ff Tue Jul 28 18:34:44 MDT 2009 Axel Dörfler <axeld@pinc-software.de> * Introduced dedicated syscalls for the deprecated BeOS fs_attr API. Before,
each attribute access needed 3 syscalls, now only one as it should.
* Renamed the new Haiku call fs_open_attr() to fs_fopen_attr(), and added a new
function fs_open_attr() that takes a path (same semantics as the
fs_[f]open_attr_dir() functions already present in BeOS).
* Merged former _kern_open_attr(), and _kern_create_attr() into one syscall.
* Cleaned up vfs.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31881 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cf43c26 Wed Mar 25 18:16:34 MDT 2009 Ingo Weinhold <ingo_weinhold@gmx.de> Removed the fs_vnode_ops::create_dir() parameter for returning the ID of the
newly created dir. The VFS really doesn't need it and for some file systems
it might not be easy to get by. Several file systems (e.g. rootfs and fat)
were ignoring the parameter anyway.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29719 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b839d85 Sat Oct 18 12:22:33 MDT 2008 Michael Lotz <mmlr@mlotz.ch> The fs_mount structure now has a constructor/destructor and is used with new and
delete. Therefore the constructor/destructor of the contained EntryCache is also
called. The manual call of the destructor to the OpenHashTable inside the
EntryCache was therefore superflous and the destructor would be called twice,
leading to bug #2869.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28232 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff ef7fb431 Wed Jul 23 15:39:48 MDT 2008 Rene Gollent <anevilyak@gmail.com> Remove extraneous put_vnode(). This would result in decrementing the vnode ref count even if an unmount operation failed, which in turn would cause other fun problems. This fixes tickets 1982 and 2538. However, there seems to be another problem remaining with unmounting: the placeholder dir in the rootfs is not removed when the volume is unmounted, or it's not reused correctly. As a consequence, on subsequent remounts of the same volume via Tracker or mountvolume, a new dir is created each time, leaving empty placeholders. (i.e. remounting the volume HaikuData results in the creation of /HaikuData1, 2, 3, 4, etc.).



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26586 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cec3084 Fri May 30 13:06:30 MDT 2008 Michael Lotz <mmlr@mlotz.ch> Fix build with TRACE_VFS enabled and also fix a warning.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25726 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/add-ons/kernel/file_systems/cdda/
H A Dkernel_interface.cppdiff 3b8f7470 Thu Apr 28 12:27:12 MDT 2011 Axel Dörfler <axeld@pinc-software.de> Applied cleaned up patch originally by Ziusudra as part of ticket #3473:
* Changed the session module to create an extra session for existing audio
tracks.
* Let cdda also recognize "audio partitions" as published by the session module.
* If there is only a single session, the session module now gives file system
drivers the chance to play with the device directly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41288 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 84719cae Wed Aug 20 07:10:18 MDT 2008 Bruno G. Albuquerque <bga@bug-br.org.br> - Add a new CD:do_lookup attribute. It will be used by userland programs (like
cddblinkd) to know if they should lookup the CD or not. It will be true
unless:

1 - The CD has CD-Text information.
2 - The user (or a userland program) changed the Volume name.
3 - The user (or a userland program) changed any track names.

- Moved (again) attribute creation to before the stored attributes are read. As
attribute operations happen in memory in cdda, we don't need to care about
the IO cost and this way it is more in line with all the other attributes
created.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27079 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b8f74706019f7e51d52087a1ec7896630c892fb Thu Apr 28 12:27:12 MDT 2011 Axel Dörfler <axeld@pinc-software.de> Applied cleaned up patch originally by Ziusudra as part of ticket #3473:
* Changed the session module to create an extra session for existing audio
tracks.
* Let cdda also recognize "audio partitions" as published by the session module.
* If there is only a single session, the session module now gives file system
drivers the chance to play with the device directly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41288 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 84719cae244ef5d9448896bc84fa1c1dd2353aef Wed Aug 20 07:10:18 MDT 2008 Bruno G. Albuquerque <bga@bug-br.org.br> - Add a new CD:do_lookup attribute. It will be used by userland programs (like
cddblinkd) to know if they should lookup the CD or not. It will be true
unless:

1 - The CD has CD-Text information.
2 - The user (or a userland program) changed the Volume name.
3 - The user (or a userland program) changed any track names.

- Moved (again) attribute creation to before the stored attributes are read. As
attribute operations happen in memory in cdda, we don't need to care about
the IO cost and this way it is more in line with all the other attributes
created.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27079 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/headers/private/kernel/
H A Dvfs.hdiff 8ae594ff Tue Jul 28 18:34:44 MDT 2009 Axel Dörfler <axeld@pinc-software.de> * Introduced dedicated syscalls for the deprecated BeOS fs_attr API. Before,
each attribute access needed 3 syscalls, now only one as it should.
* Renamed the new Haiku call fs_open_attr() to fs_fopen_attr(), and added a new
function fs_open_attr() that takes a path (same semantics as the
fs_[f]open_attr_dir() functions already present in BeOS).
* Merged former _kern_open_attr(), and _kern_create_attr() into one syscall.
* Cleaned up vfs.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31881 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c6bf0c8 Wed Sep 25 10:36:38 MDT 2002 Axel Dörfler <axeld@pinc-software.de> Renamed "calls" to "ops".
Removed unused prototype.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@1179 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3aaee8e5 Sun Jul 28 12:36:23 MDT 2002 Axel Dörfler <axeld@pinc-software.de> Beginnings of the symbolic link support, though it's not yet working.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@507 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 8ae594ffd47d73edb1c9b9e1240a1ee667197472 Tue Jul 28 18:34:44 MDT 2009 Axel Dörfler <axeld@pinc-software.de> * Introduced dedicated syscalls for the deprecated BeOS fs_attr API. Before,
each attribute access needed 3 syscalls, now only one as it should.
* Renamed the new Haiku call fs_open_attr() to fs_fopen_attr(), and added a new
function fs_open_attr() that takes a path (same semantics as the
fs_[f]open_attr_dir() functions already present in BeOS).
* Merged former _kern_open_attr(), and _kern_create_attr() into one syscall.
* Cleaned up vfs.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31881 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda3d0ca504c865533347decf27b54025b6 Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3c6bf0c8c115c68b7cda915d44ed546ed7b3fc8f Wed Sep 25 10:36:38 MDT 2002 Axel Dörfler <axeld@pinc-software.de> Renamed "calls" to "ops".
Removed unused prototype.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@1179 a95241bf-73f2-0310-859d-f6bbb57e9c96
/haiku/src/system/kernel/vm/
H A Dvm_page.cppdiff 3dbd9c11 Wed Nov 16 02:29:33 MST 2011 Michael Lotz <mmlr@mlotz.ch> Fix the LIMIT_AVAILABLE_MEMORY debug option.

When limiting the available memory by reducing the page count it may not
be enough to just limit sNumPages. Depending on the physical memory map
non existing pages between ranges (sNonExistingPages) would still be
added up and later subtracted from the sNumPages, resulting in a wrong
max page count. Also due to the fixed removal of non existing page
ranges the actually available memory would usually not be the amount
set via LIMIT_AVAILABLE_MEMORY.

Instead we now calculate the available memory when going through the
physical memory ranges and limit/exit as soon as we've reached the
desired amount of available memory (also ignoring further non-existing
pages).
diff 3e5343f2 Sat Feb 20 09:45:44 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> free_cached_page(): Removed incorrect asserts. The same ones appear a few
lines later anyway, after the cache has been locked and it has been verified
that the page is still a candidate. Fixes #5432.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35539 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cd20943 Wed Jan 06 19:37:05 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 47c40a10 Sun Oct 19 18:06:09 MDT 2008 Ingo Weinhold <ingo_weinhold@gmx.de> * Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e5b9076 Wed Jan 09 15:25:21 MST 2008 Axel Dörfler <axeld@pinc-software.de> * Fixed (or rather, worked around) a deadlock in the VM: when a file was
resized but still had dirty pages to be written back,
vm_cache_resize() (which is called with the inode lock being held)
deadlocked with the page writer.
* Now, I reintroduced busy_writing: it'll be set by everything that
writes back pages (vm_page_write_modified(), and the page writer),
and will be checked for in vm_cache_resize() - other functions are not
affected for now, AFAICT.
* vm_cache_resize() will clear that flag, and the writer will check it
again after it wrote back the page (which will fail when it's outside
the file bounds), and if it's cleared, it will get rid of the page
(if the file has been resized again in the mean time, writing it will
succeed then, and we'll keep the page around).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23334 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d2595d1 Wed Jan 09 07:56:50 MST 2008 Axel Dörfler <axeld@pinc-software.de> The modified temporary page counter could go out of sync pretty easily.
This could cause the page writer to run endlessly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23309 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3d268eda Sat Nov 10 14:19:52 MST 2007 Axel Dörfler <axeld@pinc-software.de> * Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3e9513aa Wed Oct 03 18:45:15 MDT 2007 Axel Dörfler <axeld@pinc-software.de> * fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3dbd9c1148a996711d415e6750ec71d4abdb13f2 Wed Nov 16 02:29:33 MST 2011 Michael Lotz <mmlr@mlotz.ch> Fix the LIMIT_AVAILABLE_MEMORY debug option.

When limiting the available memory by reducing the page count it may not
be enough to just limit sNumPages. Depending on the physical memory map
non existing pages between ranges (sNonExistingPages) would still be
added up and later subtracted from the sNumPages, resulting in a wrong
max page count. Also due to the fixed removal of non existing page
ranges the actually available memory would usually not be the amount
set via LIMIT_AVAILABLE_MEMORY.

Instead we now calculate the available memory when going through the
physical memory ranges and limit/exit as soon as we've reached the
desired amount of available memory (also ignoring further non-existing
pages).
H A Dvm.cppdiff c650846d Tue Mar 14 15:42:25 MDT 2023 Augustin Cavalier <waddlesplash@gmail.com> vm: Replace the VMAreas OpenHashTable with an AVLTree.

Since we used a hash table with a fixed size (1024), collisions were
obviously inevitable, meaning that while insertions would always be
fast, lookups and deletions would take linear time to search the
linked-list for the area in question. For recently-created areas,
this would be fast; for less-recently-created areas, it would get
slower and slower and slower.

A particularly pathological case was the "mmap/24-1" test from the
Open POSIX Testsuite, which creates millions of areas until it hits
ENOMEM; it then simply exits, at which point it would run for minutes
and minutes in the kernel team deletion routines; how long I don't know,
as I rebooted before it finished.

This change fixes that problem, among others, at the cost of increased
area creation time, by using an AVL tree instead of a hash. For comparison,
mmap'ing 2 million areas with the "24-1" test before this change took
around 0m2.706s of real time, while afterwards it takes about 0m3.118s,
or around a 15% increase (1.152x).

On the other hand, the total test runtime for 2 million areas went from
around 2m11.050s to 0m4.035s, or around a 97% decrease (0.031x); in other
words, with this new code, it is *32 times faster.*

Area insertion will no longer be O(1), however, so the time increase
may go up with the number of areas present on the system; but if it's
only around 3 seconds to create 2 million areas, or about 1.56 us per area,
vs. 1.35 us before, I don't think that's worth worrying about.

My nonscientific "compile HaikuDepot with 2 cores in VM" benchmark
seems to be within the realm of "noise", anyway, with most results
both before and after this change coming in around 47s real time.

Change-Id: I230e17de4f80304d082152af83db8bd5abe7b831
diff c25f6f53 Tue Mar 29 18:09:36 MDT 2022 Augustin Cavalier <waddlesplash@gmail.com> kernel/vm: Completely replace mlock() implementation.

The old implementation used the real lock_memory(). This is problematic
and does not work for a large number of reasons:

1) Various parts of the kernel assume memory is locked only very
temporarily, and will often wait on locked memory to become unlocked.
The transient nature of locks is further demonstrated by the fact that
lock_memory acquires references to structures, like the address space,
which are only released by unlock_memory

2) The VM has a hard assumption that all lock_memory calls will be
exactly balanced, and maintains internal "WiredRange" structures
on areas, etc. corresponding to the original lock_memory calls.
Maintaining separate data structures as this code did is a recipe
for even more problems when the structures are manipulated separately,
leading to confusing or incorrect behavior on unlocks.

3) Areas with locked memory cannot be deleted, nor can the pages which are
locked be removed from the areas/caches. This of course is most notable
when destroying teams which locked memory, but the problem also occurs
when just using delete_area, resize_area, mmap/munmap, etc.

Because of (2) and especially (3), adding support for mlock()-like semantics
to the existing memory locking system is just not a good option. A further
reason is that our lock_memory is much stricter than mlock(), which only
demands the pages in question must remain resident in RAM and cannot be
swapped out (or, it seems, otherwise written back to disk.)

Thus, this commit completely removes the old implementation (which
was seriously broken and did not actually automatically unlock memory
on team exit or area destruction at all, etc.) and instead adds a new
feature to VMAnonymousCache to block certain pages from being written out.
The syscall then just invokes this to do its work.

Fixes #17674. Related to #13651.

Change-Id: Id2745c51796bcf9a74ba5325fe686a95623cd521
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5147
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
diff c25f6f53 Tue Mar 29 18:09:36 MDT 2022 Augustin Cavalier <waddlesplash@gmail.com> kernel/vm: Completely replace mlock() implementation.

The old implementation used the real lock_memory(). This is problematic
and does not work for a large number of reasons:

1) Various parts of the kernel assume memory is locked only very
temporarily, and will often wait on locked memory to become unlocked.
The transient nature of locks is further demonstrated by the fact that
lock_memory acquires references to structures, like the address space,
which are only released by unlock_memory

2) The VM has a hard assumption that all lock_memory calls will be
exactly balanced, and maintains internal "WiredRange" structures
on areas, etc. corresponding to the original lock_memory calls.
Maintaining separate data structures as this code did is a recipe
for even more problems when the structures are manipulated separately,
leading to confusing or incorrect behavior on unlocks.

3) Areas with locked memory cannot be deleted, nor can the pages which are
locked be removed from the areas/caches. This of course is most notable
when destroying teams which locked memory, but the problem also occurs
when just using delete_area, resize_area, mmap/munmap, etc.

Because of (2) and especially (3), adding support for mlock()-like semantics
to the existing memory locking system is just not a good option. A further
reason is that our lock_memory is much stricter than mlock(), which only
demands the pages in question must remain resident in RAM and cannot be
swapped out (or, it seems, otherwise written back to disk.)

Thus, this commit completely removes the old implementation (which
was seriously broken and did not actually automatically unlock memory
on team exit or area destruction at all, etc.) and instead adds a new
feature to VMAnonymousCache to block certain pages from being written out.
The syscall then just invokes this to do its work.

Fixes #17674. Related to #13651.

Change-Id: Id2745c51796bcf9a74ba5325fe686a95623cd521
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5147
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
diff 3a81e944 Thu Mar 10 15:40:15 MST 2022 Augustin Cavalier <waddlesplash@gmail.com> kernel/vm: Check the area's cache_type, not the cache's, in cut_area.

It seems this is the type of the innermost source cache, not the
topmost cache (which well may be "RAM" if this is an anonymous cache
on top of a Vnode cache.)

Originally VMArea::cache_type was introduced 15 years ago
(commit 0c12332715ae3cd6dedad4270d52ccf29d2cdcbf, 2007)
merely to mirror the underlying cache's type. Somewhere along the way,
it acquired different values. As the VM has changed a lot since then,
perhaps its overall purpose should be re-evaluated?

Fixes #17556.
diff 3b802628 Mon Jul 09 04:11:38 MDT 2012 Alex Smith <alex@alex-smith.me.uk> Support ELF64 in the kernel.

This has been done by adding typedefs in elf_common.h to the correct ELF
structures for the architecture, and changing all Elf32_* uses to those
types. I don't know whether image loading works as I cannot test it yet,
there may be some 64-bit safety issues around. However, symbol lookup for
the kernel is working correctly.
diff 3fb17998 Mon Jun 27 17:24:17 MDT 2011 Michael Lotz <mmlr@mlotz.ch> When resizing an area that has individual page protections (set via mprotect),
we have to enlarge/shrink the array that holds them and assign a protection
value for the additional pages as necessary. Otherwise we'll access invalid
memory when looking up page protections for enlarged areas and get random
protection values.
Experienced with QEMU that sets page protections via mprotect on heap memory.
When the heap was later enlarged, write access to the additional memory would
result in permission denied errors and crashes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42330 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b0c1b52 Sat May 01 14:41:57 MDT 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * VMArea: Made memory_type private and added setter and getter methods.
* Don't set the VMArea's memory type in arch_vm_set_memory_type(), but let the
callers do that.
* vm_set_area_memory_type(): Does nothing, if the memory type doesn't change.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36573 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3b8c056d Mon Jan 11 18:24:40 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> Should have been part of r35023: Introduction of the vm_page::modified flag.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35024 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff 3cd20943 Wed Jan 06 19:37:05 MST 2010 Ingo Weinhold <ingo_weinhold@gmx.de> * Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
diff d5ad7629 Thu Apr 30 09:53:41 MDT 2009 Axel Dörfler <axeld@pinc-software.de> Fixed several problems of the prefetching code:
* Did claim to have reserved pages when calling vm_page_allocate_page(), but
didn't have any (copy&paste bug). We cannot use it without reserved pages,
as we need to call vm_page_allocate_page() with a cache locked.
* No longer use low_resource_state() to determine whether to precache or not,
but use the new vm_page_num_used_pages() instead.
* Also don't (try to) precache when the cache already has more than 2/3 of its
pages to safe some unnecessary work.
* The size to precache was limited to the file size incorrectly.
* When precaching failed, the cache reference was not released.
* The precaching started one page too late, causing bug #3835.
* Reenabled precaching.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30515 a95241bf-73f2-0310-859d-f6bbb57e9c96

Completed in 1884 milliseconds