History log of /haiku/src/system/kernel/cache/file_map.cpp
Revision Date Author Comments
# 425ac1b6 20-Jun-2023 Alexander von Gluck IV <kallisti5@unixzen.com>

refactor: Swap %Ld for %lld in all format usages

* %Ld is an undocumented alias for %lld in glibc.
* muslc doesn't implement it for this reason.
* While we will likely never drop %Ld support,
lets clean house and set a better example.

Change-Id: Id46dad3104abae483e80cc5c05d1464d3ecd8030
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6636
Reviewed-by: Adrien Destugues <pulkomandy@pulkomandy.tk>
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>


# 846b2f90 06-Jul-2012 Axel Dörfler <axeld@ibm.kiwi>

Changed the kernel's file_map.cpp to be usable from the fs_shell as well.

* This should reduce our maintenance burden a tiny bit :-)
* It also fixes a bug in the fs_shell, see hrev43395.


# 0e88a887 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# 01762bd5 03-Dec-2011 Michael Lotz <mmlr@mlotz.ch>

Fix possibly harmful use of stale pointer in edge case.

The call to _MakeSpace() may move the extent data from the indirect
array (kept in a heap allocation) to the direct one kept inside the
class. In that case the lastExtent pointer would become stale and
further use of it would've lead to suboptimal extents in the best case
to reading/writing at the wrong point in files and possibly corruption
of another allocation in the worst (both unlikely though).

To mitigate that we now re-initialize the pointer to the correct location
if we hit the cache limit.

Also made the use of the start variable more understandable. Instaed of
decrementing it (possibly wrapping) when an extent wasn't going to be
used and later adding the vector index again, just increment whenever
we actually move to the next extent.

For bad things to happen a few conditions needed to come together though:
1. There needed to be multiple vectors that could be combined with the
existing last extent.
2. There first needed to be more extents than the cache limit and that
number then had to decrease below the cache limit again.
3. The memory needed to stay intact after being freed up until after the
evaluation (or similar enough data had to be written to it).

At least the last one was guaranteed to not be true anymore since we
re-introduced overwritting freed memory with 0xdeadbeef in the slab,
therefore nastily hiding this. I'm not sure that the first condition is
ever met either (probably the vectors are combined beforehand so that
there never are multiple adjacent ones) at least for the normal use case
(the page writer writing back pages). I was at least unable to reproduce
an actual file corruption in my testing.

Just the out of bounds access to the stale pointer happened rather easily
though and is now at least fixed.


# e50cf876 02-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cfd22919 05-Aug-2009 Axel Dörfler <axeld@pinc-software.de>

* FileMap::Translate() did not choke on negative offsets, and _Cache() does not
necessarily return an error in this case.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32129 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9ca40777 03-Jun-2009 Axel Dörfler <axeld@pinc-software.de>

* Fixed a bug that let _InvalidateAfter() access already freed memory, and would
also mess up the file map, causing all sorts of file corruption.
* This fixes bug #3991 (and eventually some others).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30960 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e055b92a 08-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Added missing header.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29455 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c33667d4 01-Feb-2009 Michael Lotz <mmlr@mlotz.ch>

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 55ddbd7b 13-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* When a file system reports -1 offsets in its fs_get_file_map() function, this
will now accepted and regarded as sparse file data. FileMap::_Add() also
correctly joins multiple vecs with that offset together, FileMap::Translate()
will always report offset -1 even for offsets into that extent.
* read_file_io_vec_pages() (or rather, its backend common_file_io_vec_pages())
now supports sparse files, and will just clear the memory it should read from
offset -1 instead of passing a request to the vnode.
* ext2 now correctly reports sparse files. This should close bug #2889, as well
as #975.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28648 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 59dbd26f 20-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 46124337 30-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3c1a3047 04-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e700fc1d 20-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

axeld + bonefish + mmlr:
* FileMap::_InvalidateAfter():
- Off-by-one error. The last extend (which normally should be kept)
was thrown away, too, but still accessed afterwards. Worst case
could be a write to free()d memory.
- Drop the last extend when it would be truncated to zero size.
* FileMap::Translate():
- Incorrect handling of B_BUFFER_OVERFLOW case in the
vfs_get_file_map() loop. After the loop the function would return
incorrectly, making the caller think all vectors in the provided
array had been initialized correctly. This could cause a file system
implementation using the file map to read from or write to random
disk locations, in the latter case possibly corrupting the file
system.
- Some readability improvements in the final loop. Removed incorrect
check.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26523 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 36bf05ca 06-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a lock to FileMap - since the map is retrieved lazily, and file
systems like BFS only hold a read lock when reading the map, accessing the
file map wasn't safe.
* Made FileMap::Free() private, and renamed it to _Free().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26295 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 52ecda6e 06-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added some optional KDL command and statistics to the file map implementation.
* Turned them on for now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26291 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9dcc2929 07-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Made a class FileMap from struct file_map.
* Added two TODO comments to the start of the file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26287 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9b74b9cd 05-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

Vastly improved the file map implementation:
* adjacent vecs are now joined.
* partial invalidation no longer frees all cached extents.
* the array can now be larger than the needed number of entries, allowing
for a saner array allocation policy.
* it does no longer read the whole file map when the first translation is
requested, but only as much as required (it will still ask the file system
for the maximum file size, but it won't traverse further as long as the
initial request is fulfilled).
* This should help a lot with the ext2 file system that doesn't support real
file extents (but keeps a list of blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26260 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 609865fd 05-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Implemented binary search for the disk offsets.
* Moved the file_extent array into its own structure (unnamed union problem
with GCC4), and added max_count field (currently unused).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26259 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4accd841 06-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed a big bug in common_file_io_vec_pages(): vecOffset was not correctly
set when the first chunk of the file could be read in directly, causing it
to read data to a wrong place in the buffer.
* Reading in the first chunk directly would have also only worked if vecIndex
and vecOffset was 0 when calling the function.
* Applied the fs_shell changes in file_map to the kernel version as well (the
constructor already worked correctly, though).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23265 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ddd17699 13-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The file map code isn't that smart yet...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22916 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4a31d30e 13-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 637c2a85 11-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

A tiny bit more debug output when tracing is enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22902 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3d268eda 10-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c84c58e4 09-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

Extracted file_map API for later use.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22864 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 846b2f90f670827f5ddc6b22772ffb0038c70745 06-Jul-2012 Axel Dörfler <axeld@ibm.kiwi>

Changed the kernel's file_map.cpp to be usable from the fs_shell as well.

* This should reduce our maintenance burden a tiny bit :-)
* It also fixes a bug in the fs_shell, see hrev43395.


# 0e88a887b4a9ecaaf1062078d9ca9bfca78fcf3a 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# 01762bd57ff51c04b93cee907275d8e885d38519 03-Dec-2011 Michael Lotz <mmlr@mlotz.ch>

Fix possibly harmful use of stale pointer in edge case.

The call to _MakeSpace() may move the extent data from the indirect
array (kept in a heap allocation) to the direct one kept inside the
class. In that case the lastExtent pointer would become stale and
further use of it would've lead to suboptimal extents in the best case
to reading/writing at the wrong point in files and possibly corruption
of another allocation in the worst (both unlikely though).

To mitigate that we now re-initialize the pointer to the correct location
if we hit the cache limit.

Also made the use of the start variable more understandable. Instaed of
decrementing it (possibly wrapping) when an extent wasn't going to be
used and later adding the vector index again, just increment whenever
we actually move to the next extent.

For bad things to happen a few conditions needed to come together though:
1. There needed to be multiple vectors that could be combined with the
existing last extent.
2. There first needed to be more extents than the cache limit and that
number then had to decrease below the cache limit again.
3. The memory needed to stay intact after being freed up until after the
evaluation (or similar enough data had to be written to it).

At least the last one was guaranteed to not be true anymore since we
re-introduced overwritting freed memory with 0xdeadbeef in the slab,
therefore nastily hiding this. I'm not sure that the first condition is
ever met either (probably the vectors are combined beforehand so that
there never are multiple adjacent ones) at least for the normal use case
(the page writer writing back pages). I was at least unable to reproduce
an actual file corruption in my testing.

Just the out of bounds access to the stale pointer happened rather easily
though and is now at least fixed.


# e50cf8765be50a7454c9488db38b638cf90805af 02-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cfd22919f2d11a325c2c2a45a6eb97e36af4aef3 05-Aug-2009 Axel Dörfler <axeld@pinc-software.de>

* FileMap::Translate() did not choke on negative offsets, and _Cache() does not
necessarily return an error in this case.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32129 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9ca4077700dc170f847ef2887727ed0ed11e4434 03-Jun-2009 Axel Dörfler <axeld@pinc-software.de>

* Fixed a bug that let _InvalidateAfter() access already freed memory, and would
also mess up the file map, causing all sorts of file corruption.
* This fixes bug #3991 (and eventually some others).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30960 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e055b92aac1cbf2d803982ca3c228a38d4f7322c 08-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Added missing header.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29455 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c33667d400856680a8e0122300861eda77d1847a 01-Feb-2009 Michael Lotz <mmlr@mlotz.ch>

Fixing warnings under GCC4 in preparation to enable -Werror there as well:
* Replaced the use of offsetof() for structs that aren't PODs. Add a
offset_of_member() macro to util/khash.h because that's what it's used for
in our cases.
* Change the signature of add_debugger_command()/remove_debugger_command() on
GCC > 2 to avoid the depricated conversion from string constants to char *.
* Adding some "suggested" parenthesis. I know that not everyone likes that, but
it pointed out at least one bug that is fixed here as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29113 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 55ddbd7bca54878d861861e82c0c5a0d196c1b85 13-Nov-2008 Axel Dörfler <axeld@pinc-software.de>

* When a file system reports -1 offsets in its fs_get_file_map() function, this
will now accepted and regarded as sparse file data. FileMap::_Add() also
correctly joins multiple vecs with that offset together, FileMap::Translate()
will always report offset -1 even for offsets into that extent.
* read_file_io_vec_pages() (or rather, its backend common_file_io_vec_pages())
now supports sparse files, and will just clear the memory it should read from
offset -1 instead of passing a request to the vnode.
* ext2 now correctly reports sparse files. This should close bug #2889, as well
as #975.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28648 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 59dbd26f5f41a6c1272f6cac9c8cda4b19b79097 20-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved more debug macros to kernel_debug_config.h.
* Turned the checks for all those macros to "#if"s instead of "#ifdef"s.
* Introduced macro KDEBUG_LEVEL which serves as a master setting.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28248 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4612433715fc0477740693682ec4a642c1a4c6e1 30-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added parameter "size_t align" to file_map_translate(). If > 1, the
vector at the end of the file will be aligned to the given value.
* BFS uses an alignment of 512 bytes (should be block size of the
underlying device or BFS block size, whatever is less), which should
be fine, since file data are only stored in BFS blocks. This totally
avoids any partial operations at the I/O scheduler level, thus saving
disk operations. Not that I could measure any performance difference.
Theoretically it should help a lot though, particularly when dealing
with lots of small files, since we avoid using bounce buffers, which
are (a) limited in number and (b) require copying of the data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27246 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 04-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639708df9b4e2cc847b38d510149d19ec78 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e700fc1d5cf01984b8d974e5048b52da0b1a75d1 20-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

axeld + bonefish + mmlr:
* FileMap::_InvalidateAfter():
- Off-by-one error. The last extend (which normally should be kept)
was thrown away, too, but still accessed afterwards. Worst case
could be a write to free()d memory.
- Drop the last extend when it would be truncated to zero size.
* FileMap::Translate():
- Incorrect handling of B_BUFFER_OVERFLOW case in the
vfs_get_file_map() loop. After the loop the function would return
incorrectly, making the caller think all vectors in the provided
array had been initialized correctly. This could cause a file system
implementation using the file map to read from or write to random
disk locations, in the latter case possibly corrupting the file
system.
- Some readability improvements in the final loop. Removed incorrect
check.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26523 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 36bf05cac9e3cb1fa038b3341801fe675c729e41 06-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a lock to FileMap - since the map is retrieved lazily, and file
systems like BFS only hold a read lock when reading the map, accessing the
file map wasn't safe.
* Made FileMap::Free() private, and renamed it to _Free().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26295 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 52ecda6ecfe4c9710969c0ece73e267bb4702df9 06-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Added some optional KDL command and statistics to the file map implementation.
* Turned them on for now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26291 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9dcc2929fa3d3d9d7f7edb7a5d965a6a47ad5134 07-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Made a class FileMap from struct file_map.
* Added two TODO comments to the start of the file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26287 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9b74b9cda85a11073128d76d97a6491a0db46581 05-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

Vastly improved the file map implementation:
* adjacent vecs are now joined.
* partial invalidation no longer frees all cached extents.
* the array can now be larger than the needed number of entries, allowing
for a saner array allocation policy.
* it does no longer read the whole file map when the first translation is
requested, but only as much as required (it will still ask the file system
for the maximum file size, but it won't traverse further as long as the
initial request is fulfilled).
* This should help a lot with the ext2 file system that doesn't support real
file extents (but keeps a list of blocks).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26260 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 609865fd21580aa62622a0dc1f38ebebe3dcd2cc 05-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Implemented binary search for the disk offsets.
* Moved the file_extent array into its own structure (unnamed union problem
with GCC4), and added max_count field (currently unused).
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26259 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4accd841c7331a6674a4c5db4d9b2686d7eb8bac 06-Jan-2008 Axel Dörfler <axeld@pinc-software.de>

* Fixed a big bug in common_file_io_vec_pages(): vecOffset was not correctly
set when the first chunk of the file could be read in directly, causing it
to read data to a wrong place in the buffer.
* Reading in the first chunk directly would have also only worked if vecIndex
and vecOffset was 0 when calling the function.
* Applied the fs_shell changes in file_map to the kernel version as well (the
constructor already worked correctly, though).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23265 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ddd17699015f8374140444536fd807e2e2d577cb 13-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The file map code isn't that smart yet...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22916 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4a31d30e84d78b7d06b9efa6c1dd96c292262e4a 13-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* The file map needs to know the actual file size to be able to know if it has
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 637c2a855724afa75a3228879fdb7458dc7b6256 11-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

A tiny bit more debug output when tracing is enabled.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22902 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3d268eda3d0ca504c865533347decf27b54025b6 10-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c84c58e41a47a9b4d3735e845b0a7df702447508 09-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

Extracted file_map API for later use.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22864 a95241bf-73f2-0310-859d-f6bbb57e9c96