ext2: check volume state when mounting

we deny mounting read/write when not clean or with errors.

Change-Id: Ie0188319618d9d8e7199b308c007fbaeacaed87c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5400
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Jérôme Duval <jerome.duval@gmail.com>

ext2: use st_blocks from the disk inode structure

* this lets cp from coreutils use lseek to find the data chunks in a
sparse file.
* fix endian conversion macro used for num_blocks_high

Change-Id: I221d6316002b1c491ae987aeef3f25e8721b5ab9
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5218
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>

ext2: fix metadata_csum for inodes

the checksum assumes the full inode block.

Change-Id: I60ef5d0b769750107b7e104709edff95beee8085
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4118
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

ext2: implements metadata_csum and 64bit features.

* Some PVS-Studio warnings are removed.
* fixes hardlink count in source and destination folders on rename.
* tested with fstorture and fsx (one job).
* add new definitions.
* import crc32c code from BSD.
* add some consistency checks at mount.
* DirectoryIterator::_AllocateBestEntryInBlock(), the direntry is checked for
consistency before using it, which should avoid crashing on bad data.
* DirectoryIterator::_SplitIndexedBlock(): the dotdot entry length should
include the dot and dotdot entries length.

Change-Id: I0f80d73b65b1ae6ddb2e746a6f85ef806f23dbb0
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1735
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

PVS V568: strange uses of sizeof

Change-Id: I4c4e0395f579cf3b5ec4db2c30bbe2dc2d866de7
Reviewed-on: https://review.haiku-os.org/c/1608
Reviewed-by: Stephan Aßmus <superstippi@gmx.de>

PVS V570: variables assigned to themselves

Change-Id: Ia6c4af6a18be28c59dd0e5f0ceab75602c538b10
Reviewed-on: https://review.haiku-os.org/c/1609
Reviewed-by: Stephan Aßmus <superstippi@gmx.de>

ext2: check some more values for ext2_super_block

Signed-off-by: Adrien Destugues <pulkomandy@pulkomandy.tk>

Fixes #13401.

ext2: access the parent variable once checked it's non null

* cleanup
* add some inode flags we don't use

ext2: access the parent variable once checked it's non null

* cleanup
* add some inode flags we don't use

ext2: fixes 64 bit warnings

extent max length is 0x8000, not 0xffff (we only support initialized extents).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40146 a95241bf-73f2-0310-859d-f6bbb57e9c96

* manages blockgroups unused_inodes when the feature is available.
* BitmapBlock::FindMarked/FindUnmarked() tried to find a free bit
at the end of a full bitmap. This fixes #7069.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40143 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added flex group, dirnlink and gdtchecksum features for ext4.
* I reused crc_table.cpp from the UDF filesystem and switched it to have the reversed algorithm,
then generated the table in CRCTable.cpp
* added a binary search for extent tree leaves.
* fixed a check in InodeAllocator::New().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40129 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added blockgroup Flags()/SetFlags() and flag definitions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39916 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added fsblock_t and fileblock_t definitions, used them when needed.
* validate fBlockShift in superblock.
* Volume::AllocateBlocks() now uses an absolute fsblock instead of a blockgroup related fsblock.
* AllocationBlockGroup now provides absolute fsblock values.
* added support for extents feature: ExtentStream class is the equivalent for DataStream class for extent operations.
The extent tree implementation is very basic, should work for normal growing/shrinking
operations, but not for sparse files. When enlarging a file and extent tree is full,
the root is moved in a new block and a new level is added on top. Extents can usually
be extended when adjacent blocks are allocated. Shrinking happens by removing leafs
one after another.
* removed empty IndexedDirectoryIterator.*


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* switched more uint32 to off_t for 64bit support
* added error messages in InodeAllocator, Inode
* if BlockAllocator can't initialize, don't fail completely but switch to readonly
* fixed a bug in FindNextMarked() for bitmaps with a length non multiple of 32
* Inode::FindBlock() now returns an optional block_run length, useful for get_file_map()
* added flag for Inode for extents


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39234 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added some support for 64bit feature: extended struct ext2_block_group, block number types changed from uint32 to off_t
* added error traces, asserts
* BitmapBlock::CheckUnmarked() and CheckMarked() computed a wrong remainingBits and mask


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39203 a95241bf-73f2-0310-859d-f6bbb57e9c96

added some definitions for readonly features
we already support the extra isize feature, so it's now added
removed unused Inode::AttributeBlockReadAt()
group_descriptor_size was previously a reserved field in the super block


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39125 a95241bf-73f2-0310-859d-f6bbb57e9c96

added large files feature support


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39120 a95241bf-73f2-0310-859d-f6bbb57e9c96

ext3 inodes whose size is at least 256 bytes can also contain attributes (like smalldata for bfs).
We now use the Attribute class from bfs (instead of AttributeIterator) to manage small data and block attributes, though it's still readonly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39036 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for extra inode attributes in volumes with an inodesize of 256 or more:
this brings nanoseconds for access, change and modification times, and also brings creation time.
* switched off some debug output
* HTreeEntryIterator: fCount can equal fLimit.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38964 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for huge_file feature, untested (need a >2TB file)
* actually checks for readonly features if mounting read-write
* don't use a journal when mounting readonly
* DataStream::_BlocksNeeded(): in case of double or triple indirects,
compute the additional blocks needed using the difference between the
old and new indirects blocks. This was resulting in a bad NumBlocks on an inode
* BitmapBlock: mark methods missed an iteration when the startingBit was zero.
Some blocks were then allocated two times (at most 32 for each allocation run).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38950 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch from Janito Ferreira Filho (aka jjvf): this is the result of his GSoC assignment "Implement ext2/3 Read and Write Support for Haiku".
* Tested and checked most features and fs operations, while passing successfully the Linux fsck. Though the implementation still needs more testing and is to be used with
caution, it's better in my mind to have the code committed now given the size of the patch.
* Code style isn't extensively checked but is mostly OK. Code review is welcome.
Some notes from Janito:
* Sparse files aren't supported and hard links aren't supported. Write attributes methods aren't activated nor tested.
* Journaling needs more testing to make sure it behaves in a compatible way to Ext3, and support for the different modes hasn't been implemented (due to the block
and file cache incompatibility). Correct revoke management is also lacking, as is proper management of the superblock state and copies and block group copies.
* The code is partly based and inspired by the BFS implementation. Author information might need to be fixed.

I'd like to congratulate and thank Janito for his hard work to bring the implementation to the current state. I hope he'll keep on maintaining it and become a regular
contributor/committer.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38573 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch from Janito Ferreira Filho with fixes by myself: Ext3 Indexed Directory Lookup (as part of GSOC).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37295 a95241bf-73f2-0310-859d-f6bbb57e9c96

Partial implementation for attribute support. Currently listattr successfully lists attributes on files.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36148 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Applied (changed) patch by Janito to update the superblock structure to its
latest version. Thanks!
* This closes #5704.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36142 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added the new incompatible features of ext4.
* Now correctly only check for the incompatible features we do understand,
not for the ones we don't. Ie. we won't mount ext4 partitions anymore that
are using extents.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29510 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a few problems reported by Marcus - thanks for proofreading!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26218 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Seems like GCC4 does not like declaring structures inside unnamed unions;
that looks like a compiler bug to me, though.
* Pulled struct data_stream out of the union (now called ext2_data_stream),
based on a patch by Maurice, thanks!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26214 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed wrong group block offset computation: the
ext2_super_block::first_data_block must be used; the superblock offset
doesn't matter if the block size is greater than 2KB.
* Fixed block access beyond the 4GB limit - we need to cast the block to
a 64 bit type in ext2_get_file_map() before shifting it.
* Tested with various larger ext3 disks, and everything seems to work fine.
* Added some optional debug output.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26213 a95241bf-73f2-0310-859d-f6bbb57e9c96

gcc4 build fix


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26204 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added support for files larger than 4GB - not yet tested, though.
* The stat::st_nlink value is now correctly reported.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26188 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a read-only ext2 file system.
* It's not yet complete, doesn't support some ext2 stuff (like files over 4 GB),
and might have some other bugs (I only tested it with a single 20 MB ext2
image).
* To have a read/write ext2 file system, it would probably make more sense to
port GNU sources (like ext2fs lib), and use that. But a small read-only ext2
file sytem doesn't hurt, I think, and I don't know if ext2fs lib would be
feasible for kernel use (porting the file system from Linux directly would
also be an alternative, but probably more work).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26187 a95241bf-73f2-0310-859d-f6bbb57e9c96

ext2: access the parent variable once checked it's non null

* cleanup
* add some inode flags we don't use

ext2: access the parent variable once checked it's non null

* cleanup
* add some inode flags we don't use

ext2: fixes 64 bit warnings

extent max length is 0x8000, not 0xffff (we only support initialized extents).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40146 a95241bf-73f2-0310-859d-f6bbb57e9c96

* manages blockgroups unused_inodes when the feature is available.
* BitmapBlock::FindMarked/FindUnmarked() tried to find a free bit
at the end of a full bitmap. This fixes #7069.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40143 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added flex group, dirnlink and gdtchecksum features for ext4.
* I reused crc_table.cpp from the UDF filesystem and switched it to have the reversed algorithm,
then generated the table in CRCTable.cpp
* added a binary search for extent tree leaves.
* fixed a check in InodeAllocator::New().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40129 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added blockgroup Flags()/SetFlags() and flag definitions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39916 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added fsblock_t and fileblock_t definitions, used them when needed.
* validate fBlockShift in superblock.
* Volume::AllocateBlocks() now uses an absolute fsblock instead of a blockgroup related fsblock.
* AllocationBlockGroup now provides absolute fsblock values.
* added support for extents feature: ExtentStream class is the equivalent for DataStream class for extent operations.
The extent tree implementation is very basic, should work for normal growing/shrinking
operations, but not for sparse files. When enlarging a file and extent tree is full,
the root is moved in a new block and a new level is added on top. Extents can usually
be extended when adjacent blocks are allocated. Shrinking happens by removing leafs
one after another.
* removed empty IndexedDirectoryIterator.*


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* switched more uint32 to off_t for 64bit support
* added error messages in InodeAllocator, Inode
* if BlockAllocator can't initialize, don't fail completely but switch to readonly
* fixed a bug in FindNextMarked() for bitmaps with a length non multiple of 32
* Inode::FindBlock() now returns an optional block_run length, useful for get_file_map()
* added flag for Inode for extents


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39234 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added some support for 64bit feature: extended struct ext2_block_group, block number types changed from uint32 to off_t
* added error traces, asserts
* BitmapBlock::CheckUnmarked() and CheckMarked() computed a wrong remainingBits and mask


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39203 a95241bf-73f2-0310-859d-f6bbb57e9c96

added some definitions for readonly features
we already support the extra isize feature, so it's now added
removed unused Inode::AttributeBlockReadAt()
group_descriptor_size was previously a reserved field in the super block


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39125 a95241bf-73f2-0310-859d-f6bbb57e9c96

added large files feature support


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39120 a95241bf-73f2-0310-859d-f6bbb57e9c96

ext3 inodes whose size is at least 256 bytes can also contain attributes (like smalldata for bfs).
We now use the Attribute class from bfs (instead of AttributeIterator) to manage small data and block attributes, though it's still readonly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39036 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for extra inode attributes in volumes with an inodesize of 256 or more:
this brings nanoseconds for access, change and modification times, and also brings creation time.
* switched off some debug output
* HTreeEntryIterator: fCount can equal fLimit.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38964 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for huge_file feature, untested (need a >2TB file)
* actually checks for readonly features if mounting read-write
* don't use a journal when mounting readonly
* DataStream::_BlocksNeeded(): in case of double or triple indirects,
compute the additional blocks needed using the difference between the
old and new indirects blocks. This was resulting in a bad NumBlocks on an inode
* BitmapBlock: mark methods missed an iteration when the startingBit was zero.
Some blocks were then allocated two times (at most 32 for each allocation run).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38950 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch from Janito Ferreira Filho (aka jjvf): this is the result of his GSoC assignment "Implement ext2/3 Read and Write Support for Haiku".
* Tested and checked most features and fs operations, while passing successfully the Linux fsck. Though the implementation still needs more testing and is to be used with
caution, it's better in my mind to have the code committed now given the size of the patch.
* Code style isn't extensively checked but is mostly OK. Code review is welcome.
Some notes from Janito:
* Sparse files aren't supported and hard links aren't supported. Write attributes methods aren't activated nor tested.
* Journaling needs more testing to make sure it behaves in a compatible way to Ext3, and support for the different modes hasn't been implemented (due to the block
and file cache incompatibility). Correct revoke management is also lacking, as is proper management of the superblock state and copies and block group copies.
* The code is partly based and inspired by the BFS implementation. Author information might need to be fixed.

I'd like to congratulate and thank Janito for his hard work to bring the implementation to the current state. I hope he'll keep on maintaining it and become a regular
contributor/committer.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38573 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch from Janito Ferreira Filho with fixes by myself: Ext3 Indexed Directory Lookup (as part of GSOC).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37295 a95241bf-73f2-0310-859d-f6bbb57e9c96

Partial implementation for attribute support. Currently listattr successfully lists attributes on files.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36148 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Applied (changed) patch by Janito to update the superblock structure to its
latest version. Thanks!
* This closes #5704.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36142 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added the new incompatible features of ext4.
* Now correctly only check for the incompatible features we do understand,
not for the ones we don't. Ie. we won't mount ext4 partitions anymore that
are using extents.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29510 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed a few problems reported by Marcus - thanks for proofreading!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26218 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Seems like GCC4 does not like declaring structures inside unnamed unions;
that looks like a compiler bug to me, though.
* Pulled struct data_stream out of the union (now called ext2_data_stream),
based on a patch by Maurice, thanks!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26214 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Fixed wrong group block offset computation: the
ext2_super_block::first_data_block must be used; the superblock offset
doesn't matter if the block size is greater than 2KB.
* Fixed block access beyond the 4GB limit - we need to cast the block to
a 64 bit type in ext2_get_file_map() before shifting it.
* Tested with various larger ext3 disks, and everything seems to work fine.
* Added some optional debug output.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26213 a95241bf-73f2-0310-859d-f6bbb57e9c96

gcc4 build fix


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26204 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added support for files larger than 4GB - not yet tested, though.
* The stat::st_nlink value is now correctly reported.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26188 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a read-only ext2 file system.
* It's not yet complete, doesn't support some ext2 stuff (like files over 4 GB),
and might have some other bugs (I only tested it with a single 20 MB ext2
image).
* To have a read/write ext2 file system, it would probably make more sense to
port GNU sources (like ext2fs lib), and use that. But a small read-only ext2
file sytem doesn't hurt, I think, and I don't know if ext2fs lib would be
feasible for kernel use (porting the file system from Linux directly would
also be an alternative, but probably more work).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26187 a95241bf-73f2-0310-859d-f6bbb57e9c96