Bootloader: replace blacklist with blocklist

Various projects, both commercial and OSS, began to use inclusive
terminology. There is no reason to not do it.

In Haiku, bootloader uses Blacklist, which is recommended to replace
with Denylist or Blocklist. I think Blocklist is appropriate here,
since it's a list used to block offending driver at boot.

Some strings remain unchanged for compatibility with previous naming,
but this change prepares for later removal of these (once everyone has
updated their kernel and bootloader).

Change-Id: Id9105ff5e9fcb866000355089b5ef97bf63ee854
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3145
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

packagefs: Use object caches for the most used node classes.

Slab statistics from KDL show that on a stock nightly image
(i.e. no additional packages besides the standards installed)
there are 43848 *objects* (not bytes) in the PackageNodeAttribute
cache, and 25090 in the PackageFile cache, so this seems more than
worth it.

The last commit seems to reduce memory usage at boot by about 1%,
this commit seems to not affect it at all; but it is a significant
performance optimization and on systems with more packages installed
the effect may be very noticeable.

Change-Id: I676a642ed6003f82b14396e1f02684575d899362

packagefs: Open HPKG files as O_NOCACHE.

We have not one but two internal caches in packagefs for *uncompressed* data;
we do not need the underlying filesystem to cache the *compressed* data
needlessly.

This seems to be a negligible memory and boot speed improvement at best,
although it will likely help in low-memory situations in reducing
the pressure on the file cache (... actually it doesn't look like
the file cache responds to low resource notifications at all, which
may be relevant here.)

packagefs: Remove support for HPKGv1.

The current version is "v2", and it has been since the final PM branch
was merged back in 2013, so it's unclear if anyone actually uses this
file format. The "package" command can still of course read these
files, it's just that packagefs will now not be able to.

It's very unlikely anyone is impacted by this change.

Package kit: add likely-useful architectures

Required for bootstrapping them.

Change-Id: I179d8c73b08688946049bf18ff9151ef047e449b
Reviewed-on: https://review.haiku-os.org/c/1071
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

packagefs: system_packages take precedence over non-system packages.

Affects *all* nodes, i.e., both directories and files. It is still possible
to override these, but it must be done in another system_package, not just by
mounting a package with newer datestamps on the conflicting files.

This is the pure version of axeld's proposal 2(d) from #10071. However,
as the haiku_secondary packages contain application mime_db entires and
are also 'system_package's, it does not fully solve that ticket.

packagefs: Package::Open(): Also log error code

Switch package file accessor classes to use BPositionIO

* PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and
their implementation and super classes do now internally use a
BPositionIO instead of a FD to access the package file. This provides
more flexibility needed for features to come.
* BPackageReader has already grown a new Init() version with a
BPositionIO* parameter.

Remove BPackageKit::BDataOutput

Use BDataIO instead.

Revert "Move ZlibDecompressor to libshared"

This reverts commit 9af2105d36856be60528c361edf14a263f242338.

Conflicts:
src/kits/package/Jamfile

packagefs: Package: Use PackagesDirectory directly

... instead of Volume. Necessary to support using packages from
different directories.

Move ZlibDecompressor to libshared

* This will be used to implement compressed http streams
* Remove the custom BDataOutput class, and use BDataIO instead, for
easier integration with existing code.

packagefs: Support blacklisting entries in packages

In each installation location, it is now possible to create a settings
file "packages" that allows to blacklist entries contained in packages.
The format is:

Package <package name> {
EntryBlacklist {
<entry path>
...
}
}
...

<package name> is the base name (no version) of the respective package
(e.g. "haiku"), <entry path> is an installation location relative path
(e.g. "add-ons/Translators/FooTranslator").

Blacklisted entries will be ignored by packagefs, i.e. they won't appear
in the file system. This addresses the issue that it may be necessary to
remove a problematic file (e.g. driver, add-on, or library), which would
otherwise require editing the containing package file.

The settings file is not not "live". Changes take effect only after
reboot (respectively when remounting the concerned packagefs volume).

packagefs: Package: Add versionedName property

Add missing architectures for packages

The list in packagefs wasn't in sync anyway.

packagefs: CachingPackageReader: Don't delete cached heap reader

... in destructor. After we create it, it belongs to the super class and
is destroyed by it.

Add "source" package architecture

packagefs: Add caching for the package file heap reader

* ReaderImplBase:
- Add virtual CreateCachedHeapReader() which can create a cached
reader based on the given heap reader.
- Rename HeapReader() to RawHeapReader() and add HeapReader() for the
cached heap reader.
- Add DetachHeapReader() to allow a clients to remove the heap
reader(s) after deleting the ReaderImplBase object.
* packagefs:
- Add CachedDataReader class, which wraps a given
BAbstractBufferedDataReader and provides caching for it using a
VMCache. The implementation is based on the IOCache implementation.
- Use CachedDataReader to wrap the heap reader. For file data that
means they are cached twice -- in the heap reader cache and in the
file cache -- but due to the heap reader using a VMCache as well,
the pages will be recycled automatically anyway. For attribute data
the cache should be very helpful, since they weren't cached at all
before.

package: Suppress version mismatch errors where V1 is supported

* Add flags parameter to Init() of BPackageReader and friends.
* Introduce flag B_HPKG_READER_DONT_PRINT_VERSION_MISMATCH_MESSAGE and
don't print a version mismatch error when given.
* package extract/list: Use the new flag.

packagefs: Resolve error output TODOs

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.

packagefs: Add support for hpkg format V1

* Add union-like class PackageData which wraps the V1 and V2
BPackageData classes.
* GlobalFactory: Create a data reader depending on the data format
version.
* Package: Add a loader for V1 format and try that, if the other one
fails.

packagefs: Share string instances globally

Introduce a class String which refers to shared private data that is
registered in a global hash table (in class StringPool) and use the
class consequently. This eliminates duplicate allocations for the same
string and also speeds up tests for equality. There's quite a bit
overhead for the management structures (apparently even more than for
the string data itself), but due to almost all strings being used
multiple times this still almost halves the memory usage for string
data.

packagefs: Organize sources into subdirectories

packagefs: Package::Open(): Also log error code

Switch package file accessor classes to use BPositionIO

* PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and
their implementation and super classes do now internally use a
BPositionIO instead of a FD to access the package file. This provides
more flexibility needed for features to come.
* BPackageReader has already grown a new Init() version with a
BPositionIO* parameter.

Remove BPackageKit::BDataOutput

Use BDataIO instead.

Revert "Move ZlibDecompressor to libshared"

This reverts commit 9af2105d36856be60528c361edf14a263f242338.

Conflicts:
src/kits/package/Jamfile

packagefs: Package: Use PackagesDirectory directly

... instead of Volume. Necessary to support using packages from
different directories.

Move ZlibDecompressor to libshared

* This will be used to implement compressed http streams
* Remove the custom BDataOutput class, and use BDataIO instead, for
easier integration with existing code.

packagefs: Support blacklisting entries in packages

In each installation location, it is now possible to create a settings
file "packages" that allows to blacklist entries contained in packages.
The format is:

Package <package name> {
EntryBlacklist {
<entry path>
...
}
}
...

<package name> is the base name (no version) of the respective package
(e.g. "haiku"), <entry path> is an installation location relative path
(e.g. "add-ons/Translators/FooTranslator").

Blacklisted entries will be ignored by packagefs, i.e. they won't appear
in the file system. This addresses the issue that it may be necessary to
remove a problematic file (e.g. driver, add-on, or library), which would
otherwise require editing the containing package file.

The settings file is not not "live". Changes take effect only after
reboot (respectively when remounting the concerned packagefs volume).

packagefs: Package: Add versionedName property

Add missing architectures for packages

The list in packagefs wasn't in sync anyway.

packagefs: CachingPackageReader: Don't delete cached heap reader

... in destructor. After we create it, it belongs to the super class and
is destroyed by it.

Add "source" package architecture

packagefs: Add caching for the package file heap reader

* ReaderImplBase:
- Add virtual CreateCachedHeapReader() which can create a cached
reader based on the given heap reader.
- Rename HeapReader() to RawHeapReader() and add HeapReader() for the
cached heap reader.
- Add DetachHeapReader() to allow a clients to remove the heap
reader(s) after deleting the ReaderImplBase object.
* packagefs:
- Add CachedDataReader class, which wraps a given
BAbstractBufferedDataReader and provides caching for it using a
VMCache. The implementation is based on the IOCache implementation.
- Use CachedDataReader to wrap the heap reader. For file data that
means they are cached twice -- in the heap reader cache and in the
file cache -- but due to the heap reader using a VMCache as well,
the pages will be recycled automatically anyway. For attribute data
the cache should be very helpful, since they weren't cached at all
before.

package: Suppress version mismatch errors where V1 is supported

* Add flags parameter to Init() of BPackageReader and friends.
* Introduce flag B_HPKG_READER_DONT_PRINT_VERSION_MISMATCH_MESSAGE and
don't print a version mismatch error when given.
* package extract/list: Use the new flag.

packagefs: Resolve error output TODOs

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.

packagefs: Add support for hpkg format V1

* Add union-like class PackageData which wraps the V1 and V2
BPackageData classes.
* GlobalFactory: Create a data reader depending on the data format
version.
* Package: Add a loader for V1 format and try that, if the other one
fails.

packagefs: Share string instances globally

Introduce a class String which refers to shared private data that is
registered in a global hash table (in class StringPool) and use the
class consequently. This eliminates duplicate allocations for the same
string and also speeds up tests for equality. There's quite a bit
overhead for the management structures (apparently even more than for
the string data itself), but due to almost all strings being used
multiple times this still almost halves the memory usage for string
data.

packagefs: Organize sources into subdirectories