Remove ZSTD_ENABLED checks from the Package Kit.

BZstdCompressionAlgorithm already has these #ifdefs and will
return error codes appropriately if built without libzstd,
so we do not need to check again inside the Package Kit.

This should not break the no-zstd build (well, it is broken
somewhat right now anyway, but this will not break it further),
and it simplifies logic somewhat.

package kit: Add pre-uninstall scripts feature.

Just like post-install scripts which run shortly after a package is
installed, pre-uninstall scripts are run just before a package is
removed. Implements enhancement #13427

* Fix script exit code handling vs script launch errors.
* Bump package and repo file version numbers due to new attribute,
unfortunately makes new .hpkg files not backwards compatible.
* Add pre-uninstall functionality, mostly cloning the post-install
except in a few places.
* Discover that _RunQueuedScripts() is never called, a future TODO:?
* Update package documentation for pre-uninstall scripts, and use of
the boot/post-install directory.

Change-Id: I45596255ce74bc102f6e5b606cbf83e4e4347a17
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1504
Reviewed-by: Alexander G. M. Smith <agmsmith@ncf.ca>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

package kit: Skip over future package attributes.

Ignore unknown fields (also called attributes) which are from a package
file with a different minor version number. Previously it would halt
with an error when encountering such a field, even though it can safely
be skipped over (if it was unsafe, we would have incremented the major
version number).

The use case is a future package attribute for pre-uninstall scripts.
If they're not run, that just leaves some debris after uninstalling
(like symbolic link desktop icons).

* Use the B_NOT_SUPPORTED error code when reading unknown package
attributes. Don't treat it as an error if the package is a
different minor version, just skip it.
* Print unknown package attribute index numbers rather than stopping,
since they may be from future package file formats and can be safely
skipped otherwise. Mention the relevant enum so you can find it in
the source code. It's a pity that the previous abstraction layer
isn't present, since it tells us what data type the attribute is
(string, number, etc), so we could have printed its value too.

First step of two for enhancement #13427

See https://review.haiku-os.org/c/haiku/+/1504 to generate packages
with a different minor version number (second step of the enhancement).

Change-Id: I6db1897824a1713b3d5fab6fdfb990ee5923cd52
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1714
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Remove a lot of unused constants.

Spotted by Clang. No functional change intended.

Package Kit: add Zstd compression.

Change-Id: Idbdb7cf1bde659046a88ea69a76e3b5fc4cd7013
Reviewed-on: https://review.haiku-os.org/323
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

package kit: Actually add support for B_HPKG_COMPRESSION_NONE

Until now we always declared in the HPKG header that the package file is
zlib compressed. For uncompressed files we would just store all
individual chunks uncompressed. Now we handle completely uncompressed
files slightly differently: We don't write the redundant chunk size
table anymore. The size savings are minor, but it makes the uncompressed
format read-streamable which may be handy.

Switch package file accessor classes to use BPositionIO

* PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and
their implementation and super classes do now internally use a
BPositionIO instead of a FD to access the package file. This provides
more flexibility needed for features to come.
* BPackageReader has already grown a new Init() version with a
BPositionIO* parameter.

Switch package kit to BZlibCompressionAlgorithm

... and remove the Zlib{Compressor,Decompressor} API.

Move package kit Zlib* classes to support kit

Also move to B* namespace and no longer expose the zlib dependency in
the headers.

package kit: ReaderImplBase: Remove superfluous scratch buffer

ReaderImplBase::ReadSection(): Small simplification

Remove BPackageKit::BDataOutput

Use BDataIO instead.

Revert "Move ZlibDecompressor to libshared"

This reverts commit 9af2105d36856be60528c361edf14a263f242338.

Conflicts:
src/kits/package/Jamfile

Move ZlibDecompressor to libshared

* This will be used to implement compressed http streams
* Remove the custom BDataOutput class, and use BDataIO instead, for
easier integration with existing code.

Separate notification from deletion in repository-handlers.

* Add NotifyDone() to all repository-attribute handlers and invoke that
to notify any listeners.
* Unify deletion to a single implementation of Delete() in the base
class. Before, the root handler for a repository didn't do that, but
just triggered the notification.

ReaderImplBase: Handle new ...IS_WRITABLE_DIRECTORY attribute

hpkg format: Generalize user/global settings file attributes

* For all identifiers: Rename global settings file to global writable
file. We want to use the respective attribute also for other writable
files, not only settings files.
* User settings file/global writable file info/attribute: Add
isDirectory property/child attribute. This allows declaring global/
user settings directories associated with the package.

hpkg format: Add attribute for declaring post install scripts

hpkg format: Add attributes for declaring users and groups

ReaderImplBase: Fix copy and paste oversight

ReaderImplBase: Verify that the attribute type matches

... the one expected for the respective attribute. Before it was
possible that e.g. a uint was read and then interpreted as a
const char*, if a string was expected for that attribute.

hpkg format: Add attributes for declaring settings files

Global and user settings files can be declared. For global ones an
update policy can be specified. If not specified, the settings file is
not included in the package, but created by the program (or user) later.
If an update type is specified, it defines what to do with the settings
file when updating the package to a newer version.

User settings files are never included in the package; they are always
created by the program or the user. If the package contains a template/
default settings file, it can be declared, but for informative purposes
only.

Package/repository file format: Add a minor version header field

* Add minor_version to hpkg_header and hpkg_repo_header and make
heap_compression uint16.
* If the minor version of a package/repository file is greater than the
current one unknown attributes are ignored without error. This allows
introducing new harmless attributes without making the resulting files
unreadable for older package kit versions.

packagefs: Add caching for the package file heap reader

* ReaderImplBase:
- Add virtual CreateCachedHeapReader() which can create a cached
reader based on the given heap reader.
- Rename HeapReader() to RawHeapReader() and add HeapReader() for the
cached heap reader.
- Add DetachHeapReader() to allow a clients to remove the heap
reader(s) after deleting the ReaderImplBase object.
* packagefs:
- Add CachedDataReader class, which wraps a given
BAbstractBufferedDataReader and provides caching for it using a
VMCache. The implementation is based on the IOCache implementation.
- Use CachedDataReader to wrap the heap reader. For file data that
means they are cached twice -- in the heap reader cache and in the
file cache -- but due to the heap reader using a VMCache as well,
the pages will be recycled automatically anyway. For attribute data
the cache should be very helpful, since they weren't cached at all
before.

ReaderImplBase: Fix strings subsection validity check

A strings section with 0 strings is still 1 byte long. Also improve the
error output in the error case.

ReaderImplBase: Fix gcc 4 build

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.

Add "base package" package attribute

Remove resolvable provides type

* Remove BPackageResolvableType and respective property from
BPackageResolvable.
* Remove package attribute B_HPKG_ATTRIBUTE_ID_PACKAGE_PROVIDES_TYPE.

package kit: some fixes for multi-version support

* Use enums/constants/functions instead of preprocessor macros.
* Missing include in PackageInfoAttributeValue.h.
* PackageReaderImpl::Init(): Check version before header size and
return B_MISMATCHED_VALUES instead of B_BAD_DATA, if the version
doesn't match. This allows callers to determine the condition and
try a reader for a different version. A more flexible interface for
that case would be nice, but since we want to support the old package
version only temporarily, the current solution should be good enough.

BPackageVersion: Rename property release to revision

* ... to avoid confusion with the preRelease property. It's also called
"revision" in the HaikuPorts recipes.
* Update libsolv package. Was necessary due to the BPackageVersion
change, but also includes a few more changes.

Add error output in case of invalid compression ID

Add a package info "install-path" attribute

The attribute is intended for simplifying package building. The
package's install path will be used for the package's .self package
symlink, allowing installation to a temporary directory when building
the package.

Add BLowLevelPackageContentHandler section hooks

* Add hooks HandleSectionStart() and HandleSectionEnd(). They are
invoked to bracket package file section, so the handler can
discriminate which section the attributes belong to.
HandleSectionStart() features a return parameter _handleSection, which
allows to handler to pick which sections it wants to handle.
* "package dump" does now print the section names.

BLowLevelPackageContentHandler interface extension

HandleAttributeDone(): Add parentToken parameter.

Support package info [source] URL attributes

Support for reading compatible version attribute

Introduce a pre-release version component

* The version string pattern is now:
<major>[.<minor>[.<micro>]][-<pre>][-<release>]
* Introduce B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_PRE_RELEASE package
attribute.
* Add "preRelease" field to BPackageVersionData.
* Add "preRelease" property to BPackageVersion and packagefs's Version.
* Adjust package reader and writer code accordingly.

ReaderImplBase::ParseStrings(): Avoid new[0]

Fixed host build issues/warnings

Add error output in case of invalid compression ID

Add a package info "install-path" attribute

The attribute is intended for simplifying package building. The
package's install path will be used for the package's .self package
symlink, allowing installation to a temporary directory when building
the package.

Add BLowLevelPackageContentHandler section hooks

* Add hooks HandleSectionStart() and HandleSectionEnd(). They are
invoked to bracket package file section, so the handler can
discriminate which section the attributes belong to.
HandleSectionStart() features a return parameter _handleSection, which
allows to handler to pick which sections it wants to handle.
* "package dump" does now print the section names.

BLowLevelPackageContentHandler interface extension

HandleAttributeDone(): Add parentToken parameter.

Support package info [source] URL attributes

Support for reading compatible version attribute

Introduce a pre-release version component

* The version string pattern is now:
<major>[.<minor>[.<micro>]][-<pre>][-<release>]
* Introduce B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_PRE_RELEASE package
attribute.
* Add "preRelease" field to BPackageVersionData.
* Add "preRelease" property to BPackageVersion and packagefs's Version.
* Adjust package reader and writer code accordingly.

ReaderImplBase::ParseStrings(): Avoid new[0]

Fixed host build issues/warnings

* implemented repository reader and fixed some bugs in writer that
have been exposed during testing of reader

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40494 a95241bf-73f2-0310-859d-f6bbb57e9c96

Pulled reusable stuff from PackageReaderImpl into new class ReaderImplBase.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40485 a95241bf-73f2-0310-859d-f6bbb57e9c96

package kit: Actually add support for B_HPKG_COMPRESSION_NONE

Until now we always declared in the HPKG header that the package file is
zlib compressed. For uncompressed files we would just store all
individual chunks uncompressed. Now we handle completely uncompressed
files slightly differently: We don't write the redundant chunk size
table anymore. The size savings are minor, but it makes the uncompressed
format read-streamable which may be handy.

Switch package file accessor classes to use BPositionIO

* PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and
their implementation and super classes do now internally use a
BPositionIO instead of a FD to access the package file. This provides
more flexibility needed for features to come.
* BPackageReader has already grown a new Init() version with a
BPositionIO* parameter.

Switch package kit to BZlibCompressionAlgorithm

... and remove the Zlib{Compressor,Decompressor} API.

Move package kit Zlib* classes to support kit

Also move to B* namespace and no longer expose the zlib dependency in
the headers.

package kit: ReaderImplBase: Remove superfluous scratch buffer

ReaderImplBase::ReadSection(): Small simplification

Remove BPackageKit::BDataOutput

Use BDataIO instead.

Revert "Move ZlibDecompressor to libshared"

This reverts commit 9af2105d36856be60528c361edf14a263f242338.

Conflicts:
src/kits/package/Jamfile

Move ZlibDecompressor to libshared

* This will be used to implement compressed http streams
* Remove the custom BDataOutput class, and use BDataIO instead, for
easier integration with existing code.

Separate notification from deletion in repository-handlers.

* Add NotifyDone() to all repository-attribute handlers and invoke that
to notify any listeners.
* Unify deletion to a single implementation of Delete() in the base
class. Before, the root handler for a repository didn't do that, but
just triggered the notification.

ReaderImplBase: Handle new ...IS_WRITABLE_DIRECTORY attribute

hpkg format: Generalize user/global settings file attributes

* For all identifiers: Rename global settings file to global writable
file. We want to use the respective attribute also for other writable
files, not only settings files.
* User settings file/global writable file info/attribute: Add
isDirectory property/child attribute. This allows declaring global/
user settings directories associated with the package.

hpkg format: Add attribute for declaring post install scripts

hpkg format: Add attributes for declaring users and groups

ReaderImplBase: Fix copy and paste oversight

ReaderImplBase: Verify that the attribute type matches

... the one expected for the respective attribute. Before it was
possible that e.g. a uint was read and then interpreted as a
const char*, if a string was expected for that attribute.

hpkg format: Add attributes for declaring settings files

Global and user settings files can be declared. For global ones an
update policy can be specified. If not specified, the settings file is
not included in the package, but created by the program (or user) later.
If an update type is specified, it defines what to do with the settings
file when updating the package to a newer version.

User settings files are never included in the package; they are always
created by the program or the user. If the package contains a template/
default settings file, it can be declared, but for informative purposes
only.

Package/repository file format: Add a minor version header field

* Add minor_version to hpkg_header and hpkg_repo_header and make
heap_compression uint16.
* If the minor version of a package/repository file is greater than the
current one unknown attributes are ignored without error. This allows
introducing new harmless attributes without making the resulting files
unreadable for older package kit versions.

packagefs: Add caching for the package file heap reader

* ReaderImplBase:
- Add virtual CreateCachedHeapReader() which can create a cached
reader based on the given heap reader.
- Rename HeapReader() to RawHeapReader() and add HeapReader() for the
cached heap reader.
- Add DetachHeapReader() to allow a clients to remove the heap
reader(s) after deleting the ReaderImplBase object.
* packagefs:
- Add CachedDataReader class, which wraps a given
BAbstractBufferedDataReader and provides caching for it using a
VMCache. The implementation is based on the IOCache implementation.
- Use CachedDataReader to wrap the heap reader. For file data that
means they are cached twice -- in the heap reader cache and in the
file cache -- but due to the heap reader using a VMCache as well,
the pages will be recycled automatically anyway. For attribute data
the cache should be very helpful, since they weren't cached at all
before.

ReaderImplBase: Fix strings subsection validity check

A strings section with 0 strings is still 1 byte long. Also improve the
error output in the error case.

ReaderImplBase: Fix gcc 4 build

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.

Add "base package" package attribute

Remove resolvable provides type

* Remove BPackageResolvableType and respective property from
BPackageResolvable.
* Remove package attribute B_HPKG_ATTRIBUTE_ID_PACKAGE_PROVIDES_TYPE.

package kit: some fixes for multi-version support

* Use enums/constants/functions instead of preprocessor macros.
* Missing include in PackageInfoAttributeValue.h.
* PackageReaderImpl::Init(): Check version before header size and
return B_MISMATCHED_VALUES instead of B_BAD_DATA, if the version
doesn't match. This allows callers to determine the condition and
try a reader for a different version. A more flexible interface for
that case would be nice, but since we want to support the old package
version only temporarily, the current solution should be good enough.

BPackageVersion: Rename property release to revision

* ... to avoid confusion with the preRelease property. It's also called
"revision" in the HaikuPorts recipes.
* Update libsolv package. Was necessary due to the BPackageVersion
change, but also includes a few more changes.

Add error output in case of invalid compression ID

Add a package info "install-path" attribute

The attribute is intended for simplifying package building. The
package's install path will be used for the package's .self package
symlink, allowing installation to a temporary directory when building
the package.

Add BLowLevelPackageContentHandler section hooks

* Add hooks HandleSectionStart() and HandleSectionEnd(). They are
invoked to bracket package file section, so the handler can
discriminate which section the attributes belong to.
HandleSectionStart() features a return parameter _handleSection, which
allows to handler to pick which sections it wants to handle.
* "package dump" does now print the section names.

BLowLevelPackageContentHandler interface extension

HandleAttributeDone(): Add parentToken parameter.

Support package info [source] URL attributes

Support for reading compatible version attribute

Introduce a pre-release version component

* The version string pattern is now:
<major>[.<minor>[.<micro>]][-<pre>][-<release>]
* Introduce B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_PRE_RELEASE package
attribute.
* Add "preRelease" field to BPackageVersionData.
* Add "preRelease" property to BPackageVersion and packagefs's Version.
* Adjust package reader and writer code accordingly.

ReaderImplBase::ParseStrings(): Avoid new[0]

Fixed host build issues/warnings

Add error output in case of invalid compression ID

Add a package info "install-path" attribute

The attribute is intended for simplifying package building. The
package's install path will be used for the package's .self package
symlink, allowing installation to a temporary directory when building
the package.

Add BLowLevelPackageContentHandler section hooks

* Add hooks HandleSectionStart() and HandleSectionEnd(). They are
invoked to bracket package file section, so the handler can
discriminate which section the attributes belong to.
HandleSectionStart() features a return parameter _handleSection, which
allows to handler to pick which sections it wants to handle.
* "package dump" does now print the section names.

BLowLevelPackageContentHandler interface extension

HandleAttributeDone(): Add parentToken parameter.

Support package info [source] URL attributes

Support for reading compatible version attribute

Introduce a pre-release version component

* The version string pattern is now:
<major>[.<minor>[.<micro>]][-<pre>][-<release>]
* Introduce B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_PRE_RELEASE package
attribute.
* Add "preRelease" field to BPackageVersionData.
* Add "preRelease" property to BPackageVersion and packagefs's Version.
* Adjust package reader and writer code accordingly.

ReaderImplBase::ParseStrings(): Avoid new[0]

Fixed host build issues/warnings

* implemented repository reader and fixed some bugs in writer that
have been exposed during testing of reader

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40494 a95241bf-73f2-0310-859d-f6bbb57e9c96

Pulled reusable stuff from PackageReaderImpl into new class ReaderImplBase.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40485 a95241bf-73f2-0310-859d-f6bbb57e9c96