PackageKit: HPKR BMessage Format Fix

Some older repositories are having problems because
they are configured with a `url` (identifier) form
that is not actually a well-formed URL. This caused
problems when it was then interpreted as the
base-url because it did not start with "http". I
have changed this so that the base-url is not
derived from the url and can be missing.

Resolves #16149

Change-Id: I10acd8db65082ff6c72fcff1550eb63475e86133
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2931
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Package Kit: reduce identifer/base-url confusion

For historical reasons, the package kit has an "url" field that is not
actually meant to be used as an URL. Rename it in the API and user
facing output as "identifier" to make it clear what the file is used
for. This change preserves the "url" key in on-disk and online storage
(hpkr files, stored settings, etc) in an attempt to not break anything.

Fix one remaining misuse of the "url" field as an URL in
get_package_dependencies.

Add an unit test showing that BUrl does parse "tab" URIs properly (there
is just a protocol and a path segment).

Change-Id: I339ce526e5798d42d78ae650855d7e988dbb4a1a
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2542
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Fix catching by value

Pointed out by LGTM.
Change-Id: I223655c728305f6f45b3738553da5b7d7a69e8c8
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2134
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

HaikuDepot / PackageKit: Repositories 'Identifier' URL

Repositories are identified with a 'url' in the
remote 'repo.info' file. There is also a
'base url' which is the URL locally with which
the system is able to access the repository
data on. There is some confusion between these
two terms in the source. This change aims to
separate the two out and consistently name them.
The settings for the repository locally also was
not storing these values and that has been fixed.
Debug info about the repositories also did not
display the two urls consistently and will now
also do so. Finally, HaikuDepot now correlates
locally configured repositories with the data in
HaikuDepotServer using the identifier URL; this
makes the use of mirrors with HaikuDepot possible.

Fixes #13888
Change-Id: I66dfe589b05c24e1ab123a6945352e0f24b60bf1

get_package_dependencies: When adding a package fails, actually report why.

e.Details() gives "additional details" and not the real failure
message, which is in e.Message(), so actually print that.

Revert "repo rework: Re-add compatibility url field"
Revert "repo rework: Remove stubs; Breaks repo compat."
Revert "repo rework: Remove need for repos to be self-aware"

This reverts commit a2b2f4d6427914cdcdb59943dd8e4a0bfdcd53ee.
This reverts commit 602076ef82647a48fd10f4d1ec830bb4242f9eb6.
This reverts commit 5ffaf72c8a74a7eb6827e4d2b1f47c9360ddaefe.

These changes break the build on Haiku and the ability to create repo
mirrors, for the lack of a replacement for the URL (an UUID was evoked
on the mailing lists, but not implemented).

We are due for a release soon, please don't break the build.

repo rework: Remove need for repos to be self-aware

* See #12917 for details.
* Squashed to one commit to make revert easy if we
run into any issues.
* pkgman is now pre-attached to the 'current' repo
version within nightly images so they can be updated
by default.
* This shouldn't impact us keeping older sets of package
versions by commit hash for building older hrevs.
* There are XXX stubs with "Kill me". These will need
to be dropped after users are given sufficent time to
upgrade. We're dropping a previously required field (url)
so making this a slowish roll out.
* Makes the repos a lot less restrictive which should
help PM package building automation be a bit easier.
* Once this stuff smooths out, we'll add UUID's to the
repo definitions for duplicate repo detection.

get_package_dependencies: catch and report exceptions

The package kit uses exceptions for error handling, but this tool didn't
catch them so all we got in case of error is "Abort" on the error
output.

Now, the exceptions are caught and reported with the complete error
message.

Refactor reusable pkgman code into libpackage

* Move RepositoryBuilder class to libpackage and add B* prefix to name.
* Pull BPackageManager class out of PackageManager and move to
libpackage. The base class is customizable via three handler objects
responsible for transaction handling, request execution, respectively
user interaction.
* Reorganize _ApplyPackageChanges(): Now we first prepare the
transactions for all affected installation locations (downloading
files etc.) and then commit them.

Change repository directory layout

* Under the base URL there are supposed to be the repository files and a
subdirectory "packages".
* Fix the repository URL related confusion introduced earlier. The URL
in
the repository info (and thus in the repository file) is supposed to
be the base URL for the repository. It is not a (potentially)
different base URL for the package files. Package and repository
files were supposed to live in the same directory. Now, by requiring
the package files to live in a subdirectory -- which can also be a
symlink -- we gain some flexibility.
The URL in the repository config is usually the same as the in the
repository info, unless it refers to a mirror site. This allows for
mirrors to copy the original repository verbatim.
* Remove the PackageURL rule and introduce a DownloadPackage rule
instead. The URL for a package file cannot be computed in the jam
parsing phase anymore, as it contains the hash value of the package
list.
* BRepositoryConfig: Add PackagesURL() for convenience.

Switch build system from optional package to repositories

* Build libsolv and the dependency solver part of the package kit for
the build platform.
* Add build tool get_package_dependencies. Given a list of package files
and a list of repository files it determines the additional packages
that need to be retrieved from the repositories and prints their URLs.
* Add rules to work with external repositories in the build system
(build/jam/RepositoryRules):
- PackageRepository declares an external repository with all its
packages. The URL of the repository file isn't specified. It is
computed from a given base URL and the SHA256 hash of the list of
package files.
- GeneratedRepositoryPackageList generates a file containing the file
names of all packages in a repository.
- IsPackageAvailable returns whether a package is available in any
repository.
- PackageURL returns the URL for a package.
* Declare the HaikuPorts repository for x86_gcc2
(build/jam/repositories/HaikuPorts/x86_gcc2).
* Add rule AddHaikuImagePackages to add a package to the image and rule
IsHaikuImagePackageAdded to determine whether a package has been
added.
* OptionalPackages: Remove all entries that just downloaded and
installed an external package. AddHaikuImagePackages can be used
instead and is used in the remaining entries. Also move the remaining
optional package dependency declarations from
OptionalPackageDependencies here.
* ExtractBuildFeatureArchives: Instead of the URL parameter a package
name must be specified now. This allows to simplify BuildFeatures
significantly, since there's no dealing with URLs anymore. "if" out
the entries that aren't supported yet.
* build_haiku_image: For the packages installed in system and common
resolve their dependencies and download and install them as well.

Refactor reusable pkgman code into libpackage

* Move RepositoryBuilder class to libpackage and add B* prefix to name.
* Pull BPackageManager class out of PackageManager and move to
libpackage. The base class is customizable via three handler objects
responsible for transaction handling, request execution, respectively
user interaction.
* Reorganize _ApplyPackageChanges(): Now we first prepare the
transactions for all affected installation locations (downloading
files etc.) and then commit them.

Change repository directory layout

* Under the base URL there are supposed to be the repository files and a
subdirectory "packages".
* Fix the repository URL related confusion introduced earlier. The URL
in
the repository info (and thus in the repository file) is supposed to
be the base URL for the repository. It is not a (potentially)
different base URL for the package files. Package and repository
files were supposed to live in the same directory. Now, by requiring
the package files to live in a subdirectory -- which can also be a
symlink -- we gain some flexibility.
The URL in the repository config is usually the same as the in the
repository info, unless it refers to a mirror site. This allows for
mirrors to copy the original repository verbatim.
* Remove the PackageURL rule and introduce a DownloadPackage rule
instead. The URL for a package file cannot be computed in the jam
parsing phase anymore, as it contains the hash value of the package
list.
* BRepositoryConfig: Add PackagesURL() for convenience.

Switch build system from optional package to repositories

* Build libsolv and the dependency solver part of the package kit for
the build platform.
* Add build tool get_package_dependencies. Given a list of package files
and a list of repository files it determines the additional packages
that need to be retrieved from the repositories and prints their URLs.
* Add rules to work with external repositories in the build system
(build/jam/RepositoryRules):
- PackageRepository declares an external repository with all its
packages. The URL of the repository file isn't specified. It is
computed from a given base URL and the SHA256 hash of the list of
package files.
- GeneratedRepositoryPackageList generates a file containing the file
names of all packages in a repository.
- IsPackageAvailable returns whether a package is available in any
repository.
- PackageURL returns the URL for a package.
* Declare the HaikuPorts repository for x86_gcc2
(build/jam/repositories/HaikuPorts/x86_gcc2).
* Add rule AddHaikuImagePackages to add a package to the image and rule
IsHaikuImagePackageAdded to determine whether a package has been
added.
* OptionalPackages: Remove all entries that just downloaded and
installed an external package. AddHaikuImagePackages can be used
instead and is used in the remaining entries. Also move the remaining
optional package dependency declarations from
OptionalPackageDependencies here.
* ExtractBuildFeatureArchives: Instead of the URL parameter a package
name must be specified now. This allows to simplify BuildFeatures
significantly, since there's no dealing with URLs anymore. "if" out
the entries that aren't supported yet.
* build_haiku_image: For the packages installed in system and common
resolve their dependencies and download and install them as well.