Package kit: fix error handling in case of HTTP errors

HTTP errors can have some content sent with them (a message to display
to the user, usually). The package kit would not ignore this and append
the content to the end of the downloaded file. This could result in a
file larger than the expected size, and in that case, it would keep
growing infinitely by adding more error messages to it.

Also add some more specific error messages for some HTTP codes, in
particular, "invalid range" which is likely to happen if something goes
wrong with range requests. Now this case will be detected and the
download will stop.

Change-Id: I18927f361235e9f72a5701c1bd7977abda9e21ad
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4210
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

PackageKit: fix download logic for remote files

This was broken in hrev54968, as some code was lost while switching to the new
way the network services kits writes files to disk.

This should fix #16822

Change-Id: I104d82e268ded287fd64db1cb902f94b71bad53b
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3760
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

libbnetapi: BUrlRequest now outputs to BDataIO

Previously, BUrlRequest returns data received via a callback that can't
return any value. This approach have several issues:

- It's not possible to signify failures to the request.
- Users have to implement custom listeners just to handle the common
case of outputting to a buffer/file/etc.
- The received data has to be serialized into BMessage when
BUrlProtocolDispatchingListener is employed. This can cause a
noticible slowdown in real-world scenarios as evident by #10748.

With this change, BUrlRequest will output directly into a BDataIO, which
exposes a richer API for request handlers to work with (for example a
BitTorrent client can request a BPositionIO for non-linear data
delivery), as well as simplifying common cases for users.

The adaptation only requires one additional API:
BHttpRequest::SetStopOnError(). This API simply instructs the HTTP
request handler to cancel the request if an HTTP error is occurred.

Change-Id: I4160884d77bff0e7678e0a623e2587987704443a
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3084
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

libpackage.so: change UrlProtocolListener size types to off_t

Missed this use after changing the definition in hrev54926

Change-Id: I046560ec73dfb3ed6284513a34228b8343c5cf40

Package Kit: switch to libnetservices.a

Switch to the experimental services API as the copy in libbnetapi.so is
deprecated.

Change-Id: Ia0742038937d078c6a59ae13180a00a4c4eeb177
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3671
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

BHttpRequest: Treat incomplete downloads as error

When BSocket::Read() returns 0, it was treated as
a finished transfer. This is OK when we don't know
the content length, but when we do, there is no reason
not to bubble up the error. Return B_IO_ERROR in this
case.

Change-Id: I68801dbbb85bcfd2e7aa68fd6a9ded6304e82b19
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3621
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Package Kit: re-use downloads from unfinished transactions

There are three parts to this change:
- In FetchFileJob, if the request fails with a timeout or IO error
(probably because of unstable connection) attempt to resume the
download with a range request. No limit on number of retries
currently, maybe we should add one.
- In PackageManager, before downloading a file, look around in other
transaction directories in case it's already there. Partial and
complete downloads are differentiated by an attribute which the
fetch file job maintains. For complete downloads, no fetch job is
scheduled, for partial downloads, the fetch job will request the
remainder of the file.
- In BHttpRequest, the implementation of SetRangeStart() and
SetRangeEnd() have been added, along with some refactoring to
handle listener notifications consistently. This also fixed a
bug where the final notification for download progress was not
emitted for compressed data.

Fixes #12414.

Change-Id: I3e285741ed0e5651594a7c2e1c7170644a9d297d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3404
Reviewed-by: Stephan Aßmus <superstippi@gmx.de>
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

Package Kit: convert HTTP error code only on BHttpRequest

Other request types exists such as BFileRequest.

Fixes #15675.

Change-Id: Ib2e07fad4dd9f682d2b9fc0cdbf0ca60ecd3adfb
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2200
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>
Reviewed-by: François Revol <revol@free.fr>

Package Kit: Return proper errors in FetchFileJob

This should give more meaningful errors in pkgman
(not so sure, feel free to tweak the mapping),
and also avoid running the next job if fetching failed.

Change-Id: If76968f705ff25f7ecf1a5f91d88a02040d24665
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2186
Reviewed-by: Ryan Leavengood <leavengood@gmail.com>

package kit: add a NULL check on requests

This avoid crashing on malformed repository URL (like missing http://).

Remove Curl, Switch to NetApi in Package Kit

Change-Id: I92213ab60dc987175c323d1d9ed11ac8b3517f2f
Reviewed-on: https://review.haiku-os.org/475
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

CURLOPT_XFERINFOFUNCTION is available since libcurl 7.32.0 only.
Since it's just cosmetic, allow compiling with older libcurl, too.
fixes build on RedHat/CentOS 7

Change-Id: I169ad84d15e61012d1d38a9b41af7eef81683bf7

Package Kit: map some curl errors to Be error codes

Trying to get something more useful than "aborted: no error" in case of
network errors in pkgman.

Fix package download progress computation.

The download progress and total used to be doubles, but now they are
off_t. This resulted in the division being done in integers, and always
getting 0 as the result.

Fixes #11940.

Package Kit: FetchFileJob now contains package size.

* So that you know how much already was, and still has to be downloaded.
* Automatic whitespace cleanup.
* The link in FetchFileJob.h did not fetch the correct header under
Haiku anymore (since the addition of the private headers to the
image).

Let Package-Kit's FetchFileJob handle redirects.

* additionally, drop duplicate setting of CURLOPT_PROGRESSFUNCTION.

package kit: Disable curl requirement on bootstrap build

* As per the mailing list.
* Introduce HAIKU_BOOTSTRAP_BUILD define to sources
to let them know they are taking part in a bootstrap

Package Kit: Implement progress notifications.

- BJobStateListener: Add progress state and corresponding hook.
- FetchFileJob: Notify job progress hook on libcurl notifications.
- UserInteractionHandler: Add hooks for download progress and checksum
validation progress.
- PackageManager: inherit from JobStateListener and watch for job
notifications for internally generated jobs. Forward to corresponding
UserInteractionHandler hooks as needed.
- Adapt pkgman, HaikuDepot and package_daemon to above changes.
Neither HaikuDepot nor package_daemon's progress hooks are wired up to
do anything yet though.

libpackage: Use libcurl directly for downloads.

- Initial step towards allowing listening for download progress.

Mostly style-related changes to package kit:
* drop 'Haiku' namespace
* rename 'Package' namespace to 'BPackageKit'
* renamed all public classes to begin with a 'B'
* renamed BPackageKit::Roster to BPackageKit::BPackageRoster to not
clash with the BRoster from the application kit.
* fix some instances of public headers including private ones

Some functional changes, too:
* JobQueue now removes and deletes dependants of failed jobs
automatically
* JobQueue supports waiting for jobs to become runnable
* added a couple of InitCheck() methods where they make sense
and invoke those in users of these classes



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40287 a95241bf-73f2-0310-859d-f6bbb57e9c96

* more work on the package kit, repositories can now be added and
refreshed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40280 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for interactive decisions to package kit and pkgman


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40268 a95241bf-73f2-0310-859d-f6bbb57e9c96

* WIP-commit of the first parts of the package kit and the pkgman
(console-)tool


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40261 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix package download progress computation.

The download progress and total used to be doubles, but now they are
off_t. This resulted in the division being done in integers, and always
getting 0 as the result.

Fixes #11940.

Package Kit: FetchFileJob now contains package size.

* So that you know how much already was, and still has to be downloaded.
* Automatic whitespace cleanup.
* The link in FetchFileJob.h did not fetch the correct header under
Haiku anymore (since the addition of the private headers to the
image).

Let Package-Kit's FetchFileJob handle redirects.

* additionally, drop duplicate setting of CURLOPT_PROGRESSFUNCTION.

package kit: Disable curl requirement on bootstrap build

* As per the mailing list.
* Introduce HAIKU_BOOTSTRAP_BUILD define to sources
to let them know they are taking part in a bootstrap

Package Kit: Implement progress notifications.

- BJobStateListener: Add progress state and corresponding hook.
- FetchFileJob: Notify job progress hook on libcurl notifications.
- UserInteractionHandler: Add hooks for download progress and checksum
validation progress.
- PackageManager: inherit from JobStateListener and watch for job
notifications for internally generated jobs. Forward to corresponding
UserInteractionHandler hooks as needed.
- Adapt pkgman, HaikuDepot and package_daemon to above changes.
Neither HaikuDepot nor package_daemon's progress hooks are wired up to
do anything yet though.

libpackage: Use libcurl directly for downloads.

- Initial step towards allowing listening for download progress.

Mostly style-related changes to package kit:
* drop 'Haiku' namespace
* rename 'Package' namespace to 'BPackageKit'
* renamed all public classes to begin with a 'B'
* renamed BPackageKit::Roster to BPackageKit::BPackageRoster to not
clash with the BRoster from the application kit.
* fix some instances of public headers including private ones

Some functional changes, too:
* JobQueue now removes and deletes dependants of failed jobs
automatically
* JobQueue supports waiting for jobs to become runnable
* added a couple of InitCheck() methods where they make sense
and invoke those in users of these classes



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40287 a95241bf-73f2-0310-859d-f6bbb57e9c96

* more work on the package kit, repositories can now be added and
refreshed


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40280 a95241bf-73f2-0310-859d-f6bbb57e9c96

* added support for interactive decisions to package kit and pkgman


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40268 a95241bf-73f2-0310-859d-f6bbb57e9c96

* WIP-commit of the first parts of the package kit and the pkgman
(console-)tool


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40261 a95241bf-73f2-0310-859d-f6bbb57e9c96