MFC: r287580

Remove extra debug that crept in

MFC: r287579

Implement pubkey support for the bootstrap

Note that to not interfer with finger print it expects a signature on pkg itself
which is named pkg.txz.pubkeysign

To genrate it:
echo -n "$(sha256 -q pkg.txz)" | openssl dgst -sha256 -sign /thekey \
-binary -out ./pkg.txz.pubkeysig

Note the "echo -n" which prevent signing the '\n' one would get otherwise

PR: 202622

Instant-MFC r286933:

Issue warning and refuse to proceed further if the configured
repository signature_type is unsupported by bootstrap pkg(7).

Previously, when signature_type specified an unsupported method,
the bootstrap pkg(7) would proceed like when signature_type is
"none". MITM attackers may be able to use this vulnerability and
bypass validation and install their own versions of pkg(8).

At this time, only fingerprint and none are supported by the
bootstrap pkg(7).

FreeBSD's official pkg(8) repository uses the fingerprint method
and is therefore unaffected.

Errata candidate.

MFC: r281039

Allow fetching pkg(8) even if servers/proxies are not passing Content-length

MFC: r259773, r259774, r259775

Do not print a question in non-interractive mode but explain why boostrapping
has failed and how to allow it to happen

Reported by: jmmv

MFC: r258226

argc is now used

MFC: r258020 (by bdrewery)

Remove useless checks for NULL pointer before free(3)

Reported by: eadler
Approved by: eadler

MFC: r257701 (by bdrewery)

Remove condition which can never be true as the previous loop
is never exited while remote == NULL.

MFC: r278173
Test the return of fetchParseURL(3)

CID: 1125811

MFC: r278172

Plug resources leak

CID: 1125813
CID: 1125807
CID: 1125808

Merge r268728 from head:

When we fail to extract the pkg binaries (for example, / is read-only),
give a more helpful error message.

MFC: r263648, r264789, r266636

This brings:
- schema validation
- xpath-like interface for ucl objects

Adapt pkg(7) to the new libucl API

r262400,r262401,r262418

Sync pkg(7) with head

MFC r258550:
Do not create pkg.conf, as it is not necessary anymore
and packagesite is deprecated

Approved by: re (glebius)
Sponsored by: The FreeBSD Foundation

Merge r257945 from head:

Do not use just freed memory.

Approved by: re (kib)

MFC r257505:

Add -f support to 'pkg bootstrap' and 'pkg add' to force installation
of pkg(8) even if already installed.

This is useful if you somehow messup pkg(8) and need to reinstall from
remote with it already being registered in the pkg(8) /var/db/pkg database.

Also add some sanity checks to 'pkg add'.

Approved by: bapt
Approved by: re (glebius)

MFC r257377:

Add a 'pkg bootstrap' command which will bootstrap pkg(8) without
forwarding any command to it after installation.

Approved by: bapt
Approved by: re (gjb)

MFC: r256770,r257142,r257145,r257146,r257147,r257148,
r257149,r257150,r257158,r257159,r257164,r257168,
r257193

- Support checking signature for pkg bootstrap from remote
and for 'pkg add ./pkg.txz'

- Be verbose on where pkg is being bootstrapped from.

- Add support for reading configuration files from /etc/pkg.
For now only /etc/pkg/FreeBSD.conf is supported.

- Add test package signing key fingerprint into /etc/keys/pkg/trusted.

- Disable fingerprint checking by default for now as the official
packages are not yet signed.

Approved by: bapt
Approved by: re (glebius)

MFC r257051:

Add support for using "pkg+http://" for the PACKAGESITE.

pkg 1.2 is adding this support as well. This should help
lessen the confusion on why the default SRV PACKAGESITE
does not load in a browser.

Approved by: bapt
Approved by: re (glebius)

MFC: r256968, r256971, r256978

Improve SRV records support for the pkg(8) bootstrap:
- order srv records by priorities
- for all entries of the same priority, order randomly respect the weight
- select the port where to fetch from respect the port provided in the SRV
record

Allow to bootstrap by doing pkg add ./a/path/to/a/pkg_package.txz

Approved by: re (glebius)

MFC: r287580

Remove extra debug that crept in

MFC: r287579

Implement pubkey support for the bootstrap

Note that to not interfer with finger print it expects a signature on pkg itself
which is named pkg.txz.pubkeysign

To genrate it:
echo -n "$(sha256 -q pkg.txz)" | openssl dgst -sha256 -sign /thekey \
-binary -out ./pkg.txz.pubkeysig

Note the "echo -n" which prevent signing the '\n' one would get otherwise

PR: 202622

Instant-MFC r286933:

Issue warning and refuse to proceed further if the configured
repository signature_type is unsupported by bootstrap pkg(7).

Previously, when signature_type specified an unsupported method,
the bootstrap pkg(7) would proceed like when signature_type is
"none". MITM attackers may be able to use this vulnerability and
bypass validation and install their own versions of pkg(8).

At this time, only fingerprint and none are supported by the
bootstrap pkg(7).

FreeBSD's official pkg(8) repository uses the fingerprint method
and is therefore unaffected.

Errata candidate.

MFC: r281039

Allow fetching pkg(8) even if servers/proxies are not passing Content-length

MFC: r259773, r259774, r259775

Do not print a question in non-interractive mode but explain why boostrapping
has failed and how to allow it to happen

Reported by: jmmv

MFC: r258226

argc is now used

MFC: r258020 (by bdrewery)

Remove useless checks for NULL pointer before free(3)

Reported by: eadler
Approved by: eadler

MFC: r257701 (by bdrewery)

Remove condition which can never be true as the previous loop
is never exited while remote == NULL.

MFC: r278173
Test the return of fetchParseURL(3)

CID: 1125811

MFC: r278172

Plug resources leak

CID: 1125813
CID: 1125807
CID: 1125808

Merge r268728 from head:

When we fail to extract the pkg binaries (for example, / is read-only),
give a more helpful error message.

MFC: r263648, r264789, r266636

This brings:
- schema validation
- xpath-like interface for ucl objects

Adapt pkg(7) to the new libucl API

r262400,r262401,r262418

Sync pkg(7) with head

MFC r258550:
Do not create pkg.conf, as it is not necessary anymore
and packagesite is deprecated

Approved by: re (glebius)
Sponsored by: The FreeBSD Foundation

Merge r257945 from head:

Do not use just freed memory.

Approved by: re (kib)

MFC r257505:

Add -f support to 'pkg bootstrap' and 'pkg add' to force installation
of pkg(8) even if already installed.

This is useful if you somehow messup pkg(8) and need to reinstall from
remote with it already being registered in the pkg(8) /var/db/pkg database.

Also add some sanity checks to 'pkg add'.

Approved by: bapt
Approved by: re (glebius)

MFC r257377:

Add a 'pkg bootstrap' command which will bootstrap pkg(8) without
forwarding any command to it after installation.

Approved by: bapt
Approved by: re (gjb)

MFC: r256770,r257142,r257145,r257146,r257147,r257148,
r257149,r257150,r257158,r257159,r257164,r257168,
r257193

- Support checking signature for pkg bootstrap from remote
and for 'pkg add ./pkg.txz'

- Be verbose on where pkg is being bootstrapped from.

- Add support for reading configuration files from /etc/pkg.
For now only /etc/pkg/FreeBSD.conf is supported.

- Add test package signing key fingerprint into /etc/keys/pkg/trusted.

- Disable fingerprint checking by default for now as the official
packages are not yet signed.

Approved by: bapt
Approved by: re (glebius)

MFC r257051:

Add support for using "pkg+http://" for the PACKAGESITE.

pkg 1.2 is adding this support as well. This should help
lessen the confusion on why the default SRV PACKAGESITE
does not load in a browser.

Approved by: bapt
Approved by: re (glebius)

MFC: r256968, r256971, r256978

Improve SRV records support for the pkg(8) bootstrap:
- order srv records by priorities
- for all entries of the same priority, order randomly respect the weight
- select the port where to fetch from respect the port provided in the SRV
record

Allow to bootstrap by doing pkg add ./a/path/to/a/pkg_package.txz

Approved by: re (glebius)