X509_check_akid: zap stray space

Add a comment on abuse of EXFLAG_INVALID

We added things we probably shouldn't have, and so did BoringSSL and
OpenSSL. Terrible API is terrible.

discussed with jsing

Remove notBefore and notAfter cacheing.

This cache was added because our time conversion used timegm()
and gmtime() which aren't very cheap. These calls were noticably
expensive when profiling things like rpki-client which do many
X.509 validations.

Now that we convert times using julien seconds from the unix
epoch, BoringSSL style, instead of a julien days from a
Byzantine date, we no longer use timegm() and gmtime().

Since the julien seconds calculaitons are cheap for conversion,
we don't need to bother caching this, it doesn't have a noticable
performance impact.

While we are at this correct a bug where
x509_verify_asn1_time_to_time_t was not NULL safe.

Tested for performance regressions by tb@ and job@

ok tb@ job@

const correct X509_PURPOSE_get0{,_{,s}name}()

Unfortunately, PHP and rust-openssl still need this API. At least we
can make the table read-only now since we disabled its extensibility.

ok jsing

Make X509_PURPOSE opaque

Code using details of X509_PURPOSE does so by using API. So we can make
this struct opaque.

ok jsing

Remove unused parts of the purpose API

Most of this is the ability to add custom purposes. Also the astounding
X509_STORE_CTX_purpose_inherit(). The names are used by PHP, and M2Crypto
exposes X509_check_purpose(), so these remain public. Some weird, most
likely invalid, uses also remain in rust-openssl.

ok jsing

Small tweak for X509_check_purpose()

ok jsing

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Add a comment on abuse of EXFLAG_INVALID

We added things we probably shouldn't have, and so did BoringSSL and
OpenSSL. Terrible API is terrible.

discussed with jsing

Remove notBefore and notAfter cacheing.

This cache was added because our time conversion used timegm()
and gmtime() which aren't very cheap. These calls were noticably
expensive when profiling things like rpki-client which do many
X.509 validations.

Now that we convert times using julien seconds from the unix
epoch, BoringSSL style, instead of a julien days from a
Byzantine date, we no longer use timegm() and gmtime().

Since the julien seconds calculaitons are cheap for conversion,
we don't need to bother caching this, it doesn't have a noticable
performance impact.

While we are at this correct a bug where
x509_verify_asn1_time_to_time_t was not NULL safe.

Tested for performance regressions by tb@ and job@

ok tb@ job@

const correct X509_PURPOSE_get0{,_{,s}name}()

Unfortunately, PHP and rust-openssl still need this API. At least we
can make the table read-only now since we disabled its extensibility.

ok jsing

Make X509_PURPOSE opaque

Code using details of X509_PURPOSE does so by using API. So we can make
this struct opaque.

ok jsing

Remove unused parts of the purpose API

Most of this is the ability to add custom purposes. Also the astounding
X509_STORE_CTX_purpose_inherit(). The names are used by PHP, and M2Crypto
exposes X509_check_purpose(), so these remain public. Some weird, most
likely invalid, uses also remain in rust-openssl.

ok jsing

Small tweak for X509_check_purpose()

ok jsing

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

const correct X509_PURPOSE_get0{,_{,s}name}()

Unfortunately, PHP and rust-openssl still need this API. At least we
can make the table read-only now since we disabled its extensibility.

ok jsing

Make X509_PURPOSE opaque

Code using details of X509_PURPOSE does so by using API. So we can make
this struct opaque.

ok jsing

Remove unused parts of the purpose API

Most of this is the ability to add custom purposes. Also the astounding
X509_STORE_CTX_purpose_inherit(). The names are used by PHP, and M2Crypto
exposes X509_check_purpose(), so these remain public. Some weird, most
likely invalid, uses also remain in rust-openssl.

ok jsing

Small tweak for X509_check_purpose()

ok jsing

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Small tweak for X509_check_purpose()

ok jsing

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

purpose/trust: Improve comments about COUNT/MAX confusion

Remove X509_PURPOSE extensibility

Another bit of global state without lock protection. The by now familiar
complications of a stack to make this user configurable, which, of course,
no one ever did. The table is not currently const, and the API exposes its
entries directly, so anyone can modify it. This fits very well with the
safety guarantees of Rust's 'static lifetime, which is how rust-openssl
exposes it (for no good reason).

Remove the stack and make the X509_PURPOSE_add() API always fail.
Simplify the other bits accordingly.

In addition, this API inflicts the charming difference between purpose
identifiers and purpose indexes (the former minus one) onto the user.
Neither of the two obvious solutions to avoid this trap seems to have
crossed the implementer's mind.

ok jsing

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

KNF nit

Sort the NIDs by name

requested by jsing

Replace the sorted extensions lookup with a switch

If all you have is OBJ_bsearch_(), everything looks like a nail. This
changes a binary search over a list of 12 elements with a lookup via
a switch.

switch suggested by claudio
ok jsing

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Check X509_digest() return in x509v3_cache_extensions()

On failure invalidate the cert with EXFLAG_INVALID. It's unlikely that
a cert would make it through to the end of this function without setting
the flag, but it's bad style anyway.

ok jsing

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Convert some tables to C99 initializers

ok & "happy pirate day" beck

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

Avoid expensive RFC 3779 checks during cert verification

X509v3_{addr,asid}_is_canonical() check that the ipAddrBlocks and
autonomousSysIds extension conform to RFC 3779. These checks are not
cheap. Certs containing non-conformant extensions should not be
considered valid, so mark them with EXFLAG_INVALID while caching the
extension information in x509v3_cache_extensions(). This way the
expensive check while walking the chains during X509_verify_cert() is
replaced with a cheap check of the extension flags. This avoids a lot
of superfluous work when validating numerous certs with similar chains
against the same roots as is done in rpki-client.

Issue noticed and fix suggested by claudio
ok claudio inoguchi jsing

Fix X509_get_extension_flags()

Ensure that EXFLAG_INVALID is set on X509_get_purpose() failure.

ok inoguchi jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Move the now internal X.509-related structs into x509_lcl.h.
Garbage collect the now unused LIBRESSL_CRYPTO_INTERNAL and
LIBRESSL_OPAQUE_X509. Include "x509_lcl.h" where needed and
fix a couple of unnecessary reacharounds.

ok jsing

Actually error in X509_check_purpose() if x509v3_cache_extensions()
indicates failure. The previous "error return" X509_V_ERR_UNSPECIFIED
translates to 1, i.e., success. This changes to the intended behavior
of x509_purp.c r1.3 and matches OpenSSL. This will need various
adjustments in the documentation.

ok jsing

Prepare to provide X509_get_extension_flags()

ok beck jsing

Prepare to provide X509_get_{extended_,}key_usage()

ok beck jsing

Add XKU_ANYEKU #define and use it to cache the anyExtendedKeyUsage
extension. This is part of OpenSSL commit df4c395c which didn't make
it into our tree for some reason.

ok jsing

In X509_check_issued() do the same dance around x509v3_cache_extensions()
as in all other palces. Check the EXFLAG_SET flag first and if not set
grab the CRYPTO_LOCK_X509 before calling x509v3_cache_extensions().
OK tb@ beck@

Lay groundwork to support X.509 v3 extensions for IP Addresses and AS Identifiers

These extensions are defined in RFC 3779 and used in the RPKI (RFC 6482, RFC 8360).
Imported from OpenSSL 1.1.1j (aaf2fcb575cdf6491b98ab4829abf78a3dec8402b8b81efc8f23c00d443981bf)
This changeset is a no-op, as there are 10+ issues and at least 2 security issues.
Work will continue in-tree.

OK tb@, discussed with beck@

Delete some code from X509_PURPOSE_cleanup(3) that had no effect:
it called a function on static objects that returns right away
unless the argument is dynamically allocated.
OK jsing@ tb@

The useless code was independently discovered while writing documentation.
This commit is identical to:

OpenSSL commit fa3a0286d178eb3b87bf2eb5fd7af40f81453314
Author: Kurt Cancemi <kurt at x64architecture dot com>
Date: Wed Jun 8 19:15:38 2016 -0400

Fix copy-paste error in previous

Found the hard way by lists y42 org via an OCSP validation failure that
in turn caused pkg_add over TLS to fail. Detailed report by sthen.

ok sthen

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Collapse the x509v3 directory into x509.

This avoids the need to grep across directories to find functions and
prepares for further rototilling and chainsawing.

Discussed with tb@ (who also tested the release build)

Check for duplicate X.509v3 extension OIDs

Per RFC 5280, 4.2: A certificate MUST NOT include more than one instance
of a particular extension.

This implements such a check in x509v3_cache_extensions() by sorting the
list of extensions and looking for duplicate neighbors. This sidesteps
complications from extensions we do not know about and keeps algorithmic
complexity reasonable. If the check fails, EXFLAG_INVALID is set on the
certificate, which means that the verifier will not validate it.

ok jsing

Improve certificate version checks in x509v3_cache_extensions()

Only allow version v1-v3, disallow issuerUID and subjectUID in v1 certs
and require that if X509v3 extensions are present that the cert be v3.

Initial diff from job

ok job jsing

Replace X509v3_get_ext_count() with X509_get_ext_count()

Error introduced in 1.24

In the case of V1 certs, the extension count should be exactly 0

OK tb@

If extensions are encountered on a X.509 V1 cert, mark as invalid

While there, explicitly check for 0 - as X509_get_version() is a wrapper
around the less than beloved ASN1_INTEGER_get().

OK tb@

More ProxyCertInfo tentacles go to the attic

This removes ProxyCertInfo from extension caching, issuer checking
and it also drops the special path validation for proxy certs from
the legacy verifier.

ok jsing

zap stray empty line

libressl *_namespace.h: adjust *_ALIAS() to require a semicolon

LCRYPTO_ALIAS() and LSSL_ALIAS() contained a trailing semicolon.
This does not conform to style(9), breaks editors and ctags and
(most importantly) my workflow. Fix this by neutering them with
asm("") so that -Wpedantic doesn't complain. There's precedent
in libc's namespace.h

fix suggested by & ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Make internal header file names consistent

Libcrypto currently has a mess of *_lcl.h, *_locl.h, and *_local.h names
used for internal headers. Move all these headers we inherited from
OpenSSL to *_local.h, reserving the name *_internal.h for our own code.
Similarly, move dtls_locl.h and ssl_locl.h to dtls_local and ssl_local.h.
constant_time_locl.h is moved to constant_time.h since it's special.

Adjust all .c files in libcrypto, libssl and regress.

The diff is mechanical with the exception of tls13_quic.c, where
#include <ssl_locl.h> was fixed manually.

discussed with jsing,
no objection bcook

Hide public symbols in libcrypto/x509 .c files

ok tb@

X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing