Fix non-xsc path in x509_verify_potential_parent()

The combination of two bugs made this unexpectedly work as intended. To
appreciate this, let's first note that

a) check_issued(..., child, parent) checks if child was issued by parent.

b) X509_check_issued(child, parent) checks if parent was issued by child.

Now like in the real world, b) will only be true in unusual circumstances
(child is known not to be self-issued at this point). X509_check_issued()
fails by returning something different from X509_V_OK, so

return X509_check_issued(child, parent) != X509_V_OK;

will return true if child was issued by parent since then parent was indeed
not issued by child. On the other hand, if child was not issued by parent,
the verifier will notice elsewhere, e.g., in a signature check.

Fix this by reversing the order of child and parent in the above return
line and check for equality instead. This is nearly impossible to detect
in regress.

ok beck

Remove notBefore and notAfter cacheing.

This cache was added because our time conversion used timegm()
and gmtime() which aren't very cheap. These calls were noticably
expensive when profiling things like rpki-client which do many
X.509 validations.

Now that we convert times using julien seconds from the unix
epoch, BoringSSL style, instead of a julien days from a
Byzantine date, we no longer use timegm() and gmtime().

Since the julien seconds calculaitons are cheap for conversion,
we don't need to bother caching this, it doesn't have a noticable
performance impact.

While we are at this correct a bug where
x509_verify_asn1_time_to_time_t was not NULL safe.

Tested for performance regressions by tb@ and job@

ok tb@ job@

Fix the verifier to use the trust store

the trust store is yet another obscure way to add a trust anchor

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Remove a misplaced empty line

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@

Revert "Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new
verifier." (r1.27). While this may have "fixed" one corner case, it
broke expectations of Perl Net::SSLeay and Ruby OpenSSL regression
tests.

ok bcook

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Fix checks of memory caps of constraints names

x509_internal.h defines caps on the number of name constraints and
other names (such as subjectAltNames) that we want to allocate per
cert chain. These limits are checked too late. In a particularly
silly cert that jan found on ugos.ugm.ac.id 443, we ended up
allocating six times 2048 x509_constraint_name structures before
deciding that these are more than 512.

Fix this by adding a names_max member to x509_constraints_names which
is set on allocation against which each addition of a name is checked.

cluebat/ok jsing
ok inoguchi on earlier version

Set is_trusted in x509_verify_ctx_add_chain()

If we're about to add a chain we have a trust path, so we have at least
one trusted certificate. This fixes a thinko from r1.31 and fixes the
openssl(1) cms verify test.

ok jsing (who had the same diff)

Rename depth to num_untrusted so it identifies what it actually represents.

ok tb@

Avoid passing last and depth to x509_verify_cert_error() on ENOMEM.

In x509_verify_ctx_set_xsc_chain(), an ENOMEM case is currently passing
the last certificate and depth (which is no longer actually depth) to
x509_verify_cert_error(). Given we've hit an ENOMEM situation, neither
of these are useful so remove both.

ok tb@

Make the new validator check for EXFLAG_CRITICAL

As should be obvious from the name and the comment in x509_vfy.h
int last_untrusted; /* index of last untrusted cert */
last_untrusted actually counts the number of untrusted certs at the
bottom of the chain.

Unfortunately, an earlier fix introducing x509_verify_set_xsc_chain()
assumed that last_untrusted actually meant the index of the last
untrusted cert in the chain, resulting in an off-by-one, which in turn
led to x509_vfy_check_chain_extension() skipping the check for the
EXFLAG_CRITICAL flag.

A second bug in x509_verify_set_xsc_chain() assumed that it is always
called with a trusted root, which is not necessarily the case anymore.
Address this with a temporary fix which will have to be revisited once
we will allow chains with more than one trusted cert.

Reported with a test case by tobhe.

ok jsing tobhe

Set chain on xsc on chain build failure.

Prior to calling the callback, ensure that the current (invalid and likely
incomplete) chain is set on the xsc. Some things (like auto chain) depend
on this functionality.

ok beck@

Bail out early after finding an single chain if we are have been called from
x509_vfy and have an xsc. There's no point in finding more chains since that
API can not return them, and all we do is trigger buggy callbacks in
calling software.

ok jsing@

search the intermediates only after searching the root certs, clarify
this in the comments. helps avoid annoying situations with the legacy
callback

ok jsing@

Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new verifier.

Yet another mostly meaningless error value...

Noted by and ok tb@

Gracefully handle root certificates being both trusted and untrusted.

When a certificate (namely a root) is specified as both a trusted and
untrusted certificate, the new verifier will find multiple chains - the
first being back to the trusted root certificate and a second via the root
that is untrusted, followed by the trusted root certificate. This situation
can be triggered by a server that (unnecessarily) includes the root
certificate in its certificate list.

While this validates correctly (using the first chain), it means that we
encounter a failure while building the second chain due to the root
certificate already being in the chain. When this occurs we call the verify
callback indicating a bad certificate. Some sensitive software (including
bacula and icinga), treat this single bad chain callback as terminal, even
though we successfully verify the certificate.

Avoid this problem by simply dumping the chain if we encounter a situation
where the certificate is already in the chain and also a trusted root -
we'll have already picked up the trusted root as a shorter path.

Issue with icinga2 initially reported by Theodore Wynnychenko.

Fix tested by sthen@ for both bacula and icinga2.

ok tb@

Remove two reduntat memset calls.

pointed out by jsing

Plug leak in x509_verify_chain_dup()

x509_verify_chain_new() allocates a few members of a certificate chain:
an empty stack of certificates, a list of errors encountered while
validating the chain, and a list of name constraints. The function to
copy a chain would allocate a new chain using x509_verify_chain_new()
and then clobber its members by copies of the old chain. Fix this by
replacing x509_verify_chain_new() with calloc().

Found by review while investigating the report by Hanno Zysik who
found the same leak using valgrind. This is a cleaner version of
my initial fix from jsing.

ok jsing

zap ugly empty line before closing brace

Use X509_V_OK instead of 0.

ok beck@ tb@

Add back an X509_STORE_CTX error code assignment.

This was inadvertently removed in r1.19.

Spotted by tb@

ok beck@ tb@

Return the specific failure for a "self signed certificate" in the chain
in order to be compatible with the openssl error craziness in the legacy
verifier case.

This will fix a regress problem noticed by znc

ok tb@

Handle additional certificate error cases in new X.509 verifier.

With the old verifier, the verify callback can always return 1 instructing
the verifier to simply continue regardless of a certificate verification
failure (e.g. the certificate is expired or revoked). This would result
in a chain being built, however the first error encountered would be
persisted, which allows the caller to build the chain, have the
verification process succeed, yet upon inspecting the error code note
that the chain is not valid for some reason.

Mimic this behaviour by keeping track of certificate errors while building
chains - when we finish verification, find the certificate error closest
to the leaf certificate and expose that via the X509_STORE_CTX. There are
various corner cases that we also have to handle, like the fact that we
keep an certificate error until we find the issuer, at which point we have
to clear it.

Issue reported by Ilya Shipitcin due to failing haproxy regression tests.

With much discussion and input from beck@ and tb@!

ok beck@ tb@

Fix bad indent.

Hook X509_STORE_CTX get_issuer() callback from new X509 verifier.

If we fail to find a parent certificate from either the supplied roots or
intermediates and we have a X509_STORE_CTX, call its get_issuer() callback
to see if it can supply a suitable certificate. This makes things like
certificates by directory (aka by_dir) work correctly.

Issue noted by Uwe Werler <uwe@werler.is>

ok beck@ tb@

Add a safety net to ensure that we set an error on the store context.

Suggested by and discussed with beck

If x509_verify() fails, ensure that the error is also set on the store
context. This is what is returned in SSL_get_verify_result().

Spotted and initial diff from jeremy; discussed with jsing
ok beck

Make sure that x509_vfy_check_id() failure also sets ctx->error, not only
ctx->xsc->error. Will be needed in an upcoming diff.

from beck

branches: 1.13.4;
Ensure leaf is set up on X509_STORE_CTX before verification.

Previously the leaf certificate was only being set up on the X509_STORE_CTX
after two verification steps were performed, however at least one of those
steps could result in the verification callback being triggered and
existing code breaking.

Issue noticed by Raf Czlonka when attempting to connect to talk.google.com
using profanity (which does not set SNI and ends up receiving an invalid
certificate).

ok beck@ deraadt@ tb@

Ensure chain is set on the X509_STORE_CTX before triggering callback.

Various software expects the previous behaviour where the certificate chain
is available on the X509_STORE_CTX when the verify callback is triggered.

Issue hit by bket@ with lastpass-cli which has built in certificate
pinning that is checked via the verify callback.

Fix confirmed by bket@.

ok beck@

remove superfluous NULL check
ok tb@

Make check in x509_verify_ctx_set_max_signatures() consistent with others.

ok beck@

Dedup code in x509_verify_ctx_new_from_xsc().

Rather than duplicating code, have x509_verify_ctx_new_from_xsc() call
x509_verify_ctx_new(), then handle the xsc specific parts.

ok beck@

set error_depth and current_cert to make more legacy callbacks that don't check
happy

ok tb@

Deduplicate the time validation code between the legacy and new
verification code.

ok jsing@

Set error if we are given an NULL ctx in x509_verify, and set error
to X509_V_ERR_OUT_OF_MEM instead of UNSPECIFIED in chain_append when
allocation fails.

noticed by ingo@
ok tb@

nuke a stray space

remove unneded variable "time"

noticed by llvm static analyzer
ok tb@

fix bug introduced on review where refactor made it possible to
eat a NULL.

found by clang static analyzer
ok tb@

Correctly fix double free introduced on review.

the roots for a ctx are only freed in the free function, not in the
clear function, so that a ctx can be re-used with the same roots.

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Remove notBefore and notAfter cacheing.

This cache was added because our time conversion used timegm()
and gmtime() which aren't very cheap. These calls were noticably
expensive when profiling things like rpki-client which do many
X.509 validations.

Now that we convert times using julien seconds from the unix
epoch, BoringSSL style, instead of a julien days from a
Byzantine date, we no longer use timegm() and gmtime().

Since the julien seconds calculaitons are cheap for conversion,
we don't need to bother caching this, it doesn't have a noticable
performance impact.

While we are at this correct a bug where
x509_verify_asn1_time_to_time_t was not NULL safe.

Tested for performance regressions by tb@ and job@

ok tb@ job@

Fix the verifier to use the trust store

the trust store is yet another obscure way to add a trust anchor

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Remove a misplaced empty line

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@

Revert "Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new
verifier." (r1.27). While this may have "fixed" one corner case, it
broke expectations of Perl Net::SSLeay and Ruby OpenSSL regression
tests.

ok bcook

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Fix checks of memory caps of constraints names

x509_internal.h defines caps on the number of name constraints and
other names (such as subjectAltNames) that we want to allocate per
cert chain. These limits are checked too late. In a particularly
silly cert that jan found on ugos.ugm.ac.id 443, we ended up
allocating six times 2048 x509_constraint_name structures before
deciding that these are more than 512.

Fix this by adding a names_max member to x509_constraints_names which
is set on allocation against which each addition of a name is checked.

cluebat/ok jsing
ok inoguchi on earlier version

Set is_trusted in x509_verify_ctx_add_chain()

If we're about to add a chain we have a trust path, so we have at least
one trusted certificate. This fixes a thinko from r1.31 and fixes the
openssl(1) cms verify test.

ok jsing (who had the same diff)

Rename depth to num_untrusted so it identifies what it actually represents.

ok tb@

Avoid passing last and depth to x509_verify_cert_error() on ENOMEM.

In x509_verify_ctx_set_xsc_chain(), an ENOMEM case is currently passing
the last certificate and depth (which is no longer actually depth) to
x509_verify_cert_error(). Given we've hit an ENOMEM situation, neither
of these are useful so remove both.

ok tb@

Make the new validator check for EXFLAG_CRITICAL

As should be obvious from the name and the comment in x509_vfy.h
int last_untrusted; /* index of last untrusted cert */
last_untrusted actually counts the number of untrusted certs at the
bottom of the chain.

Unfortunately, an earlier fix introducing x509_verify_set_xsc_chain()
assumed that last_untrusted actually meant the index of the last
untrusted cert in the chain, resulting in an off-by-one, which in turn
led to x509_vfy_check_chain_extension() skipping the check for the
EXFLAG_CRITICAL flag.

A second bug in x509_verify_set_xsc_chain() assumed that it is always
called with a trusted root, which is not necessarily the case anymore.
Address this with a temporary fix which will have to be revisited once
we will allow chains with more than one trusted cert.

Reported with a test case by tobhe.

ok jsing tobhe

Set chain on xsc on chain build failure.

Prior to calling the callback, ensure that the current (invalid and likely
incomplete) chain is set on the xsc. Some things (like auto chain) depend
on this functionality.

ok beck@

Bail out early after finding an single chain if we are have been called from
x509_vfy and have an xsc. There's no point in finding more chains since that
API can not return them, and all we do is trigger buggy callbacks in
calling software.

ok jsing@

search the intermediates only after searching the root certs, clarify
this in the comments. helps avoid annoying situations with the legacy
callback

ok jsing@

Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new verifier.

Yet another mostly meaningless error value...

Noted by and ok tb@

Gracefully handle root certificates being both trusted and untrusted.

When a certificate (namely a root) is specified as both a trusted and
untrusted certificate, the new verifier will find multiple chains - the
first being back to the trusted root certificate and a second via the root
that is untrusted, followed by the trusted root certificate. This situation
can be triggered by a server that (unnecessarily) includes the root
certificate in its certificate list.

While this validates correctly (using the first chain), it means that we
encounter a failure while building the second chain due to the root
certificate already being in the chain. When this occurs we call the verify
callback indicating a bad certificate. Some sensitive software (including
bacula and icinga), treat this single bad chain callback as terminal, even
though we successfully verify the certificate.

Avoid this problem by simply dumping the chain if we encounter a situation
where the certificate is already in the chain and also a trusted root -
we'll have already picked up the trusted root as a shorter path.

Issue with icinga2 initially reported by Theodore Wynnychenko.

Fix tested by sthen@ for both bacula and icinga2.

ok tb@

Remove two reduntat memset calls.

pointed out by jsing

Plug leak in x509_verify_chain_dup()

x509_verify_chain_new() allocates a few members of a certificate chain:
an empty stack of certificates, a list of errors encountered while
validating the chain, and a list of name constraints. The function to
copy a chain would allocate a new chain using x509_verify_chain_new()
and then clobber its members by copies of the old chain. Fix this by
replacing x509_verify_chain_new() with calloc().

Found by review while investigating the report by Hanno Zysik who
found the same leak using valgrind. This is a cleaner version of
my initial fix from jsing.

ok jsing

zap ugly empty line before closing brace

Use X509_V_OK instead of 0.

ok beck@ tb@

Add back an X509_STORE_CTX error code assignment.

This was inadvertently removed in r1.19.

Spotted by tb@

ok beck@ tb@

Return the specific failure for a "self signed certificate" in the chain
in order to be compatible with the openssl error craziness in the legacy
verifier case.

This will fix a regress problem noticed by znc

ok tb@

Handle additional certificate error cases in new X.509 verifier.

With the old verifier, the verify callback can always return 1 instructing
the verifier to simply continue regardless of a certificate verification
failure (e.g. the certificate is expired or revoked). This would result
in a chain being built, however the first error encountered would be
persisted, which allows the caller to build the chain, have the
verification process succeed, yet upon inspecting the error code note
that the chain is not valid for some reason.

Mimic this behaviour by keeping track of certificate errors while building
chains - when we finish verification, find the certificate error closest
to the leaf certificate and expose that via the X509_STORE_CTX. There are
various corner cases that we also have to handle, like the fact that we
keep an certificate error until we find the issuer, at which point we have
to clear it.

Issue reported by Ilya Shipitcin due to failing haproxy regression tests.

With much discussion and input from beck@ and tb@!

ok beck@ tb@

Fix bad indent.

Hook X509_STORE_CTX get_issuer() callback from new X509 verifier.

If we fail to find a parent certificate from either the supplied roots or
intermediates and we have a X509_STORE_CTX, call its get_issuer() callback
to see if it can supply a suitable certificate. This makes things like
certificates by directory (aka by_dir) work correctly.

Issue noted by Uwe Werler <uwe@werler.is>

ok beck@ tb@

Add a safety net to ensure that we set an error on the store context.

Suggested by and discussed with beck

If x509_verify() fails, ensure that the error is also set on the store
context. This is what is returned in SSL_get_verify_result().

Spotted and initial diff from jeremy; discussed with jsing
ok beck

Make sure that x509_vfy_check_id() failure also sets ctx->error, not only
ctx->xsc->error. Will be needed in an upcoming diff.

from beck

branches: 1.13.4;
Ensure leaf is set up on X509_STORE_CTX before verification.

Previously the leaf certificate was only being set up on the X509_STORE_CTX
after two verification steps were performed, however at least one of those
steps could result in the verification callback being triggered and
existing code breaking.

Issue noticed by Raf Czlonka when attempting to connect to talk.google.com
using profanity (which does not set SNI and ends up receiving an invalid
certificate).

ok beck@ deraadt@ tb@

Ensure chain is set on the X509_STORE_CTX before triggering callback.

Various software expects the previous behaviour where the certificate chain
is available on the X509_STORE_CTX when the verify callback is triggered.

Issue hit by bket@ with lastpass-cli which has built in certificate
pinning that is checked via the verify callback.

Fix confirmed by bket@.

ok beck@

remove superfluous NULL check
ok tb@

Make check in x509_verify_ctx_set_max_signatures() consistent with others.

ok beck@

Dedup code in x509_verify_ctx_new_from_xsc().

Rather than duplicating code, have x509_verify_ctx_new_from_xsc() call
x509_verify_ctx_new(), then handle the xsc specific parts.

ok beck@

set error_depth and current_cert to make more legacy callbacks that don't check
happy

ok tb@

Deduplicate the time validation code between the legacy and new
verification code.

ok jsing@

Set error if we are given an NULL ctx in x509_verify, and set error
to X509_V_ERR_OUT_OF_MEM instead of UNSPECIFIED in chain_append when
allocation fails.

noticed by ingo@
ok tb@

nuke a stray space

remove unneded variable "time"

noticed by llvm static analyzer
ok tb@

fix bug introduced on review where refactor made it possible to
eat a NULL.

found by clang static analyzer
ok tb@

Correctly fix double free introduced on review.

the roots for a ctx are only freed in the free function, not in the
clear function, so that a ctx can be re-used with the same roots.

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Fix the verifier to use the trust store

the trust store is yet another obscure way to add a trust anchor

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Remove a misplaced empty line

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@

Revert "Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new
verifier." (r1.27). While this may have "fixed" one corner case, it
broke expectations of Perl Net::SSLeay and Ruby OpenSSL regression
tests.

ok bcook

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Fix checks of memory caps of constraints names

x509_internal.h defines caps on the number of name constraints and
other names (such as subjectAltNames) that we want to allocate per
cert chain. These limits are checked too late. In a particularly
silly cert that jan found on ugos.ugm.ac.id 443, we ended up
allocating six times 2048 x509_constraint_name structures before
deciding that these are more than 512.

Fix this by adding a names_max member to x509_constraints_names which
is set on allocation against which each addition of a name is checked.

cluebat/ok jsing
ok inoguchi on earlier version

Set is_trusted in x509_verify_ctx_add_chain()

If we're about to add a chain we have a trust path, so we have at least
one trusted certificate. This fixes a thinko from r1.31 and fixes the
openssl(1) cms verify test.

ok jsing (who had the same diff)

Rename depth to num_untrusted so it identifies what it actually represents.

ok tb@

Avoid passing last and depth to x509_verify_cert_error() on ENOMEM.

In x509_verify_ctx_set_xsc_chain(), an ENOMEM case is currently passing
the last certificate and depth (which is no longer actually depth) to
x509_verify_cert_error(). Given we've hit an ENOMEM situation, neither
of these are useful so remove both.

ok tb@

Make the new validator check for EXFLAG_CRITICAL

As should be obvious from the name and the comment in x509_vfy.h
int last_untrusted; /* index of last untrusted cert */
last_untrusted actually counts the number of untrusted certs at the
bottom of the chain.

Unfortunately, an earlier fix introducing x509_verify_set_xsc_chain()
assumed that last_untrusted actually meant the index of the last
untrusted cert in the chain, resulting in an off-by-one, which in turn
led to x509_vfy_check_chain_extension() skipping the check for the
EXFLAG_CRITICAL flag.

A second bug in x509_verify_set_xsc_chain() assumed that it is always
called with a trusted root, which is not necessarily the case anymore.
Address this with a temporary fix which will have to be revisited once
we will allow chains with more than one trusted cert.

Reported with a test case by tobhe.

ok jsing tobhe

Set chain on xsc on chain build failure.

Prior to calling the callback, ensure that the current (invalid and likely
incomplete) chain is set on the xsc. Some things (like auto chain) depend
on this functionality.

ok beck@

Bail out early after finding an single chain if we are have been called from
x509_vfy and have an xsc. There's no point in finding more chains since that
API can not return them, and all we do is trigger buggy callbacks in
calling software.

ok jsing@

search the intermediates only after searching the root certs, clarify
this in the comments. helps avoid annoying situations with the legacy
callback

ok jsing@

Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new verifier.

Yet another mostly meaningless error value...

Noted by and ok tb@

Gracefully handle root certificates being both trusted and untrusted.

When a certificate (namely a root) is specified as both a trusted and
untrusted certificate, the new verifier will find multiple chains - the
first being back to the trusted root certificate and a second via the root
that is untrusted, followed by the trusted root certificate. This situation
can be triggered by a server that (unnecessarily) includes the root
certificate in its certificate list.

While this validates correctly (using the first chain), it means that we
encounter a failure while building the second chain due to the root
certificate already being in the chain. When this occurs we call the verify
callback indicating a bad certificate. Some sensitive software (including
bacula and icinga), treat this single bad chain callback as terminal, even
though we successfully verify the certificate.

Avoid this problem by simply dumping the chain if we encounter a situation
where the certificate is already in the chain and also a trusted root -
we'll have already picked up the trusted root as a shorter path.

Issue with icinga2 initially reported by Theodore Wynnychenko.

Fix tested by sthen@ for both bacula and icinga2.

ok tb@

Remove two reduntat memset calls.

pointed out by jsing

Plug leak in x509_verify_chain_dup()

x509_verify_chain_new() allocates a few members of a certificate chain:
an empty stack of certificates, a list of errors encountered while
validating the chain, and a list of name constraints. The function to
copy a chain would allocate a new chain using x509_verify_chain_new()
and then clobber its members by copies of the old chain. Fix this by
replacing x509_verify_chain_new() with calloc().

Found by review while investigating the report by Hanno Zysik who
found the same leak using valgrind. This is a cleaner version of
my initial fix from jsing.

ok jsing

zap ugly empty line before closing brace

Use X509_V_OK instead of 0.

ok beck@ tb@

Add back an X509_STORE_CTX error code assignment.

This was inadvertently removed in r1.19.

Spotted by tb@

ok beck@ tb@

Return the specific failure for a "self signed certificate" in the chain
in order to be compatible with the openssl error craziness in the legacy
verifier case.

This will fix a regress problem noticed by znc

ok tb@

Handle additional certificate error cases in new X.509 verifier.

With the old verifier, the verify callback can always return 1 instructing
the verifier to simply continue regardless of a certificate verification
failure (e.g. the certificate is expired or revoked). This would result
in a chain being built, however the first error encountered would be
persisted, which allows the caller to build the chain, have the
verification process succeed, yet upon inspecting the error code note
that the chain is not valid for some reason.

Mimic this behaviour by keeping track of certificate errors while building
chains - when we finish verification, find the certificate error closest
to the leaf certificate and expose that via the X509_STORE_CTX. There are
various corner cases that we also have to handle, like the fact that we
keep an certificate error until we find the issuer, at which point we have
to clear it.

Issue reported by Ilya Shipitcin due to failing haproxy regression tests.

With much discussion and input from beck@ and tb@!

ok beck@ tb@

Fix bad indent.

Hook X509_STORE_CTX get_issuer() callback from new X509 verifier.

If we fail to find a parent certificate from either the supplied roots or
intermediates and we have a X509_STORE_CTX, call its get_issuer() callback
to see if it can supply a suitable certificate. This makes things like
certificates by directory (aka by_dir) work correctly.

Issue noted by Uwe Werler <uwe@werler.is>

ok beck@ tb@

Add a safety net to ensure that we set an error on the store context.

Suggested by and discussed with beck

If x509_verify() fails, ensure that the error is also set on the store
context. This is what is returned in SSL_get_verify_result().

Spotted and initial diff from jeremy; discussed with jsing
ok beck

Make sure that x509_vfy_check_id() failure also sets ctx->error, not only
ctx->xsc->error. Will be needed in an upcoming diff.

from beck

branches: 1.13.4;
Ensure leaf is set up on X509_STORE_CTX before verification.

Previously the leaf certificate was only being set up on the X509_STORE_CTX
after two verification steps were performed, however at least one of those
steps could result in the verification callback being triggered and
existing code breaking.

Issue noticed by Raf Czlonka when attempting to connect to talk.google.com
using profanity (which does not set SNI and ends up receiving an invalid
certificate).

ok beck@ deraadt@ tb@

Ensure chain is set on the X509_STORE_CTX before triggering callback.

Various software expects the previous behaviour where the certificate chain
is available on the X509_STORE_CTX when the verify callback is triggered.

Issue hit by bket@ with lastpass-cli which has built in certificate
pinning that is checked via the verify callback.

Fix confirmed by bket@.

ok beck@

remove superfluous NULL check
ok tb@

Make check in x509_verify_ctx_set_max_signatures() consistent with others.

ok beck@

Dedup code in x509_verify_ctx_new_from_xsc().

Rather than duplicating code, have x509_verify_ctx_new_from_xsc() call
x509_verify_ctx_new(), then handle the xsc specific parts.

ok beck@

set error_depth and current_cert to make more legacy callbacks that don't check
happy

ok tb@

Deduplicate the time validation code between the legacy and new
verification code.

ok jsing@

Set error if we are given an NULL ctx in x509_verify, and set error
to X509_V_ERR_OUT_OF_MEM instead of UNSPECIFIED in chain_append when
allocation fails.

noticed by ingo@
ok tb@

nuke a stray space

remove unneded variable "time"

noticed by llvm static analyzer
ok tb@

fix bug introduced on review where refactor made it possible to
eat a NULL.

found by clang static analyzer
ok tb@

Correctly fix double free introduced on review.

the roots for a ctx are only freed in the free function, not in the
clear function, so that a ctx can be re-used with the same roots.

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Eliminate the timegm(3) dependency in libcrypto

timegm(3) is not available on some operating systems we support in
portable. We currently use musl's implementation, for which gcc-13
decided to emit warnings (which seem incorrect in general and are
irrelevant in this case anyway). Instead of patching this up and
diverge from upstream, we can avoid reports about compiler warnings
by simply not depending on this function.

Rework the caching of notBefore and notAfter by replacing timegm(3)
with asn1_time_tm_to_time_t(3). Also make this API properly error
checkable since at the time x509v3_cache_extensions(3) is called,
nothing is known about the cert, in particular not whether it isn't
malformed one way or the other.

suggested by and ok beck

Remove a misplaced empty line

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@

Revert "Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new
verifier." (r1.27). While this may have "fixed" one corner case, it
broke expectations of Perl Net::SSLeay and Ruby OpenSSL regression
tests.

ok bcook

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Fix checks of memory caps of constraints names

x509_internal.h defines caps on the number of name constraints and
other names (such as subjectAltNames) that we want to allocate per
cert chain. These limits are checked too late. In a particularly
silly cert that jan found on ugos.ugm.ac.id 443, we ended up
allocating six times 2048 x509_constraint_name structures before
deciding that these are more than 512.

Fix this by adding a names_max member to x509_constraints_names which
is set on allocation against which each addition of a name is checked.

cluebat/ok jsing
ok inoguchi on earlier version

Set is_trusted in x509_verify_ctx_add_chain()

If we're about to add a chain we have a trust path, so we have at least
one trusted certificate. This fixes a thinko from r1.31 and fixes the
openssl(1) cms verify test.

ok jsing (who had the same diff)

Rename depth to num_untrusted so it identifies what it actually represents.

ok tb@

Avoid passing last and depth to x509_verify_cert_error() on ENOMEM.

In x509_verify_ctx_set_xsc_chain(), an ENOMEM case is currently passing
the last certificate and depth (which is no longer actually depth) to
x509_verify_cert_error(). Given we've hit an ENOMEM situation, neither
of these are useful so remove both.

ok tb@

Make the new validator check for EXFLAG_CRITICAL

As should be obvious from the name and the comment in x509_vfy.h
int last_untrusted; /* index of last untrusted cert */
last_untrusted actually counts the number of untrusted certs at the
bottom of the chain.

Unfortunately, an earlier fix introducing x509_verify_set_xsc_chain()
assumed that last_untrusted actually meant the index of the last
untrusted cert in the chain, resulting in an off-by-one, which in turn
led to x509_vfy_check_chain_extension() skipping the check for the
EXFLAG_CRITICAL flag.

A second bug in x509_verify_set_xsc_chain() assumed that it is always
called with a trusted root, which is not necessarily the case anymore.
Address this with a temporary fix which will have to be revisited once
we will allow chains with more than one trusted cert.

Reported with a test case by tobhe.

ok jsing tobhe

Set chain on xsc on chain build failure.

Prior to calling the callback, ensure that the current (invalid and likely
incomplete) chain is set on the xsc. Some things (like auto chain) depend
on this functionality.

ok beck@

Bail out early after finding an single chain if we are have been called from
x509_vfy and have an xsc. There's no point in finding more chains since that
API can not return them, and all we do is trigger buggy callbacks in
calling software.

ok jsing@

search the intermediates only after searching the root certs, clarify
this in the comments. helps avoid annoying situations with the legacy
callback

ok jsing@

Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new verifier.

Yet another mostly meaningless error value...

Noted by and ok tb@

Gracefully handle root certificates being both trusted and untrusted.

When a certificate (namely a root) is specified as both a trusted and
untrusted certificate, the new verifier will find multiple chains - the
first being back to the trusted root certificate and a second via the root
that is untrusted, followed by the trusted root certificate. This situation
can be triggered by a server that (unnecessarily) includes the root
certificate in its certificate list.

While this validates correctly (using the first chain), it means that we
encounter a failure while building the second chain due to the root
certificate already being in the chain. When this occurs we call the verify
callback indicating a bad certificate. Some sensitive software (including
bacula and icinga), treat this single bad chain callback as terminal, even
though we successfully verify the certificate.

Avoid this problem by simply dumping the chain if we encounter a situation
where the certificate is already in the chain and also a trusted root -
we'll have already picked up the trusted root as a shorter path.

Issue with icinga2 initially reported by Theodore Wynnychenko.

Fix tested by sthen@ for both bacula and icinga2.

ok tb@

Remove two reduntat memset calls.

pointed out by jsing

Plug leak in x509_verify_chain_dup()

x509_verify_chain_new() allocates a few members of a certificate chain:
an empty stack of certificates, a list of errors encountered while
validating the chain, and a list of name constraints. The function to
copy a chain would allocate a new chain using x509_verify_chain_new()
and then clobber its members by copies of the old chain. Fix this by
replacing x509_verify_chain_new() with calloc().

Found by review while investigating the report by Hanno Zysik who
found the same leak using valgrind. This is a cleaner version of
my initial fix from jsing.

ok jsing

zap ugly empty line before closing brace

Use X509_V_OK instead of 0.

ok beck@ tb@

Add back an X509_STORE_CTX error code assignment.

This was inadvertently removed in r1.19.

Spotted by tb@

ok beck@ tb@

Return the specific failure for a "self signed certificate" in the chain
in order to be compatible with the openssl error craziness in the legacy
verifier case.

This will fix a regress problem noticed by znc

ok tb@

Handle additional certificate error cases in new X.509 verifier.

With the old verifier, the verify callback can always return 1 instructing
the verifier to simply continue regardless of a certificate verification
failure (e.g. the certificate is expired or revoked). This would result
in a chain being built, however the first error encountered would be
persisted, which allows the caller to build the chain, have the
verification process succeed, yet upon inspecting the error code note
that the chain is not valid for some reason.

Mimic this behaviour by keeping track of certificate errors while building
chains - when we finish verification, find the certificate error closest
to the leaf certificate and expose that via the X509_STORE_CTX. There are
various corner cases that we also have to handle, like the fact that we
keep an certificate error until we find the issuer, at which point we have
to clear it.

Issue reported by Ilya Shipitcin due to failing haproxy regression tests.

With much discussion and input from beck@ and tb@!

ok beck@ tb@

Fix bad indent.

Hook X509_STORE_CTX get_issuer() callback from new X509 verifier.

If we fail to find a parent certificate from either the supplied roots or
intermediates and we have a X509_STORE_CTX, call its get_issuer() callback
to see if it can supply a suitable certificate. This makes things like
certificates by directory (aka by_dir) work correctly.

Issue noted by Uwe Werler <uwe@werler.is>

ok beck@ tb@

Add a safety net to ensure that we set an error on the store context.

Suggested by and discussed with beck

If x509_verify() fails, ensure that the error is also set on the store
context. This is what is returned in SSL_get_verify_result().

Spotted and initial diff from jeremy; discussed with jsing
ok beck

Make sure that x509_vfy_check_id() failure also sets ctx->error, not only
ctx->xsc->error. Will be needed in an upcoming diff.

from beck

branches: 1.13.4;
Ensure leaf is set up on X509_STORE_CTX before verification.

Previously the leaf certificate was only being set up on the X509_STORE_CTX
after two verification steps were performed, however at least one of those
steps could result in the verification callback being triggered and
existing code breaking.

Issue noticed by Raf Czlonka when attempting to connect to talk.google.com
using profanity (which does not set SNI and ends up receiving an invalid
certificate).

ok beck@ deraadt@ tb@

Ensure chain is set on the X509_STORE_CTX before triggering callback.

Various software expects the previous behaviour where the certificate chain
is available on the X509_STORE_CTX when the verify callback is triggered.

Issue hit by bket@ with lastpass-cli which has built in certificate
pinning that is checked via the verify callback.

Fix confirmed by bket@.

ok beck@

remove superfluous NULL check
ok tb@

Make check in x509_verify_ctx_set_max_signatures() consistent with others.

ok beck@

Dedup code in x509_verify_ctx_new_from_xsc().

Rather than duplicating code, have x509_verify_ctx_new_from_xsc() call
x509_verify_ctx_new(), then handle the xsc specific parts.

ok beck@

set error_depth and current_cert to make more legacy callbacks that don't check
happy

ok tb@

Deduplicate the time validation code between the legacy and new
verification code.

ok jsing@

Set error if we are given an NULL ctx in x509_verify, and set error
to X509_V_ERR_OUT_OF_MEM instead of UNSPECIFIED in chain_append when
allocation fails.

noticed by ingo@
ok tb@

nuke a stray space

remove unneded variable "time"

noticed by llvm static analyzer
ok tb@

fix bug introduced on review where refactor made it possible to
eat a NULL.

found by clang static analyzer
ok tb@

Correctly fix double free introduced on review.

the roots for a ctx are only freed in the free function, not in the
clear function, so that a ctx can be re-used with the same roots.

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Remove a misplaced empty line

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@

Revert "Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new
verifier." (r1.27). While this may have "fixed" one corner case, it
broke expectations of Perl Net::SSLeay and Ruby OpenSSL regression
tests.

ok bcook

Use EXFLAG_INVALID to handle out of memory and parse errors in
x509v3_cache_extensions().

ok tb@

Fix checks of memory caps of constraints names

x509_internal.h defines caps on the number of name constraints and
other names (such as subjectAltNames) that we want to allocate per
cert chain. These limits are checked too late. In a particularly
silly cert that jan found on ugos.ugm.ac.id 443, we ended up
allocating six times 2048 x509_constraint_name structures before
deciding that these are more than 512.

Fix this by adding a names_max member to x509_constraints_names which
is set on allocation against which each addition of a name is checked.

cluebat/ok jsing
ok inoguchi on earlier version

Set is_trusted in x509_verify_ctx_add_chain()

If we're about to add a chain we have a trust path, so we have at least
one trusted certificate. This fixes a thinko from r1.31 and fixes the
openssl(1) cms verify test.

ok jsing (who had the same diff)

Rename depth to num_untrusted so it identifies what it actually represents.

ok tb@

Avoid passing last and depth to x509_verify_cert_error() on ENOMEM.

In x509_verify_ctx_set_xsc_chain(), an ENOMEM case is currently passing
the last certificate and depth (which is no longer actually depth) to
x509_verify_cert_error(). Given we've hit an ENOMEM situation, neither
of these are useful so remove both.

ok tb@

Make the new validator check for EXFLAG_CRITICAL

As should be obvious from the name and the comment in x509_vfy.h
int last_untrusted; /* index of last untrusted cert */
last_untrusted actually counts the number of untrusted certs at the
bottom of the chain.

Unfortunately, an earlier fix introducing x509_verify_set_xsc_chain()
assumed that last_untrusted actually meant the index of the last
untrusted cert in the chain, resulting in an off-by-one, which in turn
led to x509_vfy_check_chain_extension() skipping the check for the
EXFLAG_CRITICAL flag.

A second bug in x509_verify_set_xsc_chain() assumed that it is always
called with a trusted root, which is not necessarily the case anymore.
Address this with a temporary fix which will have to be revisited once
we will allow chains with more than one trusted cert.

Reported with a test case by tobhe.

ok jsing tobhe

Set chain on xsc on chain build failure.

Prior to calling the callback, ensure that the current (invalid and likely
incomplete) chain is set on the xsc. Some things (like auto chain) depend
on this functionality.

ok beck@

Bail out early after finding an single chain if we are have been called from
x509_vfy and have an xsc. There's no point in finding more chains since that
API can not return them, and all we do is trigger buggy callbacks in
calling software.

ok jsing@

search the intermediates only after searching the root certs, clarify
this in the comments. helps avoid annoying situations with the legacy
callback

ok jsing@

Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE in new verifier.

Yet another mostly meaningless error value...

Noted by and ok tb@

Gracefully handle root certificates being both trusted and untrusted.

When a certificate (namely a root) is specified as both a trusted and
untrusted certificate, the new verifier will find multiple chains - the
first being back to the trusted root certificate and a second via the root
that is untrusted, followed by the trusted root certificate. This situation
can be triggered by a server that (unnecessarily) includes the root
certificate in its certificate list.

While this validates correctly (using the first chain), it means that we
encounter a failure while building the second chain due to the root
certificate already being in the chain. When this occurs we call the verify
callback indicating a bad certificate. Some sensitive software (including
bacula and icinga), treat this single bad chain callback as terminal, even
though we successfully verify the certificate.

Avoid this problem by simply dumping the chain if we encounter a situation
where the certificate is already in the chain and also a trusted root -
we'll have already picked up the trusted root as a shorter path.

Issue with icinga2 initially reported by Theodore Wynnychenko.

Fix tested by sthen@ for both bacula and icinga2.

ok tb@

Remove two reduntat memset calls.

pointed out by jsing

Plug leak in x509_verify_chain_dup()

x509_verify_chain_new() allocates a few members of a certificate chain:
an empty stack of certificates, a list of errors encountered while
validating the chain, and a list of name constraints. The function to
copy a chain would allocate a new chain using x509_verify_chain_new()
and then clobber its members by copies of the old chain. Fix this by
replacing x509_verify_chain_new() with calloc().

Found by review while investigating the report by Hanno Zysik who
found the same leak using valgrind. This is a cleaner version of
my initial fix from jsing.

ok jsing

zap ugly empty line before closing brace

Use X509_V_OK instead of 0.

ok beck@ tb@

Add back an X509_STORE_CTX error code assignment.

This was inadvertently removed in r1.19.

Spotted by tb@

ok beck@ tb@

Return the specific failure for a "self signed certificate" in the chain
in order to be compatible with the openssl error craziness in the legacy
verifier case.

This will fix a regress problem noticed by znc

ok tb@

Handle additional certificate error cases in new X.509 verifier.

With the old verifier, the verify callback can always return 1 instructing
the verifier to simply continue regardless of a certificate verification
failure (e.g. the certificate is expired or revoked). This would result
in a chain being built, however the first error encountered would be
persisted, which allows the caller to build the chain, have the
verification process succeed, yet upon inspecting the error code note
that the chain is not valid for some reason.

Mimic this behaviour by keeping track of certificate errors while building
chains - when we finish verification, find the certificate error closest
to the leaf certificate and expose that via the X509_STORE_CTX. There are
various corner cases that we also have to handle, like the fact that we
keep an certificate error until we find the issuer, at which point we have
to clear it.

Issue reported by Ilya Shipitcin due to failing haproxy regression tests.

With much discussion and input from beck@ and tb@!

ok beck@ tb@

Fix bad indent.

Hook X509_STORE_CTX get_issuer() callback from new X509 verifier.

If we fail to find a parent certificate from either the supplied roots or
intermediates and we have a X509_STORE_CTX, call its get_issuer() callback
to see if it can supply a suitable certificate. This makes things like
certificates by directory (aka by_dir) work correctly.

Issue noted by Uwe Werler <uwe@werler.is>

ok beck@ tb@

Add a safety net to ensure that we set an error on the store context.

Suggested by and discussed with beck

If x509_verify() fails, ensure that the error is also set on the store
context. This is what is returned in SSL_get_verify_result().

Spotted and initial diff from jeremy; discussed with jsing
ok beck

Make sure that x509_vfy_check_id() failure also sets ctx->error, not only
ctx->xsc->error. Will be needed in an upcoming diff.

from beck

branches: 1.13.4;
Ensure leaf is set up on X509_STORE_CTX before verification.

Previously the leaf certificate was only being set up on the X509_STORE_CTX
after two verification steps were performed, however at least one of those
steps could result in the verification callback being triggered and
existing code breaking.

Issue noticed by Raf Czlonka when attempting to connect to talk.google.com
using profanity (which does not set SNI and ends up receiving an invalid
certificate).

ok beck@ deraadt@ tb@

Ensure chain is set on the X509_STORE_CTX before triggering callback.

Various software expects the previous behaviour where the certificate chain
is available on the X509_STORE_CTX when the verify callback is triggered.

Issue hit by bket@ with lastpass-cli which has built in certificate
pinning that is checked via the verify callback.

Fix confirmed by bket@.

ok beck@

remove superfluous NULL check
ok tb@

Make check in x509_verify_ctx_set_max_signatures() consistent with others.

ok beck@

Dedup code in x509_verify_ctx_new_from_xsc().

Rather than duplicating code, have x509_verify_ctx_new_from_xsc() call
x509_verify_ctx_new(), then handle the xsc specific parts.

ok beck@

set error_depth and current_cert to make more legacy callbacks that don't check
happy

ok tb@

Deduplicate the time validation code between the legacy and new
verification code.

ok jsing@

Set error if we are given an NULL ctx in x509_verify, and set error
to X509_V_ERR_OUT_OF_MEM instead of UNSPECIFIED in chain_append when
allocation fails.

noticed by ingo@
ok tb@

nuke a stray space

remove unneded variable "time"

noticed by llvm static analyzer
ok tb@

fix bug introduced on review where refactor made it possible to
eat a NULL.

found by clang static analyzer
ok tb@

Correctly fix double free introduced on review.

the roots for a ctx are only freed in the free function, not in the
clear function, so that a ctx can be re-used with the same roots.

ok tb@

Add new x509 certificate chain validator in x509_verify.c

The new validator finds multiple validated chains to handle the modern
PKI cases which may frequently have multiple paths via different
intermediates to different roots. It is loosely based on golang's x509
validator

This includes integration so that the new validator can be used via
X509_verify_cert() as well as a new api x509_verify() which will
return multiple chains (similar to go).

The new validator is not enabled by default with this commit, this
will be changed in a follow on commit.

The new public API is not yet exposed, and will be finalized and
exposed with a man page and a library minor bump later.

ok tb@ inoguchi@ jsing@

Enable policy checking by default now that we are DAG implementation based.

This ensures that we will no longer silently ignore a certificate with
a critical policy extention by default.

ok tb@

Remove some dead code from the new verifier

The new verifier API is currently unused as we still operate the verifier
in legacy mode. Therefore ctx->xsc is always set and the EXFLAG_PROXY will
soon be dropped from the library, so this error on encountering proxy certs
is effectively doubly dead code.

ok jsing

Refactor x509v3_cache_extensions

Simplify x509v3_cache_extensions() by using a wrapper to avoid
duplication of code for locking and checking the EXFLAG_INVALID flag.

OK tb@

Don't do policy checking unless we were asked to do so.

ok tb@

Store errors that result from leaf certificate verification.

In the case that a verification callback is installed that tells the
verifier to continue when a certificate is invalid (e.g. expired),
any error resulting from the leaf certificate verification is not stored
and made available post verification, resulting in an incorrect error being
returned.

Also perform leaf certificate verification prior to adding the chain, which
avoids a potential memory leak (as noted by tb@).

Issue reported by Ilya Shipitsin, who encountered haproxy regress failures.

ok tb@

branches: 1.60.2;
Remove overly aggressive trust check in legacy verifier that breaks
p5-IO-Socket-SSL regress and regress/sbin/iked/live

Still passes the mutt regress that this was intended to fix.

ok tb@

Take away bogus error assignment before callback call.

Keep the depth which was needed.

This went an error too far, and broke openssl-ruby's callback
and error code sensitivity in it's tests.

With this removed, both my newly committed regress to verify
the same error codes and depths in the callback, and
openssl-ruby's tests pass again.

ok tb@

Fix the legacy verifier callback behaviour for untrusted certs.

The verifier callback is used by mutt to do a form of certificate
pinning where the callback gets fired and depending on a
cert saved to a file will decide to accept an untrusted cert.

This corrects two problems that affected this. The callback was not
getting the correct depth and chain for the error where mutt would
save the certificate in the first place, and then the callback was not
getting fired to allow it to override the failing certificate
validation.

thanks to Avon Robertson <avon.r@xtra.co.nz> for the report and
sthen@ for analysis.

"The callback is not an API, it's a gordian knot - tb@"
ok jsing@

Allow security_level to mestastasize into the verifier

The tentacles are everywhere. This checks that all certs in a chain
have keys and signature algorithms matching the requirements of the
security_level configured in the verify parameters.

ok beck jsing

Move leaf certificate checks to the last thing after chain validation.

While seemingly illogical and not what is done in Go's validator, this
mimics OpenSSL's behavior so that callback overrides for the expiry of
a certificate will not "sticky" override a failure to build a chain.

ok jsing@

KNF for a brace and zap trailing blank line

In some situations, the verifier would discard the error on an unvalidated
certificte chain. This would happen when the verification callback was
in use, instructing the verifier to continue unconditionally. This could
lead to incorrect decisions being made in software.

Put curly brace on the correct line.

In X509_STORE_CTX rename the misnamed last_untrusted to num_untrusted

ok jsing

Cache sha512 hash and parsed not_before and not_after with X509 cert.

Replace sha1 hash use with sha512 for certificate comparisons internal
to the library. use the cached sha512 for the validator's verification
cache.

Reduces our recomputation of hashes, and heavy use of time1 time
conversion functions noticed bu claudio@ in rpki client.

ok jsing@ tb@

Add RFC 3779 checks to both legacy and new verifier

OK beck@

branches: 1.49.2;
When calling the legacy callback, ensure we catch the case where it
has decided to change a succeess to a failure and change the error code.

Fixes a regression in the openssl-ruby tests which expect to test this
functionality.

ok tb@

Call the callback on success in new verifier in a compatible way
when we succeed with a chain, and ensure we do not call the callback
twice when the caller doesn't expect it. A refactor of the end of
the legacy verify code in x509_vfy is probably overdue, but this
should be done based on a piece that works. the important bit here
is this allows the perl regression tests in tree to pass.

Changes the previously committed regress tests to test the success
case callbacks to be known to pass.

ok bluhm@ tb@

Revert previous change that changed our default return for unable to
find leaf cert issuers. This breaks perl and ruby regress, as noticed
by tb that "we tried this before".

Jan's regress that cares about 21 vs 20 needs to change
ok tb@

Fix Jan's regress in openssl/x509 to do what it says it does,
then fix the only thing it still has complaints about which
is that we don't return the leaf version of the error code
when we can't verify the leaf (as opposed to the rest of the chain)

ok jan@ tb@

Don't call the verify callback twice on success.
This fixes a problem in the perl regress where it notices the
callback is called twice and complains.

ok tb@ bluhm@

Get rid of historical code to extract the roots in the legacy case.
Due to the need to support by_dir, we use the get_issuer stuff when running
in x509_vfy compatibility mode amyway - so just use it any time we are
doing that. Removes a bunch of yukky stuff and a "Don't Look Ethel"

ok tb@ jsing@

Remove the "dump_chain" flag and code. This was a workaround for a problem where
roots were not checked correctly before intermediates that has since been fixed
and is no longer necessary. It is regress checked by case 2c in
regress/lib/libcrypto/x509/verify.c
ok jsing@ tb@

Pull roots out of the trust store in the legacy xsc when building chains
to handly by_dir and fun things correctly. - fixes dlg@'s case and
by_dir regress in openssl-ruby

ok jsing@

Add a check_trust call to the legacy chain validation on chain add, remembering
the result in order to return the same errors as OpenSSL users expect to override
the generic "Untrusted cert" error.

This fixes the openssl-ruby timestamp test.

ok tb@

Refactor the legacy chain validation from the chain adding code into its
own function, in preparation for subesquent change. No functional change.

ok tb@

Use the x509_verify_cert_cache_extensions fuction instead of manually
calling the OpenSSL legacy cache extensions goo.

Requested by tb@
ok tb@

Add a bunch of workarond in the verifier to support partial chains and
the saving of the first error case so that the "autochain" craziness from
openssl will work with the new verifier. This should allow the new verification
code to work with a bunch of the autochain using cases in some software.
(and should allow us to stop using the legacy verifier with autochain)

ok tb@