README
1
2 OpenSSL 1.0.2u 20 Dec 2019
3
4 Copyright (c) 1998-2019 The OpenSSL Project
5 Copyright (c) 1995-1998 Eric A. Young, Tim J. Hudson
6 All rights reserved.
7
8 DESCRIPTION
9 -----------
10
11 The OpenSSL Project is a collaborative effort to develop a robust,
12 commercial-grade, fully featured, and Open Source toolkit implementing the
13 Secure Sockets Layer (SSLv3) and Transport Layer Security (TLS) protocols as
14 well as a full-strength general purpose cryptograpic library. The project is
15 managed by a worldwide community of volunteers that use the Internet to
16 communicate, plan, and develop the OpenSSL toolkit and its related
17 documentation.
18
19 OpenSSL is descended from the SSLeay library developed by Eric A. Young
20 and Tim J. Hudson. The OpenSSL toolkit is licensed under a dual-license (the
21 OpenSSL license plus the SSLeay license), which means that you are free to
22 get and use it for commercial and non-commercial purposes as long as you
23 fulfill the conditions of both licenses.
24
25 OVERVIEW
26 --------
27
28 The OpenSSL toolkit includes:
29
30 libssl.a:
31 Provides the client and server-side implementations for SSLv3 and TLS.
32
33 libcrypto.a:
34 Provides general cryptographic and X.509 support needed by SSL/TLS but
35 not logically part of it.
36
37 openssl:
38 A command line tool that can be used for:
39 Creation of key parameters
40 Creation of X.509 certificates, CSRs and CRLs
41 Calculation of message digests
42 Encryption and decryption
43 SSL/TLS client and server tests
44 Handling of S/MIME signed or encrypted mail
45 And more...
46
47 INSTALLATION
48 ------------
49
50 See the appropriate file:
51 INSTALL Linux, Unix, etc.
52 INSTALL.DJGPP DOS platform with DJGPP
53 INSTALL.NW Netware
54 INSTALL.OS2 OS/2
55 INSTALL.VMS VMS
56 INSTALL.W32 Windows (32bit)
57 INSTALL.W64 Windows (64bit)
58 INSTALL.WCE Windows CE
59
60 SUPPORT
61 -------
62
63 See the OpenSSL website www.openssl.org for details on how to obtain
64 commercial technical support.
65
66 If you have any problems with OpenSSL then please take the following steps
67 first:
68
69 - Download the latest version from the repository
70 to see if the problem has already been addressed
71 - Configure with no-asm
72 - Remove compiler optimisation flags
73
74 If you wish to report a bug then please include the following information
75 and create an issue on GitHub:
76
77 - On Unix systems:
78 Self-test report generated by 'make report'
79 - On other systems:
80 OpenSSL version: output of 'openssl version -a'
81 OS Name, Version, Hardware platform
82 Compiler Details (name, version)
83 - Application Details (name, version)
84 - Problem Description (steps that will reproduce the problem, if known)
85 - Stack Traceback (if the application dumps core)
86
87 Just because something doesn't work the way you expect does not mean it
88 is necessarily a bug in OpenSSL.
89
90 HOW TO CONTRIBUTE TO OpenSSL
91 ----------------------------
92
93 See CONTRIBUTING
94
95 LEGALITIES
96 ----------
97
98 A number of nations restrict the use or export of cryptography. If you
99 are potentially subject to such restrictions you should seek competent
100 professional legal advice before attempting to develop or distribute
101 cryptographic code.
102
README.ASN1
1
2OpenSSL ASN1 Revision
3=====================
4
5This document describes some of the issues relating to the new ASN1 code.
6
7Previous OpenSSL ASN1 problems
8=============================
9
10OK why did the OpenSSL ASN1 code need revising in the first place? Well
11there are lots of reasons some of which are included below...
12
131. The code is difficult to read and write. For every single ASN1 structure
14(e.g. SEQUENCE) four functions need to be written for new, free, encode and
15decode operations. This is a very painful and error prone operation. Very few
16people have ever written any OpenSSL ASN1 and those that have usually wish
17they hadn't.
18
192. Partly because of 1. the code is bloated and takes up a disproportionate
20amount of space. The SEQUENCE encoder is particularly bad: it essentially
21contains two copies of the same operation, one to compute the SEQUENCE length
22and the other to encode it.
23
243. The code is memory based: that is it expects to be able to read the whole
25structure from memory. This is fine for small structures but if you have a
26(say) 1Gb PKCS#7 signedData structure it isn't such a good idea...
27
284. The code for the ASN1 IMPLICIT tag is evil. It is handled by temporarily
29changing the tag to the expected one, attempting to read it, then changing it
30back again. This means that decode buffers have to be writable even though they
31are ultimately unchanged. This gets in the way of constification.
32
335. The handling of EXPLICIT isn't much better. It adds a chunk of code into
34the decoder and encoder for every EXPLICIT tag.
35
366. APPLICATION and PRIVATE tags aren't even supported at all.
37
387. Even IMPLICIT isn't complete: there is no support for implicitly tagged
39types that are not OPTIONAL.
40
418. Much of the code assumes that a tag will fit in a single octet. This is
42only true if the tag is 30 or less (mercifully tags over 30 are rare).
43
449. The ASN1 CHOICE type has to be largely handled manually, there aren't any
45macros that properly support it.
46
4710. Encoders have no concept of OPTIONAL and have no error checking. If the
48passed structure contains a NULL in a mandatory field it will not be encoded,
49resulting in an invalid structure.
50
5111. It is tricky to add ASN1 encoders and decoders to external applications.
52
53Template model
54==============
55
56One of the major problems with revision is the sheer volume of the ASN1 code.
57Attempts to change (for example) the IMPLICIT behaviour would result in a
58modification of *every* single decode function.
59
60I decided to adopt a template based approach. I'm using the term 'template'
61in a manner similar to SNACC templates: it has nothing to do with C++
62templates.
63
64A template is a description of an ASN1 module as several constant C structures.
65It describes in a machine readable way exactly how the ASN1 structure should
66behave. If this template contains enough detail then it is possible to write
67versions of new, free, encode, decode (and possibly others operations) that
68operate on templates.
69
70Instead of having to write code to handle each operation only a single
71template needs to be written. If new operations are needed (such as a 'print'
72operation) only a single new template based function needs to be written
73which will then automatically handle all existing templates.
74
75Plans for revision
76==================
77
78The revision will consist of the following steps. Other than the first two
79these can be handled in any order.
80
81o Design and write template new, free, encode and decode operations, initially
82memory based. *DONE*
83
84o Convert existing ASN1 code to template form. *IN PROGRESS*
85
86o Convert an existing ASN1 compiler (probably SNACC) to output templates
87in OpenSSL form.
88
89o Add support for BIO based ASN1 encoders and decoders to handle large
90structures, initially blocking I/O.
91
92o Add support for non blocking I/O: this is quite a bit harder than blocking
93I/O.
94
95o Add new ASN1 structures, such as OCSP, CRMF, S/MIME v3 (CMS), attribute
96certificates etc etc.
97
98Description of major changes
99============================
100
101The BOOLEAN type now takes three values. 0xff is TRUE, 0 is FALSE and -1 is
102absent. The meaning of absent depends on the context. If for example the
103boolean type is DEFAULT FALSE (as in the case of the critical flag for
104certificate extensions) then -1 is FALSE, if DEFAULT TRUE then -1 is TRUE.
105Usually the value will only ever be read via an API which will hide this from
106an application.
107
108There is an evil bug in the old ASN1 code that mishandles OPTIONAL with
109SEQUENCE OF or SET OF. These are both implemented as a STACK structure. The
110old code would omit the structure if the STACK was NULL (which is fine) or if
111it had zero elements (which is NOT OK). This causes problems because an empty
112SEQUENCE OF or SET OF will result in an empty STACK when it is decoded but when
113it is encoded it will be omitted resulting in different encodings. The new code
114only omits the encoding if the STACK is NULL, if it contains zero elements it
115is encoded and empty. There is an additional problem though: because an empty
116STACK was omitted, sometimes the corresponding *_new() function would
117initialize the STACK to empty so an application could immediately use it, if
118this is done with the new code (i.e. a NULL) it wont work. Therefore a new
119STACK should be allocated first. One instance of this is the X509_CRL list of
120revoked certificates: a helper function X509_CRL_add0_revoked() has been added
121for this purpose.
122
123The X509_ATTRIBUTE structure used to have an element called 'set' which took
124the value 1 if the attribute value was a SET OF or 0 if it was a single. Due
125to the behaviour of CHOICE in the new code this has been changed to a field
126called 'single' which is 0 for a SET OF and 1 for single. The old field has
127been deleted to deliberately break source compatibility. Since this structure
128is normally accessed via higher level functions this shouldn't break too much.
129
130The X509_REQ_INFO certificate request info structure no longer has a field
131called 'req_kludge'. This used to be set to 1 if the attributes field was
132(incorrectly) omitted. You can check to see if the field is omitted now by
133checking if the attributes field is NULL. Similarly if you need to omit
134the field then free attributes and set it to NULL.
135
136The top level 'detached' field in the PKCS7 structure is no longer set when
137a PKCS#7 structure is read in. PKCS7_is_detached() should be called instead.
138The behaviour of PKCS7_get_detached() is unaffected.
139
140The values of 'type' in the GENERAL_NAME structure have changed. This is
141because the old code use the ASN1 initial octet as the selector. The new
142code uses the index in the ASN1_CHOICE template.
143
144The DIST_POINT_NAME structure has changed to be a true CHOICE type.
145
146typedef struct DIST_POINT_NAME_st {
147int type;
148union {
149 STACK_OF(GENERAL_NAME) *fullname;
150 STACK_OF(X509_NAME_ENTRY) *relativename;
151} name;
152} DIST_POINT_NAME;
153
154This means that name.fullname or name.relativename should be set
155and type reflects the option. That is if name.fullname is set then
156type is 0 and if name.relativename is set type is 1.
157
158With the old code using the i2d functions would typically involve:
159
160unsigned char *buf, *p;
161int len;
162/* Find length of encoding */
163len = i2d_SOMETHING(x, NULL);
164/* Allocate buffer */
165buf = OPENSSL_malloc(len);
166if(buf == NULL) {
167 /* Malloc error */
168}
169/* Use temp variable because &p gets updated to point to end of
170 * encoding.
171 */
172p = buf;
173i2d_SOMETHING(x, &p);
174
175
176Using the new i2d you can also do:
177
178unsigned char *buf = NULL;
179int len;
180len = i2d_SOMETHING(x, &buf);
181if(len < 0) {
182 /* Malloc error */
183}
184
185and it will automatically allocate and populate a buffer with the
186encoding. After this call 'buf' will point to the start of the
187encoding which is len bytes long.
188
README.ENGINE
1 ENGINE
2 ======
3
4 With OpenSSL 0.9.6, a new component was added to support alternative
5 cryptography implementations, most commonly for interfacing with external
6 crypto devices (eg. accelerator cards). This component is called ENGINE,
7 and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
8 caused a little confusion as 0.9.6** releases were rolled in two
9 versions, a "standard" and an "engine" version. In development for 0.9.7,
10 the ENGINE code has been merged into the main branch and will be present
11 in the standard releases from 0.9.7 forwards.
12
13 There are currently built-in ENGINE implementations for the following
14 crypto devices:
15
16 o CryptoSwift
17 o Compaq Atalla
18 o nCipher CHIL
19 o Nuron
20 o Broadcom uBSec
21
22 In addition, dynamic binding to external ENGINE implementations is now
23 provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
24 section below for details.
25
26 At this stage, a number of things are still needed and are being worked on:
27
28 1 Integration of EVP support.
29 2 Configuration support.
30 3 Documentation!
31
321 With respect to EVP, this relates to support for ciphers and digests in
33 the ENGINE model so that alternative implementations of existing
34 algorithms/modes (or previously unimplemented ones) can be provided by
35 ENGINE implementations.
36
372 Configuration support currently exists in the ENGINE API itself, in the
38 form of "control commands". These allow an application to expose to the
39 user/admin the set of commands and parameter types a given ENGINE
40 implementation supports, and for an application to directly feed string
41 based input to those ENGINEs, in the form of name-value pairs. This is an
42 extensible way for ENGINEs to define their own "configuration" mechanisms
43 that are specific to a given ENGINE (eg. for a particular hardware
44 device) but that should be consistent across *all* OpenSSL-based
45 applications when they use that ENGINE. Work is in progress (or at least
46 in planning) for supporting these control commands from the CONF (or
47 NCONF) code so that applications using OpenSSL's existing configuration
48 file format can have ENGINE settings specified in much the same way.
49 Presently however, applications must use the ENGINE API itself to provide
50 such functionality. To see first hand the types of commands available
51 with the various compiled-in ENGINEs (see further down for dynamic
52 ENGINEs), use the "engine" openssl utility with full verbosity, ie;
53 openssl engine -vvvv
54
553 Documentation? Volunteers welcome! The source code is reasonably well
56 self-documenting, but some summaries and usage instructions are needed -
57 moreover, they are needed in the same POD format the existing OpenSSL
58 documentation is provided in. Any complete or incomplete contributions
59 would help make this happen.
60
61 STABILITY & BUG-REPORTS
62 =======================
63
64 What already exists is fairly stable as far as it has been tested, but
65 the test base has been a bit small most of the time. For the most part,
66 the vendors of the devices these ENGINEs support have contributed to the
67 development and/or testing of the implementations, and *usually* (with no
68 guarantees) have experience in using the ENGINE support to drive their
69 devices from common OpenSSL-based applications. Bugs and/or inexplicable
70 behaviour in using a specific ENGINE implementation should be sent to the
71 author of that implementation (if it is mentioned in the corresponding C
72 file), and in the case of implementations for commercial hardware
73 devices, also through whatever vendor support channels are available. If
74 none of this is possible, or the problem seems to be something about the
75 ENGINE API itself (ie. not necessarily specific to a particular ENGINE
76 implementation) then you should mail complete details to the relevant
77 OpenSSL mailing list. For a definition of "complete details", refer to
78 the OpenSSL "README" file. As for which list to send it to;
79
80 openssl-users: if you are *using* the ENGINE abstraction, either in an
81 pre-compiled application or in your own application code.
82
83 openssl-dev: if you are discussing problems with OpenSSL source code.
84
85 USAGE
86 =====
87
88 The default "openssl" ENGINE is always chosen when performing crypto
89 operations unless you specify otherwise. You must actively tell the
90 openssl utility commands to use anything else through a new command line
91 switch called "-engine". Also, if you want to use the ENGINE support in
92 your own code to do something similar, you must likewise explicitly
93 select the ENGINE implementation you want.
94
95 Depending on the type of hardware, system, and configuration, "settings"
96 may need to be applied to an ENGINE for it to function as expected/hoped.
97 The recommended way of doing this is for the application to support
98 ENGINE "control commands" so that each ENGINE implementation can provide
99 whatever configuration primitives it might require and the application
100 can allow the user/admin (and thus the hardware vendor's support desk
101 also) to provide any such input directly to the ENGINE implementation.
102 This way, applications do not need to know anything specific to any
103 device, they only need to provide the means to carry such user/admin
104 input through to the ENGINE in question. Ie. this connects *you* (and
105 your helpdesk) to the specific ENGINE implementation (and device), and
106 allows application authors to not get buried in hassle supporting
107 arbitrary devices they know (and care) nothing about.
108
109 A new "openssl" utility, "openssl engine", has been added in that allows
110 for testing and examination of ENGINE implementations. Basic usage
111 instructions are available by specifying the "-?" command line switch.
112
113 DYNAMIC ENGINES
114 ===============
115
116 The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
117 implementations that aren't pre-compiled and linked into OpenSSL-based
118 applications. This could be because existing compiled-in implementations
119 have known problems and you wish to use a newer version with an existing
120 application. It could equally be because the application (or OpenSSL
121 library) you are using simply doesn't have support for the ENGINE you
122 wish to use, and the ENGINE provider (eg. hardware vendor) is providing
123 you with a self-contained implementation in the form of a shared-library.
124 The other use-case for "dynamic" is with applications that wish to
125 maintain the smallest foot-print possible and so do not link in various
126 ENGINE implementations from OpenSSL, but instead leaves you to provide
127 them, if you want them, in the form of "dynamic"-loadable
128 shared-libraries. It should be possible for hardware vendors to provide
129 their own shared-libraries to support arbitrary hardware to work with
130 applications based on OpenSSL 0.9.7 or later. If you're using an
131 application based on 0.9.7 (or later) and the support you desire is only
132 announced for versions later than the one you need, ask the vendor to
133 backport their ENGINE to the version you need.
134
135 How does "dynamic" work?
136 ------------------------
137 The dynamic ENGINE has a special flag in its implementation such that
138 every time application code asks for the 'dynamic' ENGINE, it in fact
139 gets its own copy of it. As such, multi-threaded code (or code that
140 multiplexes multiple uses of 'dynamic' in a single application in any
141 way at all) does not get confused by 'dynamic' being used to do many
142 independent things. Other ENGINEs typically don't do this so there is
143 only ever 1 ENGINE structure of its type (and reference counts are used
144 to keep order). The dynamic ENGINE itself provides absolutely no
145 cryptographic functionality, and any attempt to "initialise" the ENGINE
146 automatically fails. All it does provide are a few "control commands"
147 that can be used to control how it will load an external ENGINE
148 implementation from a shared-library. To see these control commands,
149 use the command-line;
150
151 openssl engine -vvvv dynamic
152
153 The "SO_PATH" control command should be used to identify the
154 shared-library that contains the ENGINE implementation, and "NO_VCHECK"
155 might possibly be useful if there is a minor version conflict and you
156 (or a vendor helpdesk) is convinced you can safely ignore it.
157 "ID" is probably only needed if a shared-library implements
158 multiple ENGINEs, but if you know the engine id you expect to be using,
159 it doesn't hurt to specify it (and this provides a sanity check if
160 nothing else). "LIST_ADD" is only required if you actually wish the
161 loaded ENGINE to be discoverable by application code later on using the
162 ENGINE's "id". For most applications, this isn't necessary - but some
163 application authors may have nifty reasons for using it. The "LOAD"
164 command is the only one that takes no parameters and is the command
165 that uses the settings from any previous commands to actually *load*
166 the shared-library ENGINE implementation. If this command succeeds, the
167 (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
168 that has been loaded from the shared-library. As such, any control
169 commands supported by the loaded ENGINE could then be executed as per
170 normal. Eg. if ENGINE "foo" is implemented in the shared-library
171 "libfoo.so" and it supports some special control command "CMD_FOO", the
172 following code would load and use it (NB: obviously this code has no
173 error checking);
174
175 ENGINE *e = ENGINE_by_id("dynamic");
176 ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
177 ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
178 ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
179 ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
180
181 For testing, the "openssl engine" utility can be useful for this sort
182 of thing. For example the above code excerpt would achieve much the
183 same result as;
184
185 openssl engine dynamic \
186 -pre SO_PATH:/lib/libfoo.so \
187 -pre ID:foo \
188 -pre LOAD \
189 -pre "CMD_FOO:some input data"
190
191 Or to simply see the list of commands supported by the "foo" ENGINE;
192
193 openssl engine -vvvv dynamic \
194 -pre SO_PATH:/lib/libfoo.so \
195 -pre ID:foo \
196 -pre LOAD
197
198 Applications that support the ENGINE API and more specifically, the
199 "control commands" mechanism, will provide some way for you to pass
200 such commands through to ENGINEs. As such, you would select "dynamic"
201 as the ENGINE to use, and the parameters/commands you pass would
202 control the *actual* ENGINE used. Each command is actually a name-value
203 pair and the value can sometimes be omitted (eg. the "LOAD" command).
204 Whilst the syntax demonstrated in "openssl engine" uses a colon to
205 separate the command name from the value, applications may provide
206 their own syntax for making that separation (eg. a win32 registry
207 key-value pair may be used by some applications). The reason for the
208 "-pre" syntax in the "openssl engine" utility is that some commands
209 might be issued to an ENGINE *after* it has been initialised for use.
210 Eg. if an ENGINE implementation requires a smart-card to be inserted
211 during initialisation (or a PIN to be typed, or whatever), there may be
212 a control command you can issue afterwards to "forget" the smart-card
213 so that additional initialisation is no longer possible. In
214 applications such as web-servers, where potentially volatile code may
215 run on the same host system, this may provide some arguable security
216 value. In such a case, the command would be passed to the ENGINE after
217 it has been initialised for use, and so the "-post" switch would be
218 used instead. Applications may provide a different syntax for
219 supporting this distinction, and some may simply not provide it at all
220 ("-pre" is almost always what you're after, in reality).
221
222 How do I build a "dynamic" ENGINE?
223 ----------------------------------
224 This question is trickier - currently OpenSSL bundles various ENGINE
225 implementations that are statically built in, and any application that
226 calls the "ENGINE_load_builtin_engines()" function will automatically
227 have all such ENGINEs available (and occupying memory). Applications
228 that don't call that function have no ENGINEs available like that and
229 would have to use "dynamic" to load any such ENGINE - but on the other
230 hand such applications would only have the memory footprint of any
231 ENGINEs explicitly loaded using user/admin provided control commands.
232 The main advantage of not statically linking ENGINEs and only using
233 "dynamic" for hardware support is that any installation using no
234 "external" ENGINE suffers no unnecessary memory footprint from unused
235 ENGINEs. Likewise, installations that do require an ENGINE incur the
236 overheads from only *that* ENGINE once it has been loaded.
237
238 Sounds good? Maybe, but currently building an ENGINE implementation as
239 a shared-library that can be loaded by "dynamic" isn't automated in
240 OpenSSL's build process. It can be done manually quite easily however.
241 Such a shared-library can either be built with any OpenSSL code it
242 needs statically linked in, or it can link dynamically against OpenSSL
243 if OpenSSL itself is built as a shared library. The instructions are
244 the same in each case, but in the former (statically linked any
245 dependencies on OpenSSL) you must ensure OpenSSL is built with
246 position-independent code ("PIC"). The default OpenSSL compilation may
247 already specify the relevant flags to do this, but you should consult
248 with your compiler documentation if you are in any doubt.
249
250 This example will show building the "atalla" ENGINE in the
251 crypto/engine/ directory as a shared-library for use via the "dynamic"
252 ENGINE.
253 1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL
254 source tree.
255 2) Recompile at least one source file so you can see all the compiler
256 flags (and syntax) being used to build normally. Eg;
257 touch hw_atalla.c ; make
258 will rebuild "hw_atalla.o" using all such flags.
259 3) Manually enter the same compilation line to compile the
260 "hw_atalla.c" file but with the following two changes;
261 (a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
262 (b) change the output file from "hw_atalla.o" to something new,
263 eg. "tmp_atalla.o"
264 4) Link "tmp_atalla.o" into a shared-library using the top-level
265 OpenSSL libraries to resolve any dependencies. The syntax for doing
266 this depends heavily on your system/compiler and is a nightmare
267 known well to anyone who has worked with shared-library portability
268 before. 'gcc' on Linux, for example, would use the following syntax;
269 gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
270 5) Test your shared library using "openssl engine" as explained in the
271 previous section. Eg. from the top-level directory, you might try;
272 apps/openssl engine -vvvv dynamic \
273 -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
274 If the shared-library loads successfully, you will see both "-pre"
275 commands marked as "SUCCESS" and the list of control commands
276 displayed (because of "-vvvv") will be the control commands for the
277 *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
278 the "-t" switch to the utility if you want it to try and initialise
279 the atalla ENGINE for use to test any possible hardware/driver
280 issues.
281
282 PROBLEMS
283 ========
284
285 It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
286 A quick test done right before the release showed that trying "openssl speed
287 -engine cswift" generated errors. If the DSO gets enabled, an attempt is made
288 to write at memory address 0x00000002.
289
290