1 ENGINE 2 ====== 3 4 With OpenSSL 0.9.6, a new component was added to support alternative 5 cryptography implementations, most commonly for interfacing with external 6 crypto devices (eg. accelerator cards). This component is called ENGINE, 7 and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases) 8 caused a little confusion as 0.9.6** releases were rolled in two 9 versions, a "standard" and an "engine" version. In development for 0.9.7, 10 the ENGINE code has been merged into the main branch and will be present 11 in the standard releases from 0.9.7 forwards. 12 13 There are currently built-in ENGINE implementations for the following 14 crypto devices: 15 16 o CryptoSwift 17 o Compaq Atalla 18 o nCipher CHIL 19 o Nuron 20 o Broadcom uBSec 21 22 In addition, dynamic binding to external ENGINE implementations is now 23 provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" 24 section below for details. 25 26 At this stage, a number of things are still needed and are being worked on: 27 28 1 Integration of EVP support. 29 2 Configuration support. 30 3 Documentation! 31 321 With respect to EVP, this relates to support for ciphers and digests in 33 the ENGINE model so that alternative implementations of existing 34 algorithms/modes (or previously unimplemented ones) can be provided by 35 ENGINE implementations. 36 372 Configuration support currently exists in the ENGINE API itself, in the 38 form of "control commands". These allow an application to expose to the 39 user/admin the set of commands and parameter types a given ENGINE 40 implementation supports, and for an application to directly feed string 41 based input to those ENGINEs, in the form of name-value pairs. This is an 42 extensible way for ENGINEs to define their own "configuration" mechanisms 43 that are specific to a given ENGINE (eg. for a particular hardware 44 device) but that should be consistent across *all* OpenSSL-based 45 applications when they use that ENGINE. Work is in progress (or at least 46 in planning) for supporting these control commands from the CONF (or 47 NCONF) code so that applications using OpenSSL's existing configuration 48 file format can have ENGINE settings specified in much the same way. 49 Presently however, applications must use the ENGINE API itself to provide 50 such functionality. To see first hand the types of commands available 51 with the various compiled-in ENGINEs (see further down for dynamic 52 ENGINEs), use the "engine" openssl utility with full verbosity, ie; 53 openssl engine -vvvv 54 553 Documentation? Volunteers welcome! The source code is reasonably well 56 self-documenting, but some summaries and usage instructions are needed - 57 moreover, they are needed in the same POD format the existing OpenSSL 58 documentation is provided in. Any complete or incomplete contributions 59 would help make this happen. 60 61 STABILITY & BUG-REPORTS 62 ======================= 63 64 What already exists is fairly stable as far as it has been tested, but 65 the test base has been a bit small most of the time. For the most part, 66 the vendors of the devices these ENGINEs support have contributed to the 67 development and/or testing of the implementations, and *usually* (with no 68 guarantees) have experience in using the ENGINE support to drive their 69 devices from common OpenSSL-based applications. Bugs and/or inexplicable 70 behaviour in using a specific ENGINE implementation should be sent to the 71 author of that implementation (if it is mentioned in the corresponding C 72 file), and in the case of implementations for commercial hardware 73 devices, also through whatever vendor support channels are available. If 74 none of this is possible, or the problem seems to be something about the 75 ENGINE API itself (ie. not necessarily specific to a particular ENGINE 76 implementation) then you should mail complete details to the relevant 77 OpenSSL mailing list. For a definition of "complete details", refer to 78 the OpenSSL "README" file. As for which list to send it to; 79 80 openssl-users: if you are *using* the ENGINE abstraction, either in an 81 pre-compiled application or in your own application code. 82 83 openssl-dev: if you are discussing problems with OpenSSL source code. 84 85 USAGE 86 ===== 87 88 The default "openssl" ENGINE is always chosen when performing crypto 89 operations unless you specify otherwise. You must actively tell the 90 openssl utility commands to use anything else through a new command line 91 switch called "-engine". Also, if you want to use the ENGINE support in 92 your own code to do something similar, you must likewise explicitly 93 select the ENGINE implementation you want. 94 95 Depending on the type of hardware, system, and configuration, "settings" 96 may need to be applied to an ENGINE for it to function as expected/hoped. 97 The recommended way of doing this is for the application to support 98 ENGINE "control commands" so that each ENGINE implementation can provide 99 whatever configuration primitives it might require and the application 100 can allow the user/admin (and thus the hardware vendor's support desk 101 also) to provide any such input directly to the ENGINE implementation. 102 This way, applications do not need to know anything specific to any 103 device, they only need to provide the means to carry such user/admin 104 input through to the ENGINE in question. Ie. this connects *you* (and 105 your helpdesk) to the specific ENGINE implementation (and device), and 106 allows application authors to not get buried in hassle supporting 107 arbitrary devices they know (and care) nothing about. 108 109 A new "openssl" utility, "openssl engine", has been added in that allows 110 for testing and examination of ENGINE implementations. Basic usage 111 instructions are available by specifying the "-?" command line switch. 112 113 DYNAMIC ENGINES 114 =============== 115 116 The new "dynamic" ENGINE provides a low-overhead way to support ENGINE 117 implementations that aren't pre-compiled and linked into OpenSSL-based 118 applications. This could be because existing compiled-in implementations 119 have known problems and you wish to use a newer version with an existing 120 application. It could equally be because the application (or OpenSSL 121 library) you are using simply doesn't have support for the ENGINE you 122 wish to use, and the ENGINE provider (eg. hardware vendor) is providing 123 you with a self-contained implementation in the form of a shared-library. 124 The other use-case for "dynamic" is with applications that wish to 125 maintain the smallest foot-print possible and so do not link in various 126 ENGINE implementations from OpenSSL, but instead leaves you to provide 127 them, if you want them, in the form of "dynamic"-loadable 128 shared-libraries. It should be possible for hardware vendors to provide 129 their own shared-libraries to support arbitrary hardware to work with 130 applications based on OpenSSL 0.9.7 or later. If you're using an 131 application based on 0.9.7 (or later) and the support you desire is only 132 announced for versions later than the one you need, ask the vendor to 133 backport their ENGINE to the version you need. 134 135 How does "dynamic" work? 136 ------------------------ 137 The dynamic ENGINE has a special flag in its implementation such that 138 every time application code asks for the 'dynamic' ENGINE, it in fact 139 gets its own copy of it. As such, multi-threaded code (or code that 140 multiplexes multiple uses of 'dynamic' in a single application in any 141 way at all) does not get confused by 'dynamic' being used to do many 142 independent things. Other ENGINEs typically don't do this so there is 143 only ever 1 ENGINE structure of its type (and reference counts are used 144 to keep order). The dynamic ENGINE itself provides absolutely no 145 cryptographic functionality, and any attempt to "initialise" the ENGINE 146 automatically fails. All it does provide are a few "control commands" 147 that can be used to control how it will load an external ENGINE 148 implementation from a shared-library. To see these control commands, 149 use the command-line; 150 151 openssl engine -vvvv dynamic 152 153 The "SO_PATH" control command should be used to identify the 154 shared-library that contains the ENGINE implementation, and "NO_VCHECK" 155 might possibly be useful if there is a minor version conflict and you 156 (or a vendor helpdesk) is convinced you can safely ignore it. 157 "ID" is probably only needed if a shared-library implements 158 multiple ENGINEs, but if you know the engine id you expect to be using, 159 it doesn't hurt to specify it (and this provides a sanity check if 160 nothing else). "LIST_ADD" is only required if you actually wish the 161 loaded ENGINE to be discoverable by application code later on using the 162 ENGINE's "id". For most applications, this isn't necessary - but some 163 application authors may have nifty reasons for using it. The "LOAD" 164 command is the only one that takes no parameters and is the command 165 that uses the settings from any previous commands to actually *load* 166 the shared-library ENGINE implementation. If this command succeeds, the 167 (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE 168 that has been loaded from the shared-library. As such, any control 169 commands supported by the loaded ENGINE could then be executed as per 170 normal. Eg. if ENGINE "foo" is implemented in the shared-library 171 "libfoo.so" and it supports some special control command "CMD_FOO", the 172 following code would load and use it (NB: obviously this code has no 173 error checking); 174 175 ENGINE *e = ENGINE_by_id("dynamic"); 176 ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0); 177 ENGINE_ctrl_cmd_string(e, "ID", "foo", 0); 178 ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0); 179 ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0); 180 181 For testing, the "openssl engine" utility can be useful for this sort 182 of thing. For example the above code excerpt would achieve much the 183 same result as; 184 185 openssl engine dynamic \ 186 -pre SO_PATH:/lib/libfoo.so \ 187 -pre ID:foo \ 188 -pre LOAD \ 189 -pre "CMD_FOO:some input data" 190 191 Or to simply see the list of commands supported by the "foo" ENGINE; 192 193 openssl engine -vvvv dynamic \ 194 -pre SO_PATH:/lib/libfoo.so \ 195 -pre ID:foo \ 196 -pre LOAD 197 198 Applications that support the ENGINE API and more specifically, the 199 "control commands" mechanism, will provide some way for you to pass 200 such commands through to ENGINEs. As such, you would select "dynamic" 201 as the ENGINE to use, and the parameters/commands you pass would 202 control the *actual* ENGINE used. Each command is actually a name-value 203 pair and the value can sometimes be omitted (eg. the "LOAD" command). 204 Whilst the syntax demonstrated in "openssl engine" uses a colon to 205 separate the command name from the value, applications may provide 206 their own syntax for making that separation (eg. a win32 registry 207 key-value pair may be used by some applications). The reason for the 208 "-pre" syntax in the "openssl engine" utility is that some commands 209 might be issued to an ENGINE *after* it has been initialised for use. 210 Eg. if an ENGINE implementation requires a smart-card to be inserted 211 during initialisation (or a PIN to be typed, or whatever), there may be 212 a control command you can issue afterwards to "forget" the smart-card 213 so that additional initialisation is no longer possible. In 214 applications such as web-servers, where potentially volatile code may 215 run on the same host system, this may provide some arguable security 216 value. In such a case, the command would be passed to the ENGINE after 217 it has been initialised for use, and so the "-post" switch would be 218 used instead. Applications may provide a different syntax for 219 supporting this distinction, and some may simply not provide it at all 220 ("-pre" is almost always what you're after, in reality). 221 222 How do I build a "dynamic" ENGINE? 223 ---------------------------------- 224 This question is trickier - currently OpenSSL bundles various ENGINE 225 implementations that are statically built in, and any application that 226 calls the "ENGINE_load_builtin_engines()" function will automatically 227 have all such ENGINEs available (and occupying memory). Applications 228 that don't call that function have no ENGINEs available like that and 229 would have to use "dynamic" to load any such ENGINE - but on the other 230 hand such applications would only have the memory footprint of any 231 ENGINEs explicitly loaded using user/admin provided control commands. 232 The main advantage of not statically linking ENGINEs and only using 233 "dynamic" for hardware support is that any installation using no 234 "external" ENGINE suffers no unnecessary memory footprint from unused 235 ENGINEs. Likewise, installations that do require an ENGINE incur the 236 overheads from only *that* ENGINE once it has been loaded. 237 238 Sounds good? Maybe, but currently building an ENGINE implementation as 239 a shared-library that can be loaded by "dynamic" isn't automated in 240 OpenSSL's build process. It can be done manually quite easily however. 241 Such a shared-library can either be built with any OpenSSL code it 242 needs statically linked in, or it can link dynamically against OpenSSL 243 if OpenSSL itself is built as a shared library. The instructions are 244 the same in each case, but in the former (statically linked any 245 dependencies on OpenSSL) you must ensure OpenSSL is built with 246 position-independent code ("PIC"). The default OpenSSL compilation may 247 already specify the relevant flags to do this, but you should consult 248 with your compiler documentation if you are in any doubt. 249 250 This example will show building the "atalla" ENGINE in the 251 crypto/engine/ directory as a shared-library for use via the "dynamic" 252 ENGINE. 253 1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL 254 source tree. 255 2) Recompile at least one source file so you can see all the compiler 256 flags (and syntax) being used to build normally. Eg; 257 touch hw_atalla.c ; make 258 will rebuild "hw_atalla.o" using all such flags. 259 3) Manually enter the same compilation line to compile the 260 "hw_atalla.c" file but with the following two changes; 261 (a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches, 262 (b) change the output file from "hw_atalla.o" to something new, 263 eg. "tmp_atalla.o" 264 4) Link "tmp_atalla.o" into a shared-library using the top-level 265 OpenSSL libraries to resolve any dependencies. The syntax for doing 266 this depends heavily on your system/compiler and is a nightmare 267 known well to anyone who has worked with shared-library portability 268 before. 'gcc' on Linux, for example, would use the following syntax; 269 gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto 270 5) Test your shared library using "openssl engine" as explained in the 271 previous section. Eg. from the top-level directory, you might try; 272 apps/openssl engine -vvvv dynamic \ 273 -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD 274 If the shared-library loads successfully, you will see both "-pre" 275 commands marked as "SUCCESS" and the list of control commands 276 displayed (because of "-vvvv") will be the control commands for the 277 *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add 278 the "-t" switch to the utility if you want it to try and initialise 279 the atalla ENGINE for use to test any possible hardware/driver 280 issues. 281 282 PROBLEMS 283 ======== 284 285 It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. 286 A quick test done right before the release showed that trying "openssl speed 287 -engine cswift" generated errors. If the DSO gets enabled, an attempt is made 288 to write at memory address 0x00000002. 289 290