README-design.md
1Design document for the unified scheme data
2===========================================
3
4How are things connected?
5-------------------------
6
7The unified scheme takes all its data from the `build.info` files seen
8throughout the source tree. These files hold the minimum information
9needed to build end product files from diverse sources. See the
10section on `build.info` files below.
11
12From the information in `build.info` files, `Configure` builds up an
13information database as a hash table called `%unified_info`, which is
14stored in configdata.pm, found at the top of the build tree (which may
15or may not be the same as the source tree).
16
17[`Configurations/common.tmpl`](common.tmpl) uses the data from `%unified_info` to
18generate the rules for building end product files as well as
19intermediary files with the help of a few functions found in the
20build-file templates. See the section on build-file templates further
21down for more information.
22
23build.info files
24----------------
25
26As mentioned earlier, `build.info` files are meant to hold the minimum
27information needed to build output files, and therefore only (with a
28few possible exceptions [1]) have information about end products (such
29as scripts, library files and programs) and source files (such as C
30files, C header files, assembler files, etc). Intermediate files such
31as object files are rarely directly referred to in `build.info` files (and
32when they are, it's always with the file name extension `.o`), they are
33inferred by `Configure`. By the same rule of minimalism, end product
34file name extensions (such as `.so`, `.a`, `.exe`, etc) are never mentioned
35in `build.info`. Their file name extensions will be inferred by the
36build-file templates, adapted for the platform they are meant for (see
37sections on `%unified_info` and build-file templates further down).
38
39The variables `PROGRAMS`, `LIBS`, `MODULES` and `SCRIPTS` are used to declare
40end products. There are variants for them with `_NO_INST` as suffix
41(`PROGRAM_NO_INST` etc) to specify end products that shouldn't get installed.
42
43The variables `SOURCE`, `DEPEND`, `INCLUDE` and `DEFINE` are indexed by a
44produced file, and their values are the source used to produce that
45particular produced file, extra dependencies, include directories
46needed, or C macros to be defined.
47
48All their values in all the `build.info` throughout the source tree are
49collected together and form a set of programs, libraries, modules and
50scripts to be produced, source files, dependencies, etc etc etc.
51
52Let's have a pretend example, a very limited contraption of OpenSSL,
53composed of the program `apps/openssl`, the libraries `libssl` and
54`libcrypto`, an module `engines/ossltest` and their sources and
55dependencies.
56
57 # build.info
58 LIBS=libcrypto libssl
59 INCLUDE[libcrypto]=include
60 INCLUDE[libssl]=include
61 DEPEND[libssl]=libcrypto
62
63This is the top directory `build.info` file, and it tells us that two
64libraries are to be built, the include directory `include/` shall be
65used throughout when building anything that will end up in each
66library, and that the library `libssl` depend on the library
67`libcrypto` to function properly.
68
69 # apps/build.info
70 PROGRAMS=openssl
71 SOURCE[openssl]=openssl.c
72 INCLUDE[openssl]=.. ../include
73 DEPEND[openssl]=../libssl
74
75This is the `build.info` file in `apps/`, one may notice that all file
76paths mentioned are relative to the directory the `build.info` file is
77located in. This one tells us that there's a program to be built
78called `apps/openss` (the file name extension will depend on the
79platform and is therefore not mentioned in the `build.info` file). It's
80built from one source file, `apps/openssl.c`, and building it requires
81the use of `.` and `include/` include directories (both are declared
82from the point of view of the `apps/` directory), and that the program
83depends on the library `libssl` to function properly.
84
85 # crypto/build.info
86 LIBS=../libcrypto
87 SOURCE[../libcrypto]=aes.c evp.c cversion.c
88 DEPEND[cversion.o]=buildinf.h
89
90 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
91 DEPEND[buildinf.h]=../Makefile
92 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm
93
94This is the `build.info` file in `crypto/`, and it tells us a little more
95about what's needed to produce `libcrypto`. LIBS is used again to
96declare that `libcrypto` is to be produced. This declaration is
97really unnecessary as it's already mentioned in the top `build.info`
98file, but can make the info file easier to understand. This is to
99show that duplicate information isn't an issue.
100
101This `build.info` file informs us that `libcrypto` is built from a few
102source files, `crypto/aes.c`, `crypto/evp.c` and `crypto/cversion.c`.
103It also shows us that building the object file inferred from
104`crypto/cversion.c` depends on `crypto/buildinf.h`. Finally, it
105also shows the possibility to declare how some files are generated
106using some script, in this case a perl script, and how such scripts
107can be declared to depend on other files, in this case a perl module.
108
109Two things are worth an extra note:
110
111`DEPEND[cversion.o]` mentions an object file. DEPEND indexes is the
112only location where it's valid to mention them
113
114 # ssl/build.info
115 LIBS=../libssl
116 SOURCE[../libssl]=tls.c
117
118This is the build.info file in `ssl/`, and it tells us that the
119library `libssl` is built from the source file `ssl/tls.c`.
120
121 # engines/build.info
122 MODULES=dasync
123 SOURCE[dasync]=e_dasync.c
124 DEPEND[dasync]=../libcrypto
125 INCLUDE[dasync]=../include
126
127 MODULES_NO_INST=ossltest
128 SOURCE[ossltest]=e_ossltest.c
129 DEPEND[ossltest]=../libcrypto.a
130 INCLUDE[ossltest]=../include
131
132This is the `build.info` file in `engines/`, telling us that two modules
133called `engines/dasync` and `engines/ossltest` shall be built, that
134`dasync`'s source is `engines/e_dasync.c` and `ossltest`'s source is
135`engines/e_ossltest.c` and that the include directory `include/` may
136be used when building anything that will be part of these modules.
137Also, both modules depend on the library `libcrypto` to function
138properly. `ossltest` is explicitly linked with the static variant of
139the library `libcrypto`. Finally, only `dasync` is being installed, as
140`ossltest` is only for internal testing.
141
142When `Configure` digests these `build.info` files, the accumulated
143information comes down to this:
144
145 LIBS=libcrypto libssl
146 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
147 DEPEND[crypto/cversion.o]=crypto/buildinf.h
148 INCLUDE[libcrypto]=include
149 SOURCE[libssl]=ssl/tls.c
150 INCLUDE[libssl]=include
151 DEPEND[libssl]=libcrypto
152
153 PROGRAMS=apps/openssl
154 SOURCE[apps/openssl]=apps/openssl.c
155 INCLUDE[apps/openssl]=. include
156 DEPEND[apps/openssl]=libssl
157
158 MODULES=engines/dasync
159 SOURCE[engines/dasync]=engines/e_dasync.c
160 DEPEND[engines/dasync]=libcrypto
161 INCLUDE[engines/dasync]=include
162
163 MODULES_NO_INST=engines/ossltest
164 SOURCE[engines/ossltest]=engines/e_ossltest.c
165 DEPEND[engines/ossltest]=libcrypto.a
166 INCLUDE[engines/ossltest]=include
167
168 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
169 DEPEND[crypto/buildinf.h]=Makefile
170 DEPEND[util/mkbuildinf.pl]=util/Foo.pm
171
172A few notes worth mentioning:
173
174`LIBS` may be used to declare routine libraries only.
175
176`PROGRAMS` may be used to declare programs only.
177
178`MODULES` may be used to declare modules only.
179
180The indexes for `SOURCE` must only be end product files, such as
181libraries, programs or modules. The values of `SOURCE` variables must
182only be source files (possibly generated).
183
184`INCLUDE` and `DEPEND` shows a relationship between different files
185(usually produced files) or between files and directories, such as a
186program depending on a library, or between an object file and some
187extra source file.
188
189When `Configure` processes the `build.info` files, it will take it as
190truth without question, and will therefore perform very few checks.
191If the build tree is separate from the source tree, it will assume
192that all built files and up in the build directory and that all source
193files are to be found in the source tree, if they can be found there.
194`Configure` will assume that source files that can't be found in the
195source tree (such as `crypto/bildinf.h` in the example above) are
196generated and will be found in the build tree.
197
198The `%unified_info` database
199----------------------------
200
201The information in all the `build.info` get digested by `Configure` and
202collected into the `%unified_info` database, divided into the following
203indexes:
204
205 depends => a hash table containing 'file' => [ 'dependency' ... ]
206 pairs. These are directly inferred from the DEPEND
207 variables in build.info files.
208
209 modules => a list of modules. These are directly inferred from
210 the MODULES variable in build.info files.
211
212 generate => a hash table containing 'file' => [ 'generator' ... ]
213 pairs. These are directly inferred from the GENERATE
214 variables in build.info files.
215
216 includes => a hash table containing 'file' => [ 'include' ... ]
217 pairs. These are directly inferred from the INCLUDE
218 variables in build.info files.
219
220 install => a hash table containing 'type' => [ 'file' ... ] pairs.
221 The types are 'programs', 'libraries', 'modules' and
222 'scripts', and the array of files list the files of
223 that type that should be installed.
224
225 libraries => a list of libraries. These are directly inferred from
226 the LIBS variable in build.info files.
227
228 programs => a list of programs. These are directly inferred from
229 the PROGRAMS variable in build.info files.
230
231 scripts => a list of scripts. There are directly inferred from
232 the SCRIPTS variable in build.info files.
233
234 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
235 pairs. These are indirectly inferred from the SOURCE
236 variables in build.info files. Object files are
237 mentioned in this hash table, with source files from
238 SOURCE variables, and AS source files for programs and
239 libraries.
240
241 shared_sources =>
242 a hash table just like 'sources', but only as source
243 files (object files) for building shared libraries.
244
245As an example, here is how the `build.info` files example from the
246section above would be digested into a `%unified_info` table:
247
248 our %unified_info = (
249 "depends" =>
250 {
251 "apps/openssl" =>
252 [
253 "libssl",
254 ],
255 "crypto/buildinf.h" =>
256 [
257 "Makefile",
258 ],
259 "crypto/cversion.o" =>
260 [
261 "crypto/buildinf.h",
262 ],
263 "engines/dasync" =>
264 [
265 "libcrypto",
266 ],
267 "engines/ossltest" =>
268 [
269 "libcrypto.a",
270 ],
271 "libssl" =>
272 [
273 "libcrypto",
274 ],
275 "util/mkbuildinf.pl" =>
276 [
277 "util/Foo.pm",
278 ],
279 },
280 "modules" =>
281 [
282 "engines/dasync",
283 "engines/ossltest",
284 ],
285 "generate" =>
286 {
287 "crypto/buildinf.h" =>
288 [
289 "util/mkbuildinf.pl",
290 "\"\$(CC)",
291 "\$(CFLAGS)\"",
292 "\"$(PLATFORM)\"",
293 ],
294 },
295 "includes" =>
296 {
297 "apps/openssl" =>
298 [
299 ".",
300 "include",
301 ],
302 "engines/ossltest" =>
303 [
304 "include"
305 ],
306 "libcrypto" =>
307 [
308 "include",
309 ],
310 "libssl" =>
311 [
312 "include",
313 ],
314 "util/mkbuildinf.pl" =>
315 [
316 "util",
317 ],
318 }
319 "install" =>
320 {
321 "modules" =>
322 [
323 "engines/dasync",
324 ],
325 "libraries" =>
326 [
327 "libcrypto",
328 "libssl",
329 ],
330 "programs" =>
331 [
332 "apps/openssl",
333 ],
334 },
335 "libraries" =>
336 [
337 "libcrypto",
338 "libssl",
339 ],
340 "programs" =>
341 [
342 "apps/openssl",
343 ],
344 "sources" =>
345 {
346 "apps/openssl" =>
347 [
348 "apps/openssl.o",
349 ],
350 "apps/openssl.o" =>
351 [
352 "apps/openssl.c",
353 ],
354 "crypto/aes.o" =>
355 [
356 "crypto/aes.c",
357 ],
358 "crypto/cversion.o" =>
359 [
360 "crypto/cversion.c",
361 ],
362 "crypto/evp.o" =>
363 [
364 "crypto/evp.c",
365 ],
366 "engines/e_dasync.o" =>
367 [
368 "engines/e_dasync.c",
369 ],
370 "engines/dasync" =>
371 [
372 "engines/e_dasync.o",
373 ],
374 "engines/e_ossltest.o" =>
375 [
376 "engines/e_ossltest.c",
377 ],
378 "engines/ossltest" =>
379 [
380 "engines/e_ossltest.o",
381 ],
382 "libcrypto" =>
383 [
384 "crypto/aes.c",
385 "crypto/cversion.c",
386 "crypto/evp.c",
387 ],
388 "libssl" =>
389 [
390 "ssl/tls.c",
391 ],
392 "ssl/tls.o" =>
393 [
394 "ssl/tls.c",
395 ],
396 },
397 );
398
399As can be seen, everything in `%unified_info` is fairly simple suggest
400of information. Still, it tells us that to build all programs, we
401must build `apps/openssl`, and to build the latter, we will need to
402build all its sources (`apps/openssl.o` in this case) and all the
403other things it depends on (such as `libssl`). All those dependencies
404need to be built as well, using the same logic, so to build `libssl`,
405we need to build `ssl/tls.o` as well as `libcrypto`, and to build the
406latter...
407
408Build-file templates
409--------------------
410
411Build-file templates are essentially build-files (such as `Makefile` on
412Unix) with perl code fragments mixed in. Those perl code fragment
413will generate all the configuration dependent data, including all the
414rules needed to build end product files and intermediary files alike.
415At a minimum, there must be a perl code fragment that defines a set of
416functions that are used to generates specific build-file rules, to
417build static libraries from object files, to build shared libraries
418from static libraries, to programs from object files and libraries,
419etc.
420
421 generatesrc - function that produces build file lines to generate
422 a source file from some input.
423
424 It's called like this:
425
426 generatesrc(src => "PATH/TO/tobegenerated",
427 generator => [ "generatingfile", ... ]
428 generator_incs => [ "INCL/PATH", ... ]
429 generator_deps => [ "dep1", ... ]
430 incs => [ "INCL/PATH", ... ],
431 deps => [ "dep1", ... ],
432 intent => one of "libs", "dso", "bin" );
433
434 'src' has the name of the file to be generated.
435 'generator' is the command or part of command to
436 generate the file, of which the first item is
437 expected to be the file to generate from.
438 generatesrc() is expected to analyse and figure out
439 exactly how to apply that file and how to capture
440 the result. 'generator_incs' and 'generator_deps'
441 are include directories and files that the generator
442 file itself depends on. 'incs' and 'deps' are
443 include directories and files that are used if $(CC)
444 is used as an intermediary step when generating the
445 end product (the file indicated by 'src'). 'intent'
446 indicates what the generated file is going to be
447 used for.
448
449 src2obj - function that produces build file lines to build an
450 object file from source files and associated data.
451
452 It's called like this:
453
454 src2obj(obj => "PATH/TO/objectfile",
455 srcs => [ "PATH/TO/sourcefile", ... ],
456 deps => [ "dep1", ... ],
457 incs => [ "INCL/PATH", ... ]
458 intent => one of "lib", "dso", "bin" );
459
460 'obj' has the intended object file with `.o`
461 extension, src2obj() is expected to change it to
462 something more suitable for the platform.
463 'srcs' has the list of source files to build the
464 object file, with the first item being the source
465 file that directly corresponds to the object file.
466 'deps' is a list of explicit dependencies. 'incs'
467 is a list of include file directories. Finally,
468 'intent' indicates what this object file is going
469 to be used for.
470
471 obj2lib - function that produces build file lines to build a
472 static library file ("libfoo.a" in Unix terms) from
473 object files.
474
475 called like this:
476
477 obj2lib(lib => "PATH/TO/libfile",
478 objs => [ "PATH/TO/objectfile", ... ]);
479
480 'lib' has the intended library file name *without*
481 extension, obj2lib is expected to add that. 'objs'
482 has the list of object files to build this library.
483
484 libobj2shlib - backward compatibility function that's used the
485 same way as obj2shlib (described next), and was
486 expected to build the shared library from the
487 corresponding static library when that was suitable.
488 NOTE: building a shared library from a static
489 library is now DEPRECATED, as they no longer share
490 object files. Attempting to do this will fail.
491
492 obj2shlib - function that produces build file lines to build a
493 shareable object library file ("libfoo.so" in Unix
494 terms) from the corresponding object files.
495
496 called like this:
497
498 obj2shlib(shlib => "PATH/TO/shlibfile",
499 lib => "PATH/TO/libfile",
500 objs => [ "PATH/TO/objectfile", ... ],
501 deps => [ "PATH/TO/otherlibfile", ... ]);
502
503 'lib' has the base (static) library file name
504 *without* extension. This is useful in case
505 supporting files are needed (such as import
506 libraries on Windows).
507 'shlib' has the corresponding shared library name
508 *without* extension. 'deps' has the list of other
509 libraries (also *without* extension) this library
510 needs to be linked with. 'objs' has the list of
511 object files to build this library.
512
513 obj2dso - function that produces build file lines to build a
514 dynamic shared object file from object files.
515
516 called like this:
517
518 obj2dso(lib => "PATH/TO/libfile",
519 objs => [ "PATH/TO/objectfile", ... ],
520 deps => [ "PATH/TO/otherlibfile",
521 ... ]);
522
523 This is almost the same as obj2shlib, but the
524 intent is to build a shareable library that can be
525 loaded in runtime (a "plugin"...).
526
527 obj2bin - function that produces build file lines to build an
528 executable file from object files.
529
530 called like this:
531
532 obj2bin(bin => "PATH/TO/binfile",
533 objs => [ "PATH/TO/objectfile", ... ],
534 deps => [ "PATH/TO/libfile", ... ]);
535
536 'bin' has the intended executable file name
537 *without* extension, obj2bin is expected to add
538 that. 'objs' has the list of object files to build
539 this library. 'deps' has the list of library files
540 (also *without* extension) that the programs needs
541 to be linked with.
542
543 in2script - function that produces build file lines to build a
544 script file from some input.
545
546 called like this:
547
548 in2script(script => "PATH/TO/scriptfile",
549 sources => [ "PATH/TO/infile", ... ]);
550
551 'script' has the intended script file name.
552 'sources' has the list of source files to build the
553 resulting script from.
554
555Along with the build-file templates is the driving template
556[`Configurations/common.tmpl`](common.tmpl), which looks through all the
557information in `%unified_info` and generates all the rulesets to build libraries,
558programs and all intermediate files, using the rule generating
559functions defined in the build-file template.
560
561As an example with the smaller `build.info` set we've seen as an
562example, producing the rules to build `libcrypto` would result in the
563following calls:
564
565 # Note: obj2shlib will only be called if shared libraries are
566 # to be produced.
567 # Note 2: obj2shlib must convert the '.o' extension to whatever
568 # is suitable on the local platform.
569 obj2shlib(shlib => "libcrypto",
570 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ],
571 deps => [ ]);
572
573 obj2lib(lib => "libcrypto"
574 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ]);
575
576 src2obj(obj => "crypto/aes.o"
577 srcs => [ "crypto/aes.c" ],
578 deps => [ ],
579 incs => [ "include" ],
580 intent => "lib");
581
582 src2obj(obj => "crypto/evp.o"
583 srcs => [ "crypto/evp.c" ],
584 deps => [ ],
585 incs => [ "include" ],
586 intent => "lib");
587
588 src2obj(obj => "crypto/cversion.o"
589 srcs => [ "crypto/cversion.c" ],
590 deps => [ "crypto/buildinf.h" ],
591 incs => [ "include" ],
592 intent => "lib");
593
594 generatesrc(src => "crypto/buildinf.h",
595 generator => [ "util/mkbuildinf.pl", "\"$(CC)",
596 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ],
597 generator_incs => [ "util" ],
598 generator_deps => [ "util/Foo.pm" ],
599 incs => [ ],
600 deps => [ ],
601 intent => "lib");
602
603The returned strings from all those calls are then concatenated
604together and written to the resulting build-file.
605
README.md
1Intro
2=====
3
4This directory contains a few sets of files that are used for
5configuration in diverse ways:
6
7 *.conf Target platform configurations, please read
8 'Configurations of OpenSSL target platforms' for more
9 information.
10 *.tmpl Build file templates, please read 'Build-file
11 programming with the "unified" build system' as well
12 as 'Build info files' for more information.
13 *.pm Helper scripts / modules for the main `Configure`
14 script. See 'Configure helper scripts for more
15 information.
16
17Configurations of OpenSSL target platforms
18==========================================
19
20Configuration targets are a collection of facts that we know about
21different platforms and their capabilities. We organise them in a
22hash table, where each entry represent a specific target.
23
24Note that configuration target names must be unique across all config
25files. The Configure script does check that a config file doesn't
26have config targets that shadow config targets from other files.
27
28In each table entry, the following keys are significant:
29
30 inherit_from => Other targets to inherit values from.
31 Explained further below. [1]
32 template => Set to 1 if this isn't really a platform
33 target. Instead, this target is a template
34 upon which other targets can be built.
35 Explained further below. [1]
36
37 sys_id => System identity for systems where that
38 is difficult to determine automatically.
39
40 enable => Enable specific configuration features.
41 This MUST be an array of words.
42 disable => Disable specific configuration features.
43 This MUST be an array of words.
44 Note: if the same feature is both enabled
45 and disabled, disable wins.
46
47 as => The assembler command. This is not always
48 used (for example on Unix, where the C
49 compiler is used instead).
50 asflags => Default assembler command flags [4].
51 cpp => The C preprocessor command, normally not
52 given, as the build file defaults are
53 usually good enough.
54 cppflags => Default C preprocessor flags [4].
55 defines => As an alternative, macro definitions may be
56 given here instead of in 'cppflags' [4].
57 If given here, they MUST be as an array of
58 the string such as "MACRO=value", or just
59 "MACRO" for definitions without value.
60 includes => As an alternative, inclusion directories
61 may be given here instead of in 'cppflags'
62 [4]. If given here, the MUST be an array
63 of strings, one directory specification
64 each.
65 cc => The C compiler command, usually one of "cc",
66 "gcc" or "clang". This command is normally
67 also used to link object files and
68 libraries into the final program.
69 cxx => The C++ compiler command, usually one of
70 "c++", "g++" or "clang++". This command is
71 also used when linking a program where at
72 least one of the object file is made from
73 C++ source.
74 cflags => Defaults C compiler flags [4].
75 cxxflags => Default C++ compiler flags [4]. If unset,
76 it gets the same value as cflags.
77
78 (linking is a complex thing, see [3] below)
79 ld => Linker command, usually not defined
80 (meaning the compiler command is used
81 instead).
82 (NOTE: this is here for future use, it's
83 not implemented yet)
84 lflags => Default flags used when linking apps,
85 shared libraries or DSOs [4].
86 ex_libs => Extra libraries that are needed when
87 linking shared libraries, DSOs or programs.
88 The value is also assigned to Libs.private
89 in $(libdir)/pkgconfig/libcrypto.pc.
90
91 shared_cppflags => Extra C preprocessor flags used when
92 processing C files for shared libraries.
93 shared_cflag => Extra C compiler flags used when compiling
94 for shared libraries, typically something
95 like "-fPIC".
96 shared_ldflag => Extra linking flags used when linking
97 shared libraries.
98 module_cppflags
99 module_cflags
100 module_ldflags => Has the same function as the corresponding
101 'shared_' attributes, but for building DSOs.
102 When unset, they get the same values as the
103 corresponding 'shared_' attributes.
104
105 ar => The library archive command, the default is
106 "ar".
107 (NOTE: this is here for future use, it's
108 not implemented yet)
109 arflags => Flags to be used with the library archive
110 command. On Unix, this includes the
111 command letter, 'r' by default.
112
113 ranlib => The library archive indexing command, the
114 default is 'ranlib' it it exists.
115
116 unistd => An alternative header to the typical
117 '<unistd.h>'. This is very rarely needed.
118
119 shared_extension => File name extension used for shared
120 libraries.
121 obj_extension => File name extension used for object files.
122 On unix, this defaults to ".o" (NOTE: this
123 is here for future use, it's not
124 implemented yet)
125 exe_extension => File name extension used for executable
126 files. On unix, this defaults to "" (NOTE:
127 this is here for future use, it's not
128 implemented yet)
129 shlib_variant => A "variant" identifier inserted between the base
130 shared library name and the extension. On "unixy"
131 platforms (BSD, Linux, Solaris, MacOS/X, ...) this
132 supports installation of custom OpenSSL libraries
133 that don't conflict with other builds of OpenSSL
134 installed on the system. The variant identifier
135 becomes part of the SONAME of the library and also
136 any symbol versions (symbol versions are not used or
137 needed with MacOS/X). For example, on a system
138 where a default build would normally create the SSL
139 shared library as 'libssl.so -> libssl.so.1.1' with
140 the value of the symlink as the SONAME, a target
141 definition that sets 'shlib_variant => "-abc"' will
142 create 'libssl.so -> libssl-abc.so.1.1', again with
143 an SONAME equal to the value of the symlink. The
144 symbol versions associated with the variant library
145 would then be 'OPENSSL_ABC_<version>' rather than
146 the default 'OPENSSL_<version>'. The string inserted
147 into symbol versions is obtained by mapping all
148 letters in the "variant" identifier to upper case
149 and all non-alphanumeric characters to '_'.
150
151 thread_scheme => The type of threads is used on the
152 configured platform. Currently known
153 values are "(unknown)", "pthreads",
154 "uithreads" (a.k.a solaris threads) and
155 "winthreads". Except for "(unknown)", the
156 actual value is currently ignored but may
157 be used in the future. See further notes
158 below [2].
159 dso_scheme => The type of dynamic shared objects to build
160 for. This mostly comes into play with
161 modules, but can be used for other purposes
162 as well. Valid values are "DLFCN"
163 (dlopen() et al), "DLFCN_NO_H" (for systems
164 that use dlopen() et al but do not have
165 fcntl.h), "DL" (shl_load() et al), "WIN32"
166 and "VMS".
167 asm_arch => The architecture to be used for compiling assembly
168 source. This acts as a selector in build.info files.
169 uplink_arch => The architecture to be used for compiling uplink
170 source. This acts as a selector in build.info files.
171 This is separate from asm_arch because it's compiled
172 even when 'no-asm' is given, even though it contains
173 assembler source.
174 perlasm_scheme => The perlasm method used to create the
175 assembler files used when compiling with
176 assembler implementations.
177 shared_target => The shared library building method used.
178 This serves multiple purposes:
179 - as index for targets found in shared_info.pl.
180 - as linker script generation selector.
181 To serve both purposes, the index for shared_info.pl
182 should end with '-shared', and this suffix will be
183 removed for use as a linker script generation
184 selector. Note that the latter is only used if
185 'shared_defflag' is defined.
186 build_scheme => The scheme used to build up a Makefile.
187 In its simplest form, the value is a string
188 with the name of the build scheme.
189 The value may also take the form of a list
190 of strings, if the build_scheme is to have
191 some options. In this case, the first
192 string in the list is the name of the build
193 scheme.
194 Currently recognised build scheme is "unified".
195 For the "unified" build scheme, this item
196 *must* be an array with the first being the
197 word "unified" and the second being a word
198 to identify the platform family.
199
200 multilib => On systems that support having multiple
201 implementations of a library (typically a
202 32-bit and a 64-bit variant), this is used
203 to have the different variants in different
204 directories.
205
206 bn_ops => Building options (was just bignum options in
207 the earlier history of this option, hence the
208 name). This is a string of words that describe
209 algorithms' implementation parameters that
210 are optimal for the designated target platform,
211 such as the type of integers used to build up
212 the bignum, different ways to implement certain
213 ciphers and so on. To fully comprehend the
214 meaning, the best is to read the affected
215 source.
216 The valid words are:
217
218 THIRTY_TWO_BIT bignum limbs are 32 bits,
219 this is default if no
220 option is specified, it
221 works on any supported
222 system [unless "wider"
223 limb size is implied in
224 assembly code];
225 BN_LLONG bignum limbs are 32 bits,
226 but 64-bit 'unsigned long
227 long' is used internally
228 in calculations;
229 SIXTY_FOUR_BIT_LONG bignum limbs are 64 bits
230 and sizeof(long) is 8;
231 SIXTY_FOUR_BIT bignums limbs are 64 bits,
232 but execution environment
233 is ILP32;
234 RC4_CHAR RC4 key schedule is made
235 up of 'unsigned char's;
236 Note: should not be used
237 for new configuration
238 targets
239 RC4_INT RC4 key schedule is made
240 up of 'unsigned int's;
241 Note: should not be used
242 for new configuration
243 targets
244
245[1] as part of the target configuration, one can have a key called
246 `inherit_from` that indicates what other configurations to inherit
247 data from. These are resolved recursively.
248
249 Inheritance works as a set of default values that can be overridden
250 by corresponding key values in the inheriting configuration.
251
252 Note 1: any configuration table can be used as a template.
253 Note 2: pure templates have the attribute `template => 1` and
254 cannot be used as build targets.
255
256 If several configurations are given in the `inherit_from` array,
257 the values of same attribute are concatenated with space
258 separation. With this, it's possible to have several smaller
259 templates for different configuration aspects that can be combined
260 into a complete configuration.
261
262 Instead of a scalar value or an array, a value can be a code block
263 of the form `sub { /* your code here */ }`. This code block will
264 be called with the list of inherited values for that key as
265 arguments. In fact, the concatenation of strings is really done
266 by using `sub { join(" ",@_) }` on the list of inherited values.
267
268 An example:
269
270 "foo" => {
271 template => 1,
272 haha => "ha ha",
273 hoho => "ho",
274 ignored => "This should not appear in the end result",
275 },
276 "bar" => {
277 template => 1,
278 haha => "ah",
279 hoho => "haho",
280 hehe => "hehe"
281 },
282 "laughter" => {
283 inherit_from => [ "foo", "bar" ],
284 hehe => sub { join(" ",(@_,"!!!")) },
285 ignored => "",
286 }
287
288 The entry for "laughter" will become as follows after processing:
289
290 "laughter" => {
291 haha => "ha ha ah",
292 hoho => "ho haho",
293 hehe => "hehe !!!",
294 ignored => ""
295 }
296
297[2] OpenSSL is built with threading capabilities unless the user
298 specifies `no-threads`. The value of the key `thread_scheme` may
299 be `(unknown)`, in which case the user MUST give some compilation
300 flags to `Configure`.
301
302[3] OpenSSL has three types of things to link from object files or
303 static libraries:
304
305 - shared libraries; that would be libcrypto and libssl.
306 - shared objects (sometimes called dynamic libraries); that would
307 be the modules.
308 - applications; those are apps/openssl and all the test apps.
309
310 Very roughly speaking, linking is done like this (words in braces
311 represent the configuration settings documented at the beginning
312 of this file):
313
314 shared libraries:
315 {ld} $(CFLAGS) {lflags} {shared_ldflag} -o libfoo.so \
316 foo/something.o foo/somethingelse.o {ex_libs}
317
318 shared objects:
319 {ld} $(CFLAGS) {lflags} {module_ldflags} -o libeng.so \
320 blah1.o blah2.o -lcrypto {ex_libs}
321
322 applications:
323 {ld} $(CFLAGS) {lflags} -o app \
324 app1.o utils.o -lssl -lcrypto {ex_libs}
325
326[4] There are variants of these attribute, prefixed with `lib_`,
327 `dso_` or `bin_`. Those variants replace the unprefixed attribute
328 when building library, DSO or program modules specifically.
329
330Historically, the target configurations came in form of a string with
331values separated by colons. This use is deprecated. The string form
332looked like this:
333
334 "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:
335 {bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:
336 {bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:
337 {rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:
338 {padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:
339 {shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:
340 {arflags}:{multilib}"
341
342Build info files
343================
344
345The `build.info` files that are spread over the source tree contain the
346minimum information needed to build and distribute OpenSSL. It uses a
347simple and yet fairly powerful language to determine what needs to be
348built, from what sources, and other relationships between files.
349
350For every `build.info` file, all file references are relative to the
351directory of the `build.info` file for source files, and the
352corresponding build directory for built files if the build tree
353differs from the source tree.
354
355When processed, every line is processed with the perl module
356Text::Template, using the delimiters `{-` and `-}`. The hashes
357`%config` and `%target` are passed to the perl fragments, along with
358$sourcedir and $builddir, which are the locations of the source
359directory for the current `build.info` file and the corresponding build
360directory, all relative to the top of the build tree.
361
362`Configure` only knows inherently about the top `build.info` file. For
363any other directory that has one, further directories to look into
364must be indicated like this:
365
366 SUBDIRS=something someelse
367
368On to things to be built; they are declared by setting specific
369variables:
370
371 PROGRAMS=foo bar
372 LIBS=libsomething
373 MODULES=libeng
374 SCRIPTS=myhack
375
376Note that the files mentioned for PROGRAMS, LIBS and MODULES *must* be
377without extensions. The build file templates will figure them out.
378
379For each thing to be built, it is then possible to say what sources
380they are built from:
381
382 PROGRAMS=foo bar
383 SOURCE[foo]=foo.c common.c
384 SOURCE[bar]=bar.c extra.c common.c
385
386It's also possible to tell some other dependencies:
387
388 DEPEND[foo]=libsomething
389 DEPEND[libbar]=libsomethingelse
390
391(it could be argued that 'libsomething' and 'libsomethingelse' are
392source as well. However, the files given through SOURCE are expected
393to be located in the source tree while files given through DEPEND are
394expected to be located in the build tree)
395
396It's also possible to depend on static libraries explicitly:
397
398 DEPEND[foo]=libsomething.a
399 DEPEND[libbar]=libsomethingelse.a
400
401This should be rarely used, and care should be taken to make sure it's
402only used when supported. For example, native Windows build doesn't
403support building static libraries and DLLs at the same time, so using
404static libraries on Windows can only be done when configured
405`no-shared`.
406
407In some cases, it's desirable to include some source files in the
408shared form of a library only:
409
410 SHARED_SOURCE[libfoo]=dllmain.c
411
412For any file to be built, it's also possible to tell what extra
413include paths the build of their source files should use:
414
415 INCLUDE[foo]=include
416
417It's also possible to specify C macros that should be defined:
418
419 DEFINE[foo]=FOO BAR=1
420
421In some cases, one might want to generate some source files from
422others, that's done as follows:
423
424 GENERATE[foo.s]=asm/something.pl $(CFLAGS)
425 GENERATE[bar.s]=asm/bar.S
426
427The value of each GENERATE line is a command line or part of it.
428Configure places no rules on the command line, except that the first
429item must be the generator file. It is, however, entirely up to the
430build file template to define exactly how those command lines should
431be handled, how the output is captured and so on.
432
433Sometimes, the generator file itself depends on other files, for
434example if it is a perl script that depends on other perl modules.
435This can be expressed using DEPEND like this:
436
437 DEPEND[asm/something.pl]=../perlasm/Foo.pm
438
439There may also be cases where the exact file isn't easily specified,
440but an inclusion directory still needs to be specified. INCLUDE can
441be used in that case:
442
443 INCLUDE[asm/something.pl]=../perlasm
444
445NOTE: GENERATE lines are limited to one command only per GENERATE.
446
447Finally, you can have some simple conditional use of the `build.info`
448information, looking like this:
449
450 IF[1]
451 something
452 ELSIF[2]
453 something other
454 ELSE
455 something else
456 ENDIF
457
458The expression in square brackets is interpreted as a string in perl,
459and will be seen as true if perl thinks it is, otherwise false. For
460example, the above would have "something" used, since 1 is true.
461
462Together with the use of Text::Template, this can be used as
463conditions based on something in the passed variables, for example:
464
465 IF[{- $disabled{shared} -}]
466 LIBS=libcrypto
467 SOURCE[libcrypto]=...
468 ELSE
469 LIBS=libfoo
470 SOURCE[libfoo]=...
471 ENDIF
472
473Build-file programming with the "unified" build system
474======================================================
475
476"Build files" are called `Makefile` on Unix-like operating systems,
477`descrip.mms` for MMS on VMS, `makefile` for `nmake` on Windows, etc.
478
479To use the "unified" build system, the target configuration needs to
480set the three items `build_scheme`, `build_file` and `build_command`.
481In the rest of this section, we will assume that `build_scheme` is set
482to "unified" (see the configurations documentation above for the
483details).
484
485For any name given by `build_file`, the "unified" system expects a
486template file in `Configurations/` named like the build file, with
487`.tmpl` appended, or in case of possible ambiguity, a combination of
488the second `build_scheme` list item and the `build_file` name. For
489example, if `build_file` is set to `Makefile`, the template could be
490`Configurations/Makefile.tmpl` or `Configurations/unix-Makefile.tmpl`.
491In case both `Configurations/unix-Makefile.tmpl` and
492`Configurations/Makefile.tmpl` are present, the former takes precedence.
493
494The build-file template is processed with the perl module
495Text::Template, using `{-` and `-}` as delimiters that enclose the
496perl code fragments that generate configuration-dependent content.
497Those perl fragments have access to all the hash variables from
498configdata.pem.
499
500The build-file template is expected to define at least the following
501perl functions in a perl code fragment enclosed with `{-` and `-}`.
502They are all expected to return a string with the lines they produce.
503
504 generatesrc - function that produces build file lines to generate
505 a source file from some input.
506
507 It's called like this:
508
509 generatesrc(src => "PATH/TO/tobegenerated",
510 generator => [ "generatingfile", ... ]
511 generator_incs => [ "INCL/PATH", ... ]
512 generator_deps => [ "dep1", ... ]
513 generator => [ "generatingfile", ... ]
514 incs => [ "INCL/PATH", ... ],
515 deps => [ "dep1", ... ],
516 intent => one of "libs", "dso", "bin" );
517
518 'src' has the name of the file to be generated.
519 'generator' is the command or part of command to
520 generate the file, of which the first item is
521 expected to be the file to generate from.
522 generatesrc() is expected to analyse and figure out
523 exactly how to apply that file and how to capture
524 the result. 'generator_incs' and 'generator_deps'
525 are include directories and files that the generator
526 file itself depends on. 'incs' and 'deps' are
527 include directories and files that are used if $(CC)
528 is used as an intermediary step when generating the
529 end product (the file indicated by 'src'). 'intent'
530 indicates what the generated file is going to be
531 used for.
532
533 src2obj - function that produces build file lines to build an
534 object file from source files and associated data.
535
536 It's called like this:
537
538 src2obj(obj => "PATH/TO/objectfile",
539 srcs => [ "PATH/TO/sourcefile", ... ],
540 deps => [ "dep1", ... ],
541 incs => [ "INCL/PATH", ... ]
542 intent => one of "lib", "dso", "bin" );
543
544 'obj' has the intended object file with '.o'
545 extension, src2obj() is expected to change it to
546 something more suitable for the platform.
547 'srcs' has the list of source files to build the
548 object file, with the first item being the source
549 file that directly corresponds to the object file.
550 'deps' is a list of explicit dependencies. 'incs'
551 is a list of include file directories. Finally,
552 'intent' indicates what this object file is going
553 to be used for.
554
555 obj2lib - function that produces build file lines to build a
556 static library file ("libfoo.a" in Unix terms) from
557 object files.
558
559 called like this:
560
561 obj2lib(lib => "PATH/TO/libfile",
562 objs => [ "PATH/TO/objectfile", ... ]);
563
564 'lib' has the intended library file name *without*
565 extension, obj2lib is expected to add that. 'objs'
566 has the list of object files to build this library.
567
568 libobj2shlib - backward compatibility function that's used the
569 same way as obj2shlib (described next), and was
570 expected to build the shared library from the
571 corresponding static library when that was suitable.
572 NOTE: building a shared library from a static
573 library is now DEPRECATED, as they no longer share
574 object files. Attempting to do this will fail.
575
576 obj2shlib - function that produces build file lines to build a
577 shareable object library file ("libfoo.so" in Unix
578 terms) from the corresponding object files.
579
580 called like this:
581
582 obj2shlib(shlib => "PATH/TO/shlibfile",
583 lib => "PATH/TO/libfile",
584 objs => [ "PATH/TO/objectfile", ... ],
585 deps => [ "PATH/TO/otherlibfile", ... ]);
586
587 'lib' has the base (static) library ffile name
588 *without* extension. This is useful in case
589 supporting files are needed (such as import
590 libraries on Windows).
591 'shlib' has the corresponding shared library name
592 *without* extension. 'deps' has the list of other
593 libraries (also *without* extension) this library
594 needs to be linked with. 'objs' has the list of
595 object files to build this library.
596
597 obj2dso - function that produces build file lines to build a
598 dynamic shared object file from object files.
599
600 called like this:
601
602 obj2dso(lib => "PATH/TO/libfile",
603 objs => [ "PATH/TO/objectfile", ... ],
604 deps => [ "PATH/TO/otherlibfile",
605 ... ]);
606
607 This is almost the same as obj2shlib, but the
608 intent is to build a shareable library that can be
609 loaded in runtime (a "plugin"...).
610
611 obj2bin - function that produces build file lines to build an
612 executable file from object files.
613
614 called like this:
615
616 obj2bin(bin => "PATH/TO/binfile",
617 objs => [ "PATH/TO/objectfile", ... ],
618 deps => [ "PATH/TO/libfile", ... ]);
619
620 'bin' has the intended executable file name
621 *without* extension, obj2bin is expected to add
622 that. 'objs' has the list of object files to build
623 this library. 'deps' has the list of library files
624 (also *without* extension) that the programs needs
625 to be linked with.
626
627 in2script - function that produces build file lines to build a
628 script file from some input.
629
630 called like this:
631
632 in2script(script => "PATH/TO/scriptfile",
633 sources => [ "PATH/TO/infile", ... ]);
634
635 'script' has the intended script file name.
636 'sources' has the list of source files to build the
637 resulting script from.
638
639In all cases, file file paths are relative to the build tree top, and
640the build file actions run with the build tree top as current working
641directory.
642
643Make sure to end the section with these functions with a string that
644you thing is appropriate for the resulting build file. If nothing
645else, end it like this:
646
647 ""; # Make sure no lingering values end up in the Makefile
648 -}
649
650Configure helper scripts
651========================
652
653Configure uses helper scripts in this directory:
654
655Checker scripts
656---------------
657
658These scripts are per platform family, to check the integrity of the
659tools used for configuration and building. The checker script used is
660either `{build_platform}-{build_file}-checker.pm` or
661`{build_platform}-checker.pm`, where `{build_platform}` is the second
662`build_scheme` list element from the configuration target data, and
663`{build_file}` is `build_file` from the same target data.
664
665If the check succeeds, the script is expected to end with a non-zero
666expression. If the check fails, the script can end with a zero, or
667with a `die`.
668