BUILDING revision 1.37
1BUILDING(8) NetBSD System Manager's Manual BUILDING(8) 2 3NAME 4 BUILDING - Procedure for building NetBSD from source code. 5 6STATUS 7 This document is a work-in-progress. As such, the information described 8 here may not match the reality of the build system as of this writing. 9 Once this document is completely in sync with reality, this paragraph 10 will be removed. 11 12 Discrepancies between this documentation and the current reality of im- 13 plementation are noted specially, as with the note below: 14 15 Note: This document applies only to platforms which use the new toolchain 16 as indicated by the default setting of TOOLCHAIN_MISSING in <bsd.own.mk>. 17 Platforms which have not yet been switched to the new toolchain should 18 continue building traditionally, using the notes specified in the file 19 UPDATING. 20 21REQUIREMENTS 22 NetBSD is designed to be buildable on most POSIX-compliant host systems. 23 The basic build procedure is the same whether compiling natively (on the 24 same NetBSD architecture) or cross compiling (on another architecture or 25 OS). 26 27 This source tree contains a special subtree, ``tools'', which uses the 28 host system to create a build toolchain for the target architecture. The 29 host system must have at least C and C++ compilers in order to create the 30 toolchain (make is not required); all other tools are created as part of 31 the NetBSD build process. 32 33 Note: A couple of host toolchain components are not yet available 34 in the tools directory. Also, some tools use non-POSIX, non-ANSI C 35 extensions and need to be standardized. As a result, cross-compil- 36 ing from systems other than NetBSD is not currently supported. 37 38FILES 39 Source tree layout 40 doc/BUILDING.mdoc 41 This document (in -mdoc troff format; the original copy). 42 43 BUILDING This document (in plaintext). 44 45 Makefile The main Makefile for NetBSD; should only be run for na- 46 tive builds with an appropriately up-to-date version of 47 NetBSD make(1). (For building from out-of-date systems or 48 on a non-native host, see the build.sh shell script.) 49 50 UPDATING Special notes for updating from an earlier revision of 51 NetBSD. It is important to read this file before every 52 build of an updated source tree. 53 54 build.sh Bourne-compatible shell script used for building the host 55 build tools and the NetBSD system from scratch. Can be 56 used for both native and cross builds, and should be used 57 instead of make(1) for any source tree that is updated and 58 recompiled regularly. 59 60 crypto/dist/, dist/, gnu/dist/ 61 Sources imported verbatim from third parties, without man- 62 gling the existing build structure. Other source trees in 63 bin through usr.sbin use the NetBSD make(1) ``reachover'' 64 Makefile semantics when building these programs for a na- 65 tive host. 66 67 distrib/, etc/ 68 Sources for items used when making a full release snap- 69 shot, such as files installed in DESTDIR/etc on the desti- 70 nation system, boot media, and release notes. 71 72 regress/ Regression test harness. Can be cross-compiled, but only 73 run natively. 74 75 sys/ NetBSD kernel sources. 76 77 tools/ ``Reachover'' build structure for the host build tools. 78 This has a special method of determining out-of-date sta- 79 tus. 80 81 bin/ ... usr.sbin/ 82 Sources to the NetBSD userland (non-kernel) programs. If 83 any of these directories are missing, they will be skipped 84 during the build. 85 86 Build tree layout 87 The NetBSD build tree is described in hier(7), and the release layout is 88 described in release(7). 89 90CONFIGURATION 91 Environment variables 92 Several environment variables control the behaviour of NetBSD builds. 93 94 MACHINE Machine type. 95 96 MACHINE_ARCH Machine architecture. 97 98 MAKE Path name to invoke make(1) as. 99 100 MAKEFLAGS Flags to invoke make(1) with. 101 102 MAKEOBJDIR Directory to use as the .OBJDIR for the current direc- 103 tory. Used only if MAKEOBJDIRPREFIX is not defined. 104 MAKEOBJDIR can only be provided in the environment or 105 via the -M flag of build.sh. 106 107 MAKEOBJDIRPREFIX Top level directory of the object directory tree. If 108 this is defined, ${MAKEOBJDIRPREFIX}/${.CURDIR} is used 109 as the .OBJDIR for the current directory. The current 110 directory may be read only. MAKEOBJDIRPREFIX can only 111 be provided in the environment or via the -M flag of 112 build.sh. 113 114 "make" variables 115 Several variables control the behavior of NetBSD builds. Unless other- 116 wise specified, these variables may be set in either the process environ- 117 ment or the make(1) configuration file specified by MAKECONF. 118 119 BUILDID Identifier for the build. The identifier will be appended to 120 object directory names, and can be consulted in the make(1) 121 configuration file in order to set additional build parame- 122 ters, such as compiler flags. 123 124 DESTDIR Directory to contain the built NetBSD system. If set, spe- 125 cial options are passed to the compilation tools to prevent 126 their default use of the host system's /usr/include, 127 /usr/lib, and so forth. This pathname should not end with a 128 slash (/) character (for installation into the system's root 129 directory, set DESTDIR to an empty string). The directory 130 must reside on a file system which supports long file names 131 and hard links. 132 133 Default: Empty string if USETOOLS is ``yes''; unset other- 134 wise. 135 136 Note: build.sh will provide a default of destdir.MACHINE (in 137 the top-level .OBJDIR) unless run in `expert' mode 138 139 MAKECONF The name of the make(1) configuration file. Only settable in 140 the process environment. 141 142 Default: ``/etc/mk.conf'' 143 144 MKCATPAGES Can be set to ``yes'' or ``no''. Indicates whether prefor- 145 matted plaintext manual pages will be created during a build. 146 147 Default: ``yes'' 148 149 MKCRYPTO Can be set to ``yes'' or ``no''. Indicates whether crypto- 150 graphic code will be included in a build; provided for the 151 benefit of countries that do not allow strong cryptography. 152 Will not affect use of the standard low-security password en- 153 cryption system, crypt(3). 154 155 Default: ``yes'' 156 157 MKDOC Can be set to ``yes'' or ``no''. Indicates whether system 158 documentation destined for DESTDIR/usr/share/doc will be in- 159 stalled during a build. 160 161 Default: ``yes'' 162 163 MKHOSTOBJ Can be set to ``yes'' or ``no''. If set to ``yes'', then for 164 programs intended to be run on the compile host, the name, 165 release, and architecture of the host operating system will 166 be suffixed to the name of the object directory created by 167 ``make obj''. (This allows multiple host systems to compile 168 NetBSD for a single target.) If set to ``no'', then programs 169 built to be run on the compile host will use the same object 170 directory names as programs built to be run on the target. 171 172 Default: ``no'' 173 174 MKINFO Can be set to ``yes'' or ``no''. Indicates whether GNU Info 175 files, used for the documentation for most of the compilation 176 tools, will be created and installed during a build. 177 178 Default: ``yes'' 179 180 MKLINT Can be set to ``yes'' or ``no''. Indicates whether lint(1) 181 will be run against portions of the NetBSD source code during 182 the build, and whether lint libraries will be installed into 183 DESTDIR/usr/libdata/lint. 184 185 Default: ``yes'' 186 187 MKMAN Can be set to ``yes'' or ``no''. Indicates whether manual 188 pages will be installed during a build. 189 190 Default: ``yes'' 191 192 MKNLS Can be set to ``yes'' or ``no''. Indicates whether Native 193 Language System locale zone files will be compiled and in- 194 stalled during a build. 195 196 Default: ``yes'' 197 198 MKOBJ Can be set to ``yes'' or ``no''. Indicates whether object 199 directories will be created when running ``make obj''. If 200 set to ``no'', then all built files will be located inside 201 the regular source tree. 202 203 Default: ``yes'' 204 205 MKPIC Can be set to ``yes'' or ``no''. Indicates whether shared 206 objects and libraries will be created and installed during a 207 build. If set to ``no'', the entire built system will be 208 statically linked. 209 210 Default: Platform dependent. As of this writing, all plat- 211 forms except sh3 default to ``yes''. 212 213 MKPICINSTALL 214 Can be set to ``yes'' or ``no''. Indicates whether the ar(1) 215 format libraries (lib*_pic.a), used to generate shared li- 216 braries, are installed during a build. 217 218 Default: ``yes'' 219 220 MKPROFILE Can be set to ``yes'' or ``no''. Indicates whether profiled 221 libraries (lib*_p.a) will be built and installed during a 222 build. 223 224 Default: ``yes''; however, some platforms turn off MKPROFILE 225 by default at times due to toolchain problems with profiled 226 code. 227 228 MKSHARE Can be set to ``yes'' or ``no''. Indicates whether files 229 destined to reside in DESTDIR/usr/share will be built and in- 230 stalled during a build. If set to ``no'', then all of 231 MKCATPAGES, MKDOC, MKINFO, MKMAN, and MKNLS will be set to 232 ``no'' unconditionally. 233 234 Default: ``yes'' 235 236 TOOLDIR Directory to hold the host tools, once built. This directory 237 should be unique to a given host system and NetBSD source 238 tree. (However, multiple targets may share the same TOOLDIR; 239 the target-dependent files have unique names.) If unset, a 240 default based on the uname(1) information of the host plat- 241 form will be created in the .OBJDIR of src/tools. 242 243 Default: Unset. 244 245 UNPRIVED If set, then an unprivileged install will occur. The user, 246 group, permissions, and file flags, will not be set on the 247 installed item; instead the information will be appended to a 248 file called METALOG in DESTDIR. The contents of METALOG is 249 used during the generation of the distribution tar files to 250 ensure that the appropriate file ownership is stored. 251 252 Default: Unset. 253 254 UPDATE If set, then all install operations intended to write to 255 DESTDIR will compare file timestamps before installing, and 256 skip the install phase if the destination files are up-to- 257 date. This also has implications on full builds (see next 258 subsection). 259 260 Default: Unset. 261 262 USETOOLS Indicates whether the tools specified by TOOLDIR should be 263 used as part of a build in progress. Must be set to ``yes'' 264 if cross-compiling. 265 266 yes Use the tools from TOOLDIR. 267 268 no Do not use the tools from TOOLDIR, but refuse to build 269 native compilation tool components that are version- 270 specific for that tool. 271 272 never Do not use the tools from TOOLDIR, even when building 273 native tool components. This is similar to the tradi- 274 tional NetBSD build method, but does not verify that 275 the compilation tools in use are up-to-date enough in 276 order to build the tree successfully. This may cause 277 build or runtime problems when building the whole 278 NetBSD source tree. 279 280 Default: ``yes'' if building all or part of a whole NetBSD 281 source tree (detected automatically); ``no'' otherwise (to 282 preserve traditional semantics of the <bsd.*.mk> make(1) in- 283 clude files). 284 285 "make" variables for full builds 286 These variables only affect the top level ``Makefile'' and do not affect 287 manually building subtrees of the NetBSD source code. 288 289 INSTALLWORLDDIR Location for the ``make installworld'' target to install 290 to. 291 292 Default: ``/'' 293 294 MKOBJDIRS Can be set to ``yes'' or ``no''. Indicates whether ob- 295 ject directories will be created automatically (via a 296 ``make obj'' pass) at the start of a build. 297 298 Default: ``no'' 299 300 NBUILDJOBS Now obsolete. Use the make(1) option -j, instead (see 301 below) 302 303 Default: Unset. 304 305 NOCLEANDIR If set, avoids the ``make cleandir'' phase of a full 306 build. This has the effect of allowing only changed 307 files in a source tree to be recompiled. This can speed 308 up builds when updating only a few files in the tree. 309 310 Default: Unset. 311 312 NODISTRIBDIRS If set, avoids the ``make distrib-dirs'' phase of a full 313 build. This skips running mtree(8) on DESTDIR, useful 314 on systems where building as an unprivileged user, or 315 where it is known that the system-wide mtree files have 316 not changed. 317 318 Default: Unset. 319 320 NOINCLUDES If set, avoids the ``make includes'' phase of a full 321 build. This has the effect of preventing make(1) from 322 thinking that some programs are out-of-date simply be- 323 cause the system include files have changed. However, 324 this option should not be used when updating the entire 325 NetBSD source tree arbitrarily; it is suggested to use 326 UPDATE in that case. 327 328 Default: Unset. 329 330 RELEASEDIR If set, specifies the directory to which a release(7) 331 layout will be written at the end of a ``make release''. 332 333 Default: Unset. 334 335 Note: build.sh will provide a default of releasedir (in 336 the top-level .OBJDIR) unless run in `expert' mode 337 338 UPDATE If set, then in addition to the effects described for 339 UPDATE above, this implies the effects of NOCLEANDIR 340 (i.e., ``make cleandir'' is avoided). 341 342BUILDING 343 "make" command line options 344 This is only a summary of options available to make(1); only the options 345 used most frequently with NetBSD builds are listed here. 346 347 -j njob Run up to njob make(1) subjobs in parallel. Makefiles should 348 use .WAIT or have explicit dependancies as necessary to en- 349 force build ordering. If you see build failures with -j, 350 please save complete build logs so the failures can be ana- 351 lyzed. 352 353 -m dir Specify the default directory for searching for system Make- 354 file segments, mainly the <bsd.*.mk> files. When building any 355 full NetBSD source tree, this should be set to the 356 ``share/mk'' directory in the source tree. (This is set auto- 357 matically when building from the top level.) 358 359 -n Display the commands that would have been executed, but do not 360 actually execute them. This will still cause recursion to 361 take place. 362 363 -v var Print make(1)'s idea of the value of var. Does not build any 364 targets. 365 366 var=value Set the variable var to value, overriding any setting speci- 367 fied by the process environment, the MAKECONF configuration 368 file, or the system Makefile segments. 369 370 "make" targets 371 These default targets may be built by running make(1) in any subtree of 372 the NetBSD source code. It is recommended that none of these be used 373 from the top level Makefile; as a specific exception, ``make obj'' and 374 ``make cleandir'' are useful in that context. 375 376 all Build programs, libraries, and preformatted documentation. 377 378 clean Remove program and library object code files. 379 380 cleandir Same as clean, but also remove preformatted documentation, de- 381 pendency files generated by ``make depend'', and any other 382 files known to be created at build time. ``make distclean'' 383 may be used as a synonym, for familiarity with a similar well- 384 known convention. 385 386 depend Create dependency files (.depend) containing more detailed in- 387 formation about the dependencies of source code on header 388 files. Allows programs to be recompiled automatically when a 389 dependency changes. 390 391 dependall Does a ``make depend'' immediately followed by a ``make all''. 392 This improves cache locality of the build since both passes 393 read the source files in their entirety. 394 395 includes Build and install system header files. Typically needed be- 396 fore any system libraries or programs can be built. 397 398 install Install programs, libraries, and documentation into DESTDIR. 399 Few files will be installed to DESTDIR/dev, DESTDIR/etc, 400 DESTDIR/root or DESTDIR/var in order to prevent user supplied 401 configuration data from being overwritten. 402 403 lint Run lint(1) against the C source code, where appropriate, and 404 generate system-installed lint libraries. 405 406 obj Create object directories to be used for built files, instead 407 of building directly in the source tree. 408 409 tags Create ctags(1) searchable function lists usable by the ex(1) 410 and vi(1) text editors. 411 412 "make" targets for the top level 413 Additional make(1) targets are usable specifically from the top source 414 level to facilitate building the entire NetBSD source tree. 415 416 build Build the entire NetBSD system. This orders portions of 417 the source tree such that prerequisites will be built in 418 the proper order. 419 420 distribution Do a ``make build'', and then install a full distribution 421 into DESTDIR, including files in DESTDIR/dev, DESTDIR/etc, 422 DESTDIR/root and DESTDIR/var. 423 424 buildworld As per ``make distribution'', except that it ensures that 425 DESTDIR is not the root directory. 426 427 installworld Install the distribution from DESTDIR to INSTALLWORLDDIR 428 (which defaults to the root directory). Ensures that 429 INSTALLWORLDDIR is the not root directory if cross compil- 430 ing. 431 432 Note: It is highly recommended that you upgrade your kernel 433 and reboot before performing this operation. 434 435 sets Create distribution sets from DESTDIR into 436 RELEASEDIR/MACHINE/binary/sets. Should be run after ``make 437 distribution'' (as ``make build'' does not install all of 438 the required files). 439 440 sourcesets Create source sets of the source tree into 441 RELEASEDIR/source/sets. 442 443 release Do a ``make distribution'', build kernels, distribution me- 444 dia, and install sets (this as per ``make sets''), and then 445 package the system into a standard release layout as de- 446 scribed by release(7). This requires that RELEASEDIR be 447 set (see above). 448 449 regression-tests 450 Can only be run after building the regression tests in the 451 directory ``regress''. Runs the compiled regression tests 452 on the local host. 453 454 The "build.sh" script 455 This script file is a Bourne shell script designed to build the entire 456 NetBSD system on any host with a Bourne shell in /bin/sh, including many 457 that are not POSIX compliant. Note that if a host system's /bin/sh is 458 unusually old and broken, the Korn Shell (/bin/ksh), if available, may be 459 a usable alternative. 460 461 All cross-compile builds, and most native builds, of the entire system 462 should make use of build.sh rather than just running ``make''. This way, 463 the make(1) program will be bootstrapped properly, in case the host sys- 464 tem has an older or incompatible ``make'' program. 465 466 When compiling the entire system via build.sh, many make(1) variables are 467 set for you in order to help encapsulate the build process. In the list 468 of options below, variables that are automatically set by build.sh are 469 noted where applicable. 470 471 The following operations are supported by build.sh: 472 473 build Build the system as per ``make build''. This option im- 474 plies the obj and tools operations. 475 476 distribution Build a full distribution as per ``make distribution''. 477 This option implies the build operation. 478 479 release Build a full release as per ``make release''. This option 480 implies the distribution operation. 481 482 makewrapper Create the nbmake-MACHINE wrapper. This operation is auto- 483 matically performed for any of the other operations. 484 485 obj Perform ``make obj''. 486 487 tools Build and install the host tools from src/tools. 488 489 install=idir Install the contents of DESTDIR to idir, using ``make 490 installworld''. 491 492 kernel=kconf Build a new kernel. The kconf argument is the name of a 493 configuration file suitable for use by config(8). If kconf 494 does not contain any `/' characters, the configuration file 495 is expected to be found in the KERNCONFDIR directory, which 496 is typically sys/arch/MACHINE/conf. The new kernel will be 497 built in a subdirectory of KERNOBJDIR, which is typically 498 sys/arch/MACHINE/compile or an associated object directory. 499 In order to ensure that the kernel is built using up-to- 500 date tools, it is strongly recommended that the tools be 501 rebuilt (using the tools operation). 502 503 releasekernel=kconf 504 Install a gzip(1)ed copy of the kernel built by 505 kernel=kconf into RELEASEDIR/MACHINE/binary/kernel, usually 506 as netbsd-kconf.gz, although the ``netbsd'' prefix is de- 507 termined from the ``config'' directives in kconf. 508 509 sets Perform ``make sets''. 510 511 sourcesets Perform ``make sourcesets''. 512 513 The following command line options alter the behaviour of the above oper- 514 ations: The following command line options alter the behaviour of the 515 build.sh operations described above: 516 517 -a arch Set the value of MACHINE_ARCH to arch. 518 519 -B buildid 520 Set the value of BUILDID to buildid. This will also append the 521 build idenfitier to the name of the ``make'' wrapper script so 522 that the resulting name is of the form 523 ``nbmake-MACHINE-BUILDID''. 524 525 -D dest Set the value of DESTDIR to dest. 526 527 -E Set `expert' mode. This overrides various sanity checks, and 528 allows: DESTDIR does not have to be set to a non-root path for 529 builds, and UNPRIVED does not have to be set when building as a 530 non-root user. 531 532 Note: It is highly recommended that you know what you are doing 533 when you use this option. 534 535 -j njob Passed through to make(1). Makefiles should use .WAIT or have 536 explicit dependancies as necessary to enforce build ordering. 537 If you see build failures with -j, please save complete build 538 logs so the failures can be analyzed. 539 540 -M obj Set MAKEOBJDIRPREFIX to obj. 541 542 -m mach Set the value of MACHINE to mach. This will also override any 543 value of MACHINE_ARCH in the process environment with a value 544 deduced from mach, unless -a is specified, or mach is a special 545 case listed below. All cross builds require -m, but if unset 546 on a NetBSD host, the host's value of MACHINE will be detected 547 and used automatically. 548 549 Some machines support multiple values for MACHINE_ARCH. For a 550 given value of mach, the following MACHINE and MACHINE_ARCH 551 values will result: 552 553 mach MACHINE MACHINE_ARCH 554 evbmips evbmips (not set) 555 evbmips-eb evbmips mipseb 556 evbmips-el evbmips mipsel 557 evbsh3 evbsh3 (not set) 558 evbsh3-eb evbsh3 sh3eb 559 evbsh3-el evbsh3 sh3el 560 sbmips sbmips (not set) 561 sbmips-eb sbmips mipseb 562 sbmips-el sbmips mipsel 563 564 -n Show the commands that would be executed by build.sh, but do 565 not make any changes. This is similar in concept to ``make 566 -n''. 567 568 -O obj Create an appropriate transform macro for MAKEOBJDIR that will 569 place the built object files under obj. For instance, a set- 570 ting of /usr/obj will place build-time files under 571 /usr/obj/bin, /usr/obj/lib, and so forth. 572 573 -o Set the value of MKOBJDIRS to ``no''. Otherwise, it will be 574 automatically set to ``yes'' (which is opposite to the default 575 behaviour). 576 577 -R rel Set the value of RELEASEDIR to rel. 578 579 -r Remove the contents of DESTDIR and TOOLDIR before building 580 (provides a clean starting point). This will skip deleting 581 DESTDIR if building on a native system to the root directory. 582 583 -T tools Set the value of TOOLDIR to tools. If set, the bootstrap 584 ``make'' will only be rebuilt as needed (when the source files 585 for make(1) change). 586 587 -U Set the UNPRIVED variable. 588 589 -u Set the UPDATE variable. 590 591 -V var=[value] 592 Set the variable var to value (which is optional). 593 594 -w wrapper 595 Create the nbmake wrapper script (see below) in a custom loca- 596 tion, specified by wrapper. This allows, for instance, to 597 place the wrapper in PATH automatically. Note that wrapper is 598 the full name of the file, not just a directory name. 599 600 The "nbmake-MACHINE" wrapper script 601 If using the build.sh script to build NetBSD, a nbmake-MACHINE script 602 will be created in TOOLDIR/bin upon the first build to assist in building 603 subtrees on a cross-compile host. 604 605 nbmake-MACHINE can be invoked in lieu of make(1), and will instead call 606 the up-to-date version of ``nbmake'' installed into TOOLDIR/bin with sev- 607 eral key variables pre-set, including MACHINE, MACHINE_ARCH, and TOOLDIR. 608 This script can be symlinked into a directory listed in PATH, or called 609 with an absolute path. 610 611EXAMPLES 612 1. ./build.sh tools kernel=GENERIC 613 614 Build a new toolchain, and use the new toolchain to configure and 615 build a new GENERIC kernel. 616 617 2. ./build.sh -U distribution 618 619 Using unprivileged mode, build a complete distribution to a DESTDIR 620 directory that build.sh selects (and will display). 621 622 3. # ./build.sh -U install=/ 623 624 As root, install to / the distribution that was built by example 2. 625 Even though this is run as root, -U is required so that the permis- 626 sions stored in DESTDIR/METALOG are correctly applied to the files 627 as they're copied to /. 628 629 4. ./build.sh -U -u release 630 631 Using unprivileged mode, build a complete release to DESTDIR and 632 RELEASEDIR directories that build.sh selects (and will display). 633 UPDATE (-u) is set to prevent the ``make cleandir'', so that if this 634 is run after example 2, it doesn't need to redo that portion of the 635 release build. 636 637OBSOLETE VARIABLES 638 NBUILDJOBS Use the make(1) option -j, instead. 639 640 USE_NEW_TOOLCHAIN 641 The new toolchain is now the default. To disable, use 642 TOOLCHAIN_MISSING=yes. 643 644SEE ALSO 645 make(1), hier(7), release(7) 646 647HISTORY 648 The build.sh based build scheme was introduced for NetBSD 1.6 as 649 USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that. 650 651BUGS 652 A few platforms are not yet using this build system. 653 654NetBSD July 3, 2003 NetBSD 655