BUILDING revision 1.20
1BUILDING(8)             NetBSD System Manager's Manual             BUILDING(8)
2
3NAME
4     BUILDING - Procedure for building NetBSD from source code.
5
6STATUS
7     This document is a work-in-progress.  As such, the information described
8     here may not match the reality of the build system as of this writing.
9     Once this document is completely in sync with reality, this paragraph
10     will be removed.
11
12     Discrepancies between this documentation and the current reality of im-
13     plementation are noted specially, as with the note below:
14
15     Note: This document applies only to platforms which use the new toolchain
16     as indicated by the default setting of TOOLCHAIN_MISSING in <bsd.own.mk>.
17     Platforms which have not yet been switched to the new toolchain should
18     continue building traditionally, using the notes specified in the file
19     UPDATING.
20
21REQUIREMENTS
22     NetBSD is designed to be buildable on most POSIX-compliant host systems.
23     The basic build procedure is the same whether compiling natively (on the
24     same NetBSD architecture) or cross compiling (on another architecture or
25     OS).
26
27     This source tree contains a special subtree, ``tools'', which uses the
28     host system to create a build toolchain for the target architecture.  The
29     host system must have at least C and C++ compilers in order to create the
30     toolchain (make is not required); all other tools are created as part of
31     the NetBSD build process.
32
33           Note: A couple host toolchain components are not yet available in
34           the tools directory.  Also, some tools use non-POSIX, non-ANSI C
35           extensions and need to be standardized.  As a result, cross-compil-
36           ing from systems other than NetBSD is not currently supported.
37
38FILES
39   Source tree layout
40
41     doc/BUILDING.mdoc
42                    This document (in -mdoc troff format; the original copy).
43
44     BUILDING       This document (in plaintext).
45
46     Makefile       The main Makefile for NetBSD; should only be run for na-
47                    tive builds with an appropriately up-to-date version of
48                    NetBSD make(1).  (For building from out-of-date systems or
49                    on a non-native host, see the build.sh shell script.)
50
51     UPDATING       Special notes for updating from an earlier revision of
52                    NetBSD.  It is important to read this file before every
53                    build of an updated source tree.
54
55     build.sh       Bourne-compatible shell script used for building the host
56                    build tools and the NetBSD system from scratch.  Can be
57                    used for both native and cross builds, and should be used
58                    instead of make(1) for any source tree that is updated and
59                    recompiled regularly.
60
61     crypto/dist/, dist/, gnu/dist/
62                    Sources imported verbatim from third parties, without man-
63                    gling the existing build structure.  Other source trees in
64                    bin through usr.sbin use the NetBSD make(1) ``reachover''
65                    Makefile semantics when building these programs for a na-
66                    tive host.
67
68     distrib/, etc/
69                    Sources for items used when making a full release snap-
70                    shot, such as files installed in /etc on the destination
71                    system, boot media, and release notes.
72
73     regress/       Regression test harness.  Can be cross-compiled, but only
74                    run natively.
75
76     sys/           NetBSD kernel sources.
77
78     tools/         ``Reachover'' build structure for the host build tools.
79                    This has a special method of determining out-of-date sta-
80                    tus.
81
82     bin/ ... usr.sbin/
83                    Sources to the NetBSD userland (non-kernel) programs.  If
84                    any of these directories are missing, they will be skipped
85                    during the build.
86
87   Build tree layout
88     The NetBSD build tree is described in hier(7), and the release layout is
89     described in release(7).
90
91CONFIGURATION
92   Environment variables
93     Several environment variables control the behaviour of NetBSD builds.
94
95     MACHINE           Machine type.
96
97     MACHINE_ARCH      Machine architecture.
98
99     MAKE              Path name to invoke make(1) as.
100
101     MAKEFLAGS         Flags to invoke make(1) with.
102
103     MAKEOBJDIR        Directory to use as the .OBJDIR for the current direc-
104                       tory.  Used only if MAKEOBJDIRPREFIX is not defined.
105                       MAKEOBJDIR can only be provided in the environment.
106
107     MAKEOBJDIRPREFIX  Top level directory of the object directory tree.  If
108                       this is defined, ${MAKEOBJDIRPREFIX}/${.CURDIR} is used
109                       as the .OBJDIR for the current directory.  The current
110                       directory may be read only.  MAKEOBJDIRPREFIX can only
111                       be provided in the environment.
112
113   "make" variables
114     Several variables control the behavior of NetBSD builds.  Unless other-
115     wise specified, these variables may be set in either the process environ-
116     ment or the make(1) configuration file specified by MAKECONF.
117
118     BUILDID     Identifier for the build.  The identifier will be appended to
119                 object directory names, and can be consulted in the make(1)
120                 configuration file in order to set additional build parame-
121                 ters, such as compiler flags.
122
123     DESTDIR     Directory to contain the built NetBSD system.  If set, spe-
124                 cial options are passed to the compilation tools to prevent
125                 their default use of the host system's /usr/include,
126                 /usr/lib, and so forth.  This pathname should not end with a
127                 slash (/) character (for installation into the system's root
128                 directory, set DESTDIR to an empty string).  The directory
129                 must reside on a file system which supports long file names
130                 and hard links.
131
132                 Default: Empty string if USETOOLS is ``yes''; unset other-
133                 wise.
134
135     MAKECONF    The name of the make(1) configuration file.  Only settable in
136                 the process environment.
137
138                 Default: ``/etc/mk.conf''
139
140     MKCATPAGES  Can be set to ``yes'' or ``no''.  Indicates whether prefor-
141                 matted plaintext manual pages will be created during a build.
142
143                 Default: ``yes''
144
145     MKCRYPTO    Can be set to ``yes'' or ``no''.  Indicates whether crypto-
146                 graphic code will be included in a build; provided for the
147                 benefit of countries that do not allow strong cryptography.
148                 Will not affect use of the standard low-security password en-
149                 cryption system, crypt(3).
150
151                 Default: ``yes''
152
153     MKDOC       Can be set to ``yes'' or ``no''.  Indicates whether system
154                 documentation destined for /usr/share/doc will be installed
155                 during a build.
156
157                 Default: ``yes''
158
159     MKHOSTOBJ   Can be set to ``yes'' or ``no''.  If set to ``yes'', then for
160                 programs intended to be run on the compile host, the name,
161                 release, and architecture of the host operating system will
162                 be suffixed to the name of the object directory created by
163                 ``make obj''.  (This allows multiple host systems to compile
164                 NetBSD for a single target.)  If set to ``no'', then programs
165                 built to be run on the compile host will use the same object
166                 directory names as programs built to be run on the target.
167
168                 Default: ``no''
169
170     MKINFO      Can be set to ``yes'' or ``no''.  Indicates whether GNU Info
171                 files, used for the documentation for most of the compilation
172                 tools, will be created and installed during a build.
173
174                 Default: ``yes''
175
176     MKLINT      Can be set to ``yes'' or ``no''.  Indicates whether lint(1)
177                 will be run against portions of the NetBSD source code during
178                 the build, and whether lint libraries will be installed into
179                 /usr/libdata/lint.
180
181                 Default: ``yes''
182
183     MKMAN       Can be set to ``yes'' or ``no''.  Indicates whether manual
184                 pages will be installed during a build.
185
186                 Default: ``yes''
187
188     MKNLS       Can be set to ``yes'' or ``no''.  Indicates whether Native
189                 Language System locale zone files will be compiled and in-
190                 stalled during a build.
191
192                 Default: ``yes''
193
194     MKOBJ       Can be set to ``yes'' or ``no''.  Indicates whether object
195                 directories will be created when running ``make obj''.  If
196                 set to ``no'', then all built files will be located inside
197                 the regular source tree.
198
199                 Default: ``yes''
200
201     MKPIC       Can be set to ``yes'' or ``no''.  Indicates whether shared
202                 objects and libraries will be created and installed during a
203                 build.  If set to ``no'', the entire built system will be
204                 statically linked.
205
206                 Default: Platform dependent.  As of this writing, all plat-
207                 forms except sh3 default to ``yes''.
208
209     MKPICINSTALL
210                 Can be set to ``yes'' or ``no''.  Indicates whether the ar(1)
211                 format libraries (lib*_pic.a), used to generate shared li-
212                 braries, are installed during a build.
213
214                 Default: ``yes''
215
216     MKPROFILE   Can be set to ``yes'' or ``no''.  Indicates whether profiled
217                 libraries (lib*_p.a) will be built and installed during a
218                 build.
219
220                 Default: ``yes''; however, some platforms turn off MKPROFILE
221                 by default at times due to toolchain problems with profiled
222                 code.
223
224     MKSHARE     Can be set to ``yes'' or ``no''.  Indicates whether files
225                 destined to reside in /usr/share will be built and installed
226                 during a build.  If set to ``no'', then all of MKCATPAGES,
227                 MKDOC, MKINFO, MKMAN, and MKNLS will be set to ``no'' uncon-
228                 ditionally.
229
230                 Default: ``yes''
231
232     TOOLDIR     Directory to hold the host tools, once built.  This directory
233                 should be unique to a given host system and NetBSD source
234                 tree.  (However, multiple targets may share the same TOOLDIR;
235                 the target-dependent files have unique names.)  If unset, a
236                 default based on the uname(1) information of the host plat-
237                 form will be created in the .OBJDIR of src/tools.
238
239                 Default: Unset.
240
241     UNPRIVED    If set, then an unprivileged install will occur.  The user,
242                 group, permissions, and file flags, will not be set on the
243                 installed item; instead the information will be appended to a
244                 file called METALOG in DESTDIR.  The contents of METALOG is
245                 used during the generation of the distribution tar files to
246                 ensure that the appropriate file ownership is stored.
247
248                 Default: Unset.
249
250     UPDATE      If set, then all install operations intended to write to
251                 DESTDIR will compare file timestamps before installing, and
252                 skip the install phase if the destination files are up-to-
253                 date.  This also has implications on full builds (see next
254                 subsection).
255
256                 Default: Unset.
257
258     USETOOLS    Indicates whether the tools specified by TOOLDIR should be
259                 used as part of a build in progress.  Must be set to ``yes''
260                 if cross-compiling.
261
262                 yes    Use the tools from TOOLDIR.
263
264                 no     Do not use the tools from TOOLDIR, but refuse to build
265                        native compilation tool components that are version-
266                        specific for that tool.
267
268                 never  Do not use the tools from TOOLDIR, even when building
269                        native tool components.  This is similar to the tradi-
270                        tional NetBSD build method, but does not verify that
271                        the compilation tools in use are up-to-date enough in
272                        order to build the tree successfully.  This may cause
273                        build or runtime problems when building the whole
274                        NetBSD source tree.
275
276                 Default: ``yes'' if building all or part of a whole NetBSD
277                 source tree (detected automatically); ``no'' otherwise (to
278                 preserve traditional semantics of the <bsd.*.mk> make(1) in-
279                 clude files).
280
281   "make" variables for full builds
282     These variables only affect the top level ``Makefile'' and do not affect
283     manually building subtrees of the NetBSD source code.
284
285     INSTALLWORLDDIR  Location for the ``make installworld'' target to install
286                      to.
287
288                      Default: ``/''
289
290     MKOBJDIRS        Can be set to ``yes'' or ``no''.  Indicates whether ob-
291                      ject directories will be created automatically (via a
292                      ``make obj'' pass) at the start of a build.
293
294                      Default: ``yes''
295
296     NBUILDJOBS       Now obsolete.  Use the make(1) option -j, instead (see
297                      below)
298
299                      Default: Unset.
300
301     NOCLEANDIR       If set, avoids the ``make cleandir'' phase of a full
302                      build.  This has the effect of allowing only changed
303                      files in a source tree to be recompiled.  This can speed
304                      up builds when updating only a few files in the tree.
305
306                      Default: Unset.
307
308     NODISTRIBDIRS    If set, avoids the ``make distrib-dirs'' phase of a full
309                      build.  This skips running mtree(8) on DESTDIR, useful
310                      on systems where building as an unprivileged user, or
311                      where it is known that the system-wide mtree files have
312                      not changed.
313
314                      Default: Unset.
315
316     NOINCLUDES       If set, avoids the ``make includes'' phase of a full
317                      build.  This has the effect of preventing make(1) from
318                      thinking that some programs are out-of-date simply be-
319                      cause the system include files have changed.  However,
320                      this option should not be used when updating the entire
321                      NetBSD source tree arbitrarily; it is suggested to use
322                      UPDATE in that case.
323
324                      Default: Unset.
325
326     RELEASEDIR       If set, specifies the directory to which a release(7)
327                      layout will be written at the end of a ``make release''.
328
329                      Default: Unset.
330
331     UPDATE           If set, then in addition to the effects described for
332                      UPDATE above, this implies the effects of NOCLEANDIR.
333
334BUILDING
335   "make" command line options
336     This is only a summary of options available to make(1); only the options
337     used most frequently with NetBSD builds are listed here.
338
339     -j njob    Run up to njob make(1) subjobs in parallel.  Makefiles should
340                use .WAIT or have explicit dependancies as necessary to en-
341                force build ordering.  If you see build failures with -j,
342                please save complete build logs so the failures can be ana-
343                lyzed.
344
345     -m dir     Specify the default directory for searching for system Make-
346                file segments, mainly the <bsd.*.mk> files.  When building any
347                full NetBSD source tree, this should be set to the
348                ``share/mk'' directory in the source tree.  (This is set auto-
349                matically when building from the top level.)
350
351     -n         Display the commands that would have been executed, but do not
352                actually execute them.  This will still cause recursion to
353                take place.
354
355     -v var     Print make(1)'s idea of the value of var.  Does not build any
356                targets.
357
358     var=value  Set the variable var to value, overriding any setting speci-
359                fied by the process environment, the MAKECONF configuration
360                file, or the system Makefile segments.
361
362   "make" targets
363     These default targets may be built by running make(1) in any subtree of
364     the NetBSD source code.  It is recommended that none of these be used
365     from the top level Makefile; as a specific exception, ``make obj'' and
366     ``make cleandir'' are useful in that context.
367
368     all        Build programs, libraries, and preformatted documentation.
369
370     clean      Remove program and library object code files.
371
372     cleandir   Same as clean, but also remove preformatted documentation, de-
373                pendency files generated by ``make depend'', and any other
374                files known to be created at build time.  ``make distclean''
375                may be used as a synonym, for familiarity with a similar well-
376                known convention.
377
378     depend     Create dependency files (.depend) containing more detailed in-
379                formation about the dependencies of source code on header
380                files.  Allows programs to be recompiled automatically when a
381                dependency changes.
382
383     dependall  Does a ``make depend'' immediately followed by a ``make all''.
384                This improves cache locality of the build since both passes
385                read the source files in their entirety.
386
387     includes   Build and install system header files.  Typically needed be-
388                fore any system libraries or programs can be built.
389
390     install    Install programs, libraries, and documentation into DESTDIR.
391                Few files will be installed to /dev, /etc, /root or /var in
392                order to prevent user supplied configuration data from being
393                overwritten.
394
395     lint       Run lint(1) against the C source code, where appropriate, and
396                generate system-installed lint libraries.
397
398     obj        Create object directories to be used for built files, instead
399                of building directly in the source tree.
400
401     tags       Create ctags(1) searchable function lists usable by the ex(1)
402                and vi(1) text editors.
403
404   "make" targets for the top level
405     Additional make(1) targets are usable specifically from the top source
406     level to facilitate building the entire NetBSD source tree.
407
408     build         Build the entire NetBSD system.  This orders portions of
409                   the source tree such that prerequisites will be built in
410                   the proper order.
411
412     distribution  Do a ``make build'', and then install a full distribution
413                   into DESTDIR.
414
415     buildworld    As per ``make distribution'', except that it ensures that
416                   DESTDIR is not the root directory.
417
418     installworld  Install the distribution from DESTDIR to INSTALLWORLDDIR
419                   (which defaults to the root directory).  Ensures that
420                   INSTALLWORLDDIR is the not root directory if cross compil-
421                   ing.
422
423                   Note: It is highly recommended that you upgrade your kernel
424                   and reboot before performing this operation.
425
426     release       Do a ``make build'', then package the system into a stan-
427                   dard release layout as described by release(7).  This re-
428                   quires that RELEASEDIR be set (see above).
429
430     regression-tests
431                   Can only be run after building the regression tests in the
432                   directory ``regress''.  Runs the compiled regression tests
433                   on the local host.
434
435   The "build.sh" script
436     This script file is a Bourne shell script designed to build the entire
437     NetBSD system on any host with a Bourne shell in /bin/sh, including many
438     that are not POSIX compliant.  Note that if a host system's /bin/sh is
439     unusually old and broken, the Korn Shell (/bin/ksh), if available, may be
440     a usable alternative.
441
442     All cross-compile builds, and most native builds, of the entire system
443     should make use of build.sh rather than just running ``make''.  This way,
444     the make(1) program will be bootstrapped properly, in case the host sys-
445     tem has an older or incompatible ``make'' program.
446
447     When compiling the entire system via build.sh, many make(1) variables are
448     set for you in order to help encapsulate the build process.  In the list
449     of options below, variables that are automatically set by build.sh are
450     noted where applicable.
451
452     The following are available command line options that may be supplied to
453     build.sh:
454
455     -a arch   Set the value of MACHINE_ARCH to arch.
456
457     -B buildid
458               Set the value of BUILDID to buildid.  This will also append the
459               build idenfitier to the name of the ``make'' wrapper script so
460               that the resulting name is of the form ``nbmake-MACHINE-
461               BUILDID''.
462
463     -b        Bootstrap ``make'' and create a nbmake-MACHINE script (see be-
464               low).
465
466     -D dest   Set the value of DESTDIR to dest.
467
468     -d        Build a full distribution.  This differs from a default build
469               in that files will also be installed to /dev, /etc, /root and
470               /var.  Note this does not build a ``release''; no release sets
471               are placed in ${RELEASEDIR}.  -d is implied by -R.
472
473     -E        Set `expert' mode; DESTDIR does not have to be set to a non-
474               root path for builds when this is set.
475
476     -i installworlddir
477               Install the contents of DESTDIR to installworlddir after all
478               other operations have completed, using the top level
479               ``installworld'' target.
480
481     -j njob   Passed through to make(1).  Makefiles should use .WAIT or have
482               explicit dependancies as necessary to enforce build ordering.
483               If you see build failures with -j, please save complete build
484               logs so the failures can be analyzed.
485
486     -k kernel
487               Build a new kernel.  The kernel argument is the name of a con-
488               figuration file suitable for use by config(8).  If kernel does
489               not contain any `/' characters, the configuration file is ex-
490               pected to be found in the KERNCONFDIR directory, which is typi-
491               cally sys/arch/MACHINE/conf.  The new kernel will be built in a
492               subdirectory of KERNOBJDIR, which is typically
493               sys/arch/MACHINE/compile or an associated object directory.  In
494               order to ensure that the kernel is built using up-to-date
495               tools, it is strongly recommended that the tools be rebuilt
496               (using the -t option) in a separate invocation of build.sh pri-
497               or to using the -k option, or that the -t and -k options be
498               used together in a single invocation of build.sh.
499
500     -M obj    Set MAKEOBJDIRPREFIX to obj.
501
502     -m mach   Set the value of MACHINE to mach.  This will also override any
503               value of MACHINE_ARCH in the process environment with a value
504               deduced from mach, unless -a is specified.  All cross builds
505               require -m, but if unset on a NetBSD host, the host's value of
506               MACHINE will be detected and used automatically.
507
508     -n        Show the commands that would be executed by build.sh, but do
509               not make any changes.  This is similar in concept to ``make
510               -n''.
511
512     -O obj    Create an appropriate transform macro for MAKEOBJDIR that will
513               place the built object files under obj.  For instance, a set-
514               ting of /usr/obj will place build-time files under
515               /usr/obj/bin, /usr/obj/lib, and so forth.
516
517     -o        Set the value of MKOBJDIRS to ``no''.
518
519     -R rel    Set the value of RELEASEDIR to rel.  Setting this option will
520               cause build.sh to run ``make release'' instead of ``make
521               build''.
522
523     -r        Remove the contents of DESTDIR and TOOLDIR before building
524               (provides a clean starting point).  This will skip deleting
525               DESTDIR if building on a native system to the root directory.
526
527     -T tools  Set the value of TOOLDIR to tools.  If set, the bootstrap
528               ``make'' will only be rebuilt as needed (when the source files
529               for make(1) change).
530
531     -t        Build and install the host tools from src/tools only.  This op-
532               tion implies -b.
533
534     -U        Set the UNPRIVED variable.
535
536     -u        Set the UPDATE variable.
537
538     -w wrapper
539               Create the nbmake wrapper script (see below) in a custom loca-
540               tion, specified by wrapper.  This allows, for instance, to
541               place the wrapper in PATH automatically.  Note that wrapper is
542               the full name of the file, not just a directory name.
543
544   The "nbmake-MACHINE" wrapper script
545     If using the build.sh script to build NetBSD, a nbmake-MACHINE script
546     will be created in TOOLDIR/bin upon the first build to assist in building
547     subtrees on a cross-compile host.
548
549     nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
550     the up-to-date version of ``nbmake'' installed into TOOLDIR/bin with sev-
551     eral key variables pre-set, including MACHINE, MACHINE_ARCH, and TOOLDIR.
552     This script can be symlinked into a directory listed in PATH, or called
553     with an absolute path.
554
555EXAMPLES
556     ./build.sh -t
557               Build a new toolchain.
558
559     cd ${KERNCONFDIR} ; ${TOOLDIR}/bin/nbconfig GENERIC
560               Use the new version of config(8) to prepare to build a new
561               GENERIC kernel.
562
563     cd ${KERNOBJDIR}/GENERIC ; ${TOOLDIR}/bin/nbmake-${MACHINE} dependall
564               Use the new toolchain to build a new GENERIC kernel.
565
566     ./build.sh -t -k GENERIC
567               Build a new toolchain, and use the new toolchain to configure
568               and build a new GENERIC kernel.
569
570     ./build.sh -U -d
571               Using unprivileged mode, build a complete distribution in
572               DESTDIR.
573
574     ./build.sh -U -R /some/dir/RELEASE
575               Using unprivileged mode, build a complete release in the speci-
576               fied release directory.
577
578OBSOLETE VARIABLES
579     NBUILDJOBS  Use the make(1) option -j, instead.
580
581     USE_NEW_TOOLCHAIN
582                 The new toolchain is now the default.  To disable, use
583                 TOOLCHAIN_MISSING=yes.
584
585SEE ALSO
586     make(1), hier(7), release(7)
587
588HISTORY
589     The build.sh based build scheme was introduced for NetBSD 1.6 as
590     USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
591
592BUGS
593     A few platforms are not yet using this build system.
594
595NetBSD                         December 25, 2002                             9
596