1.. SPDX-License-Identifier: GPL-2.0+
2
3Buildman build tool
4===================
5
6Quick-start
7-----------
8
9If you just want to quickly set up buildman so you can build something (for
10example Raspberry Pi 2):
11
12.. code-block:: bash
13
14 cd /path/to/u-boot
15 PATH=$PATH:`pwd`/tools/buildman
16 buildman --fetch-arch arm
17 buildman -k rpi_2
18 ls ../current/rpi_2
19 # u-boot.bin is the output image
20
21
22What is this?
23-------------
24
25This tool handles building U-Boot to check that you have not broken it
26with your patch series. It can build each individual commit and report
27which boards fail on which commits, and which errors come up. It aims
28to make full use of multi-processor machines.
29
30A key feature of buildman is its output summary, which allows warnings,
31errors or image size increases in a particular commit or board to be
32quickly identified and the offending commit pinpointed. This can be a big
33help for anyone working with >10 patches at a time.
34
35
36Caveats
37-------
38
39Buildman can be stopped and restarted, in which case it will continue
40where it left off. This should happen cleanly and without side-effects.
41If not, it is a bug, for which a patch would be welcome.
42
43Buildman gets so tied up in its work that it can ignore the outside world.
44You may need to press Ctrl-C several times to quit it. Also it will print
45out various exceptions when stopped. You may have to kill it since the
46Ctrl-C handling is somewhat broken.
47
48
49Theory of Operation
50-------------------
51
52(please read this section in full twice or you will be perpetually confused)
53
54Buildman is a builder. It is not make, although it runs make. It does not
55produce any useful output on the terminal while building, except for
56progress information (but see -v below). All the output (errors, warnings and
57binaries if you ask for them) is stored in output directories, which you can
58look at from a separate 'buildman -s' instance while the build is progressing,
59or when it is finished.
60
61Buildman is designed to build entire git branches, i.e. muliple commits. It
62can be run repeatedly on the same branch after making changes to commits on
63that branch. In this case it will automatically rebuild commits which have
64changed (and remove its old results for that commit). It is possible to build
65a branch for one board, then later build it for another board. This adds to
66the output, so now you have results for two boards. If you want buildman to
67re-build a commit it has already built (e.g. because of a toolchain update),
68use the -f flag.
69
70Buildman produces a concise summary of which boards succeeded and failed.
71It shows which commit introduced which board failure using a simple
72red/green colour coding (with yellow/cyan for warnings). Full error
73information can be requested, in which case it is de-duped and displayed
74against the commit that introduced the error. An example workflow is below.
75
76Buildman stores image size information and can report changes in image size
77from commit to commit. An example of this is below.
78
79Buildman starts multiple threads, and each thread builds for one board at
80a time. A thread starts at the first commit, configures the source for your
81board and builds it. Then it checks out the next commit and does an
82incremental build (i.e. not using 'make xxx_defconfig' unless you use -C).
83Eventually the thread reaches the last commit and stops. If a commit causes
84an error or warning, buildman will try it again after reconfiguring (but see
85-Q). Thus some commits may be built twice, with the first result silently
86discarded. Lots of errors and warnings will causes lots of reconfigures and your
87build will be very slow. This is because a file that produces just a warning
88would not normally be rebuilt in an incremental build. Once a thread finishes
89building all the commits for a board, it starts on the commits for another
90board.
91
92Buildman works in an entirely separate place from your U-Boot repository.
93It creates a separate working directory for each thread, and puts the
94output files in the working directory, organised by commit name and board
95name, in a two-level hierarchy (but see -P).
96
97Buildman is invoked in your U-Boot directory, the one with the .git
98directory. It clones this repository into a copy for each thread, and the
99threads do not affect the state of your git repository. Any checkouts done
100by the thread affect only the working directory for that thread.
101
102Buildman automatically selects the correct tool chain for each board. You
103must supply suitable tool chains (see --fetch-arch), but buildman takes care
104of selecting the right one.
105
106Buildman generally builds a branch (with the -b flag), and in this case
107builds the upstream commit as well, for comparison. So even if you have one
108commit in your branch, two commits will be built. Put all your commits in a
109branch, set the branch's upstream to a valid value, and all will be well.
110Otherwise buildman will perform random actions. Use -n to check what the
111random actions might be.
112
113Buildman effectively has two modes: without -s it builds, with -s it
114summarises the results of previous (or active) builds.
115
116If you just want to build the current source tree, leave off the -b flag.
117This will display results and errors as they happen. You can still look at
118them later using -se. Note that buildman will assume that the source has
119changed, and will build all specified boards in this case.
120
121Buildman is optimised for building many commits at once, for many boards.
122On multi-core machines, Buildman is fast because it uses most of the
123available CPU power. When it gets to the end, or if you are building just
124a few commits or boards, it will be pretty slow. As a tip, if you don't
125plan to use your machine for anything else, you can use -T to increase the
126number of threads beyond the default.
127
128
129Selecting which boards to build
130-------------------------------
131
132Buildman lets you build all boards, or a subset. Specify the subset by passing
133command-line arguments that list the desired build target, architecture,
134CPU, board name, vendor, SoC or options. Multiple arguments are allowed. Each
135argument will be interpreted as a regular expression, so behaviour is a superset
136of exact or substring matching. Examples are:
137
138- 'tegra20' - all boards with a Tegra20 SoC
139- 'tegra' - all boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...)
140- '^tegra[23]0$' - all boards with either Tegra20 or Tegra30 SoC
141- 'powerpc' - all PowerPC boards
142
143While the default is to OR the terms together, you can also make use of
144the '&' operator to limit the selection:
145
146- 'freescale & arm sandbox' - all Freescale boards with ARM architecture, plus
147 sandbox
148
149You can also use -x to specifically exclude some boards. For example:
150
151 buildman arm -x nvidia,freescale,.*ball$
152
153means to build all arm boards except nvidia, freescale and anything ending
154with 'ball'.
155
156For building specific boards you can use the --boards (or --bo) option, which
157takes a comma-separated list of board target names and be used multiple times
158on the command line:
159
160.. code-block:: bash
161
162 buildman --boards sandbox,snow --boards firefly-rk3399
163
164It is convenient to use the -n option to see what will be built based on
165the subset given. Use -v as well to get an actual list of boards.
166
167Buildman does not store intermediate object files. It optionally copies
168the binary output into a directory when a build is successful (-k). Size
169information is always recorded. It needs a fair bit of disk space to work,
170typically 250MB per thread.
171
172
173Setting up
174----------
175
176#. Get the U-Boot source. You probably already have it, but if not these
177 steps should get you started with a repo and some commits for testing.
178
179 .. code-block:: bash
180
181 cd /path/to/u-boot
182 git clone git://git.denx.de/u-boot.git .
183 git checkout -b my-branch origin/master
184 # Add some commits to the branch, reading for testing
185
186#. Create ~/.buildman to tell buildman where to find tool chains (see
187 buildman_settings_ for details). As an example::
188
189 # Buildman settings file
190
191 [toolchain]
192 root: /
193 rest: /toolchains/*
194 eldk: /opt/eldk-4.2
195 arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
196 aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
197
198 [toolchain-prefix]
199 arc = /opt/arc/arc_gnu_2021.03_prebuilt_elf32_le_linux_install/bin/arc-elf32-
200
201 [toolchain-alias]
202 riscv = riscv32
203 sh = sh4
204 x86: i386
205
206
207 This selects the available toolchain paths. Add the base directory for
208 each of your toolchains here. Buildman will search inside these directories
209 and also in any '/usr' and '/usr/bin' subdirectories.
210
211 Make sure the tags (here root: rest: and eldk:) are unique.
212
213 The toolchain-alias section indicates that the i386 toolchain should be used
214 to build x86 commits.
215
216 Note that you can also specific exactly toolchain prefixes if you like::
217
218 [toolchain-prefix]
219 arm: /opt/arm-eabi-4.6/bin/arm-eabi-
220
221 or even::
222
223 [toolchain-prefix]
224 arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc
225
226 This tells buildman that you want to use this exact toolchain for the arm
227 architecture. This will override any toolchains found by searching using the
228 [toolchain] settings.
229
230 Since the toolchain prefix is an explicit request, buildman will report an
231 error if a toolchain is not found with that prefix. The current PATH will be
232 searched, so it is possible to use::
233
234 [toolchain-prefix]
235 arm: arm-none-eabi-
236
237 and buildman will find arm-none-eabi-gcc in /usr/bin if you have it
238 installed.
239
240 Another example::
241
242 [toolchain-wrapper]
243 wrapper: ccache
244
245 This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In
246 this example, ccache. It doesn't affect the toolchain scan. The wrapper is
247 added when CROSS_COMPILE environtal variable is set. The name in this
248 section is ignored. If more than one line is provided, only the last one
249 is taken.
250
251#. Make sure you have the require Python pre-requisites
252
253 Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
254 urllib2. These should normally be available, but if you get an error like
255 this then you will need to obtain those modules::
256
257 ImportError: No module named multiprocessing
258
259
260#. Check the available toolchains
261
262 Run this check to make sure that you have a toolchain for every architecture::
263
264 $ ./tools/buildman/buildman --list-tool-chains
265 Scanning for tool chains
266 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-'
267 Tool chain test: OK, arch='x86', priority 1
268 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-'
269 Tool chain test: OK, arch='arm', priority 1
270 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux'
271 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.'
272 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin'
273 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc'
274 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin'
275 Tool chain test: OK, arch='i386', priority 4
276 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux'
277 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.'
278 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin'
279 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc'
280 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin'
281 Tool chain test: OK, arch='aarch64', priority 4
282 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux'
283 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.'
284 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin'
285 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc'
286 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin'
287 Tool chain test: OK, arch='microblaze', priority 4
288 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux'
289 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.'
290 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin'
291 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc'
292 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin'
293 Tool chain test: OK, arch='mips64', priority 4
294 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux'
295 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.'
296 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin'
297 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc'
298 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin'
299 Tool chain test: OK, arch='sparc64', priority 4
300 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi'
301 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.'
302 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin'
303 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc'
304 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin'
305 Tool chain test: OK, arch='arm', priority 3
306 Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1
307 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux'
308 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.'
309 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin'
310 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc'
311 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin'
312 Tool chain test: OK, arch='sparc', priority 4
313 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux'
314 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.'
315 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin'
316 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc'
317 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin'
318 Tool chain test: OK, arch='mips', priority 4
319 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux'
320 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.'
321 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin'
322 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc'
323 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc'
324 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin'
325 Tool chain test: OK, arch='x86_64', priority 4
326 Tool chain test: OK, arch='x86_64', priority 4
327 Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
328 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux'
329 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.'
330 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin'
331 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc'
332 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin'
333 Tool chain test: OK, arch='m68k', priority 4
334 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux'
335 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.'
336 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin'
337 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
338 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin'
339 Tool chain test: OK, arch='powerpc', priority 4
340 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux'
341 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.'
342 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin'
343 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc'
344 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin'
345 Tool chain test: OK, arch='bfin', priority 6
346 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux'
347 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.'
348 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin'
349 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc'
350 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin'
351 Tool chain test: OK, arch='sparc', priority 4
352 Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4
353 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux'
354 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.'
355 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin'
356 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc'
357 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin'
358 Tool chain test: OK, arch='mips', priority 4
359 Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4
360 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux'
361 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.'
362 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin'
363 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc'
364 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin'
365 Tool chain test: OK, arch='m68k', priority 4
366 Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4
367 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux'
368 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.'
369 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin'
370 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
371 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin'
372 Tool chain test: OK, arch='powerpc', priority 4
373 Tool chain test: OK, arch='or32', priority 4
374 - scanning path '/'
375 - looking in '/.'
376 - looking in '/bin'
377 - looking in '/usr/bin'
378 - found '/usr/bin/i586-mingw32msvc-gcc'
379 - found '/usr/bin/c89-gcc'
380 - found '/usr/bin/x86_64-linux-gnu-gcc'
381 - found '/usr/bin/gcc'
382 - found '/usr/bin/c99-gcc'
383 - found '/usr/bin/arm-linux-gnueabi-gcc'
384 - found '/usr/bin/aarch64-linux-gnu-gcc'
385 - found '/usr/bin/winegcc'
386 - found '/usr/bin/arm-linux-gnueabihf-gcc'
387 Tool chain test: OK, arch='i586', priority 11
388 Tool chain test: OK, arch='c89', priority 11
389 Tool chain test: OK, arch='x86_64', priority 4
390 Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
391 Tool chain test: OK, arch='sandbox', priority 11
392 Tool chain test: OK, arch='c99', priority 11
393 Tool chain test: OK, arch='arm', priority 4
394 Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
395 Tool chain test: OK, arch='aarch64', priority 4
396 Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4
397 Tool chain test: OK, arch='sandbox', priority 11
398 Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11
399 Tool chain test: OK, arch='arm', priority 4
400 Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
401 List of available toolchains (34):
402 aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
403 alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc
404 am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
405 arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc
406 bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
407 c89 : /usr/bin/c89-gcc
408 c99 : /usr/bin/c99-gcc
409 frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc
410 h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc
411 hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc
412 hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
413 i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc
414 i586 : /usr/bin/i586-mingw32msvc-gcc
415 ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc
416 m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc
417 m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc
418 microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
419 mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc
420 mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc
421 or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
422 powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
423 powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
424 ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
425 s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
426 sandbox : /usr/bin/gcc
427 sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc
428 sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc
429 sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
430 tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc
431 x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc
432 x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
433
434
435 You can see that everything is covered, even some strange ones that won't
436 be used (c88 and c99). This is a feature.
437
438
439#. Install new toolchains if needed
440
441 You can download toolchains and update the [toolchain] section of the
442 settings file to find them.
443
444 To make this easier, buildman can automatically download and install
445 toolchains from kernel.org. First list the available architectures::
446
447 $ ./tools/buildman/buildman --fetch-arch list
448 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
449 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
450 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
451 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
452 Available architectures: alpha am33_2.0 arm bfin cris crisv32 frv h8300
453 hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
454 sparc sparc64 tilegx x86_64 xtensa
455
456 Then pick one and download it::
457
458 $ ./tools/buildman/buildman --fetch-arch or32
459 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
460 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
461 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
462 Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
463 Unpacking to: /home/sjg/.buildman-toolchains
464 Testing
465 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
466 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
467 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
468 Tool chain test: OK
469
470 Or download them all from kernel.org and move them to /toolchains directory:
471
472 .. code-block:: bash
473
474 ./tools/buildman/buildman --fetch-arch all
475 sudo mkdir -p /toolchains
476 sudo mv ~/.buildman-toolchains/*/* /toolchains/
477
478 Buildman should now be set up to use your new toolchain.
479
480 At the time of writing, U-Boot has these architectures:
481
482 arc, arm, m68k, microblaze, mips, nios2, powerpc, sandbox, sh, x86, xtensa
483
484
485How to run it
486-------------
487
488First do a dry run using the -n flag: (replace <branch> with a real, local
489branch with a valid upstream):
490
491.. code-block:: bash
492
493 ./tools/buildman/buildman -b <branch> -n
494
495If it can't detect the upstream branch, try checking out the branch, and
496doing something like 'git branch --set-upstream-to upstream/master'
497or something similar. Buildman will try to guess a suitable upstream branch
498if it can't find one (you will see a message like "Guessing upstream as ...").
499You can also use the -c option to manually specify the number of commits to
500build.
501
502As an example::
503
504 Dry run, so not doing much. But I would do this:
505
506 Building 18 commits for 1059 boards (4 threads, 1 job per thread)
507 Build directory: ../lcd9b
508 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm
509 c18f1b4 tegra: Use const for pinmux_config_pingroup/table()
510 2f043ae tegra: Add display support to funcmux
511 e349900 tegra: fdt: Add pwm binding and node
512 424a5f0 tegra: fdt: Add LCD definitions for Tegra
513 0636ccf tegra: Add support for PWM
514 a994fe7 tegra: Add SOC support for display/lcd
515 fcd7350 tegra: Add LCD driver
516 4d46e9d tegra: Add LCD support to Nvidia boards
517 991bd48 arm: Add control over cachability of memory regions
518 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
519 d92aff7 lcd: Add support for flushing LCD fb from dcache after update
520 dbd0677 tegra: Align LCD frame buffer to section boundary
521 0cff9b8 tegra: Support control of cache settings for LCD
522 9c56900 tegra: fdt: Add LCD definitions for Seaboard
523 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
524 cac5a23 tegra: Enable display/lcd support on Seaboard
525 49ff541 wip
526
527 Total boards to build for each commit: 1059
528
529This shows that it will build all 1059 boards, using 4 threads (because
530we have a 4-core CPU). Each thread will run with -j1, meaning that each
531make job will use a single CPU. The list of commits to be built helps you
532confirm that things look about right. Notice that buildman has chosen a
533'base' directory for you, immediately above your source tree.
534
535Buildman works entirely inside the base directory, here ../lcd9b,
536creating a working directory for each thread, and creating output
537directories for each commit and board.
538
539
540Suggested Workflow
541------------------
542
543To run the build for real, take off the -n:
544
545.. code-block:: bash
546
547 ./tools/buildman/buildman -b <branch>
548
549Buildman will set up some working directories, and get started. After a
550minute or so it will settle down to a steady pace, with a display like this::
551
552 Building 18 commits for 1059 boards (4 threads, 1 job per thread)
553 528 36 124 /19062 -18374 1:13:30 : SIMPC8313_SP
554
555This means that it is building 19062 board/commit combinations. So far it
556has managed to successfully build 528. Another 36 have built with warnings,
557and 124 more didn't build at all. It has 18374 builds left to complete.
558Buildman expects to complete the process in around an hour and a quarter.
559Use this time to buy a faster computer.
560
561
562To find out how the build went, ask for a summary with -s. You can do this
563either before the build completes (presumably in another terminal) or
564afterwards. Let's work through an example of how this is used::
565
566 $ ./tools/buildman/buildman -b lcd9b -s
567 ...
568 01: Merge branch 'master' of git://git.denx.de/u-boot-arm
569 powerpc: + galaxy5200_LOWBOOT
570 02: tegra: Use const for pinmux_config_pingroup/table()
571 03: tegra: Add display support to funcmux
572 04: tegra: fdt: Add pwm binding and node
573 05: tegra: fdt: Add LCD definitions for Tegra
574 06: tegra: Add support for PWM
575 07: tegra: Add SOC support for display/lcd
576 08: tegra: Add LCD driver
577 09: tegra: Add LCD support to Nvidia boards
578 10: arm: Add control over cachability of memory regions
579 11: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
580 12: lcd: Add support for flushing LCD fb from dcache after update
581 arm: + lubbock
582 13: tegra: Align LCD frame buffer to section boundary
583 14: tegra: Support control of cache settings for LCD
584 15: tegra: fdt: Add LCD definitions for Seaboard
585 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
586 17: tegra: Enable display/lcd support on Seaboard
587 18: wip
588
589This shows which commits have succeeded and which have failed. In this case
590the build is still in progress so many boards are not built yet (use -u to
591see which ones). But already we can see a few failures. The galaxy5200_LOWBOOT
592never builds correctly. This could be a problem with our toolchain, or it
593could be a bug in the upstream. The good news is that we probably don't need
594to blame our commits. The bad news is that our commits are not tested on that
595board.
596
597Commit 12 broke lubbock. That's what the '+ lubbock', in red, means. The
598failure is never fixed by a later commit, or you would see lubbock again, in
599green, without the +.
600
601To see the actual error::
602
603 $ ./tools/buildman/buildman -b <branch> -se
604 ...
605 12: lcd: Add support for flushing LCD fb from dcache after update
606 arm: + lubbock
607 +common/libcommon.o: In function `lcd_sync':
608 +common/lcd.c:120: undefined reference to `flush_dcache_range'
609 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
610 +make: *** [build/u-boot] Error 139
611 13: tegra: Align LCD frame buffer to section boundary
612 14: tegra: Support control of cache settings for LCD
613 15: tegra: fdt: Add LCD definitions for Seaboard
614 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
615 -common/lcd.c:120: undefined reference to `flush_dcache_range'
616 +common/lcd.c:125: undefined reference to `flush_dcache_range'
617 17: tegra: Enable display/lcd support on Seaboard
618 18: wip
619
620So the problem is in lcd.c, due to missing cache operations. This information
621should be enough to work out what that commit is doing to break these
622boards. (In this case pxa did not have cache operations defined).
623
624Note that if there were other boards with errors, the above command would
625show their errors also. Each line is shown only once. So if lubbock and snow
626produce the same error, we just see::
627
628 12: lcd: Add support for flushing LCD fb from dcache after update
629 arm: + lubbock snow
630 +common/libcommon.o: In function `lcd_sync':
631 +common/lcd.c:120: undefined reference to `flush_dcache_range'
632 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
633 +make: *** [build/u-boot] Error 139
634
635But if you did want to see just the errors for lubbock, use:
636
637.. code-block:: bash
638
639 ./tools/buildman/buildman -b <branch> -se lubbock
640
641If you see error lines marked with '-', that means that the errors were fixed
642by that commit. Sometimes commits can be in the wrong order, so that a
643breakage is introduced for a few commits and fixed by later commits. This
644shows up clearly with buildman. You can then reorder the commits and try
645again.
646
647At commit 16, the error moves: you can see that the old error at line 120
648is fixed, but there is a new one at line 126. This is probably only because
649we added some code and moved the broken line further down the file.
650
651As mentioned, if many boards have the same error, then -e will display the
652error only once. This makes the output as concise as possible. To see which
653boards have each error, use -l. So it is safe to omit the board name - you
654will not get lots of repeated output for every board.
655
656Buildman tries to distinguish warnings from errors, and shows warning lines
657separately with a 'w' prefix. Warnings introduced show as yellow. Warnings
658fixed show as cyan.
659
660The full build output in this case is available in::
661
662 ../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
663
664Files:
665
666done
667 Indicates the build was done, and holds the return code from make. This is 0
668 for a good build, typically 2 for a failure.
669
670err
671 Output from stderr, if any. Errors and warnings appear here.
672
673log
674 Output from stdout. Normally there isn't any since buildman runs in silent
675 mode. Use -V to force a verbose build (this passes V=1 to 'make')
676
677toolchain
678 Shows information about the toolchain used for the build.
679
680sizes
681 Shows image size information.
682
683It is possible to get the build binary output there also. Use the -k option
684for this. In that case you will also see some output files, like:
685
686- System.map
687- toolchain
688- u-boot
689- u-boot.bin
690- u-boot.map
691- autoconf.mk
692- SPL/TPL versions like u-boot-spl and u-boot-spl.bin if available
693
694
695Checking Image Sizes
696--------------------
697
698A key requirement for U-Boot is that you keep code/data size to a minimum.
699Where a new feature increases this noticeably it should normally be put
700behind a CONFIG flag so that boards can leave it disabled and keep the image
701size more or less the same with each new release.
702
703To check the impact of your commits on image size, use -S. For example::
704
705 $ ./tools/buildman/buildman -b us-x86 -sS
706 Summary of 10 commits for 1066 boards (4 threads, 1 job per thread)
707 01: MAKEALL: add support for per architecture toolchains
708 02: x86: Add function to get top of usable ram
709 x86: (for 1/3 boards) text -272.0 rodata +41.0
710 03: x86: Add basic cache operations
711 04: x86: Permit bootstage and timer data to be used prior to relocation
712 x86: (for 1/3 boards) data +16.0
713 05: x86: Add an __end symbol to signal the end of the U-Boot binary
714 x86: (for 1/3 boards) text +76.0
715 06: x86: Rearrange the output input to remove BSS
716 x86: (for 1/3 boards) bss -2140.0
717 07: x86: Support relocation of FDT on start-up
718 x86: + coreboot-x86
719 08: x86: Add error checking to x86 relocation code
720 09: x86: Adjust link device tree include file
721 10: x86: Enable CONFIG_OF_CONTROL on coreboot
722
723
724You can see that image size only changed on x86, which is good because this
725series is not supposed to change any other board. From commit 7 onwards the
726build fails so we don't get code size numbers. The numbers are fractional
727because they are an average of all boards for that architecture. The
728intention is to allow you to quickly find image size problems introduced by
729your commits.
730
731Note that the 'text' region and 'rodata' are split out. You should add the
732two together to get the total read-only size (reported as the first column
733in the output from binutil's 'size' utility).
734
735A useful option is --step which lets you skip some commits. For example
736--step 2 will show the image sizes for only every 2nd commit (so it will
737compare the image sizes of the 1st, 3rd, 5th... commits). You can also use
738--step 0 which will compare only the first and last commits. This is useful
739for an overview of how your entire series affects code size. It will build
740only the upstream commit and your final branch commit.
741
742You can also use -d to see a detailed size breakdown for each board. This
743list is sorted in order from largest growth to largest reduction.
744
745It is even possible to go a little further with the -B option (--bloat). This
746shows where U-Boot has bloated, breaking the size change down to the function
747level. Example output is below::
748
749 $ ./tools/buildman/buildman -b us-mem4 -sSdB
750 ...
751 19: Roll crc32 into hash infrastructure
752 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6
753 paz00 : all +23 bss -4 rodata -29 text +56
754 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64)
755 function old new delta
756 hash_command 80 160 +80
757 crc32_wd_buf - 56 +56
758 ext4fs_read_file 540 568 +28
759 insert_var_value_sub 688 692 +4
760 run_list_real 1996 1992 -4
761 do_mem_crc 168 68 -100
762 trimslice : all -9 bss +16 rodata -29 text +4
763 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
764 function old new delta
765 hash_command 80 160 +80
766 crc32_wd_buf - 56 +56
767 ext4fs_iterate_dir 672 668 -4
768 ext4fs_read_file 568 548 -20
769 do_mem_crc 168 68 -100
770 whistler : all -9 bss +16 rodata -29 text +4
771 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
772 function old new delta
773 hash_command 80 160 +80
774 crc32_wd_buf - 56 +56
775 ext4fs_iterate_dir 672 668 -4
776 ext4fs_read_file 568 548 -20
777 do_mem_crc 168 68 -100
778 seaboard : all -9 bss -28 rodata -29 text +48
779 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56)
780 function old new delta
781 hash_command 80 160 +80
782 crc32_wd_buf - 56 +56
783 ext4fs_read_file 548 568 +20
784 run_list_real 1996 2000 +4
785 do_nandboot 760 756 -4
786 do_mem_crc 168 68 -100
787 colibri_t20 : all -9 rodata -29 text +20
788 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
789 function old new delta
790 hash_command 80 160 +80
791 crc32_wd_buf - 56 +56
792 read_abs_bbt 204 208 +4
793 do_nandboot 760 756 -4
794 ext4fs_read_file 576 568 -8
795 do_mem_crc 168 68 -100
796 ventana : all -37 bss -12 rodata -29 text +4
797 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
798 function old new delta
799 hash_command 80 160 +80
800 crc32_wd_buf - 56 +56
801 ext4fs_iterate_dir 672 668 -4
802 ext4fs_read_file 568 548 -20
803 do_mem_crc 168 68 -100
804 harmony : all -37 bss -16 rodata -29 text +8
805 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16)
806 function old new delta
807 hash_command 80 160 +80
808 crc32_wd_buf - 56 +56
809 nand_write_oob_syndrome 428 432 +4
810 ext4fs_iterate_dir 672 668 -4
811 ext4fs_read_file 568 548 -20
812 do_mem_crc 168 68 -100
813 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336
814 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
815 function old new delta
816 crc32_wd_buf - 56 +56
817 do_fat_read_at 2872 2904 +32
818 hash_algo 16 - -16
819 do_mem_crc 168 68 -100
820 hash_command 420 160 -260
821 tec : all -449 bss -4 data -16 rodata -93 text -336
822 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
823 function old new delta
824 crc32_wd_buf - 56 +56
825 do_fat_read_at 2872 2904 +32
826 hash_algo 16 - -16
827 do_mem_crc 168 68 -100
828 hash_command 420 160 -260
829 plutux : all -481 bss +16 data -16 rodata -93 text -388
830 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340)
831 function old new delta
832 crc32_wd_buf - 56 +56
833 do_load_serial_bin 1688 1700 +12
834 hash_algo 16 - -16
835 do_fat_read_at 2904 2872 -32
836 do_mem_crc 168 68 -100
837 hash_command 420 160 -260
838 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4
839 MPC8610HPCD : all +55 rodata -29 text +84
840 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
841 function old new delta
842 hash_command - 176 +176
843 do_mem_crc 184 88 -96
844 MPC8641HPCN : all +55 rodata -29 text +84
845 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
846 function old new delta
847 hash_command - 176 +176
848 do_mem_crc 184 88 -96
849 MPC8641HPCN_36BIT: all +55 rodata -29 text +84
850 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
851 function old new delta
852 hash_command - 176 +176
853 do_mem_crc 184 88 -96
854 sbc8641d : all +55 rodata -29 text +84
855 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
856 function old new delta
857 hash_command - 176 +176
858 do_mem_crc 184 88 -96
859 xpedite517x : all -33 data -16 rodata -93 text +76
860 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64)
861 function old new delta
862 hash_command - 176 +176
863 hash_algo 16 - -16
864 do_mem_crc 184 88 -96
865 ...
866
867
868This shows that commit 19 has reduced codesize for arm slightly and increased
869it for powerpc. This increase was offset in by reductions in rodata and
870data/bss.
871
872Shown below the summary lines are the sizes for each board. Below each board
873are the sizes for each function. This information starts with:
874
875add
876 number of functions added / removed
877
878grow
879 number of functions which grew / shrunk
880
881bytes
882 number of bytes of code added to / removed from all functions, plus the total
883 byte change in brackets
884
885The change seems to be that hash_command() has increased by more than the
886do_mem_crc() function has decreased. The function sizes typically add up to
887roughly the text area size, but note that every read-only section except
888rodata is included in 'text', so the function total does not exactly
889correspond.
890
891It is common when refactoring code for the rodata to decrease as the text size
892increases, and vice versa.
893
894
895.. _buildman_settings:
896
897The .buildman settings file
898---------------------------
899
900The .buildman file provides information about the available toolchains and
901also allows build flags to be passed to 'make'. It consists of several
902sections, with the section name in square brackets. Within each section are
903a set of (tag, value) pairs.
904
905'[global]' section
906 allow-missing
907 Indicates the policy to use for missing blobs. Note that the flags
908 ``--allow-missing`` (``-M``) and ``--no-allow-missing`` (``--no-a``)
909 override these setting.
910
911 always
912 Run with ``-M`` by default.
913
914 multiple
915 Run with ``-M`` if more than one board is being built.
916
917 branch
918 Run with ``-M`` if a branch is being built.
919
920 Note that the last two can be given together::
921
922 allow-missing = multiple branch
923
924'[toolchain]' section
925 This lists the available toolchains. The tag here doesn't matter, but
926 make sure it is unique. The value is the path to the toolchain. Buildman
927 will look in that path for a file ending in 'gcc'. It will then execute
928 it to check that it is a C compiler, passing only the --version flag to
929 it. If the return code is 0, buildman assumes that it is a valid C
930 compiler. It uses the first part of the name as the architecture and
931 strips off the last part when setting the CROSS_COMPILE environment
932 variable (parts are delimited with a hyphen).
933
934 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
935 and CROSS_COMPILE will be set to powerpc-linux- when using it.
936
937'[toolchain-alias]' section
938 This converts toolchain architecture names to U-Boot names. For example,
939 if an x86 toolchains is called i386-linux-gcc it will not normally be
940 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
941 will tell buildman that the i386 and x86_64 toolchains can be used for
942 the x86 architecture.
943
944'[make-flags]' section
945 U-Boot's build system supports a few flags (such as BUILD_TAG) which
946 affect the build product. These flags can be specified in the buildman
947 settings file. They can also be useful when building U-Boot against other
948 open source software.
949
950 [make-flags]
951 at91-boards=ENABLE_AT91_TEST=1
952 snapper9260=${at91-boards} BUILD_TAG=442
953 snapper9g45=${at91-boards} BUILD_TAG=443
954
955 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
956 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
957 variable ${target} is available to access the target name (snapper9260
958 and snapper9g20 in this case). Variables are resolved recursively. Note
959 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
960 and underscore (_).
961
962 It is expected that any variables added are dealt with in U-Boot's
963 config.mk file and documented in the README.
964
965 Note that you can pass ad-hoc options to the build using environment
966 variables, for example:
967
968 SOME_OPTION=1234 ./tools/buildman/buildman my_board
969
970
971Quick Sanity Check
972------------------
973
974If you have made changes and want to do a quick sanity check of the
975currently checked-out source, run buildman without the -b flag. This will
976build the selected boards and display build status as it runs (i.e. -v is
977enabled automatically). Use -e to see errors/warnings as well.
978
979
980Building Ranges
981---------------
982
983You can build a range of commits by specifying a range instead of a branch
984when using the -b flag. For example::
985
986 buildman -b upstream/master..us-buildman
987
988will build commits in us-buildman that are not in upstream/master.
989
990
991Building Faster
992---------------
993
994By default, buildman doesn't execute 'make mrproper' prior to building the
995first commit for each board. This reduces the amount of work 'make' does, and
996hence speeds up the build. To force use of 'make mrproper', use -the -m flag.
997This flag will slow down any buildman invocation, since it increases the amount
998of work done on any build.
999
1000One possible application of buildman is as part of a continual edit, build,
1001edit, build, ... cycle; repeatedly applying buildman to the same change or
1002series of changes while making small incremental modifications to the source
1003each time. This provides quick feedback regarding the correctness of recent
1004modifications. In this scenario, buildman's default choice of build directory
1005causes more build work to be performed than strictly necessary.
1006
1007By default, each buildman thread uses a single directory for all builds. When a
1008thread builds multiple boards, the configuration built in this directory will
1009cycle through various different configurations, one per board built by the
1010thread. Variations in the configuration will force a rebuild of affected source
1011files when a thread switches between boards. Ideally, such buildman-induced
1012rebuilds would not happen, thus allowing the build to operate as efficiently as
1013the build system and source changes allow. buildman's -P flag may be used to
1014enable this; -P causes each board to be built in a separate (board-specific)
1015directory, thus avoiding any buildman-induced configuration changes in any
1016build directory.
1017
1018U-Boot's build system embeds information such as a build timestamp into the
1019final binary. This information varies each time U-Boot is built. This causes
1020various files to be rebuilt even if no source changes are made, which in turn
1021requires that the final U-Boot binary be re-linked. This unnecessary work can
1022be avoided by turning off the timestamp feature. This can be achieved using
1023the `-r` flag, which enables reproducible builds by setting
1024`SOURCE_DATE_EPOCH=0` when building.
1025
1026Combining all of these options together yields the command-line shown below.
1027This will provide the quickest possible feedback regarding the current content
1028of the source tree, thus allowing rapid tested evolution of the code::
1029
1030 ./tools/buildman/buildman -Pr tegra
1031
1032
1033Checking configuration
1034----------------------
1035
1036A common requirement when converting CONFIG options to Kconfig is to check
1037that the effective configuration has not changed due to the conversion.
1038Buildman supports this with the -K option, used after a build. This shows
1039differences in effective configuration between one commit and the next.
1040
1041For example::
1042
1043 $ buildman -b kc4 -sK
1044 ...
1045 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig
1046 arm:
1047 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1
1048 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1
1049 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1
1050 am335x_evm_usbspl :
1051 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1
1052 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1
1053 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1
1054 44: Convert CONFIG_SPL_USB_HOST to Kconfig
1055 ...
1056
1057This shows that commit 44 enabled three new options for the board
1058am335x_evm_usbspl which were not enabled in commit 43. There is also a
1059summary for 'arm' showing all the changes detected for that architecture.
1060In this case there is only one board with changes, so 'arm' output is the
1061same as 'am335x_evm_usbspl'/
1062
1063The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg
1064files which are produced by a build. If all you want is to check the
1065configuration you can in fact avoid doing a full build, using --config-only.
1066This tells buildman to configuration U-Boot and create the .cfg files, but not
1067actually build the source. This is 5-10 times faster than doing a full build.
1068
1069By default buildman considers the follow two configuration methods
1070equivalent::
1071
1072 #define CONFIG_SOME_OPTION
1073
1074 CONFIG_SOME_OPTION=y
1075
1076The former would appear in a header filer and the latter in a defconfig
1077file. The achieve this, buildman considers 'y' to be '1' in configuration
1078variables. This avoids lots of useless output when converting a CONFIG
1079option to Kconfig. To disable this behaviour, use --squash-config-y.
1080
1081
1082Checking the environment
1083------------------------
1084
1085When converting CONFIG options which manipulate the default environment,
1086a common requirement is to check that the default environment has not
1087changed due to the conversion. Buildman supports this with the -U option,
1088used after a build. This shows differences in the default environment
1089between one commit and the next.
1090
1091For example::
1092
1093 $ buildman -b squash brppt1 -sU
1094 Summary of 2 commits for 3 boards (3 threads, 3 jobs per thread)
1095 01: Migrate bootlimit to Kconfig
1096 02: Squashed commit of the following:
1097 c brppt1_mmc: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1098 c brppt1_spi: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1099 + brppt1_nand: altbootcmd=run usbscript
1100 - brppt1_nand: altbootcmd=run usbscript
1101 (no errors to report)
1102
1103This shows that commit 2 modified the value of 'altbootcmd' for 'brppt1_mmc'
1104and 'brppt1_spi', removing a trailing semicolon. 'brppt1_nand' gained an a
1105value for 'altbootcmd', but lost one for ' altbootcmd'.
1106
1107The -U option uses the u-boot.env files which are produced by a build.
1108Internally, buildman writes out an out-env file into the build directory for
1109later comparison.
1110
1111
1112Building with clang
1113-------------------
1114
1115To build with clang (sandbox only), use the -O option to override the
1116toolchain. For example:
1117
1118.. code-block:: bash
1119
1120 buildman -O clang-7 --board sandbox
1121
1122
1123Building without LTO
1124--------------------
1125
1126Link-time optimisation (LTO) is designed to reduce code size by globally
1127optimising the U-Boot build. Unfortunately this can dramatically slow down
1128builds. This is particularly noticeable when running a lot of builds.
1129
1130Use the -L (--no-lto) flag to disable LTO.
1131
1132.. code-block:: bash
1133
1134 buildman -L --board sandbox
1135
1136
1137Doing a simple build
1138--------------------
1139
1140In some cases you just want to build a single board and get the full output, use
1141the -w option, for example:
1142
1143.. code-block:: bash
1144
1145 buildman -o /tmp/build --board sandbox -w
1146
1147This will write the full build into /tmp/build including object files. You must
1148specify the output directory with -o when using -w.
1149
1150
1151Support for IDEs (Integrated Development Environments)
1152------------------------------------------------------
1153
1154Normally buildman summarises the output and shows information indicating the
1155meaning of each line of output. For example a '+' symbol appears at the start of
1156each error line. Also, buildman prints information about what it is about to do,
1157along with a summary at the end.
1158
1159When using buildman from an IDE, it is helpful to drop this behaviour. Use the
1160-I/--ide option for that. You might find -W helpful also so that warnings do
1161not cause the build to fail:
1162
1163.. code-block:: bash
1164
1165 buildman -o /tmp/build --board sandbox -wWI
1166
1167
1168Support for binary blobs
1169------------------------
1170
1171U-Boot is moving to using Binman (see :doc:`../develop/package/binman`) for
1172dealing with the complexities of packaging U-Boot along with binary files from
1173other projects. These are called 'external blobs' by Binman.
1174
1175Typically a missing external blob causes a build failure. For build testing of
1176a lot of boards, or boards for which you do not have the blobs, you can use the
1177-M flag to allow missing blobs. This marks the build as if it succeeded,
1178although with warnings shown, including 'Some images are invalid'. If any boards
1179fail in this way, buildman exits with status 101.
1180
1181To convert warnings to errors, use -E. To make buildman return success with
1182these warnings, use -W.
1183
1184It is generally safe to default to enabling -M for all runs of buildman, so long
1185as you check the exit code. To do this, add::
1186
1187 allow-missing = "always"
1188
1189to the top of the buildman_settings_ file.
1190
1191
1192Changing the configuration
1193--------------------------
1194
1195Sometimes it is useful to change the CONFIG options for a build on the fly. This
1196can be used to build a board (or multiple) with a few changes to see the impact.
1197The -a option supports this:
1198
1199.. code-block:: bash
1200
1201 -a <cfg>
1202
1203where <cfg> is a CONFIG option (with or without the `CONFIG_` prefix) to enable.
1204For example:
1205
1206.. code-block:: bash
1207
1208 buildman -a CMD_SETEXPR_FMT
1209
1210will build with CONFIG_CMD_SETEXPR_FMT enabled.
1211
1212You can disable options by preceding them with tilde (~). You can specify the
1213-a option multiple times:
1214
1215.. code-block:: bash
1216
1217 buildman -a CMD_SETEXPR_FMT -a ~CMDLINE
1218
1219Some options have values, in which case you can change them:
1220
1221.. code-block:: bash
1222
1223 buildman -a 'BOOTCOMMAND="echo hello"' CONFIG_SYS_LOAD_ADDR=0x1000
1224
1225Note that you must put quotes around string options and the whole thing must be
1226in single quotes, to make sure the shell leave it alone.
1227
1228If you try to set an option that does not exist, or that cannot be changed for
1229some other reason (e.g. it is 'selected' by another option), then buildman
1230shows an error::
1231
1232 $ buildman --board sandbox -a FRED
1233 Building current source for 1 boards (1 thread, 32 jobs per thread)
1234 0 0 0 /1 -1 (starting)errs
1235 Some CONFIG adjustments did not take effect. This may be because
1236 the request CONFIGs do not exist or conflict with others.
1237
1238 Failed adjustments:
1239
1240 FRED Missing expected line: CONFIG_FRED=y
1241
1242
1243One major caveat with this feature with branches (-b) is that buildman does not
1244name the output directories differently when you change the configuration, so
1245doing the same build again with different configuration will not trigger a
1246rebuild. You can use -f to work around that.
1247
1248
1249Other options
1250-------------
1251
1252Buildman has various other command-line options. Try --help to see them.
1253
1254To find out what toolchain prefix buildman will use for a build, use the -A
1255option.
1256
1257To request that compiler warnings be promoted to errors, use -E. This passes the
1258-Werror flag to the compiler. Note that the build can still produce warnings
1259with -E, e.g. the migration warnings::
1260
1261 ===================== WARNING ======================
1262 This board does not use CONFIG_DM_MMC. Please update
1263 ...
1264 ====================================================
1265
1266When doing builds, Buildman's return code will reflect the overall result::
1267
1268 0 (success) No errors or warnings found
1269 100 Errors found
1270 101 Warnings found (only if no -W)
1271
1272You can use -W to tell Buildman to return 0 (success) instead of 101 when
1273warnings are found. Note that it can be useful to combine -E and -W. This means
1274that all compiler warnings will produce failures (code 100) and all other
1275warnings will produce success (since 101 is changed to 0).
1276
1277If there are both warnings and errors, errors win, so buildman returns 100.
1278
1279The -y option is provided (for use with -s) to ignore the bountiful device-tree
1280warnings. Similarly, -Y tells buildman to ignore the migration warnings.
1281
1282Sometimes you might get an error in a thread that is not handled by buildman,
1283perhaps due to a failure of a tool that it calls. You might see the output, but
1284then buildman hangs. Failing to handle any eventuality is a bug in buildman and
1285should be reported. But you can use -T0 to disable threading and hopefully
1286figure out the root cause of the build failure.
1287
1288Build summary
1289-------------
1290
1291When buildman finishes it shows a summary, something like this::
1292
1293 Completed: 5 total built, duration 0:00:21, rate 0.24
1294
1295This shows that a total of 5 builds were done across all selected boards, it
1296took 21 seconds and the builds happened at the rate of 0.24 per second. The
1297latter number depends on the speed of your machine and the efficiency of the
1298U-Boot build.
1299
1300
1301Using boards.cfg
1302----------------
1303
1304This file is no-longer needed by buildman but it is still generated in the
1305working directory. This helps avoid a delay on every build, since scanning all
1306the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force
1307regeneration of the file - in that case buildman exits after writing the file
1308with exit code 2 if there was an error in the maintainer files. To use the
1309default filename, use a hyphen, i.e. `-R -`.
1310
1311You should use 'buildman -nv <criteria>' instead of greoing the boards.cfg file,
1312since it may be dropped altogether in future.
1313
1314
1315Checking maintainers
1316--------------------
1317
1318Sometimes a board is added without a corresponding entry in a MAINTAINERS file.
1319Use the `--maintainer-check` option to check this::
1320
1321 $ buildman --maintainer-check
1322 WARNING: board/mikrotik/crs3xx-98dx3236/MAINTAINERS: missing defconfig ending at line 7
1323 WARNING: no maintainers for 'clearfog_spi'
1324
1325Buildman returns with an exit code of 2 if there area any warnings.
1326
1327An experimental `--full-check option` also checks for boards which don't have a
1328CONFIG_TARGET_xxx where xxx corresponds to their defconfig filename. This is
1329not strictly necessary, but may be useful information.
1330
1331
1332Checking the command
1333--------------------
1334
1335Buildman writes out the toolchain information to a `toolchain` file within the
1336output directory. It also writes the commands used to build U-Boot in an
1337`out-cmd` file. You can check these if you suspect something strange is
1338happening.
1339
1340TODO
1341----
1342
1343Many improvements have been made over the years. There is still quite a bit of
1344scope for more though, e.g.:
1345
1346- easier access to log files
1347- 'hunting' for problems, perhaps by building a few boards for each arch, or
1348 checking commits for changed files and building only boards which use those
1349 files
1350
1351
1352Credits
1353-------
1354
1355Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving
1356the build speed by building all commits for a board instead of the other
1357way around.
1358
1359.. sectionauthor:: Simon Glass
1360.. sectionauthor:: Copyright (c) 2013 The Chromium OS Authors.
1361.. sectionauthor:: sjg@chromium.org
1362.. Halloween 2012
1363.. Updated 12-12-12
1364.. Updated 23-02-13
1365.. Updated 09-04-20
1366