1.. SPDX-License-Identifier: GPL-2.0+ 2.. Copyright (c) 2016 Google, Inc 3 4Introduction 5============ 6 7Firmware often consists of several components which must be packaged together. 8For example, we may have SPL, U-Boot, a device tree and an environment area 9grouped together and placed in MMC flash. When the system starts, it must be 10able to find these pieces. 11 12Building firmware should be separate from packaging it. Many of the complexities 13of modern firmware build systems come from trying to do both at once. With 14binman, you build all the pieces that are needed, using whatever assortment of 15projects and build systems are needed, then use binman to stitch everything 16together. 17 18 19What it does 20------------ 21 22Binman reads your board's device tree and finds a node which describes the 23required image layout. It uses this to work out what to place where. 24 25Binman provides a mechanism for building images, from simple SPL + U-Boot 26combinations, to more complex arrangements with many parts. It also allows 27users to inspect images, extract and replace binaries within them, repacking if 28needed. 29 30 31Features 32-------- 33 34Apart from basic padding, alignment and positioning features, Binman supports 35hierarchical images, compression, hashing and dealing with the binary blobs 36which are a sad trend in open-source firmware at present. 37 38Executable binaries can access the location of other binaries in an image by 39using special linker symbols (zero-overhead but somewhat limited) or by reading 40the devicetree description of the image. 41 42Binman is designed primarily for use with U-Boot and associated binaries such 43as ARM Trusted Firmware, but it is suitable for use with other projects, such 44as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS, 45vblocks and the like. 46 47Binman provides a way to process binaries before they are included, by adding a 48Python plug-in. 49 50Binman is intended for use with U-Boot but is designed to be general enough 51to be useful in other image-packaging situations. 52 53 54Motivation 55---------- 56 57As mentioned above, packaging of firmware is quite a different task from 58building the various parts. In many cases the various binaries which go into 59the image come from separate build systems. For example, ARM Trusted Firmware 60is used on ARMv8 devices but is not built in the U-Boot tree. If a Linux kernel 61is included in the firmware image, it is built elsewhere. 62 63It is of course possible to add more and more build rules to the U-Boot 64build system to cover these cases. It can shell out to other Makefiles and 65build scripts. But it seems better to create a clear divide between building 66software and packaging it. 67 68At present this is handled by manual instructions, different for each board, 69on how to create images that will boot. By turning these instructions into a 70standard format, we can support making valid images for any board without 71manual effort, lots of READMEs, etc. 72 73Benefits: 74 75 - Each binary can have its own build system and tool chain without creating 76 any dependencies between them 77 - Avoids the need for a single-shot build: individual parts can be updated 78 and brought in as needed 79 - Provides for a standard image description available in the build and at 80 run-time 81 - SoC-specific image-signing tools can be accommodated 82 - Avoids cluttering the U-Boot build system with image-building code 83 - The image description is automatically available at run-time in U-Boot, 84 SPL. It can be made available to other software also 85 - The image description is easily readable (it's a text file in device-tree 86 format) and permits flexible packing of binaries 87 88 89Terminology 90----------- 91 92Binman uses the following terms: 93 94- image - an output file containing a firmware image 95- binary - an input binary that goes into the image 96 97 98Installation 99------------ 100 101You can install binman using:: 102 103 pip install binary-manager 104 105The name is chosen since binman conflicts with an existing package. 106 107If you are using binman within the U-Boot tree, it may be easiest to add a 108symlink from your local `~/.bin` directory to `/path/to/tools/binman/binman`. 109 110 111Relationship to FIT 112------------------- 113 114FIT is U-Boot's official image format. It supports multiple binaries with 115load / execution addresses, compression. It also supports verification 116through hashing and RSA signatures. 117 118FIT was originally designed to support booting a Linux kernel (with an 119optional ramdisk) and device tree chosen from various options in the FIT. 120Now that U-Boot supports configuration via device tree, it is possible to 121load U-Boot from a FIT, with the device tree chosen by SPL. 122 123Binman considers FIT to be one of the binaries it can place in the image. 124 125Where possible it is best to put as much as possible in the FIT, with binman 126used to deal with cases not covered by FIT. Examples include initial 127execution (since FIT itself does not have an executable header) and dealing 128with device boundaries, such as the read-only/read-write separation in SPI 129flash. 130 131For U-Boot, binman should not be used to create ad-hoc images in place of 132FIT. 133 134Note that binman can itself create a FIT. This helps to move mkimage 135invocations out of the Makefile and into binman image descriptions. It also 136helps by removing the need for ad-hoc tools like `make_fit_atf.py`. 137 138 139Relationship to mkimage 140----------------------- 141 142The mkimage tool provides a means to create a FIT. Traditionally it has 143needed an image description file: a device tree, like binman, but in a 144different format. More recently it has started to support a '-f auto' mode 145which can generate that automatically. 146 147More relevant to binman, mkimage also permits creation of many SoC-specific 148image types. These can be listed by running 'mkimage -T list'. Examples 149include 'rksd', the Rockchip SD/MMC boot format. The mkimage tool is often 150called from the U-Boot build system for this reason. 151 152Binman considers the output files created by mkimage to be binary blobs 153which it can place in an image. Binman does not replace the mkimage tool or 154this purpose. It would be possible in some situations to create a new entry 155type for the images in mkimage, but this would not add functionality. It 156seems better to use the mkimage tool to generate binaries and avoid blurring 157the boundaries between building input files (mkimage) and packaging then 158into a final image (binman). 159 160Note that binman can itself invoke mkimage. This helps to move mkimage 161invocations out of the Makefile and into binman image descriptions. 162 163 164Using binman 165============ 166 167Example use of binman in U-Boot 168------------------------------- 169 170Binman aims to replace some of the ad-hoc image creation in the U-Boot 171build system. 172 173Consider sunxi. It has the following steps: 174 175 #. It uses a custom mksunxiboot tool to build an SPL image called 176 sunxi-spl.bin. This should probably move into mkimage. 177 178 #. It uses mkimage to package U-Boot into a legacy image file (so that it can 179 hold the load and execution address) called u-boot.img. 180 181 #. It builds a final output image called u-boot-sunxi-with-spl.bin which 182 consists of sunxi-spl.bin, some padding and u-boot.img. 183 184Binman is intended to replace the last step. The U-Boot build system builds 185u-boot.bin and sunxi-spl.bin. Binman can then take over creation of 186sunxi-spl.bin by calling mksunxiboot or mkimage. In any case, it would then 187create the image from the component parts. 188 189This simplifies the U-Boot Makefile somewhat, since various pieces of logic 190can be replaced by a call to binman. 191 192 193Invoking binman within U-Boot 194----------------------------- 195 196Within U-Boot, binman is invoked by the build system, i.e. when you type 'make' 197or use buildman to build U-Boot. There is no need to run binman independently 198during development. Everything happens automatically and is set up for your 199SoC or board so that binman produced the right things. 200 201The general policy is that the Makefile builds all the binaries in INPUTS-y 202(the 'inputs' rule), then binman is run to produce the final images (the 'all' 203rule). 204 205There should be only one invocation of binman in Makefile, the very last step 206that pulls everything together. At present there are some arch-specific 207invocations as well, but these should be dropped when those architectures are 208converted to use binman properly. 209 210As above, the term 'binary' is used for something in INPUTS-y and 'image' is 211used for the things that binman creates. So the binaries are inputs to the 212image(s) and it is the image that is actually loaded on the board. 213 214Again, at present, there are a number of things created in Makefile which should 215be done by binman (when we get around to it), like `u-boot-ivt.img`, 216`lpc32xx-spl.img`, `u-boot-with-nand-spl.imx`, `u-boot-spl-padx4.sfp` and 217`u-boot-mtk.bin`, just to pick on a few. When completed this will remove about 218400 lines from `Makefile`. 219 220Since binman is invoked only once, it must of course create all the images that 221are needed, in that one invocation. It does this by working through the image 222descriptions one by one, collecting the input binaries, processing them as 223needed and producing the final images. 224 225The same binaries may be used by multiple images. For example binman may be used 226to produce an SD-card image and a SPI-flash image. In this case the binaries 227going into the process are the same, but binman produces slightly different 228images in each case. 229 230For some SoCs, U-Boot is not the only project that produces the necessary 231binaries. For example, ARM Trusted Firmware (ATF) is a project that produces 232binaries which must be incorporate, such as `bl31.elf` or `bl31.bin`. For this 233to work you must have built ATF before you build U-Boot and you must tell U-Boot 234where to find the bl31 image, using the BL31 environment variable. 235 236How do you know how to incorporate ATF? It is handled by the atf-bl31 entry type 237(etype). An etype is an implementation of reading a binary into binman, in this 238case the `bl31.bin` file. When you build U-Boot but do not set the BL31 239environment variable, binman provides a help message, which comes from 240`missing-blob-help`:: 241 242 See the documentation for your board. You may need to build ARM Trusted 243 Firmware and build with BL31=/path/to/bl31.bin 244 245The mechanism by which binman is advised of this is also in the Makefile. See 246the `-a atf-bl31-path=${BL31}` piece in `cmd_binman`. This tells binman to 247set the EntryArg `atf-bl31-path` to the value of the `BL31` environment 248variable. Within binman, this EntryArg is picked up by the `Entry_atf_bl31` 249etype. An EntryArg is simply an argument to the entry. The `atf-bl31-path` 250name is documented in :ref:`etype_atf_bl31`. 251 252Taking this a little further, when binman is used to create a FIT, it supports 253using an ELF file, e.g. `bl31.elf` and splitting it into separate pieces (with 254`fit,operation = "split-elf"`), each with its own load address. 255 256 257Invoking binman outside U-Boot 258------------------------------ 259 260While binman is invoked from within the U-Boot build system, it is also possible 261to invoke it separately. This is typically used in a production build system, 262where signing is completed (with real keys) and any missing binaries are 263provided. 264 265For example, for build testing there is no need to provide a real signature, 266nor is there any need to provide a real ATF BL31 binary (for example). These can 267be added later by invoking binman again, providing all the required inputs 268from the first time, plus any that were missing or placeholders. 269 270So in practice binman is often used twice: 271 272- once within the U-Boot build system, for development and testing 273- again outside U-Boot to assembly and final production images 274 275While the same input binaries are used in each case, you will of course you will 276need to create your own binman command line, similar to that in `cmd_binman` in 277the Makefile. You may find the -I and --toolpath options useful. The 278device tree file is provided to binman in binary form, so there is no need to 279have access to the original `.dts` sources. 280 281 282Assembling the image description 283-------------------------------- 284 285Since binman uses the device tree for its image description, you can use the 286same files that describe your board's hardware to describe how the image is 287assembled. Typically the images description is in a common file used by all 288boards with a particular SoC (e.g. `imx8mp-u-boot.dtsi`). 289 290Where a particular boards needs to make changes, it can override properties in 291the SoC file, just as it would for any other device tree property. It can also 292add a image that is specific to the board. 293 294Another way to control the image description to make use of CONFIG options in 295the description. For example, if the start offset of a particular entry varies 296by board, you can add a Kconfig for that and reference it in the description:: 297 298 u-boot-spl { 299 }; 300 301 fit { 302 offset = <CONFIG_SPL_PAD_TO>; 303 ... 304 }; 305 306The SoC can provide a default value but boards can override that as needed and 307binman will take care of it. 308 309It is even possible to control which entries appear in the image, by using the 310C preprocessor:: 311 312 #ifdef CONFIG_HAVE_MRC 313 intel-mrc { 314 offset = <CFG_X86_MRC_ADDR>; 315 }; 316 #endif 317 318Only boards which enable `HAVE_MRC` will include this entry. 319 320Obviously a similar approach can be used to control which images are produced, 321with a Kconfig option to enable a SPI image, for example. However there is 322generally no harm in producing an image that is not used. If a board uses MMC 323but not SPI, but the SoC supports booting from both, then both images can be 324produced, with only on or other being used by particular boards. This can help 325reduce the need for having multiple defconfig targets for a board where the 326only difference is the boot media, enabling / disabling secure boot, etc. 327 328Of course you can use the device tree itself to pass any board-specific 329information that is needed by U-Boot at runtime (see binman_syms_ for how to 330make binman insert these values directly into executables like SPL). 331 332There is one more way this can be done: with individual .dtsi files for each 333image supported by the SoC. Then the board `.dts` file can include the ones it 334wants. This is not recommended, since it is likely to be difficult to maintain 335and harder to understand the relationship between the different boards. 336 337 338Producing images for multiple boards 339------------------------------------ 340 341When invoked within U-Boot, binman only builds a single set of images, for 342the chosen board. This is set by the `CONFIG_DEFAULT_DEVICE_TREE` option. 343 344However, U-Boot generally builds all the device tree files associated with an 345SoC. These are written to the (e.g. for ARM) `arch/arm/dts` directory. Each of 346these contains the full binman description for that board. Often the best 347approach is to build a single image that includes all these device tree binaries 348and allow SPL to select the correct one on boot. 349 350However, it is also possible to build separate images for each board, simply by 351invoking binman multiple times, once for each device tree file, using a 352different output directory. This will produce one set of images for each board. 353 354 355Example use of binman for x86 356----------------------------- 357 358In most cases x86 images have a lot of binary blobs, 'black-box' code 359provided by Intel which must be run for the platform to work. Typically 360these blobs are not relocatable and must be placed at fixed areas in the 361firmware image. 362 363Currently this is handled by ifdtool, which places microcode, FSP, MRC, VGA 364BIOS, reference code and Intel ME binaries into a u-boot.rom file. 365 366Binman is intended to replace all of this, with ifdtool left to handle only 367the configuration of the Intel-format descriptor. 368 369 370Installing binman 371----------------- 372 373First install prerequisites, e.g: 374 375.. code-block:: bash 376 377 sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \ 378 liblz4-tool 379 380You can run binman directly if you put it on your PATH. But if you want to 381install into your `~/.local` Python directory, use: 382 383.. code-block:: bash 384 385 pip install tools/patman tools/dtoc tools/binman 386 387Note that binman makes use of libraries from patman and dtoc, which is why these 388need to be installed. Also you need `libfdt` and `pylibfdt` which can be 389installed like this: 390 391.. code-block:: bash 392 393 git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git 394 cd dtc 395 pip install . 396 make NO_PYTHON=1 install 397 398This installs the `libfdt.so` library into `~/lib` so you can use 399`LD_LIBRARY_PATH=~/lib` when running binman. If you want to install it in the 400system-library directory, replace the last line with: 401 402.. code-block:: bash 403 404 make NO_PYTHON=1 PREFIX=/ install 405 406Running binman 407-------------- 408 409Type: 410 411.. code-block:: bash 412 413 make NO_PYTHON=1 PREFIX=/ install 414 binman build -b <board_name> 415 416to build an image for a board. The board name is the same name used when 417configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox'). 418Binman assumes that the input files for the build are in ../b/<board_name>. 419 420Or you can specify this explicitly: 421 422.. code-block:: bash 423 424 make NO_PYTHON=1 PREFIX=/ install 425 binman build -I <build_path> 426 427where <build_path> is the build directory containing the output of the U-Boot 428build. 429 430(Future work will make this more configurable) 431 432In either case, binman picks up the device tree file (u-boot.dtb) and looks 433for its instructions in the 'binman' node. 434 435Binman has a few other options which you can see by running 'binman -h'. 436 437 438Enabling binman for a board 439--------------------------- 440 441At present binman is invoked from a rule in the main Makefile. You should be 442able to enable CONFIG_BINMAN to enable this rule. 443 444The output file is typically named image.bin and is located in the output 445directory. If input files are needed to you add these to INPUTS-y either in the 446main Makefile or in a config.mk file in your arch subdirectory. 447 448Once binman is executed it will pick up its instructions from a device-tree 449file, typically <soc>-u-boot.dtsi, where <soc> is your CONFIG_SYS_SOC value. 450You can use other, more specific CONFIG options - see 'Automatic .dtsi 451inclusion' below. 452 453.. _binman_syms: 454 455Access to binman entry offsets at run time (symbols) 456---------------------------------------------------- 457 458Binman assembles images and determines where each entry is placed in the image. 459This information may be useful to U-Boot at run time. For example, in SPL it 460is useful to be able to find the location of U-Boot so that it can be executed 461when SPL is finished. 462 463Binman allows you to declare symbols in the SPL image which are filled in 464with their correct values during the build. For example: 465 466.. code-block:: c 467 468 binman_sym_declare(ulong, u_boot_any, image_pos); 469 470declares a ulong value which will be assigned to the image-pos of any U-Boot 471image (u-boot.bin, u-boot.img, u-boot-nodtb.bin) that is present in the image. 472You can access this value with something like: 473 474.. code-block:: c 475 476 ulong u_boot_offset = binman_sym(ulong, u_boot_any, image_pos); 477 478Thus u_boot_offset will be set to the image-pos of U-Boot in memory, assuming 479that the whole image has been loaded, or is available in flash. You can then 480jump to that address to start U-Boot. 481 482At present this feature is only supported in SPL and TPL. In principle it is 483possible to fill in such symbols in U-Boot proper, as well, but a future C 484library is planned for this instead, to read from the device tree. 485 486As well as image-pos, it is possible to read the size of an entry and its 487offset (which is the start position of the entry within its parent). 488 489A small technical note: Binman automatically adds the base address of the image 490(i.e. __image_copy_start) to the value of the image-pos symbol, so that when the 491image is loaded to its linked address, the value will be correct and actually 492point into the image. 493 494For example, say SPL is at the start of the image and linked to start at address 49580108000. If U-Boot's image-pos is 0x8000 then binman will write an image-pos 496for U-Boot of 80110000 into the SPL binary, since it assumes the image is loaded 497to 80108000, with SPL at 80108000 and U-Boot at 80110000. 498 499For x86 devices (with the end-at-4gb property) this base address is not added 500since it is assumed that images are XIP and the offsets already include the 501address. 502 503While U-Boot's symbol updating is handled automatically by the u-boot-spl 504entry type (and others), it is possible to use this feature with any blob. To 505do this, add a `write-symbols` (boolean) property to the node, set the ELF 506filename using `elf-filename` and set 'elf-base-sym' to the base symbol for the 507start of the binary image (this defaults to `__image_copy_start` which is what 508U-Boot uses). See `testBlobSymbol()` for an example. 509 510.. _binman_fdt: 511 512Access to binman entry offsets at run time (fdt) 513------------------------------------------------ 514 515Binman can update the U-Boot FDT to include the final position and size of 516each entry in the images it processes. The option to enable this is -u and it 517causes binman to make sure that the 'offset', 'image-pos' and 'size' properties 518are set correctly for every entry. Since it is not necessary to specify these in 519the image definition, binman calculates the final values and writes these to 520the device tree. These can be used by U-Boot at run-time to find the location 521of each entry. 522 523Alternatively, an FDT map entry can be used to add a special FDT containing 524just the information about the image. This is preceded by a magic string so can 525be located anywhere in the image. An image header (typically at the start or end 526of the image) can be used to point to the FDT map. See fdtmap and image-header 527entries for more information. 528 529Map files 530--------- 531 532The -m option causes binman to output a .map file for each image that it 533generates. This shows the offset and size of each entry. For example:: 534 535 Offset Size Name 536 00000000 00000028 main-section 537 00000000 00000010 section@0 538 00000000 00000004 u-boot 539 00000010 00000010 section@1 540 00000000 00000004 u-boot 541 542This shows a hierarchical image with two sections, each with a single entry. The 543offsets of the sections are absolute hex byte offsets within the image. The 544offsets of the entries are relative to their respective sections. The size of 545each entry is also shown, in bytes (hex). The indentation shows the entries 546nested inside their sections. 547 548 549Passing command-line arguments to entries 550----------------------------------------- 551 552Sometimes it is useful to pass binman the value of an entry property from the 553command line. For example some entries need access to files and it is not 554always convenient to put these filenames in the image definition (device tree). 555 556The -a option supports this:: 557 558 -a <prop>=<value> 559 560where:: 561 562 <prop> is the property to set 563 <value> is the value to set it to 564 565Not all properties can be provided this way. Only some entries support it, 566typically for filenames. 567 568 569Image description format 570======================== 571 572The binman node is called 'binman'. An example image description is shown 573below:: 574 575 binman { 576 filename = "u-boot-sunxi-with-spl.bin"; 577 pad-byte = <0xff>; 578 blob { 579 filename = "spl/sunxi-spl.bin"; 580 }; 581 u-boot { 582 offset = <CONFIG_SPL_PAD_TO>; 583 }; 584 }; 585 586 587This requests binman to create an image file called u-boot-sunxi-with-spl.bin 588consisting of a specially formatted SPL (spl/sunxi-spl.bin, built by the 589normal U-Boot Makefile), some 0xff padding, and a U-Boot legacy image. The 590padding comes from the fact that the second binary is placed at 591CONFIG_SPL_PAD_TO. If that line were omitted then the U-Boot binary would 592immediately follow the SPL binary. 593 594The binman node describes an image. The sub-nodes describe entries in the 595image. Each entry represents a region within the overall image. The name of 596the entry (blob, u-boot) tells binman what to put there. For 'blob' we must 597provide a filename. For 'u-boot', binman knows that this means 'u-boot.bin'. 598 599Entries are normally placed into the image sequentially, one after the other. 600The image size is the total size of all entries. As you can see, you can 601specify the start offset of an entry using the 'offset' property. 602 603Note that due to a device tree requirement, all entries must have a unique 604name. If you want to put the same binary in the image multiple times, you can 605use any unique name, with the 'type' property providing the type. 606 607The attributes supported for entries are described below. 608 609offset: 610 This sets the offset of an entry within the image or section containing 611 it. The first byte of the image is normally at offset 0. If 'offset' is 612 not provided, binman sets it to the end of the previous region, or the 613 start of the image's entry area (normally 0) if there is no previous 614 region. 615 616align: 617 This sets the alignment of the entry. The entry offset is adjusted 618 so that the entry starts on an aligned boundary within the containing 619 section or image. For example 'align = <16>' means that the entry will 620 start on a 16-byte boundary. This may mean that padding is added before 621 the entry. The padding is part of the containing section but is not 622 included in the entry, meaning that an empty space may be created before 623 the entry starts. Alignment should be a power of 2. If 'align' is not 624 provided, no alignment is performed. 625 626size: 627 This sets the size of the entry. The contents will be padded out to 628 this size. If this is not provided, it will be set to the size of the 629 contents. 630 631min-size: 632 Sets the minimum size of the entry. This size includes explicit padding 633 ('pad-before' and 'pad-after'), but not padding added to meet alignment 634 requirements. While this does not affect the contents of the entry within 635 binman itself (the padding is performed only when its parent section is 636 assembled), the end result will be that the entry ends with the padding 637 bytes, so may grow. Defaults to 0. 638 639pad-before: 640 Padding before the contents of the entry. Normally this is 0, meaning 641 that the contents start at the beginning of the entry. This can be used 642 to offset the entry contents a little. While this does not affect the 643 contents of the entry within binman itself (the padding is performed 644 only when its parent section is assembled), the end result will be that 645 the entry starts with the padding bytes, so may grow. Defaults to 0. 646 647pad-after: 648 Padding after the contents of the entry. Normally this is 0, meaning 649 that the entry ends at the last byte of content (unless adjusted by 650 other properties). This allows room to be created in the image for 651 this entry to expand later. While this does not affect the contents of 652 the entry within binman itself (the padding is performed only when its 653 parent section is assembled), the end result will be that the entry ends 654 with the padding bytes, so may grow. Defaults to 0. 655 656align-size: 657 This sets the alignment of the entry size. For example, to ensure 658 that the size of an entry is a multiple of 64 bytes, set this to 64. 659 While this does not affect the contents of the entry within binman 660 itself (the padding is performed only when its parent section is 661 assembled), the end result is that the entry ends with the padding 662 bytes, so may grow. If 'align-size' is not provided, no alignment is 663 performed. 664 665align-end: 666 This sets the alignment of the end of an entry with respect to the 667 containing section. Some entries require that they end on an alignment 668 boundary, regardless of where they start. This does not move the start 669 of the entry, so the contents of the entry will still start at the 670 beginning. But there may be padding at the end. While this does not 671 affect the contents of the entry within binman itself (the padding is 672 performed only when its parent section is assembled), the end result 673 is that the entry ends with the padding bytes, so may grow. 674 If 'align-end' is not provided, no alignment is performed. 675 676filename: 677 For 'blob' types this provides the filename containing the binary to 678 put into the entry. If binman knows about the entry type (like 679 u-boot-bin), then there is no need to specify this. 680 681type: 682 Sets the type of an entry. This defaults to the entry name, but it is 683 possible to use any name, and then add (for example) 'type = "u-boot"' 684 to specify the type. 685 686offset-unset: 687 Indicates that the offset of this entry should not be set by placing 688 it immediately after the entry before. Instead, is set by another 689 entry which knows where this entry should go. When this boolean 690 property is present, binman will give an error if another entry does 691 not set the offset (with the GetOffsets() method). 692 693image-pos: 694 This cannot be set on entry (or at least it is ignored if it is), but 695 with the -u option, binman will set it to the absolute image position 696 for each entry. This makes it easy to find out exactly where the entry 697 ended up in the image, regardless of parent sections, etc. 698 699extend-size: 700 Extend the size of this entry to fit available space. This space is only 701 limited by the size of the image/section and the position of the next 702 entry. 703 704compress: 705 Sets the compression algortihm to use (for blobs only). See the entry 706 documentation for details. 707 708missing-msg: 709 Sets the tag of the message to show if this entry is missing. This is 710 used for external blobs. When they are missing it is helpful to show 711 information about what needs to be fixed. See missing-blob-help for the 712 message for each tag. 713 714assume-size: 715 Sets the assumed size of a blob entry if it is missing. This allows for a 716 check that the rest of the image fits into the available space, even when 717 the contents are not available. If the entry is missing, Binman will use 718 this assumed size for the entry size, including creating a fake file of that 719 size if requested. 720 721no-expanded: 722 By default binman substitutes entries with expanded versions if available, 723 so that a `u-boot` entry type turns into `u-boot-expanded`, for example. The 724 `--no-expanded` command-line option disables this globally. The 725 `no-expanded` property disables this just for a single entry. Put the 726 `no-expanded` boolean property in the node to select this behaviour. 727 728optional: 729 External blobs are normally required to be present for the image to be 730 built (but see `External blobs`_). This properly allows an entry to be 731 optional, so that when it is cannot be found, this problem is ignored and 732 an empty file is used for this blob. This should be used only when the blob 733 is entirely optional and is not needed for correct operation of the image. 734 Note that missing, optional blobs do not produce a non-zero exit code from 735 binman, although it does show a warning about the missing external blob. 736 737insert-template: 738 This is not strictly speaking an entry property, since it is processed early 739 in Binman before the entries are read. It is a list of phandles of nodes to 740 include in the current (target) node. For each node, its subnodes and their 741 properties are brought into the target node. See Templates_ below for 742 more information. 743 744The attributes supported for images and sections are described below. Several 745are similar to those for entries. 746 747size: 748 Sets the image size in bytes, for example 'size = <0x100000>' for a 749 1MB image. 750 751offset: 752 This is similar to 'offset' in entries, setting the offset of a section 753 within the image or section containing it. The first byte of the section 754 is normally at offset 0. If 'offset' is not provided, binman sets it to 755 the end of the previous region, or the start of the image's entry area 756 (normally 0) if there is no previous region. 757 758align-size: 759 This sets the alignment of the image size. For example, to ensure 760 that the image ends on a 512-byte boundary, use 'align-size = <512>'. 761 If 'align-size' is not provided, no alignment is performed. 762 763pad-before: 764 This sets the padding before the image entries. The first entry will 765 be positioned after the padding. This defaults to 0. 766 767pad-after: 768 This sets the padding after the image entries. The padding will be 769 placed after the last entry. This defaults to 0. 770 771pad-byte: 772 This specifies the pad byte to use when padding in the image. It 773 defaults to 0. To use 0xff, you would add 'pad-byte = <0xff>'. 774 775filename: 776 This specifies the image filename. It defaults to 'image.bin'. 777 778sort-by-offset: 779 This causes binman to reorder the entries as needed to make sure they 780 are in increasing positional order. This can be used when your entry 781 order may not match the positional order. A common situation is where 782 the 'offset' properties are set by CONFIG options, so their ordering is 783 not known a priori. 784 785 This is a boolean property so needs no value. To enable it, add a 786 line 'sort-by-offset;' to your description. 787 788multiple-images: 789 Normally only a single image is generated. To create more than one 790 image, put this property in the binman node. For example, this will 791 create image1.bin containing u-boot.bin, and image2.bin containing 792 both spl/u-boot-spl.bin and u-boot.bin:: 793 794 binman { 795 multiple-images; 796 image1 { 797 u-boot { 798 }; 799 }; 800 801 image2 { 802 spl { 803 }; 804 u-boot { 805 }; 806 }; 807 }; 808 809end-at-4gb: 810 For x86 machines the ROM offsets start just before 4GB and extend 811 up so that the image finished at the 4GB boundary. This boolean 812 option can be enabled to support this. The image size must be 813 provided so that binman knows when the image should start. For an 814 8MB ROM, the offset of the first entry would be 0xfff80000 with 815 this option, instead of 0 without this option. 816 817skip-at-start: 818 This property specifies the entry offset of the first entry. 819 820 For PowerPC mpc85xx based CPU, CONFIG_TEXT_BASE is the entry 821 offset of the first entry. It can be 0xeff40000 or 0xfff40000 for 822 nor flash boot, 0x201000 for sd boot etc. 823 824 'end-at-4gb' property is not applicable where CONFIG_TEXT_BASE + 825 Image size != 4gb. 826 827align-default: 828 Specifies the default alignment for entries in this section, if they do 829 not specify an alignment. Note that this only applies to top-level entries 830 in the section (direct subentries), not any subentries of those entries. 831 This means that each section must specify its own default alignment, if 832 required. 833 834symlink: 835 Adds a symlink to the image with string given in the symlink property. 836 837overlap: 838 Indicates that this entry overlaps with others in the same section. These 839 entries should appear at the end of the section. Overlapping entries are not 840 packed with other entries, but their contents are written over other entries 841 in the section. Overlapping entries must have an explicit offset and size. 842 843write-symbols: 844 Indicates that the blob should be updated with symbol values calculated by 845 binman. This is automatic for certain entry types, e.g. `u-boot-spl`. See 846 binman_syms_ for more information. 847 848no-write-symbols: 849 Disables symbol writing for this entry. This can be used in entry types 850 where symbol writing is automatic. For example, if `u-boot-spl` refers to 851 the `u_boot_any_image_pos` symbol but U-Boot is not available in the image 852 containing SPL, this can be used to disable the writing. Quite likely this 853 indicates a bug in your setup. 854 855elf-filename: 856 Sets the file name of a blob's associated ELF file. For example, if the 857 blob is `zephyr.bin` then the ELF file may be `zephyr.elf`. This allows 858 binman to locate symbols and understand the structure of the blob. See 859 binman_syms_ for more information. 860 861elf-base-sym: 862 Sets the name of the ELF symbol that points to the start of a blob. For 863 U-Boot this is `__image_copy_start` and that is the default used by binman 864 if this property is missing. For other projects, a difference symbol may be 865 needed. Add this symbol to the properties for the blob so that symbols can 866 be read correctly. See binman_syms_ for more information. 867 868offset-from-elf: 869 Sets the offset of an entry based on a symbol value in an another entry. 870 The format is <&phandle>, "sym_name", <offset> where phandle is the entry 871 containing the blob (with associated ELF file providing symbols), <sym_name> 872 is the symbol to lookup (relative to elf-base-sym) and <offset> is an offset 873 to add to that value. 874 875preserve: 876 Indicates that this entry should be preserved by any firmware updates. This 877 flag should be checked by the updater when it is deciding which entries to 878 update. This flag is normally attached to sections but can be attached to 879 a single entry in a section if the updater supports it. Not that binman 880 itself has no control over the updater's behaviour, so this is just a 881 signal. It is not enforced by binman. 882 883Examples of the above options can be found in the tests. See the 884tools/binman/test directory. 885 886It is possible to have the same binary appear multiple times in the image, 887either by using a unit number suffix (u-boot@0, u-boot@1) or by using a 888different name for each and specifying the type with the 'type' attribute. 889 890 891Sections and hierachical images 892------------------------------- 893 894Sometimes it is convenient to split an image into several pieces, each of which 895contains its own set of binaries. An example is a flash device where part of 896the image is read-only and part is read-write. We can set up sections for each 897of these, and place binaries in them independently. The image is still produced 898as a single output file. 899 900This feature provides a way of creating hierarchical images. For example here 901is an example image with two copies of U-Boot. One is read-only (ro), intended 902to be written only in the factory. Another is read-write (rw), so that it can be 903upgraded in the field. The sizes are fixed so that the ro/rw boundary is known 904and can be programmed:: 905 906 binman { 907 section@0 { 908 read-only; 909 name-prefix = "ro-"; 910 size = <0x100000>; 911 u-boot { 912 }; 913 }; 914 section@1 { 915 name-prefix = "rw-"; 916 size = <0x100000>; 917 u-boot { 918 }; 919 }; 920 }; 921 922This image could be placed into a SPI flash chip, with the protection boundary 923set at 1MB. 924 925A few special properties are provided for sections: 926 927read-only: 928 Indicates that this section is read-only. This has no impact on binman's 929 operation, but his property can be read at run time. 930 931name-prefix: 932 This string is prepended to all the names of the binaries in the 933 section. In the example above, the 'u-boot' binaries which actually be 934 renamed to 'ro-u-boot' and 'rw-u-boot'. This can be useful to 935 distinguish binaries with otherwise identical names. 936 937filename: 938 This allows the contents of the section to be written to a file in the 939 output directory. This can sometimes be useful to use the data in one 940 section in different image, since there is currently no way to share data 941 beteen images other than through files. 942 943Image Properties 944---------------- 945 946Image nodes act like sections but also have a few extra properties: 947 948filename: 949 Output filename for the image. This defaults to image.bin (or in the 950 case of multiple images <nodename>.bin where <nodename> is the name of 951 the image node. 952 953allow-repack: 954 Create an image that can be repacked. With this option it is possible 955 to change anything in the image after it is created, including updating 956 the position and size of image components. By default this is not 957 permitted since it is not possibly to know whether this might violate a 958 constraint in the image description. For example, if a section has to 959 increase in size to hold a larger binary, that might cause the section 960 to fall out of its allow region (e.g. read-only portion of flash). 961 962 Adding this property causes the original offset and size values in the 963 image description to be stored in the FDT and fdtmap. 964 965 966Image dependencies 967------------------ 968 969Binman does not currently support images that depend on each other. For example, 970if one image creates `fred.bin` and then the next uses this `fred.bin` to 971produce a final `image.bin`, then the behaviour is undefined. It may work, or it 972may produce an error about `fred.bin` being missing, or it may use a version of 973`fred.bin` from a previous run. 974 975Often this can be handled by incorporating the dependency into the second 976image. For example, instead of:: 977 978 binman { 979 multiple-images; 980 981 fred { 982 u-boot { 983 }; 984 fill { 985 size = <0x100>; 986 }; 987 }; 988 989 image { 990 blob { 991 filename = "fred.bin"; 992 }; 993 u-boot-spl { 994 }; 995 }; 996 997you can do this:: 998 999 binman { 1000 image { 1001 fred { 1002 type = "section"; 1003 u-boot { 1004 }; 1005 fill { 1006 size = <0x100>; 1007 }; 1008 }; 1009 u-boot-spl { 1010 }; 1011 }; 1012 1013 1014 1015Hashing Entries 1016--------------- 1017 1018It is possible to ask binman to hash the contents of an entry and write that 1019value back to the device-tree node. For example:: 1020 1021 binman { 1022 u-boot { 1023 hash { 1024 algo = "sha256"; 1025 }; 1026 }; 1027 }; 1028 1029Here, a new 'value' property will be written to the 'hash' node containing 1030the hash of the 'u-boot' entry. Only SHA256 is supported at present. Whole 1031sections can be hased if desired, by adding the 'hash' node to the section. 1032 1033The has value can be chcked at runtime by hashing the data actually read and 1034comparing this has to the value in the device tree. 1035 1036 1037Expanded entries 1038---------------- 1039 1040Binman automatically replaces 'u-boot' with an expanded version of that, i.e. 1041'u-boot-expanded'. This means that when you write:: 1042 1043 u-boot { 1044 }; 1045 1046you actually get:: 1047 1048 u-boot { 1049 type = "u-boot-expanded'; 1050 }; 1051 1052which in turn expands to:: 1053 1054 u-boot { 1055 type = "section"; 1056 1057 u-boot-nodtb { 1058 }; 1059 1060 u-boot-dtb { 1061 }; 1062 }; 1063 1064U-Boot's various phase binaries actually comprise two or three pieces. 1065For example, u-boot.bin has the executable followed by a devicetree. 1066 1067With binman we want to be able to update that devicetree with full image 1068information so that it is accessible to the executable. This is tricky 1069if it is not clear where the devicetree starts. 1070 1071The above feature ensures that the devicetree is clearly separated from the 1072U-Boot executable and can be updated separately by binman as needed. It can be 1073disabled with the --no-expanded flag if required. 1074 1075The same applies for u-boot-spl and u-boot-tpl. In those cases, the expansion 1076includes the BSS padding, so for example:: 1077 1078 spl { 1079 type = "u-boot-spl" 1080 }; 1081 1082you actually get:: 1083 1084 spl { 1085 type = "u-boot-expanded'; 1086 }; 1087 1088which in turn expands to:: 1089 1090 spl { 1091 type = "section"; 1092 1093 u-boot-spl-nodtb { 1094 }; 1095 1096 u-boot-spl-bss-pad { 1097 }; 1098 1099 u-boot-spl-dtb { 1100 }; 1101 }; 1102 1103Of course we should not expand SPL if it has no devicetree. Also if the BSS 1104padding is not needed (because BSS is in RAM as with CONFIG_SPL_SEPARATE_BSS), 1105the 'u-boot-spl-bss-pad' subnode should not be created. The use of the expaned 1106entry type is controlled by the UseExpanded() method. In the SPL case it checks 1107the 'spl-dtb' entry arg, which is 'y' or '1' if SPL has a devicetree. 1108 1109For the BSS case, a 'spl-bss-pad' entry arg controls whether it is present. All 1110entry args are provided by the U-Boot Makefile. 1111 1112 1113Optional entries 1114---------------- 1115 1116Some entries need to exist only if certain conditions are met. For example, an 1117entry may want to appear in the image only if a file has a particular format. 1118Obviously the entry must exist in the image description for it to be processed 1119at all, so a way needs to be found to have the entry remove itself. 1120 1121To handle this, when entry.ObtainContents() is called, the entry can call 1122entry.mark_absent() to mark itself as absent, passing a suitable message as the 1123reason. 1124 1125Any absent entries are dropped immediately after ObtainContents() has been 1126called on all entries. 1127 1128It is not possible for an entry to mark itself absent at any other point in the 1129processing. It must happen in the ObtainContents() method. 1130 1131The effect is as if the entry had never been present at all, since the image 1132is packed without it and it disappears from the list of entries. 1133 1134 1135Compression 1136----------- 1137 1138Binman support compression for 'blob' entries (those of type 'blob' and 1139derivatives). To enable this for an entry, add a 'compress' property:: 1140 1141 blob { 1142 filename = "datafile"; 1143 compress = "lz4"; 1144 }; 1145 1146The entry will then contain the compressed data, using the 'lz4' compression 1147algorithm. Currently this is the only one that is supported. The uncompressed 1148size is written to the node in an 'uncomp-size' property, if -u is used. 1149 1150Compression is also supported for sections. In that case the entire section is 1151compressed in one block, including all its contents. This means that accessing 1152an entry from the section required decompressing the entire section. Also, the 1153size of a section indicates the space that it consumes in its parent section 1154(and typically the image). With compression, the section may contain more data, 1155and the uncomp-size property indicates that, as above. The contents of the 1156section is compressed first, before any padding is added. This ensures that the 1157padding itself is not compressed, which would be a waste of time. 1158 1159 1160Automatic .dtsi inclusion 1161------------------------- 1162 1163It is sometimes inconvenient to add a 'binman' node to the .dts file for each 1164board. This can be done by using #include to bring in a common file. Another 1165approach supported by the U-Boot build system is to automatically include 1166a common header. You can then put the binman node (and anything else that is 1167specific to U-Boot, such as bootph-all properies) in that header file. 1168 1169Binman will search for the following files in arch/<arch>/dts:: 1170 1171 <dts>-u-boot.dtsi where <dts> is the base name of the .dts file 1172 <CONFIG_SYS_SOC>-u-boot.dtsi 1173 <CONFIG_SYS_CPU>-u-boot.dtsi 1174 <CONFIG_SYS_VENDOR>-u-boot.dtsi 1175 u-boot.dtsi 1176 1177U-Boot will only use the first one that it finds. If you need to include a 1178more general file you can do that from the more specific file using #include. 1179If you are having trouble figuring out what is going on, you can use 1180`DEVICE_TREE_DEBUG=1` with your build:: 1181 1182 make DEVICE_TREE_DEBUG=1 1183 scripts/Makefile.lib:334: Automatic .dtsi inclusion: options: 1184 arch/arm/dts/juno-r2-u-boot.dtsi arch/arm/dts/-u-boot.dtsi 1185 arch/arm/dts/armv8-u-boot.dtsi arch/arm/dts/armltd-u-boot.dtsi 1186 arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" 1187 1188 1189Templates 1190========= 1191 1192Sometimes multiple images need to be created which have all have a common 1193part. For example, a board may generate SPI and eMMC images which both include 1194a FIT. Since the FIT includes many entries, it is tedious to repeat them twice 1195in the image description. 1196 1197Templates provide a simple way to handle this:: 1198 1199 binman { 1200 multiple-images; 1201 common_part: template-1 { 1202 some-property; 1203 fit { 1204 ... lots of entries in here 1205 }; 1206 1207 text { 1208 text = "base image"; 1209 }; 1210 }; 1211 1212 spi-image { 1213 filename = "image-spi.bin"; 1214 insert-template = <&common_part>; 1215 1216 /* things specific to SPI follow */ 1217 footer { 1218 ]; 1219 1220 text { 1221 text = "SPI image"; 1222 }; 1223 }; 1224 1225 mmc-image { 1226 filename = "image-mmc.bin"; 1227 insert-template = <&common_part>; 1228 1229 /* things specific to MMC follow */ 1230 footer { 1231 ]; 1232 1233 text { 1234 text = "MMC image"; 1235 }; 1236 }; 1237 }; 1238 1239The template node name must start with 'template', so it is not considered to be 1240an image itself. 1241 1242The mechanism is very simple. For each phandle in the 'insert-templates' 1243property, the source node is looked up. Then the subnodes of that source node 1244are copied into the target node, i.e. the one containing the `insert-template` 1245property. 1246 1247If the target node has a node with the same name as a template, its properties 1248override corresponding properties in the template. This allows the template to 1249be uses as a base, with the node providing updates to the properties as needed. 1250The overriding happens recursively. 1251 1252Template nodes appear first in each node that they are inserted into and 1253ordering of template nodes is preserved. Other nodes come afterwards. If a 1254template node also appears in the target node, then the template node sets the 1255order. Thus the template can be used to set the ordering, even if the target 1256node provides all the properties. In the above example, `fit` and `text` appear 1257first in the `spi-image` and `mmc-image` images, followed by `footer`. 1258 1259Where there are multiple template nodes, they are inserted in that order. so 1260the first template node appears first, then the second. 1261 1262Properties in the template node are inserted into the destination node if they 1263do not exist there. In the example above, `some-property` is added to each of 1264`spi-image` and `mmc-image`. 1265 1266Note that template nodes are removed from the binman description after 1267processing and before binman builds the image descriptions. 1268 1269The initial devicetree produced by the templating process is written to the 1270`u-boot.dtb.tmpl1` file. This can be useful to see what is going on if there is 1271a failure before the final `u-boot.dtb.out` file is written. A second 1272`u-boot.dtb.tmpl2` file is written when the templates themselves are removed. 1273 1274Dealing with phandles 1275--------------------- 1276 1277Templates can contain phandles and these are copied to the destination node. 1278However this should be used with care, since if a template is instantiated twice 1279then the phandle will be copied twice, resulting in a devicetree with duplicate 1280phandles, i.e. the same phandle used by two different nodes. Binman detects this 1281situation and produces an error, for example:: 1282 1283 Duplicate phandle 1 in nodes /binman/image/fit/images/atf/atf-bl31 and 1284 /binman/image-2/fit/images/atf/atf-bl31 1285 1286In this case an atf-bl31 node containing a phandle has been copied into two 1287different target nodes, resulting in the same phandle for each. See 1288testTemplatePhandleDup() for the test case. 1289 1290The solution is typically to put the phandles in the corresponding target nodes 1291(one for each) and remove the phandle from the template. 1292 1293Updating an ELF file 1294==================== 1295 1296For the EFI app, where U-Boot is loaded from UEFI and runs as an app, there is 1297no way to update the devicetree after U-Boot is built. Normally this works by 1298creating a new u-boot.dtb.out with he updated devicetree, which is automatically 1299built into the output image. With ELF this is not possible since the ELF is 1300not part of an image, just a stand-along file. We must create an updated ELF 1301file with the new devicetree. 1302 1303This is handled by the --update-fdt-in-elf option. It takes four arguments, 1304separated by comma: 1305 1306 infile - filename of input ELF file, e.g. 'u-boot's 1307 outfile - filename of output ELF file, e.g. 'u-boot.out' 1308 begin_sym - symbol at the start of the embedded devicetree, e.g. 1309 '__dtb_dt_begin' 1310 end_sym - symbol at the start of the embedded devicetree, e.g. 1311 '__dtb_dt_end' 1312 1313When this flag is used, U-Boot does all the normal packaging, but as an 1314additional step, it creates a new ELF file with the new devicetree embedded in 1315it. 1316 1317If logging is enabled you will see a message like this:: 1318 1319 Updating file 'u-boot' with data length 0x400a (16394) between symbols 1320 '__dtb_dt_begin' and '__dtb_dt_end' 1321 1322There must be enough space for the updated devicetree. If not, an error like 1323the following is produced:: 1324 1325 ValueError: Not enough space in 'u-boot' for data length 0x400a (16394); 1326 size is 0x1744 (5956) 1327 1328 1329Entry Documentation 1330=================== 1331 1332For details on the various entry types supported by binman and how to use them, 1333see entries.rst which is generated from the source code using: 1334 1335 binman entry-docs >tools/binman/entries.rst 1336 1337.. toctree:: 1338 :maxdepth: 2 1339 1340 entries 1341 1342 1343Managing images 1344=============== 1345 1346Listing images 1347-------------- 1348 1349It is possible to list the entries in an existing firmware image created by 1350binman, provided that there is an 'fdtmap' entry in the image. For example:: 1351 1352 $ binman ls -i image.bin 1353 Name Image-pos Size Entry-type Offset Uncomp-size 1354 ---------------------------------------------------------------------- 1355 main-section c00 section 0 1356 u-boot 0 4 u-boot 0 1357 section 5fc section 4 1358 cbfs 100 400 cbfs 0 1359 u-boot 138 4 u-boot 38 1360 u-boot-dtb 180 108 u-boot-dtb 80 3b5 1361 u-boot-dtb 500 1ff u-boot-dtb 400 3b5 1362 fdtmap 6fc 381 fdtmap 6fc 1363 image-header bf8 8 image-header bf8 1364 1365This shows the hierarchy of the image, the position, size and type of each 1366entry, the offset of each entry within its parent and the uncompressed size if 1367the entry is compressed. 1368 1369It is also possible to list just some files in an image, e.g.:: 1370 1371 $ binman ls -i image.bin section/cbfs 1372 Name Image-pos Size Entry-type Offset Uncomp-size 1373 -------------------------------------------------------------------- 1374 cbfs 100 400 cbfs 0 1375 u-boot 138 4 u-boot 38 1376 u-boot-dtb 180 108 u-boot-dtb 80 3b5 1377 1378or with wildcards:: 1379 1380 $ binman ls -i image.bin "*cb*" "*head*" 1381 Name Image-pos Size Entry-type Offset Uncomp-size 1382 ---------------------------------------------------------------------- 1383 cbfs 100 400 cbfs 0 1384 u-boot 138 4 u-boot 38 1385 u-boot-dtb 180 108 u-boot-dtb 80 3b5 1386 image-header bf8 8 image-header bf8 1387 1388If an older version of binman is used to list images created by a newer one, it 1389is possible that it will contain entry types that are not supported. These still 1390show with the correct type, but binman just sees them as blobs (plain binary 1391data). Any special features of that etype are not supported by the old binman. 1392 1393 1394Extracting files from images 1395---------------------------- 1396 1397You can extract files from an existing firmware image created by binman, 1398provided that there is an 'fdtmap' entry in the image. For example:: 1399 1400 $ binman extract -i image.bin section/cbfs/u-boot 1401 1402which will write the uncompressed contents of that entry to the file 'u-boot' in 1403the current directory. You can also extract to a particular file, in this case 1404u-boot.bin:: 1405 1406 $ binman extract -i image.bin section/cbfs/u-boot -f u-boot.bin 1407 1408It is possible to extract all files into a destination directory, which will 1409put files in subdirectories matching the entry hierarchy:: 1410 1411 $ binman extract -i image.bin -O outdir 1412 1413or just a selection:: 1414 1415 $ binman extract -i image.bin "*u-boot*" -O outdir 1416 1417Some entry types have alternative formats, for example fdtmap which allows 1418extracted just the devicetree binary without the fdtmap header:: 1419 1420 $ binman extract -i /tmp/b/odroid-c4/image.bin -f out.dtb -F fdt fdtmap 1421 $ fdtdump out.dtb 1422 /dts-v1/; 1423 // magic: 0xd00dfeed 1424 // totalsize: 0x8ab (2219) 1425 // off_dt_struct: 0x38 1426 // off_dt_strings: 0x82c 1427 // off_mem_rsvmap: 0x28 1428 // version: 17 1429 // last_comp_version: 2 1430 // boot_cpuid_phys: 0x0 1431 // size_dt_strings: 0x7f 1432 // size_dt_struct: 0x7f4 1433 1434 / { 1435 image-node = "binman"; 1436 image-pos = <0x00000000>; 1437 size = <0x0011162b>; 1438 ... 1439 1440Use `-F list` to see what alternative formats are available:: 1441 1442 $ binman extract -i /tmp/b/odroid-c4/image.bin -F list 1443 Flag (-F) Entry type Description 1444 fdt fdtmap Extract the devicetree blob from the fdtmap 1445 1446 1447Replacing files in an image 1448--------------------------- 1449 1450You can replace files in an existing firmware image created by binman, provided 1451that there is an 'fdtmap' entry in the image. For example:: 1452 1453 $ binman replace -i image.bin section/cbfs/u-boot 1454 1455which will write the contents of the file 'u-boot' from the current directory 1456to the that entry, compressing if necessary. If the entry size changes, you must 1457add the 'allow-repack' property to the original image before generating it (see 1458above), otherwise you will get an error. 1459 1460You can also use a particular file, in this case u-boot.bin:: 1461 1462 $ binman replace -i image.bin section/cbfs/u-boot -f u-boot.bin 1463 1464It is possible to replace all files from a source directory which uses the same 1465hierarchy as the entries:: 1466 1467 $ binman replace -i image.bin -I indir 1468 1469Files that are missing will generate a warning. 1470 1471You can also replace just a selection of entries:: 1472 1473 $ binman replace -i image.bin "*u-boot*" -I indir 1474 1475It is possible to replace whole sections as well, but in that case any 1476information about entries within the section may become outdated. This is 1477because Binman cannot know whether things have moved around or resized within 1478the section, once you have updated its data. 1479 1480Technical note: With 'allow-repack', Binman writes information about the 1481original offset and size properties of each entry, if any were specified, in 1482the 'orig-offset' and 'orig-size' properties. This allows Binman to distinguish 1483between an entry which ended up being packed at an offset (or assigned a size) 1484and an entry which had a particular offset / size requested in the Binman 1485configuration. Where are particular offset / size was requested, this is treated 1486as set in stone, so Binman will ensure it doesn't change. Without this feature, 1487repacking an entry might cause it to disobey the original constraints provided 1488when it was created. 1489 1490 1491Signing FIT container with private key in an image 1492-------------------------------------------------- 1493 1494You can sign FIT container with private key in your image. 1495For example:: 1496 1497 $ binman sign -i image.bin -k privatekey -a sha256,rsa4096 fit 1498 1499binman will extract FIT container, sign and replace it immediately. 1500 1501If you want to sign and replace FIT container in place:: 1502 1503 $ binman sign -i image.bin -k privatekey -a sha256,rsa4096 -f fit.fit fit 1504 1505which will sign FIT container with private key and replace it immediately 1506inside your image. 1507 1508.. _`BinmanLogging`: 1509 1510Logging 1511------- 1512 1513Binman normally operates silently unless there is an error, in which case it 1514just displays the error. The -D/--debug option can be used to create a full 1515backtrace when errors occur. You can use BINMAN_DEBUG=1 when building to select 1516this. 1517 1518Internally binman logs some output while it is running. This can be displayed 1519by increasing the -v/--verbosity from the default of 1: 1520 1521 0: silent 1522 1: warnings only 1523 2: notices (important messages) 1524 3: info about major operations 1525 4: detailed information about each operation 1526 5: debug (all output) 1527 1528You can use BINMAN_VERBOSE=5 (for example) when building to select this. 1529 1530 1531Bintools 1532======== 1533 1534`Bintool` is the name binman gives to a binary tool which it uses to create and 1535manipulate binaries that binman cannot handle itself. Bintools are often 1536necessary since Binman only supports a subset of the available file formats 1537natively. 1538 1539Many SoC vendors invent ways to load code into their SoC using new file formats, 1540sometimes changing the format with successive SoC generations. Sometimes the 1541tool is available as Open Source. Sometimes it is a pre-compiled binary that 1542must be downloaded from the vendor's website. Sometimes it is available in 1543source form but difficult or slow to build. 1544 1545Even for images that use bintools, binman still assembles the image from its 1546image description. It may handle parts of the image natively and part with 1547various bintools. 1548 1549Binman relies on these tools so provides various features to manage them: 1550 1551- Determining whether the tool is currently installed 1552- Downloading or building the tool 1553- Determining the version of the tool that is installed 1554- Deciding which tools are needed to build an image 1555 1556The Bintool class is an interface to the tool, a thin level of abstration, using 1557Python functions to run the tool for each purpose (e.g. creating a new 1558structure, adding a file to an existing structure) rather than just lists of 1559string arguments. 1560 1561As with external blobs, bintools (which are like 'external' tools) can be 1562missing. When building an image requires a bintool and it is not installed, 1563binman detects this and reports the problem, but continues to build an image. 1564This is useful in CI systems which want to check that everything is correct but 1565don't have access to the bintools. 1566 1567To make this work, all calls to bintools (e.g. with Bintool.run_cmd()) must cope 1568with the tool being missing, i.e. when None is returned, by: 1569 1570- Calling self.record_missing_bintool() 1571- Setting up some fake contents so binman can continue 1572 1573Of course the image will not work, but binman reports which bintools are needed 1574and also provide a way to fetch them. 1575 1576To see the available bintools, use:: 1577 1578 binman tool --list 1579 1580To fetch tools which are missing, use:: 1581 1582 binman tool --fetch missing 1583 1584You can also use `--fetch all` to fetch all tools or `--fetch <tool>` to fetch 1585a particular tool. Some tools are built from source code, in which case you will 1586need to have at least the `build-essential` and `git` packages installed. 1587 1588Tools are fetched into the `~/.binman-tools` directory. This directory is 1589automatically added to the toolpath so there is no need to use `--toolpath` to 1590specify it. If you want to use these tools outside binman, you may want to 1591add this directory to your `PATH`. For example, if you use bash, add this to 1592the end of `.bashrc`:: 1593 1594 PATH="$HOME/.binman-tools:$PATH" 1595 1596To select a custom directory, use the `--tooldir` option. 1597 1598Bintool Documentation 1599===================== 1600 1601To provide details on the various bintools supported by binman, bintools.rst is 1602generated from the source code using: 1603 1604 binman bintool-docs >tools/binman/bintools.rst 1605 1606.. toctree:: 1607 :maxdepth: 2 1608 1609 bintools 1610 1611Binman commands and arguments 1612============================= 1613 1614Usage:: 1615 1616 binman [-h] [-B BUILD_DIR] [-D] [--tooldir TOOLDIR] [-H] 1617 [--toolpath TOOLPATH] [-T THREADS] [--test-section-timeout] 1618 [-v VERBOSITY] [-V] 1619 {build,bintool-docs,entry-docs,ls,extract,replace,test,tool} ... 1620 1621Binman provides the following commands: 1622 1623- **build** - build images 1624- **bintools-docs** - generate documentation about bintools 1625- **entry-docs** - generate documentation about entry types 1626- **ls** - list an image 1627- **extract** - extract files from an image 1628- **replace** - replace one or more entries in an image 1629- **test** - run tests 1630- **tool** - manage bintools 1631 1632Options: 1633 1634-h, --help 1635 Show help message and exit 1636 1637-B BUILD_DIR, --build-dir BUILD_DIR 1638 Directory containing the build output 1639 1640-D, --debug 1641 Enabling debugging (provides a full traceback on error) 1642 1643--tooldir TOOLDIR Set the directory to store tools 1644 1645-H, --full-help 1646 Display the README file 1647 1648--toolpath TOOLPATH 1649 Add a path to the list of directories containing tools 1650 1651-T THREADS, --threads THREADS 1652 Number of threads to use (0=single-thread). Note that -T0 is useful for 1653 debugging since everything runs in one thread. 1654 1655-v VERBOSITY, --verbosity VERBOSITY 1656 Control verbosity: 0=silent, 1=warnings, 2=notices, 3=info, 4=detail, 1657 5=debug 1658 1659-V, --version 1660 Show the binman version 1661 1662Test options: 1663 1664--test-section-timeout 1665 Use a zero timeout for section multi-threading (for testing) 1666 1667Commands are described below. 1668 1669binman build 1670------------ 1671 1672This builds one or more images using the provided image description. 1673 1674Usage:: 1675 1676 binman build [-h] [-a ENTRY_ARG] [-b BOARD] [-d DT] [--fake-dtb] 1677 [--fake-ext-blobs] [--force-missing-bintools FORCE_MISSING_BINTOOLS] 1678 [-i IMAGE] [-I INDIR] [-m] [-M] [-n] [-O OUTDIR] [-p] [-u] 1679 [--update-fdt-in-elf UPDATE_FDT_IN_ELF] [-W] 1680 1681Options: 1682 1683-h, --help 1684 Show help message and exit 1685 1686-a ENTRY_ARG, --entry-arg ENTRY_ARG 1687 Set argument value `arg=value`. See 1688 `Passing command-line arguments to entries`_. 1689 1690-b BOARD, --board BOARD 1691 Board name to build. This can be used instead of `-d`, in which case the 1692 file `u-boot.dtb` is used, within the build directory's board subdirectory. 1693 1694-d DT, --dt DT 1695 Configuration file (.dtb) to use. This must have a top-level node called 1696 `binman`. See `Image description format`_. 1697 1698-i IMAGE, --image IMAGE 1699 Image filename to build (if not specified, build all) 1700 1701-I INDIR, --indir INDIR 1702 Add a path to the list of directories to use for input files. This can be 1703 specified multiple times to add more than one path. 1704 1705-m, --map 1706 Output a map file for each image. See `Map files`_. 1707 1708-M, --allow-missing 1709 Allow external blobs and bintools to be missing. See `External blobs`_. 1710 1711-n, --no-expanded 1712 Don't use 'expanded' versions of entries where available; normally 'u-boot' 1713 becomes 'u-boot-expanded', for example. See `Expanded entries`_. 1714 1715-O OUTDIR, --outdir OUTDIR 1716 Path to directory to use for intermediate and output files 1717 1718-p, --preserve 1719 Preserve temporary output directory even if option -O is not given 1720 1721-u, --update-fdt 1722 Update the binman node with offset/size info. See 1723 `Access to binman entry offsets at run time (fdt)`_. 1724 1725--update-fdt-in-elf UPDATE_FDT_IN_ELF 1726 Update an ELF file with the output dtb. The argument is a string consisting 1727 of four parts, separated by commas. See `Updating an ELF file`_. 1728 1729-W, --ignore-missing 1730 Return success even if there are missing blobs/bintools (requires -M) 1731 1732Options used only for testing: 1733 1734--fake-dtb 1735 Use fake device tree contents 1736 1737--fake-ext-blobs 1738 Create fake ext blobs with dummy content 1739 1740--force-missing-bintools FORCE_MISSING_BINTOOLS 1741 Comma-separated list of bintools to consider missing 1742 1743binman bintool-docs 1744------------------- 1745 1746Usage:: 1747 1748 binman bintool-docs [-h] 1749 1750This outputs documentation for the bintools in rST format. See 1751`Bintool Documentation`_. 1752 1753binman entry-docs 1754----------------- 1755 1756Usage:: 1757 1758 binman entry-docs [-h] 1759 1760This outputs documentation for the entry types in rST format. See 1761`Entry Documentation`_. 1762 1763binman ls 1764--------- 1765 1766Usage:: 1767 1768 binman ls [-h] -i IMAGE [paths ...] 1769 1770Positional arguments: 1771 1772paths 1773 Paths within file to list (wildcard) 1774 1775Pptions: 1776 1777-h, --help 1778 show help message and exit 1779 1780-i IMAGE, --image IMAGE 1781 Image filename to list 1782 1783This lists an image, showing its contents. See `Listing images`_. 1784 1785binman extract 1786-------------- 1787 1788Usage:: 1789 1790 binman extract [-h] [-F FORMAT] -i IMAGE [-f FILENAME] [-O OUTDIR] [-U] 1791 [paths ...] 1792 1793Positional arguments: 1794 1795Paths 1796 Paths within file to extract (wildcard) 1797 1798Options: 1799 1800-h, --help 1801 show help message and exit 1802 1803-F FORMAT, --format FORMAT 1804 Select an alternative format for extracted data 1805 1806-i IMAGE, --image IMAGE 1807 Image filename to extract 1808 1809-f FILENAME, --filename FILENAME 1810 Output filename to write to 1811 1812-O OUTDIR, --outdir OUTDIR 1813 Path to directory to use for output files 1814 1815-U, --uncompressed 1816 Output raw uncompressed data for compressed entries 1817 1818This extracts the contents of entries from an image. See 1819`Extracting files from images`_. 1820 1821binman replace 1822-------------- 1823 1824Usage:: 1825 1826 binman replace [-h] [-C] -i IMAGE [-f FILENAME] [-F] [-I INDIR] [-m] 1827 [paths ...] 1828 1829Positional arguments: 1830 1831paths 1832 Paths within file to replace (wildcard) 1833 1834Options: 1835 1836-h, --help 1837 show help message and exit 1838 1839-C, --compressed 1840 Input data is already compressed if needed for the entry 1841 1842-i IMAGE, --image IMAGE 1843 Image filename to update 1844 1845-f FILENAME, --filename FILENAME 1846 Input filename to read from 1847 1848-F, --fix-size 1849 Don't allow entries to be resized 1850 1851-I INDIR, --indir INDIR 1852 Path to directory to use for input files 1853 1854-m, --map 1855 Output a map file for the updated image 1856 1857-O OUTDIR, --outdir OUTDIR 1858 Path to directory to use for intermediate and output files 1859 1860-p, --preserve 1861 Preserve temporary output directory even if option -O is not given 1862 1863This replaces one or more entries in an existing image. See 1864`Replacing files in an image`_. 1865 1866binman test 1867----------- 1868 1869Usage:: 1870 1871 binman test [-h] [-P PROCESSES] [-T] [-X] [tests ...] 1872 1873Positional arguments: 1874 1875tests 1876 Test names to run (omit for all) 1877 1878Options: 1879 1880-h, --help 1881 show help message and exit 1882 1883-P PROCESSES, --processes PROCESSES 1884 set number of processes to use for running tests. This defaults to the 1885 number of CPUs on the machine 1886 1887-T, --test-coverage 1888 run tests and check for 100% coverage 1889 1890-X, --test-preserve-dirs 1891 Preserve and display test-created input directories; also preserve the 1892 output directory if a single test is run (pass test name at the end of the 1893 command line 1894 1895binman sign 1896----------- 1897 1898Usage:: 1899 1900 binman sign [-h] -a ALGO [-f FILE] -i IMAGE -k KEY [paths ...] 1901 1902positional arguments: 1903 1904paths 1905 Paths within file to sign (wildcard) 1906 1907options: 1908 1909-h, --help 1910 show this help message and exit 1911 1912-a ALGO, --algo ALGO 1913 Hash algorithm e.g. sha256,rsa4096 1914 1915-f FILE, --file FILE 1916 Input filename to sign 1917 1918-i IMAGE, --image IMAGE 1919 Image filename to update 1920 1921-k KEY, --key KEY 1922 Private key file for signing 1923 1924binman tool 1925----------- 1926 1927Usage:: 1928 1929 binman tool [-h] [-l] [-f] [bintools ...] 1930 1931Positional arguments: 1932 1933bintools 1934 Bintools to process 1935 1936Options: 1937 1938-h, --help 1939 show help message and exit 1940 1941-l, --list 1942 List all known bintools 1943 1944-f, --fetch 1945 Fetch a bintool from a known location. Use `all` to fetch all and `missing` 1946 to fetch any missing tools. 1947 1948 1949Technical details 1950================= 1951 1952Order of image creation 1953----------------------- 1954 1955Image creation proceeds in the following order, for each entry in the image. 1956 19571. AddMissingProperties() - binman can add calculated values to the device 1958tree as part of its processing, for example the offset and size of each 1959entry. This method adds any properties associated with this, expanding the 1960device tree as needed. These properties can have placeholder values which are 1961set later by SetCalculatedProperties(). By that stage the size of sections 1962cannot be changed (since it would cause the images to need to be repacked), 1963but the correct values can be inserted. 1964 19652. ProcessFdt() - process the device tree information as required by the 1966particular entry. This may involve adding or deleting properties. If the 1967processing is complete, this method should return True. If the processing 1968cannot complete because it needs the ProcessFdt() method of another entry to 1969run first, this method should return False, in which case it will be called 1970again later. 1971 19723. GetEntryContents() - the contents of each entry are obtained, normally by 1973reading from a file. This calls the Entry.ObtainContents() to read the 1974contents. The default version of Entry.ObtainContents() calls 1975Entry.GetDefaultFilename() and then reads that file. So a common mechanism 1976to select a file to read is to override that function in the subclass. The 1977functions must return True when they have read the contents. Binman will 1978retry calling the functions a few times if False is returned, allowing 1979dependencies between the contents of different entries. 1980 19814. GetEntryOffsets() - calls Entry.GetOffsets() for each entry. This can 1982return a dict containing entries that need updating. The key should be the 1983entry name and the value is a tuple (offset, size). This allows an entry to 1984provide the offset and size for other entries. The default implementation 1985of GetEntryOffsets() returns {}. 1986 19875. PackEntries() - calls Entry.Pack() which figures out the offset and 1988size of an entry. The 'current' image offset is passed in, and the function 1989returns the offset immediately after the entry being packed. The default 1990implementation of Pack() is usually sufficient. 1991 1992Note: for sections, this also checks that the entries do not overlap, nor extend 1993outside the section. If the section does not have a defined size, the size is 1994set large enough to hold all the entries. For entries that are explicitly marked 1995as overlapping, this check is skipped. 1996 19976. SetImagePos() - sets the image position of every entry. This is the absolute 1998position 'image-pos', as opposed to 'offset' which is relative to the containing 1999section. This must be done after all offsets are known, which is why it is quite 2000late in the ordering. 2001 20027. SetCalculatedProperties() - update any calculated properties in the device 2003tree. This sets the correct 'offset' and 'size' vaues, for example. 2004 20058. ProcessEntryContents() - this calls Entry.ProcessContents() on each entry. 2006The default implementatoin does nothing. This can be overriden to adjust the 2007contents of an entry in some way. For example, it would be possible to create 2008an entry containing a hash of the contents of some other entries. At this 2009stage the offset and size of entries should not be adjusted unless absolutely 2010necessary, since it requires a repack (going back to PackEntries()). 2011 20129. ResetForPack() - if the ProcessEntryContents() step failed, in that an entry 2013has changed its size, then there is no alternative but to go back to step 5 and 2014try again, repacking the entries with the updated size. ResetForPack() removes 2015the fixed offset/size values added by binman, so that the packing can start from 2016scratch. 2017 201810. WriteSymbols() - write the value of symbols into the U-Boot SPL binary. 2019See 'Access to binman entry offsets at run time' below for a description of 2020what happens in this stage. 2021 202211. BuildImage() - builds the image and writes it to a file 2023 202412. WriteMap() - writes a text file containing a map of the image. This is the 2025final step. 2026 2027 2028.. _`External tools`: 2029 2030External tools 2031-------------- 2032 2033Binman can make use of external command-line tools to handle processing of 2034entry contents or to generate entry contents. These tools are executed using 2035the 'tools' module's Run() method. The tools generally must exist on the PATH, 2036but the --toolpath option can be used to specify additional search paths to 2037use. This option can be specified multiple times to add more than one path. 2038 2039For some compile tools binman will use the versions specified by commonly-used 2040environment variables like CC and HOSTCC for the C compiler, based on whether 2041the tool's output will be used for the target or for the host machine. If those 2042aren't given, it will also try to derive target-specific versions from the 2043CROSS_COMPILE environment variable during a cross-compilation. 2044 2045If the tool is not available in the path you can use BINMAN_TOOLPATHS to specify 2046a space-separated list of paths to search, e.g.:: 2047 2048 BINMAN_TOOLPATHS="/tools/g12a /tools/tegra" binman ... 2049 2050 2051.. _`External blobs`: 2052 2053External blobs 2054-------------- 2055 2056Binary blobs, even if the source code is available, complicate building 2057firmware. The instructions can involve multiple steps and the binaries may be 2058hard to build or obtain. Binman at least provides a unified description of how 2059to build the final image, no matter what steps are needed to get there. 2060 2061Binman also provides a `blob-ext` entry type that pulls in a binary blob from an 2062external file. If the file is missing, binman can optionally complete the build 2063and just report a warning. Use the `-M/--allow-missing` option to enble this. 2064This is useful in CI systems which want to check that everything is correct but 2065don't have access to the blobs. 2066 2067If the blobs are in a different directory, you can specify this with the `-I` 2068option. 2069 2070For U-Boot, you can set the BINMAN_INDIRS environment variable to provide a 2071space-separated list of directories to search for binary blobs:: 2072 2073 BINMAN_INDIRS="odroid-c4/fip/g12a \ 2074 odroid-c4/build/board/hardkernel/odroidc4/firmware \ 2075 odroid-c4/build/scp_task" binman ... 2076 2077Note that binman fails with exit code 103 when there are missing blobs. If you 2078wish binman to continue anyway, you can pass `-W` to binman. 2079 2080 2081Code coverage 2082------------- 2083 2084Binman is a critical tool and is designed to be very testable. Entry 2085implementations target 100% test coverage. Run 'binman test -T' to check this. 2086 2087To enable Python test coverage on Debian-type distributions (e.g. Ubuntu):: 2088 2089 $ sudo apt-get install python-coverage python3-coverage python-pytest 2090 2091 2092Exit status 2093----------- 2094 2095Binman produces the following exit codes: 2096 20970 2098 Success 2099 21001 2101 Any sort of failure - see output for more details 2102 2103103 2104 There are missing external blobs or bintools. This is only returned if 2105 -M is passed to binman, otherwise missing blobs return an exit status of 1. 2106 Note, if -W is passed as well as -M, then this is converted into a warning 2107 and will return an exit status of 0 instead. 2108 2109 2110U-Boot environment variables for binman 2111--------------------------------------- 2112 2113The U-Boot Makefile supports various environment variables to control binman. 2114All of these are set within the Makefile and result in passing various 2115environment variables (or make flags) to binman: 2116 2117BINMAN_DEBUG 2118 Enables backtrace debugging by adding a `-D` argument. See 2119 :ref:`BinmanLogging`. 2120 2121BINMAN_INDIRS 2122 Sets the search path for input files used by binman by adding one or more 2123 `-I` arguments. See :ref:`External blobs`. 2124 2125BINMAN_TOOLPATHS 2126 Sets the search path for external tool used by binman by adding one or more 2127 `--toolpath` arguments. See :ref:`External tools`. 2128 2129BINMAN_VERBOSE 2130 Sets the logging verbosity of binman by adding a `-v` argument. See 2131 :ref:`BinmanLogging`. 2132 2133 2134Error messages 2135-------------- 2136 2137This section provides some guidance for some of the less obvious error messages 2138produced by binman. 2139 2140 2141Expected __bss_size symbol 2142~~~~~~~~~~~~~~~~~~~~~~~~~~ 2143 2144Example:: 2145 2146 binman: Node '/binman/u-boot-spl-ddr/u-boot-spl/u-boot-spl-bss-pad': 2147 Expected __bss_size symbol in spl/u-boot-spl 2148 2149This indicates that binman needs the `__bss_size` symbol to be defined in the 2150SPL binary, where `spl/u-boot-spl` is the ELF file containing the symbols. The 2151symbol tells binman the size of the BSS region, in bytes. It needs this to be 2152able to pad the image so that the following entries do not overlap the BSS, 2153which would cause them to be overwritte by variable access in SPL. 2154 2155This symbols is normally defined in the linker script, immediately after 2156_bss_start and __bss_end are defined, like this:: 2157 2158 __bss_size = __bss_end - __bss_start; 2159 2160You may need to add it to your linker script if you get this error. 2161 2162 2163Concurrent tests 2164---------------- 2165 2166Binman tries to run tests concurrently. This means that the tests make use of 2167all available CPUs to run. 2168 2169 To enable this:: 2170 2171 $ sudo apt-get install python-subunit python3-subunit 2172 2173Use '-P 1' to disable this. It is automatically disabled when code coverage is 2174being used (-T) since they are incompatible. 2175 2176 2177Debugging tests 2178--------------- 2179 2180Sometimes when debugging tests it is useful to keep the input and output 2181directories so they can be examined later. Use -X or --test-preserve-dirs for 2182this. 2183 2184 2185Running tests on non-x86 architectures 2186-------------------------------------- 2187 2188Binman's tests have been written under the assumption that they'll be run on a 2189x86-like host and there hasn't been an attempt to make them portable yet. 2190However, it's possible to run the tests by cross-compiling to x86. 2191 2192To install an x86 cross-compiler on Debian-type distributions (e.g. Ubuntu):: 2193 2194 $ sudo apt-get install gcc-x86-64-linux-gnu 2195 2196Then, you can run the tests under cross-compilation:: 2197 2198 $ CROSS_COMPILE=x86_64-linux-gnu- binman test -T 2199 2200You can also use gcc-i686-linux-gnu similar to the above. 2201 2202 2203Writing new entries and debugging 2204--------------------------------- 2205 2206The behaviour of entries is defined by the Entry class. All other entries are 2207a subclass of this. An important subclass is Entry_blob which takes binary 2208data from a file and places it in the entry. In fact most entry types are 2209subclasses of Entry_blob. 2210 2211Each entry type is a separate file in the tools/binman/etype directory. Each 2212file contains a class called Entry_<type> where <type> is the entry type. 2213New entry types can be supported by adding new files in that directory. 2214These will automatically be detected by binman when needed. 2215 2216Entry properties are documented in entry.py. The entry subclasses are free 2217to change the values of properties to support special behaviour. For example, 2218when Entry_blob loads a file, it sets content_size to the size of the file. 2219Entry classes can adjust other entries. For example, an entry that knows 2220where other entries should be positioned can set up those entries' offsets 2221so they don't need to be set in the binman decription. It can also adjust 2222entry contents. 2223 2224Most of the time such essoteric behaviour is not needed, but it can be 2225essential for complex images. 2226 2227If you need to specify a particular device-tree compiler to use, you can define 2228the DTC environment variable. This can be useful when the system dtc is too 2229old. 2230 2231To enable a full backtrace and other debugging features in binman, pass 2232BINMAN_DEBUG=1 to your build:: 2233 2234 make qemu-x86_defconfig 2235 make BINMAN_DEBUG=1 2236 2237To enable verbose logging from binman, base BINMAN_VERBOSE to your build, which 2238adds a -v<level> option to the call to binman:: 2239 2240 make qemu-x86_defconfig 2241 make BINMAN_VERBOSE=5 2242 2243 2244Building sections in parallel 2245----------------------------- 2246 2247By default binman uses multiprocessing to speed up compilation of large images. 2248This works at a section level, with one thread for each entry in the section. 2249This can speed things up if the entries are large and use compression. 2250 2251This feature can be disabled with the '-T' flag, which defaults to a suitable 2252value for your machine. This depends on the Python version, e.g on v3.8 it uses 225312 threads on an 8-core machine. See ConcurrentFutures_ for more details. 2254 2255The special value -T0 selects single-threaded mode, useful for debugging during 2256development, since dealing with exceptions and problems in threads is more 2257difficult. This avoids any use of ThreadPoolExecutor. 2258 2259 2260Collecting data for an entry type 2261--------------------------------- 2262 2263Some entry types deal with data obtained from others. For example, 2264`Entry_mkimage` calls the `mkimage` tool with data from its subnodes:: 2265 2266 mkimage { 2267 args = "-n test -T script"; 2268 2269 u-boot-spl { 2270 }; 2271 2272 u-boot { 2273 }; 2274 }; 2275 2276This shows mkimage being passed a file consisting of SPL and U-Boot proper. It 2277is created by calling `Entry.collect_contents_to_file()`. Note that in this 2278case, the data is passed to mkimage for processing but does not appear 2279separately in the image. It may not appear at all, depending on what mkimage 2280does. The contents of the `mkimage` entry are entirely dependent on the 2281processing done by the entry, with the provided subnodes (`u-boot-spl` and 2282`u-boot`) simply providing the input data for that processing. 2283 2284Note that `Entry.collect_contents_to_file()` simply concatenates the data from 2285the different entries together, with no control over alignment, etc. Another 2286approach is to subclass `Entry_section` so that those features become available, 2287such as `size` and `pad-byte`. Then the contents of the entry can be obtained by 2288calling `super().BuildSectionData()` in the entry's BuildSectionData() 2289implementation to get the input data, then write it to a file and process it 2290however is desired. 2291 2292There are other ways to obtain data also, depending on the situation. If the 2293entry type is simply signing data which exists elsewhere in the image, then 2294you can use `Entry_collection` as a base class. It lets you use a property 2295called `content` which lists the entries containing data to be processed. This 2296is used by `Entry_vblock`, for example:: 2297 2298 u_boot: u-boot { 2299 }; 2300 2301 vblock { 2302 content = <&u_boot &dtb>; 2303 keyblock = "firmware.keyblock"; 2304 signprivate = "firmware_data_key.vbprivk"; 2305 version = <1>; 2306 kernelkey = "kernel_subkey.vbpubk"; 2307 preamble-flags = <1>; 2308 }; 2309 2310 dtb: u-boot-dtb { 2311 }; 2312 2313which shows an image containing `u-boot` and `u-boot-dtb`, with the `vblock` 2314image collecting their contents to produce input for its signing process, 2315without affecting those entries, which still appear in the final image 2316untouched. 2317 2318Another example is where an entry type needs several independent pieces of input 2319to function. For example, `Entry_fip` allows a number of different binary blobs 2320to be placed in their own individual places in a custom data structure in the 2321output image. To make that work you can add subnodes for each of them and call 2322`Entry.Create()` on each subnode, as `Entry_fip` does. Then the data for each 2323blob can come from any suitable place, such as an `Entry_u_boot` or an 2324`Entry_blob` or anything else:: 2325 2326 atf-fip { 2327 fip-hdr-flags = /bits/ 64 <0x123>; 2328 soc-fw { 2329 fip-flags = /bits/ 64 <0x123456789abcdef>; 2330 filename = "bl31.bin"; 2331 }; 2332 2333 u-boot { 2334 fip-uuid = [fc 65 13 92 4a 5b 11 ec 2335 94 35 ff 2d 1c fc 79 9c]; 2336 }; 2337 }; 2338 2339The `soc-fw` node is a `blob-ext` (i.e. it reads in a named binary file) whereas 2340`u-boot` is a normal entry type. This works because `Entry_fip` selects the 2341`blob-ext` entry type if the node name (here `soc-fw`) is recognised as being 2342a known blob type. 2343 2344When adding new entry types you are encouraged to use subnodes to provide the 2345data for processing, unless the `content` approach is more suitable. Consider 2346whether the input entries are contained within (or consumed by) the entry, vs 2347just being 'referenced' by the entry. In the latter case, the `content` approach 2348makes more sense. Ad-hoc properties and other methods of obtaining data are 2349discouraged, since it adds to confusion for users. 2350 2351History / Credits 2352----------------- 2353 2354Binman takes a lot of inspiration from a Chrome OS tool called 2355'cros_bundle_firmware', which I wrote some years ago. That tool was based on 2356a reasonably simple and sound design but has expanded greatly over the 2357years. In particular its handling of x86 images is convoluted. 2358 2359Quite a few lessons have been learned which are hopefully applied here. 2360 2361 2362Design notes 2363------------ 2364 2365On the face of it, a tool to create firmware images should be fairly simple: 2366just find all the input binaries and place them at the right place in the 2367image. The difficulty comes from the wide variety of input types (simple 2368flat binaries containing code, packaged data with various headers), packing 2369requirments (alignment, spacing, device boundaries) and other required 2370features such as hierarchical images. 2371 2372The design challenge is to make it easy to create simple images, while 2373allowing the more complex cases to be supported. For example, for most 2374images we don't much care exactly where each binary ends up, so we should 2375not have to specify that unnecessarily. 2376 2377New entry types should aim to provide simple usage where possible. If new 2378core features are needed, they can be added in the Entry base class. 2379 2380 2381To do 2382----- 2383 2384Some ideas: 2385 2386- Use of-platdata to make the information available to code that is unable 2387 to use device tree (such as a very small SPL image). For now, limited info is 2388 available via linker symbols 2389- Allow easy building of images by specifying just the board name 2390- Support building an image for a board (-b) more completely, with a 2391 configurable build directory 2392- Detect invalid properties in nodes 2393- Sort the fdtmap by offset 2394- Output temporary files to a different directory 2395- Rationalise the fdt, fdt_util and pylibfdt modules which currently have some 2396 overlapping and confusing functionality 2397- Update the fdt library to use a better format for Prop.value (the current one 2398 is useful for dtoc but not much else) 2399- Figure out how to make Fdt support changing the node order, so that 2400 Node.AddSubnode() can support adding a node before another, existing node. 2401 Perhaps it should completely regenerate the flat tree? 2402- Support images which depend on each other 2403 2404-- 2405Simon Glass <sjg@chromium.org> 24067/7/2016 2407 2408.. _ConcurrentFutures: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor 2409