Binman is designed primarily for use with U-Boot and associated binaries such
as ARM Trusted Firmware, but it is suitable for use with other projects, such
as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS,
-vblocks and and the like.
+vblocks and the like.
Binman provides a way to process binaries before they are included, by adding a
Python plug-in.
- binary - an input binary that goes into the image
+Installation
+------------
+
+You can install binman using::
+
+ pip install binary-manager
+
+The name is chosen since binman conflicts with an existing package.
+
+If you are using binman within the U-Boot tree, it may be easiest to add a
+symlink from your local `~/.bin` directory to `/path/to/tools/binman/binman`.
+
+
Relationship to FIT
-------------------
For U-Boot, binman should not be used to create ad-hoc images in place of
FIT.
+Note that binman can itself create a FIT. This helps to move mkimage
+invocations out of the Makefile and into binman image descriptions. It also
+helps by removing the need for ad-hoc tools like `make_fit_atf.py`.
+
Relationship to mkimage
-----------------------
the boundaries between building input files (mkimage) and packaging then
into a final image (binman).
+Note that binman can itself invoke mkimage. This helps to move mkimage
+invocations out of the Makefile and into binman image descriptions.
+
Using binman
============
Binman is intended to replace the last step. The U-Boot build system builds
u-boot.bin and sunxi-spl.bin. Binman can then take over creation of
-sunxi-spl.bin (by calling mksunxiboot, or hopefully one day mkimage). In any
-case, it would then create the image from the component parts.
+sunxi-spl.bin by calling mksunxiboot or mkimage. In any case, it would then
+create the image from the component parts.
This simplifies the U-Boot Makefile somewhat, since various pieces of logic
can be replaced by a call to binman.
+Invoking binman within U-Boot
+-----------------------------
+
+Within U-Boot, binman is invoked by the build system, i.e. when you type 'make'
+or use buildman to build U-Boot. There is no need to run binman independently
+during development. Everything happens automatically and is set up for your
+SoC or board so that binman produced the right things.
+
+The general policy is that the Makefile builds all the binaries in INPUTS-y
+(the 'inputs' rule), then binman is run to produce the final images (the 'all'
+rule).
+
+There should be only one invocation of binman in Makefile, the very last step
+that pulls everything together. At present there are some arch-specific
+invocations as well, but these should be dropped when those architectures are
+converted to use binman properly.
+
+As above, the term 'binary' is used for something in INPUTS-y and 'image' is
+used for the things that binman creates. So the binaries are inputs to the
+image(s) and it is the image that is actually loaded on the board.
+
+Again, at present, there are a number of things created in Makefile which should
+be done by binman (when we get around to it), like `u-boot-ivt.img`,
+`lpc32xx-spl.img`, `u-boot-with-nand-spl.imx`, `u-boot-spl-padx4.sfp` and
+`u-boot-mtk.bin`, just to pick on a few. When completed this will remove about
+400 lines from `Makefile`.
+
+Since binman is invoked only once, it must of course create all the images that
+are needed, in that one invocation. It does this by working through the image
+descriptions one by one, collecting the input binaries, processing them as
+needed and producing the final images.
+
+The same binaries may be used by multiple images. For example binman may be used
+to produce an SD-card image and a SPI-flash image. In this case the binaries
+going into the process are the same, but binman produces slightly different
+images in each case.
+
+For some SoCs, U-Boot is not the only project that produces the necessary
+binaries. For example, ARM Trusted Firmware (ATF) is a project that produces
+binaries which must be incorporate, such as `bl31.elf` or `bl31.bin`. For this
+to work you must have built ATF before you build U-Boot and you must tell U-Boot
+where to find the bl31 image, using the BL31 environment variable.
+
+How do you know how to incorporate ATF? It is handled by the atf-bl31 entry type
+(etype). An etype is an implementation of reading a binary into binman, in this
+case the `bl31.bin` file. When you build U-Boot but do not set the BL31
+environment variable, binman provides a help message, which comes from
+`missing-blob-help`::
+
+ See the documentation for your board. You may need to build ARM Trusted
+ Firmware and build with BL31=/path/to/bl31.bin
+
+The mechanism by which binman is advised of this is also in the Makefile. See
+the `-a atf-bl31-path=${BL31}` piece in `cmd_binman`. This tells binman to
+set the EntryArg `atf-bl31-path` to the value of the `BL31` environment
+variable. Within binman, this EntryArg is picked up by the `Entry_atf_bl31`
+etype. An EntryArg is simply an argument to the entry. The `atf-bl31-path`
+name is documented in :ref:`etype_atf_bl31`.
+
+Taking this a little further, when binman is used to create a FIT, it supports
+using an ELF file, e.g. `bl31.elf` and splitting it into separate pieces (with
+`fit,operation = "split-elf"`), each with its own load address.
+
+
+Invoking binman outside U-Boot
+------------------------------
+
+While binman is invoked from within the U-Boot build system, it is also possible
+to invoke it separately. This is typically used in a production build system,
+where signing is completed (with real keys) and any missing binaries are
+provided.
+
+For example, for build testing there is no need to provide a real signature,
+nor is there any need to provide a real ATF BL31 binary (for example). These can
+be added later by invoking binman again, providing all the required inputs
+from the first time, plus any that were missing or placeholders.
+
+So in practice binman is often used twice:
+
+- once within the U-Boot build system, for development and testing
+- again outside U-Boot to assembly and final production images
+
+While the same input binaries are used in each case, you will of course you will
+need to create your own binman command line, similar to that in `cmd_binman` in
+the Makefile. You may find the -I and --toolpath options useful. The
+device tree file is provided to binman in binary form, so there is no need to
+have access to the original `.dts` sources.
+
+
+Assembling the image description
+--------------------------------
+
+Since binman uses the device tree for its image description, you can use the
+same files that describe your board's hardware to describe how the image is
+assembled. Typically the images description is in a common file used by all
+boards with a particular SoC (e.g. `imx8mp-u-boot.dtsi`).
+
+Where a particular boards needs to make changes, it can override properties in
+the SoC file, just as it would for any other device tree property. It can also
+add a image that is specific to the board.
+
+Another way to control the image description to make use of CONFIG options in
+the description. For example, if the start offset of a particular entry varies
+by board, you can add a Kconfig for that and reference it in the description::
+
+ u-boot-spl {
+ };
+
+ fit {
+ offset = <CONFIG_SPL_PAD_TO>;
+ ...
+ };
+
+The SoC can provide a default value but boards can override that as needed and
+binman will take care of it.
+
+It is even possible to control which entries appear in the image, by using the
+C preprocessor::
+
+ #ifdef CONFIG_HAVE_MRC
+ intel-mrc {
+ offset = <CFG_X86_MRC_ADDR>;
+ };
+ #endif
+
+Only boards which enable `HAVE_MRC` will include this entry.
+
+Obviously a similar approach can be used to control which images are produced,
+with a Kconfig option to enable a SPI image, for example. However there is
+generally no harm in producing an image that is not used. If a board uses MMC
+but not SPI, but the SoC supports booting from both, then both images can be
+produced, with only on or other being used by particular boards. This can help
+reduce the need for having multiple defconfig targets for a board where the
+only difference is the boot media, enabling / disabling secure boot, etc.
+
+Of course you can use the device tree itself to pass any board-specific
+information that is needed by U-Boot at runtime (see binman_syms_ for how to
+make binman insert these values directly into executables like SPL).
+
+There is one more way this can be done: with individual .dtsi files for each
+image supported by the SoC. Then the board `.dts` file can include the ones it
+wants. This is not recommended, since it is likely to be difficult to maintain
+and harder to understand the relationship between the different boards.
+
+
+Producing images for multiple boards
+------------------------------------
+
+When invoked within U-Boot, binman only builds a single set of images, for
+the chosen board. This is set by the `CONFIG_DEFAULT_DEVICE_TREE` option.
+
+However, U-Boot generally builds all the device tree files associated with an
+SoC. These are written to the (e.g. for ARM) `arch/arm/dts` directory. Each of
+these contains the full binman description for that board. Often the best
+approach is to build a single image that includes all these device tree binaries
+and allow SPL to select the correct one on boot.
+
+However, it is also possible to build separate images for each board, simply by
+invoking binman multiple times, once for each device tree file, using a
+different output directory. This will produce one set of images for each board.
+
+
Example use of binman for x86
-----------------------------
Installing binman
-----------------
-First install prerequisites, e.g::
+First install prerequisites, e.g:
+
+.. code-block:: bash
sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \
liblz4-tool
You can run binman directly if you put it on your PATH. But if you want to
-install into your `~/.local` Python directory, use::
+install into your `~/.local` Python directory, use:
+
+.. code-block:: bash
pip install tools/patman tools/dtoc tools/binman
Note that binman makes use of libraries from patman and dtoc, which is why these
need to be installed. Also you need `libfdt` and `pylibfdt` which can be
-installed like this::
+installed like this:
+
+.. code-block:: bash
git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git
cd dtc
This installs the `libfdt.so` library into `~/lib` so you can use
`LD_LIBRARY_PATH=~/lib` when running binman. If you want to install it in the
-system-library directory, replace the last line with::
+system-library directory, replace the last line with:
+
+.. code-block:: bash
make NO_PYTHON=1 PREFIX=/ install
Running binman
--------------
-Type::
+Type:
+
+.. code-block:: bash
+ make NO_PYTHON=1 PREFIX=/ install
binman build -b <board_name>
to build an image for a board. The board name is the same name used when
configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox').
Binman assumes that the input files for the build are in ../b/<board_name>.
-Or you can specify this explicitly::
+Or you can specify this explicitly:
+
+.. code-block:: bash
+ make NO_PYTHON=1 PREFIX=/ install
binman build -I <build_path>
where <build_path> is the build directory containing the output of the U-Boot
You can use other, more specific CONFIG options - see 'Automatic .dtsi
inclusion' below.
+.. _binman_syms:
Access to binman entry offsets at run time (symbols)
----------------------------------------------------
when SPL is finished.
Binman allows you to declare symbols in the SPL image which are filled in
-with their correct values during the build. For example::
+with their correct values during the build. For example:
+
+.. code-block:: c
binman_sym_declare(ulong, u_boot_any, image_pos);
declares a ulong value which will be assigned to the image-pos of any U-Boot
image (u-boot.bin, u-boot.img, u-boot-nodtb.bin) that is present in the image.
-You can access this value with something like::
+You can access this value with something like:
+
+.. code-block:: c
ulong u_boot_offset = binman_sym(ulong, u_boot_any, image_pos);
since it is assumed that images are XIP and the offsets already include the
address.
+While U-Boot's symbol updating is handled automatically by the u-boot-spl
+entry type (and others), it is possible to use this feature with any blob. To
+do this, add a `write-symbols` (boolean) property to the node, set the ELF
+filename using `elf-filename` and set 'elf-base-sym' to the base symbol for the
+start of the binary image (this defaults to `__image_copy_start` which is what
+U-Boot uses). See `testBlobSymbol()` for an example.
+
+.. _binman_fdt:
Access to binman entry offsets at run time (fdt)
------------------------------------------------
of the image) can be used to point to the FDT map. See fdtmap and image-header
entries for more information.
-
Map files
---------
this size. If this is not provided, it will be set to the size of the
contents.
+min-size:
+ Sets the minimum size of the entry. This size includes explicit padding
+ ('pad-before' and 'pad-after'), but not padding added to meet alignment
+ requirements. While this does not affect the contents of the entry within
+ binman itself (the padding is performed only when its parent section is
+ assembled), the end result will be that the entry ends with the padding
+ bytes, so may grow. Defaults to 0.
+
pad-before:
Padding before the contents of the entry. Normally this is 0, meaning
that the contents start at the beginning of the entry. This can be used
for each entry. This makes it easy to find out exactly where the entry
ended up in the image, regardless of parent sections, etc.
-expand-size:
- Expand the size of this entry to fit available space. This space is only
+extend-size:
+ Extend the size of this entry to fit available space. This space is only
limited by the size of the image/section and the position of the next
entry.
`no-expanded` property disables this just for a single entry. Put the
`no-expanded` boolean property in the node to select this behaviour.
+optional:
+ External blobs are normally required to be present for the image to be
+ built (but see `External blobs`_). This properly allows an entry to be
+ optional, so that when it is cannot be found, this problem is ignored and
+ an empty file is used for this blob. This should be used only when the blob
+ is entirely optional and is not needed for correct operation of the image.
+ Note that missing, optional blobs do not produce a non-zero exit code from
+ binman, although it does show a warning about the missing external blob.
+
+insert-template:
+ This is not strictly speaking an entry property, since it is processed early
+ in Binman before the entries are read. It is a list of phandles of nodes to
+ include in the current (target) node. For each node, its subnodes and their
+ properties are brought into the target node. See Templates_ below for
+ more information.
+
The attributes supported for images and sections are described below. Several
are similar to those for entries.
skip-at-start:
This property specifies the entry offset of the first entry.
- For PowerPC mpc85xx based CPU, CONFIG_SYS_TEXT_BASE is the entry
+ For PowerPC mpc85xx based CPU, CONFIG_TEXT_BASE is the entry
offset of the first entry. It can be 0xeff40000 or 0xfff40000 for
nor flash boot, 0x201000 for sd boot etc.
- 'end-at-4gb' property is not applicable where CONFIG_SYS_TEXT_BASE +
+ 'end-at-4gb' property is not applicable where CONFIG_TEXT_BASE +
Image size != 4gb.
align-default:
This means that each section must specify its own default alignment, if
required.
+symlink:
+ Adds a symlink to the image with string given in the symlink property.
+
+overlap:
+ Indicates that this entry overlaps with others in the same section. These
+ entries should appear at the end of the section. Overlapping entries are not
+ packed with other entries, but their contents are written over other entries
+ in the section. Overlapping entries must have an explicit offset and size.
+
+write-symbols:
+ Indicates that the blob should be updated with symbol values calculated by
+ binman. This is automatic for certain entry types, e.g. `u-boot-spl`. See
+ binman_syms_ for more information.
+
+no-write-symbols:
+ Disables symbol writing for this entry. This can be used in entry types
+ where symbol writing is automatic. For example, if `u-boot-spl` refers to
+ the `u_boot_any_image_pos` symbol but U-Boot is not available in the image
+ containing SPL, this can be used to disable the writing. Quite likely this
+ indicates a bug in your setup.
+
+elf-filename:
+ Sets the file name of a blob's associated ELF file. For example, if the
+ blob is `zephyr.bin` then the ELF file may be `zephyr.elf`. This allows
+ binman to locate symbols and understand the structure of the blob. See
+ binman_syms_ for more information.
+
+elf-base-sym:
+ Sets the name of the ELF symbol that points to the start of a blob. For
+ U-Boot this is `__image_copy_start` and that is the default used by binman
+ if this property is missing. For other projects, a difference symbol may be
+ needed. Add this symbol to the properties for the blob so that symbols can
+ be read correctly. See binman_syms_ for more information.
+
+offset-from-elf:
+ Sets the offset of an entry based on a symbol value in an another entry.
+ The format is <&phandle>, "sym_name", <offset> where phandle is the entry
+ containing the blob (with associated ELF file providing symbols), <sym_name>
+ is the symbol to lookup (relative to elf-base-sym) and <offset> is an offset
+ to add to that value.
+
+preserve:
+ Indicates that this entry should be preserved by any firmware updates. This
+ flag should be checked by the updater when it is deciding which entries to
+ update. This flag is normally attached to sections but can be attached to
+ a single entry in a section if the updater supports it. Not that binman
+ itself has no control over the updater's behaviour, so this is just a
+ signal. It is not enforced by binman.
+
Examples of the above options can be found in the tests. See the
tools/binman/test directory.
renamed to 'ro-u-boot' and 'rw-u-boot'. This can be useful to
distinguish binaries with otherwise identical names.
+filename:
+ This allows the contents of the section to be written to a file in the
+ output directory. This can sometimes be useful to use the data in one
+ section in different image, since there is currently no way to share data
+ beteen images other than through files.
Image Properties
----------------
image description to be stored in the FDT and fdtmap.
+Image dependencies
+------------------
+
+Binman does not currently support images that depend on each other. For example,
+if one image creates `fred.bin` and then the next uses this `fred.bin` to
+produce a final `image.bin`, then the behaviour is undefined. It may work, or it
+may produce an error about `fred.bin` being missing, or it may use a version of
+`fred.bin` from a previous run.
+
+Often this can be handled by incorporating the dependency into the second
+image. For example, instead of::
+
+ binman {
+ multiple-images;
+
+ fred {
+ u-boot {
+ };
+ fill {
+ size = <0x100>;
+ };
+ };
+
+ image {
+ blob {
+ filename = "fred.bin";
+ };
+ u-boot-spl {
+ };
+ };
+
+you can do this::
+
+ binman {
+ image {
+ fred {
+ type = "section";
+ u-boot {
+ };
+ fill {
+ size = <0x100>;
+ };
+ };
+ u-boot-spl {
+ };
+ };
+
+
+
Hashing Entries
---------------
U-Boot executable and can be updated separately by binman as needed. It can be
disabled with the --no-expanded flag if required.
-The same applies for u-boot-spl and u-boot-spl. In those cases, the expansion
+The same applies for u-boot-spl and u-boot-tpl. In those cases, the expansion
includes the BSS padding, so for example::
spl {
entry args are provided by the U-Boot Makefile.
+Optional entries
+----------------
+
+Some entries need to exist only if certain conditions are met. For example, an
+entry may want to appear in the image only if a file has a particular format.
+Obviously the entry must exist in the image description for it to be processed
+at all, so a way needs to be found to have the entry remove itself.
+
+To handle this, when entry.ObtainContents() is called, the entry can call
+entry.mark_absent() to mark itself as absent, passing a suitable message as the
+reason.
+
+Any absent entries are dropped immediately after ObtainContents() has been
+called on all entries.
+
+It is not possible for an entry to mark itself absent at any other point in the
+processing. It must happen in the ObtainContents() method.
+
+The effect is as if the entry had never been present at all, since the image
+is packed without it and it disappears from the list of entries.
+
+
Compression
-----------
board. This can be done by using #include to bring in a common file. Another
approach supported by the U-Boot build system is to automatically include
a common header. You can then put the binman node (and anything else that is
-specific to U-Boot, such as u-boot,dm-pre-reloc properies) in that header
-file.
+specific to U-Boot, such as bootph-all properies) in that header file.
Binman will search for the following files in arch/<arch>/dts::
arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi"
+Templates
+=========
+
+Sometimes multiple images need to be created which have all have a common
+part. For example, a board may generate SPI and eMMC images which both include
+a FIT. Since the FIT includes many entries, it is tedious to repeat them twice
+in the image description.
+
+Templates provide a simple way to handle this::
+
+ binman {
+ multiple-images;
+ common_part: template-1 {
+ some-property;
+ fit {
+ ... lots of entries in here
+ };
+
+ text {
+ text = "base image";
+ };
+ };
+
+ spi-image {
+ filename = "image-spi.bin";
+ insert-template = <&fit>;
+
+ /* things specific to SPI follow */
+ footer {
+ ];
+
+ text {
+ text = "SPI image";
+ };
+ };
+
+ mmc-image {
+ filename = "image-mmc.bin";
+ insert-template = <&fit>;
+
+ /* things specific to MMC follow */
+ footer {
+ ];
+
+ text {
+ text = "MMC image";
+ };
+ };
+ };
+
+The template node name must start with 'template', so it is not considered to be
+an image itself.
+
+The mechanism is very simple. For each phandle in the 'insert-templates'
+property, the source node is looked up. Then the subnodes of that source node
+are copied into the target node, i.e. the one containing the `insert-template`
+property.
+
+If the target node has a node with the same name as a template, its properties
+override corresponding properties in the template. This allows the template to
+be uses as a base, with the node providing updates to the properties as needed.
+The overriding happens recursively.
+
+Template nodes appear first in each node that they are inserted into and
+ordering of template nodes is preserved. Other nodes come afterwards. If a
+template node also appears in the target node, then the template node sets the
+order. Thus the template can be used to set the ordering, even if the target
+node provides all the properties. In the above example, `fit` and `text` appear
+first in the `spi-image` and `mmc-image` images, followed by `footer`.
+
+Where there are multiple template nodes, they are inserted in that order. so
+the first template node appears first, then the second.
+
+Properties in the template node are inserted into the destination node if they
+do not exist there. In the example above, `some-property` is added to each of
+`spi-image` and `mmc-image`.
+
+Note that template nodes are removed from the binman description after
+processing and before binman builds the image descriptions.
+
+The initial devicetree produced by the templating process is written to the
+`u-boot.dtb.tmpl1` file. This can be useful to see what is going on if there is
+a failure before the final `u-boot.dtb.out` file is written. A second
+`u-boot.dtb.tmpl2` file is written when the templates themselves are removed.
+
+Dealing with phandles
+---------------------
+
+Templates can contain phandles and these are copied to the destination node.
+However this should be used with care, since if a template is instantiated twice
+then the phandle will be copied twice, resulting in a devicetree with duplicate
+phandles, i.e. the same phandle used by two different nodes. Binman detects this
+situation and produces an error, for example::
+
+ Duplicate phandle 1 in nodes /binman/image/fit/images/atf/atf-bl31 and
+ /binman/image-2/fit/images/atf/atf-bl31
+
+In this case an atf-bl31 node containing a phandle has been copied into two
+different target nodes, resulting in the same phandle for each. See
+testTemplatePhandleDup() for the test case.
+
+The solution is typically to put the phandles in the corresponding target nodes
+(one for each) and remove the phandle from the template.
+
Updating an ELF file
====================
$ binman replace -i image.bin "*u-boot*" -I indir
+It is possible to replace whole sections as well, but in that case any
+information about entries within the section may become outdated. This is
+because Binman cannot know whether things have moved around or resized within
+the section, once you have updated its data.
+
+Technical note: With 'allow-repack', Binman writes information about the
+original offset and size properties of each entry, if any were specified, in
+the 'orig-offset' and 'orig-size' properties. This allows Binman to distinguish
+between an entry which ended up being packed at an offset (or assigned a size)
+and an entry which had a particular offset / size requested in the Binman
+configuration. Where are particular offset / size was requested, this is treated
+as set in stone, so Binman will ensure it doesn't change. Without this feature,
+repacking an entry might cause it to disobey the original constraints provided
+when it was created.
+
+
+Signing FIT container with private key in an image
+--------------------------------------------------
+
+You can sign FIT container with private key in your image.
+For example::
+
+ $ binman sign -i image.bin -k privatekey -a sha256,rsa4096 fit
+
+binman will extract FIT container, sign and replace it immediately.
+
+If you want to sign and replace FIT container in place::
+
+ $ binman sign -i image.bin -k privatekey -a sha256,rsa4096 -f fit.fit fit
+
+which will sign FIT container with private key and replace it immediately
+inside your image.
+
+.. _`BinmanLogging`:
Logging
-------
You can use BINMAN_VERBOSE=5 (for example) when building to select this.
+Bintools
+========
+
+`Bintool` is the name binman gives to a binary tool which it uses to create and
+manipulate binaries that binman cannot handle itself. Bintools are often
+necessary since Binman only supports a subset of the available file formats
+natively.
+
+Many SoC vendors invent ways to load code into their SoC using new file formats,
+sometimes changing the format with successive SoC generations. Sometimes the
+tool is available as Open Source. Sometimes it is a pre-compiled binary that
+must be downloaded from the vendor's website. Sometimes it is available in
+source form but difficult or slow to build.
+
+Even for images that use bintools, binman still assembles the image from its
+image description. It may handle parts of the image natively and part with
+various bintools.
+
+Binman relies on these tools so provides various features to manage them:
+
+- Determining whether the tool is currently installed
+- Downloading or building the tool
+- Determining the version of the tool that is installed
+- Deciding which tools are needed to build an image
+
+The Bintool class is an interface to the tool, a thin level of abstration, using
+Python functions to run the tool for each purpose (e.g. creating a new
+structure, adding a file to an existing structure) rather than just lists of
+string arguments.
+
+As with external blobs, bintools (which are like 'external' tools) can be
+missing. When building an image requires a bintool and it is not installed,
+binman detects this and reports the problem, but continues to build an image.
+This is useful in CI systems which want to check that everything is correct but
+don't have access to the bintools.
+
+To make this work, all calls to bintools (e.g. with Bintool.run_cmd()) must cope
+with the tool being missing, i.e. when None is returned, by:
+
+- Calling self.record_missing_bintool()
+- Setting up some fake contents so binman can continue
+
+Of course the image will not work, but binman reports which bintools are needed
+and also provide a way to fetch them.
+
+To see the available bintools, use::
+
+ binman tool --list
+
+To fetch tools which are missing, use::
+
+ binman tool --fetch missing
+
+You can also use `--fetch all` to fetch all tools or `--fetch <tool>` to fetch
+a particular tool. Some tools are built from source code, in which case you will
+need to have at least the `build-essential` and `git` packages installed.
+
+Tools are fetched into the `~/.binman-tools` directory. This directory is
+automatically added to the toolpath so there is no need to use `--toolpath` to
+specify it. If you want to use these tools outside binman, you may want to
+add this directory to your `PATH`. For example, if you use bash, add this to
+the end of `.bashrc`::
+
+ PATH="$HOME/.binman-tools:$PATH"
+
+To select a custom directory, use the `--tooldir` option.
+
+Bintool Documentation
+=====================
+
+To provide details on the various bintools supported by binman, bintools.rst is
+generated from the source code using:
+
+ binman bintool-docs >tools/binman/bintools.rst
+
+.. toctree::
+ :maxdepth: 2
+
+ bintools
+
+Binman commands and arguments
+=============================
+
+Usage::
+
+ binman [-h] [-B BUILD_DIR] [-D] [--tooldir TOOLDIR] [-H]
+ [--toolpath TOOLPATH] [-T THREADS] [--test-section-timeout]
+ [-v VERBOSITY] [-V]
+ {build,bintool-docs,entry-docs,ls,extract,replace,test,tool} ...
+
+Binman provides the following commands:
+
+- **build** - build images
+- **bintools-docs** - generate documentation about bintools
+- **entry-docs** - generate documentation about entry types
+- **ls** - list an image
+- **extract** - extract files from an image
+- **replace** - replace one or more entries in an image
+- **test** - run tests
+- **tool** - manage bintools
+
+Options:
+
+-h, --help
+ Show help message and exit
+
+-B BUILD_DIR, --build-dir BUILD_DIR
+ Directory containing the build output
+
+-D, --debug
+ Enabling debugging (provides a full traceback on error)
+
+--tooldir TOOLDIR Set the directory to store tools
+
+-H, --full-help
+ Display the README file
+
+--toolpath TOOLPATH
+ Add a path to the list of directories containing tools
+
+-T THREADS, --threads THREADS
+ Number of threads to use (0=single-thread). Note that -T0 is useful for
+ debugging since everything runs in one thread.
+
+-v VERBOSITY, --verbosity VERBOSITY
+ Control verbosity: 0=silent, 1=warnings, 2=notices, 3=info, 4=detail,
+ 5=debug
+
+-V, --version
+ Show the binman version
+
+Test options:
+
+--test-section-timeout
+ Use a zero timeout for section multi-threading (for testing)
+
+Commands are described below.
+
+binman build
+------------
+
+This builds one or more images using the provided image description.
+
+Usage::
+
+ binman build [-h] [-a ENTRY_ARG] [-b BOARD] [-d DT] [--fake-dtb]
+ [--fake-ext-blobs] [--force-missing-bintools FORCE_MISSING_BINTOOLS]
+ [-i IMAGE] [-I INDIR] [-m] [-M] [-n] [-O OUTDIR] [-p] [-u]
+ [--update-fdt-in-elf UPDATE_FDT_IN_ELF] [-W]
+
+Options:
+
+-h, --help
+ Show help message and exit
+
+-a ENTRY_ARG, --entry-arg ENTRY_ARG
+ Set argument value `arg=value`. See
+ `Passing command-line arguments to entries`_.
+
+-b BOARD, --board BOARD
+ Board name to build. This can be used instead of `-d`, in which case the
+ file `u-boot.dtb` is used, within the build directory's board subdirectory.
+
+-d DT, --dt DT
+ Configuration file (.dtb) to use. This must have a top-level node called
+ `binman`. See `Image description format`_.
+
+-i IMAGE, --image IMAGE
+ Image filename to build (if not specified, build all)
+
+-I INDIR, --indir INDIR
+ Add a path to the list of directories to use for input files. This can be
+ specified multiple times to add more than one path.
+
+-m, --map
+ Output a map file for each image. See `Map files`_.
+
+-M, --allow-missing
+ Allow external blobs and bintools to be missing. See `External blobs`_.
+
+-n, --no-expanded
+ Don't use 'expanded' versions of entries where available; normally 'u-boot'
+ becomes 'u-boot-expanded', for example. See `Expanded entries`_.
+
+-O OUTDIR, --outdir OUTDIR
+ Path to directory to use for intermediate and output files
+
+-p, --preserve
+ Preserve temporary output directory even if option -O is not given
+
+-u, --update-fdt
+ Update the binman node with offset/size info. See
+ `Access to binman entry offsets at run time (fdt)`_.
+
+--update-fdt-in-elf UPDATE_FDT_IN_ELF
+ Update an ELF file with the output dtb. The argument is a string consisting
+ of four parts, separated by commas. See `Updating an ELF file`_.
+
+-W, --ignore-missing
+ Return success even if there are missing blobs/bintools (requires -M)
+
+Options used only for testing:
+
+--fake-dtb
+ Use fake device tree contents
+
+--fake-ext-blobs
+ Create fake ext blobs with dummy content
+
+--force-missing-bintools FORCE_MISSING_BINTOOLS
+ Comma-separated list of bintools to consider missing
+
+binman bintool-docs
+-------------------
+
+Usage::
+
+ binman bintool-docs [-h]
+
+This outputs documentation for the bintools in rST format. See
+`Bintool Documentation`_.
+
+binman entry-docs
+-----------------
+
+Usage::
+
+ binman entry-docs [-h]
+
+This outputs documentation for the entry types in rST format. See
+`Entry Documentation`_.
+
+binman ls
+---------
+
+Usage::
+
+ binman ls [-h] -i IMAGE [paths ...]
+
+Positional arguments:
+
+paths
+ Paths within file to list (wildcard)
+
+Pptions:
+
+-h, --help
+ show help message and exit
+
+-i IMAGE, --image IMAGE
+ Image filename to list
+
+This lists an image, showing its contents. See `Listing images`_.
+
+binman extract
+--------------
+
+Usage::
+
+ binman extract [-h] [-F FORMAT] -i IMAGE [-f FILENAME] [-O OUTDIR] [-U]
+ [paths ...]
+
+Positional arguments:
+
+Paths
+ Paths within file to extract (wildcard)
+
+Options:
+
+-h, --help
+ show help message and exit
+
+-F FORMAT, --format FORMAT
+ Select an alternative format for extracted data
+
+-i IMAGE, --image IMAGE
+ Image filename to extract
+
+-f FILENAME, --filename FILENAME
+ Output filename to write to
+
+-O OUTDIR, --outdir OUTDIR
+ Path to directory to use for output files
+
+-U, --uncompressed
+ Output raw uncompressed data for compressed entries
+
+This extracts the contents of entries from an image. See
+`Extracting files from images`_.
+
+binman replace
+--------------
+
+Usage::
+
+ binman replace [-h] [-C] -i IMAGE [-f FILENAME] [-F] [-I INDIR] [-m]
+ [paths ...]
+
+Positional arguments:
+
+paths
+ Paths within file to replace (wildcard)
+
+Options:
+
+-h, --help
+ show help message and exit
+
+-C, --compressed
+ Input data is already compressed if needed for the entry
+
+-i IMAGE, --image IMAGE
+ Image filename to update
+
+-f FILENAME, --filename FILENAME
+ Input filename to read from
+
+-F, --fix-size
+ Don't allow entries to be resized
+
+-I INDIR, --indir INDIR
+ Path to directory to use for input files
+
+-m, --map
+ Output a map file for the updated image
+
+-O OUTDIR, --outdir OUTDIR
+ Path to directory to use for intermediate and output files
+
+-p, --preserve
+ Preserve temporary output directory even if option -O is not given
+
+This replaces one or more entries in an existing image. See
+`Replacing files in an image`_.
+
+binman test
+-----------
+
+Usage::
+
+ binman test [-h] [-P PROCESSES] [-T] [-X] [tests ...]
+
+Positional arguments:
+
+tests
+ Test names to run (omit for all)
+
+Options:
+
+-h, --help
+ show help message and exit
+
+-P PROCESSES, --processes PROCESSES
+ set number of processes to use for running tests. This defaults to the
+ number of CPUs on the machine
+
+-T, --test-coverage
+ run tests and check for 100% coverage
+
+-X, --test-preserve-dirs
+ Preserve and display test-created input directories; also preserve the
+ output directory if a single test is run (pass test name at the end of the
+ command line
+
+binman sign
+-----------
+
+Usage::
+
+ binman sign [-h] -a ALGO [-f FILE] -i IMAGE -k KEY [paths ...]
+
+positional arguments:
+
+paths
+ Paths within file to sign (wildcard)
+
+options:
+
+-h, --help
+ show this help message and exit
+
+-a ALGO, --algo ALGO
+ Hash algorithm e.g. sha256,rsa4096
+
+-f FILE, --file FILE
+ Input filename to sign
+
+-i IMAGE, --image IMAGE
+ Image filename to update
+
+-k KEY, --key KEY
+ Private key file for signing
+
+binman tool
+-----------
+
+Usage::
+
+ binman tool [-h] [-l] [-f] [bintools ...]
+
+Positional arguments:
+
+bintools
+ Bintools to process
+
+Options:
+
+-h, --help
+ show help message and exit
+
+-l, --list
+ List all known bintools
+
+-f, --fetch
+ Fetch a bintool from a known location. Use `all` to fetch all and `missing`
+ to fetch any missing tools.
+
+
Technical details
=================
Note: for sections, this also checks that the entries do not overlap, nor extend
outside the section. If the section does not have a defined size, the size is
-set large enough to hold all the entries.
+set large enough to hold all the entries. For entries that are explicitly marked
+as overlapping, this check is skipped.
6. SetImagePos() - sets the image position of every entry. This is the absolute
position 'image-pos', as opposed to 'offset' which is relative to the containing
final step.
+.. _`External tools`:
+
External tools
--------------
BINMAN_TOOLPATHS="/tools/g12a /tools/tegra" binman ...
+.. _`External blobs`:
+
External blobs
--------------
odroid-c4/build/board/hardkernel/odroidc4/firmware \
odroid-c4/build/scp_task" binman ...
+Note that binman fails with exit code 103 when there are missing blobs. If you
+wish binman to continue anyway, you can pass `-W` to binman.
+
+
Code coverage
-------------
$ sudo apt-get install python-coverage python3-coverage python-pytest
+Exit status
+-----------
+
+Binman produces the following exit codes:
+
+0
+ Success
+
+1
+ Any sort of failure - see output for more details
+
+103
+ There are missing external blobs or bintools. This is only returned if
+ -M is passed to binman, otherwise missing blobs return an exit status of 1.
+ Note, if -W is passed as well as -M, then this is converted into a warning
+ and will return an exit status of 0 instead.
+
+
+U-Boot environment variables for binman
+---------------------------------------
+
+The U-Boot Makefile supports various environment variables to control binman.
+All of these are set within the Makefile and result in passing various
+environment variables (or make flags) to binman:
+
+BINMAN_DEBUG
+ Enables backtrace debugging by adding a `-D` argument. See
+ :ref:`BinmanLogging`.
+
+BINMAN_INDIRS
+ Sets the search path for input files used by binman by adding one or more
+ `-I` arguments. See :ref:`External blobs`.
+
+BINMAN_TOOLPATHS
+ Sets the search path for external tool used by binman by adding one or more
+ `--toolpath` arguments. See :ref:`External tools`.
+
+BINMAN_VERBOSE
+ Sets the logging verbosity of binman by adding a `-v` argument. See
+ :ref:`BinmanLogging`.
+
+
+Error messages
+--------------
+
+This section provides some guidance for some of the less obvious error messages
+produced by binman.
+
+
+Expected __bss_size symbol
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Example::
+
+ binman: Node '/binman/u-boot-spl-ddr/u-boot-spl/u-boot-spl-bss-pad':
+ Expected __bss_size symbol in spl/u-boot-spl
+
+This indicates that binman needs the `__bss_size` symbol to be defined in the
+SPL binary, where `spl/u-boot-spl` is the ELF file containing the symbols. The
+symbol tells binman the size of the BSS region, in bytes. It needs this to be
+able to pad the image so that the following entries do not overlap the BSS,
+which would cause them to be overwritte by variable access in SPL.
+
+This symbols is normally defined in the linker script, immediately after
+_bss_start and __bss_end are defined, like this::
+
+ __bss_size = __bss_end - __bss_start;
+
+You may need to add it to your linker script if you get this error.
+
+
Concurrent tests
----------------
difficult. This avoids any use of ThreadPoolExecutor.
+Collecting data for an entry type
+---------------------------------
+
+Some entry types deal with data obtained from others. For example,
+`Entry_mkimage` calls the `mkimage` tool with data from its subnodes::
+
+ mkimage {
+ args = "-n test -T script";
+
+ u-boot-spl {
+ };
+
+ u-boot {
+ };
+ };
+
+This shows mkimage being passed a file consisting of SPL and U-Boot proper. It
+is created by calling `Entry.collect_contents_to_file()`. Note that in this
+case, the data is passed to mkimage for processing but does not appear
+separately in the image. It may not appear at all, depending on what mkimage
+does. The contents of the `mkimage` entry are entirely dependent on the
+processing done by the entry, with the provided subnodes (`u-boot-spl` and
+`u-boot`) simply providing the input data for that processing.
+
+Note that `Entry.collect_contents_to_file()` simply concatenates the data from
+the different entries together, with no control over alignment, etc. Another
+approach is to subclass `Entry_section` so that those features become available,
+such as `size` and `pad-byte`. Then the contents of the entry can be obtained by
+calling `super().BuildSectionData()` in the entry's BuildSectionData()
+implementation to get the input data, then write it to a file and process it
+however is desired.
+
+There are other ways to obtain data also, depending on the situation. If the
+entry type is simply signing data which exists elsewhere in the image, then
+you can use `Entry_collection` as a base class. It lets you use a property
+called `content` which lists the entries containing data to be processed. This
+is used by `Entry_vblock`, for example::
+
+ u_boot: u-boot {
+ };
+
+ vblock {
+ content = <&u_boot &dtb>;
+ keyblock = "firmware.keyblock";
+ signprivate = "firmware_data_key.vbprivk";
+ version = <1>;
+ kernelkey = "kernel_subkey.vbpubk";
+ preamble-flags = <1>;
+ };
+
+ dtb: u-boot-dtb {
+ };
+
+which shows an image containing `u-boot` and `u-boot-dtb`, with the `vblock`
+image collecting their contents to produce input for its signing process,
+without affecting those entries, which still appear in the final image
+untouched.
+
+Another example is where an entry type needs several independent pieces of input
+to function. For example, `Entry_fip` allows a number of different binary blobs
+to be placed in their own individual places in a custom data structure in the
+output image. To make that work you can add subnodes for each of them and call
+`Entry.Create()` on each subnode, as `Entry_fip` does. Then the data for each
+blob can come from any suitable place, such as an `Entry_u_boot` or an
+`Entry_blob` or anything else::
+
+ atf-fip {
+ fip-hdr-flags = /bits/ 64 <0x123>;
+ soc-fw {
+ fip-flags = /bits/ 64 <0x123456789abcdef>;
+ filename = "bl31.bin";
+ };
+
+ u-boot {
+ fip-uuid = [fc 65 13 92 4a 5b 11 ec
+ 94 35 ff 2d 1c fc 79 9c];
+ };
+ };
+
+The `soc-fw` node is a `blob-ext` (i.e. it reads in a named binary file) whereas
+`u-boot` is a normal entry type. This works because `Entry_fip` selects the
+`blob-ext` entry type if the node name (here `soc-fw`) is recognised as being
+a known blob type.
+
+When adding new entry types you are encouraged to use subnodes to provide the
+data for processing, unless the `content` approach is more suitable. Consider
+whether the input entries are contained within (or consumed by) the entry, vs
+just being 'referenced' by the entry. In the latter case, the `content` approach
+makes more sense. Ad-hoc properties and other methods of obtaining data are
+discouraged, since it adds to confusion for users.
+
History / Credits
-----------------
- Detect invalid properties in nodes
- Sort the fdtmap by offset
- Output temporary files to a different directory
+- Rationalise the fdt, fdt_util and pylibfdt modules which currently have some
+ overlapping and confusing functionality
+- Update the fdt library to use a better format for Prop.value (the current one
+ is useful for dtoc but not much else)
+- Figure out how to make Fdt support changing the node order, so that
+ Node.AddSubnode() can support adding a node before another, existing node.
+ Perhaps it should completely regenerate the flat tree?
+- Support images which depend on each other
--
Simon Glass <sjg@chromium.org>
projects / platform / kernel / u-boot.git / blobdiff