grouped together and placed in MMC flash. When the system starts, it must be
able to find these pieces.
-So far U-Boot has not provided a way to handle creating such images in a
-general way. Each SoC does what it needs to build an image, often packing or
-concatenating images in the U-Boot build system.
-
-Binman aims to provide a mechanism for building images, from simple
-SPL + U-Boot combinations, to more complex arrangements with many parts.
+Building firmware should be separate from packaging it. Many of the complexities
+of modern firmware build systems come from trying to do both at once. With
+binman, you build all the pieces that are needed, using whatever assortment of
+projects and build systems are needed, then use binman to stitch everything
+together.
What it does
------------
Binman reads your board's device tree and finds a node which describes the
-required image layout. It uses this to work out what to place where. The
-output file normally contains the device tree, so it is in principle possible
-to read an image and extract its constituent parts.
+required image layout. It uses this to work out what to place where.
+
+Binman provides a mechanism for building images, from simple SPL + U-Boot
+combinations, to more complex arrangements with many parts. It also allows
+users to inspect images, extract and replace binaries within them, repacking if
+needed.
Features
--------
-So far binman is pretty simple. It supports binary blobs, such as 'u-boot',
-'spl' and 'fdt'. It supports empty entries (such as setting to 0xff). It can
-place entries at a fixed location in the image, or fit them together with
-suitable padding and alignment. It provides a way to process binaries before
-they are included, by adding a Python plug-in. The device tree is available
-to U-Boot at run-time so that the images can be interpreted.
+Apart from basic padding, alignment and positioning features, Binman supports
+hierarchical images, compression, hashing and dealing with the binary blobs
+which are a sad trend in open-source firmware at present.
+
+Executable binaries can access the location of other binaries in an image by
+using special linker symbols (zero-overhead but somewhat limited) or by reading
+the devicetree description of the image.
-Binman can update the device tree with the final location of everything when it
-is done. Entry positions can be provided to U-Boot SPL as run-time symbols,
-avoiding device-tree code overhead.
+Binman is designed primarily for use with U-Boot and associated binaries such
+as ARM Trusted Firmware, but it is suitable for use with other projects, such
+as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS,
+vblocks and and the like.
-Binman can also support incorporating filesystems in the image if required.
-For example x86 platforms may use CBFS in some cases.
+Binman provides a way to process binaries before they are included, by adding a
+Python plug-in.
Binman is intended for use with U-Boot but is designed to be general enough
to be useful in other image-packaging situations.
Motivation
----------
-Packaging of firmware is quite a different task from building the various
-parts. In many cases the various binaries which go into the image come from
-separate build systems. For example, ARM Trusted Firmware is used on ARMv8
-devices but is not built in the U-Boot tree. If a Linux kernel is included
-in the firmware image, it is built elsewhere.
+As mentioned above, packaging of firmware is quite a different task from
+building the various parts. In many cases the various binaries which go into
+the image come from separate build systems. For example, ARM Trusted Firmware
+is used on ARMv8 devices but is not built in the U-Boot tree. If a Linux kernel
+is included in the firmware image, it is built elsewhere.
It is of course possible to add more and more build rules to the U-Boot
build system to cover these cases. It can shell out to other Makefiles and
Binman is intended to replace the last step. The U-Boot build system builds
u-boot.bin and sunxi-spl.bin. Binman can then take over creation of
-sunxi-spl.bin (by calling mksunxiboot, or hopefully one day mkimage). In any
-case, it would then create the image from the component parts.
+sunxi-spl.bin by calling mksunxiboot or mkimage. In any case, it would then
+create the image from the component parts.
This simplifies the U-Boot Makefile somewhat, since various pieces of logic
can be replaced by a call to binman.
the configuration of the Intel-format descriptor.
-Running binman
---------------
+Installing binman
+-----------------
First install prerequisites, e.g::
sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \
liblz4-tool
+You can run binman directly if you put it on your PATH. But if you want to
+install into your `~/.local` Python directory, use::
+
+ pip install tools/patman tools/dtoc tools/binman
+
+Note that binman makes use of libraries from patman and dtoc, which is why these
+need to be installed. Also you need `libfdt` and `pylibfdt` which can be
+installed like this::
+
+ git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git
+ cd dtc
+ pip install .
+ make NO_PYTHON=1 install
+
+This installs the `libfdt.so` library into `~/lib` so you can use
+`LD_LIBRARY_PATH=~/lib` when running binman. If you want to install it in the
+system-library directory, replace the last line with::
+
+ make NO_PYTHON=1 PREFIX=/ install
+
+Running binman
+--------------
+
Type::
binman build -b <board_name>
Enabling binman for a board
---------------------------
-At present binman is invoked from a rule in the main Makefile. Typically you
-will have a rule like::
-
- ifneq ($(CONFIG_ARCH_<something>),)
- u-boot-<your_suffix>.bin: <input_file_1> <input_file_2> checkbinman FORCE
- $(call if_changed,binman)
- endif
+At present binman is invoked from a rule in the main Makefile. You should be
+able to enable CONFIG_BINMAN to enable this rule.
-This assumes that u-boot-<your_suffix>.bin is a target, and is the final file
-that you need to produce. You can make it a target by adding it to INPUTS-y
-either in the main Makefile or in a config.mk file in your arch subdirectory.
+The output file is typically named image.bin and is located in the output
+directory. If input files are needed to you add these to INPUTS-y either in the
+main Makefile or in a config.mk file in your arch subdirectory.
Once binman is executed it will pick up its instructions from a device-tree
file, typically <soc>-u-boot.dtsi, where <soc> is your CONFIG_SYS_SOC value.
command line. For example some entries need access to files and it is not
always convenient to put these filenames in the image definition (device tree).
-The-a option supports this::
+The -a option supports this::
- -a<prop>=<value>
where::
information about what needs to be fixed. See missing-blob-help for the
message for each tag.
+no-expanded:
+ By default binman substitutes entries with expanded versions if available,
+ so that a `u-boot` entry type turns into `u-boot-expanded`, for example. The
+ `--no-expanded` command-line option disables this globally. The
+ `no-expanded` property disables this just for a single entry. Put the
+ `no-expanded` boolean property in the node to select this behaviour.
+
The attributes supported for images and sections are described below. Several
are similar to those for entries.
'end-at-4gb' property is not applicable where CONFIG_SYS_TEXT_BASE +
Image size != 4gb.
+align-default:
+ Specifies the default alignment for entries in this section, if they do
+ not specify an alignment. Note that this only applies to top-level entries
+ in the section (direct subentries), not any subentries of those entries.
+ This means that each section must specify its own default alignment, if
+ required.
+
Examples of the above options can be found in the tests. See the
tools/binman/test directory.
U-Boot executable and can be updated separately by binman as needed. It can be
disabled with the --no-expanded flag if required.
-The same applies for u-boot-spl and u-boot-spl. In those cases, the expansion
+The same applies for u-boot-spl and u-boot-tpl. In those cases, the expansion
includes the BSS padding, so for example::
spl {
U-Boot will only use the first one that it finds. If you need to include a
more general file you can do that from the more specific file using #include.
-If you are having trouble figuring out what is going on, you can uncomment
-the 'warning' line in scripts/Makefile.lib to see what it has found::
+If you are having trouble figuring out what is going on, you can use
+`DEVICE_TREE_DEBUG=1` with your build::
+
+ make DEVICE_TREE_DEBUG=1
+ scripts/Makefile.lib:334: Automatic .dtsi inclusion: options:
+ arch/arm/dts/juno-r2-u-boot.dtsi arch/arm/dts/-u-boot.dtsi
+ arch/arm/dts/armv8-u-boot.dtsi arch/arm/dts/armltd-u-boot.dtsi
+ arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi"
+
+
+Updating an ELF file
+====================
+
+For the EFI app, where U-Boot is loaded from UEFI and runs as an app, there is
+no way to update the devicetree after U-Boot is built. Normally this works by
+creating a new u-boot.dtb.out with he updated devicetree, which is automatically
+built into the output image. With ELF this is not possible since the ELF is
+not part of an image, just a stand-along file. We must create an updated ELF
+file with the new devicetree.
- # Uncomment for debugging
- # This shows all the files that were considered and the one that we chose.
- # u_boot_dtsi_options_debug = $(u_boot_dtsi_options_raw)
+This is handled by the --update-fdt-in-elf option. It takes four arguments,
+separated by comma:
+
+ infile - filename of input ELF file, e.g. 'u-boot's
+ outfile - filename of output ELF file, e.g. 'u-boot.out'
+ begin_sym - symbol at the start of the embedded devicetree, e.g.
+ '__dtb_dt_begin'
+ end_sym - symbol at the start of the embedded devicetree, e.g.
+ '__dtb_dt_end'
+
+When this flag is used, U-Boot does all the normal packaging, but as an
+additional step, it creates a new ELF file with the new devicetree embedded in
+it.
+
+If logging is enabled you will see a message like this::
+
+ Updating file 'u-boot' with data length 0x400a (16394) between symbols
+ '__dtb_dt_begin' and '__dtb_dt_end'
+
+There must be enough space for the updated devicetree. If not, an error like
+the following is produced::
+
+ ValueError: Not enough space in 'u-boot' for data length 0x400a (16394);
+ size is 0x1744 (5956)
Entry Documentation
--------------------
+===================
For details on the various entry types supported by binman and how to use them,
-see README.entries. This is generated from the source code using:
+see entries.rst which is generated from the source code using:
+
+ binman entry-docs >tools/binman/entries.rst
+
+.. toctree::
+ :maxdepth: 2
- binman entry-docs >tools/binman/README.entries
+ entries
Managing images
u-boot-dtb 180 108 u-boot-dtb 80 3b5
image-header bf8 8 image-header bf8
+If an older version of binman is used to list images created by a newer one, it
+is possible that it will contain entry types that are not supported. These still
+show with the correct type, but binman just sees them as blobs (plain binary
+data). Any special features of that etype are not supported by the old binman.
+
Extracting files from images
----------------------------
$ binman extract -i image.bin "*u-boot*" -O outdir
+Some entry types have alternative formats, for example fdtmap which allows
+extracted just the devicetree binary without the fdtmap header::
+
+ $ binman extract -i /tmp/b/odroid-c4/image.bin -f out.dtb -F fdt fdtmap
+ $ fdtdump out.dtb
+ /dts-v1/;
+ // magic: 0xd00dfeed
+ // totalsize: 0x8ab (2219)
+ // off_dt_struct: 0x38
+ // off_dt_strings: 0x82c
+ // off_mem_rsvmap: 0x28
+ // version: 17
+ // last_comp_version: 2
+ // boot_cpuid_phys: 0x0
+ // size_dt_strings: 0x7f
+ // size_dt_struct: 0x7f4
+
+ / {
+ image-node = "binman";
+ image-pos = <0x00000000>;
+ size = <0x0011162b>;
+ ...
+
+Use `-F list` to see what alternative formats are available::
+
+ $ binman extract -i /tmp/b/odroid-c4/image.bin -F list
+ Flag (-F) Entry type Description
+ fdt fdtmap Extract the devicetree blob from the fdtmap
+
Replacing files in an image
---------------------------
You can replace files in an existing firmware image created by binman, provided
-that there is an 'fdtmap' entry in the image. For example:
+that there is an 'fdtmap' entry in the image. For example::
$ binman replace -i image.bin section/cbfs/u-boot
You can use BINMAN_VERBOSE=5 (for example) when building to select this.
+Bintools
+========
+
+`Bintool` is the name binman gives to a binary tool which it uses to create and
+manipulate binaries that binman cannot handle itself. Bintools are often
+necessary since Binman only supports a subset of the available file formats
+natively.
+
+Many SoC vendors invent ways to load code into their SoC using new file formats,
+sometimes changing the format with successive SoC generations. Sometimes the
+tool is available as Open Source. Sometimes it is a pre-compiled binary that
+must be downloaded from the vendor's website. Sometimes it is available in
+source form but difficult or slow to build.
+
+Even for images that use bintools, binman still assembles the image from its
+image description. It may handle parts of the image natively and part with
+various bintools.
+
+Binman relies on these tools so provides various features to manage them:
+
+- Determining whether the tool is currently installed
+- Downloading or building the tool
+- Determining the version of the tool that is installed
+- Deciding which tools are needed to build an image
+
+The Bintool class is an interface to the tool, a thin level of abstration, using
+Python functions to run the tool for each purpose (e.g. creating a new
+structure, adding a file to an existing structure) rather than just lists of
+string arguments.
+
+As with external blobs, bintools (which are like 'external' tools) can be
+missing. When building an image requires a bintool and it is not installed,
+binman detects this and reports the problem, but continues to build an image.
+This is useful in CI systems which want to check that everything is correct but
+don't have access to the bintools.
+
+To make this work, all calls to bintools (e.g. with Bintool.run_cmd()) must cope
+with the tool being missing, i.e. when None is returned, by:
+
+- Calling self.record_missing_bintool()
+- Setting up some fake contents so binman can continue
+
+Of course the image will not work, but binman reports which bintools are needed
+and also provide a way to fetch them.
+
+To see the available bintools, use::
+
+ binman tool --list
+
+To fetch tools which are missing, use::
+
+ binman tool --fetch missing
+
+You can also use `--fetch all` to fetch all tools or `--fetch <tool>` to fetch
+a particular tool. Some tools are built from source code, in which case you will
+need to have at least the `build-essential` and `git` packages installed.
+
+Bintool Documentation
+=====================
+
+To provide details on the various bintools supported by binman, bintools.rst is
+generated from the source code using:
+
+ binman bintool-docs >tools/binman/bintools.rst
+
+.. toctree::
+ :maxdepth: 2
+
+ bintools
+
+
Technical details
=================
aren't given, it will also try to derive target-specific versions from the
CROSS_COMPILE environment variable during a cross-compilation.
+If the tool is not available in the path you can use BINMAN_TOOLPATHS to specify
+a space-separated list of paths to search, e.g.::
+
+ BINMAN_TOOLPATHS="/tools/g12a /tools/tegra" binman ...
+
+
+External blobs
+--------------
+
+Binary blobs, even if the source code is available, complicate building
+firmware. The instructions can involve multiple steps and the binaries may be
+hard to build or obtain. Binman at least provides a unified description of how
+to build the final image, no matter what steps are needed to get there.
+
+Binman also provides a `blob-ext` entry type that pulls in a binary blob from an
+external file. If the file is missing, binman can optionally complete the build
+and just report a warning. Use the `-M/--allow-missing` option to enble this.
+This is useful in CI systems which want to check that everything is correct but
+don't have access to the blobs.
+
+If the blobs are in a different directory, you can specify this with the `-I`
+option.
+
+For U-Boot, you can use set the BINMAN_INDIRS environment variable to provide a
+space-separated list of directories to search for binary blobs::
+
+ BINMAN_INDIRS="odroid-c4/fip/g12a \
+ odroid-c4/build/board/hardkernel/odroidc4/firmware \
+ odroid-c4/build/scp_task" binman ...
Code coverage
-------------
$ sudo apt-get install python-coverage python3-coverage python-pytest
+Error messages
+--------------
+
+This section provides some guidance for some of the less obvious error messages
+produced by binman.
+
+
+Expected __bss_size symbol
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Example::
+
+ binman: Node '/binman/u-boot-spl-ddr/u-boot-spl/u-boot-spl-bss-pad':
+ Expected __bss_size symbol in spl/u-boot-spl
+
+This indicates that binman needs the `__bss_size` symbol to be defined in the
+SPL binary, where `spl/u-boot-spl` is the ELF file containing the symbols. The
+symbol tells binman the size of the BSS region, in bytes. It needs this to be
+able to pad the image so that the following entries do not overlap the BSS,
+which would cause them to be overwritte by variable access in SPL.
+
+This symbols is normally defined in the linker script, immediately after
+_bss_start and __bss_end are defined, like this::
+
+ __bss_size = __bss_end - __bss_start;
+
+You may need to add it to your linker script if you get this error.
+
+
Concurrent tests
----------------
make BINMAN_VERBOSE=5
+Building sections in parallel
+-----------------------------
+
+By default binman uses multiprocessing to speed up compilation of large images.
+This works at a section level, with one thread for each entry in the section.
+This can speed things up if the entries are large and use compression.
+
+This feature can be disabled with the '-T' flag, which defaults to a suitable
+value for your machine. This depends on the Python version, e.g on v3.8 it uses
+12 threads on an 8-core machine. See ConcurrentFutures_ for more details.
+
+The special value -T0 selects single-threaded mode, useful for debugging during
+development, since dealing with exceptions and problems in threads is more
+difficult. This avoids any use of ThreadPoolExecutor.
+
+
History / Credits
-----------------
Some ideas:
- Use of-platdata to make the information available to code that is unable
- to use device tree (such as a very small SPL image)
+ to use device tree (such as a very small SPL image). For now, limited info is
+ available via linker symbols
- Allow easy building of images by specifying just the board name
- Support building an image for a board (-b) more completely, with a
configurable build directory
--
Simon Glass <sjg@chromium.org>
7/7/2016
+
+.. _ConcurrentFutures: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor
projects / platform / kernel / u-boot.git / blobdiff