"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<chapter id="user-manual-metadata">
- <title>Metadata</title>
+ <title>Syntax and Operators</title>
- <section>
- <title>Overview</title>
-
- <para>
- The BitBake task executor together with various types of configuration
- files form the OpenEmbedded Core.
- This section provides an overview of the task executor and the
- configuration files by describing their use and interaction.
- </para>
-
- <para>
- BitBake handles the parsing and execution of the data files.
- The data itself is of various types:
- <itemizedlist>
- <listitem><para><emphasis>Recipes:</emphasis>
- Details about particular pieces of software.
- </para></listitem>
- <listitem><para><emphasis>Class Data:</emphasis>
- An abstraction of common build information
- (e.g. how to build a Linux kernel).
- </para></listitem>
- <listitem><para><emphasis>Configuration Data:</emphasis>
- Machine-specific settings, policy decisions,
- and so forth.
- Configuration data acts as the glue to bind everything
- together.</para></listitem>
- </itemizedlist>
- The remainder of this chapter provides examples of BitBake metadata.
- Any syntax not supported in any of the previously listed areas
- is documented as such.
- </para>
- </section>
+ <para>
+ Bitbake files have their own syntax.
+ The syntax has similarities to several
+ other languages but also has some unique features.
+ This section describes the available syntax and operators
+ as well as provides examples.
+ </para>
<section id='basic-syntax'>
<title>Basic Syntax</title>
</para></listitem>
</itemizedlist>
</para>
-
- <section id='setscene'>
- <title>Setscene</title>
-
- <para>
- This section needs to get the concept of the setscene across.
- The reader needs to know what it is and what it is used for during
- the build process.
- </para>
- </section>
-
- <section id='checksums'>
- <title>Checksums (Signatures)</title>
-
- <para>
- A checksum is a unique signature of a task's inputs.
- The setscene code uses a checksum to determine if a task needs
- to be run.
- Because it is a change in a task's inputs that triggers running
- the task, the process needs to detect all the inputs to a given task.
- For shell tasks, this turns out to be fairly easy because
- BitBake generates a "run" shell script for each task and
- it is possible to create a checksum that gives you a good idea of when
- the task's data changes.
- </para>
-
- <para>
- To complicate the problem, some things should not be included in
- the checksum.
- First, there is the actual specific build path of a given task -
- the working directory.
- It does not matter if the working directory changes because it should not
- affect the output for target packages.
- The simplistic approach for excluding the working directory is to set
- it to some fixed value and create the checksum for the "run" script.
- </para>
-
- <para>
- Another problem results from the "run" scripts containing functions that
- might or might not get called.
- The incremental build solution contains code that figures out dependencies
- between shell functions.
- This code is used to prune the "run" scripts down to the minimum set,
- thereby alleviating this problem and making the "run" scripts much more
- readable as a bonus.
- </para>
-
- <para>
- So far we have solutions for shell scripts.
- What about Python tasks?
- The same approach applies even though these tasks are more difficult.
- The process needs to figure out what variables a Python function accesses
- and what functions it calls.
- Again, the incremental build solution contains code that first figures out
- the variable and function dependencies, and then creates a checksum for the data
- used as the input to the task.
- </para>
-
- <para>
- Like the working directory case, situations exist where dependencies
- should be ignored.
- For these cases, you can instruct the build process to ignore a dependency
- by using a line like the following:
- <literallayout class='monospaced'>
- PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
- </literallayout>
- This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not
- depend on the value of <filename>MACHINE</filename>, even if it does reference it.
- </para>
-
- <para>
- Equally, there are cases where we need to add dependencies BitBake
- is not able to find.
- You can accomplish this by using a line like the following:
- <literallayout class='monospaced'>
- PACKAGE_ARCHS[vardeps] = "MACHINE"
- </literallayout>
- This example explicitly adds the <filename>MACHINE</filename> variable as a
- dependency for <filename>PACKAGE_ARCHS</filename>.
- </para>
-
- <para>
- Consider a case with in-line Python, for example, where BitBake is not
- able to figure out dependencies.
- When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
- produces output when it discovers something for which it cannot figure out
- dependencies.
- </para>
-
- <para>
- Thus far, this section has limited discussion to the direct inputs into a task.
- Information based on direct inputs is referred to as the "basehash" in the
- code.
- However, there is still the question of a task's indirect inputs - the
- things that were already built and present in the build directory.
- The checksum (or signature) for a particular task needs to add the hashes
- of all the tasks on which the particular task depends.
- Choosing which dependencies to add is a policy decision.
- However, the effect is to generate a master checksum that combines the basehash
- and the hashes of the task's dependencies.
- </para>
-
- <para>
- At the code level, there are a variety of ways both the basehash and the
- dependent task hashes can be influenced.
- Within the BitBake configuration file, we can give BitBake some extra information
- to help it construct the basehash.
- The following statement effectively results in a list of global variable
- dependency excludes - variables never included in any checksum.
- This example uses variables from OpenEmbedded to help illustrate
- the concept:
- <literallayout class='monospaced'>
- BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
- SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
- USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
- PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
- CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
- </literallayout>
- The previous example excludes the work directory, which is part of
- <filename>TMPDIR</filename>.
- </para>
-
- <para>
- The rules for deciding which hashes of dependent tasks to include through
- dependency chains are more complex and are generally accomplished with a
- Python function.
- The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
- of this and also illustrates how you can insert your own policy into the system
- if so desired.
- This file defines the two basic signature generators OpenEmbedded Core
- uses: "OEBasic" and "OEBasicHash".
- By default, there is a dummy "noop" signature handler enabled in BitBake.
- This means that behavior is unchanged from previous versions.
- <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
- through this setting in the <filename>bitbake.conf</filename> file:
- <literallayout class='monospaced'>
- BB_SIGNATURE_HANDLER ?= "OEBasicHash"
- </literallayout>
- The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
- "OEBasic" version but adds the task hash to the stamp files.
- This results in any metadata change that changes the task hash, automatically
- causing the task to be run again.
- This removes the need to bump
- <link linkend='var-PR'><filename>PR</filename></link>
- values, and changes to metadata automatically ripple across the build.
- </para>
-
- <para>
- It is also worth noting that the end result of these signature generators is to
- make some dependency and hash information available to the build.
- This information includes:
- <itemizedlist>
- <listitem><para><filename>BB_BASEHASH_task-<taskname></filename>:
- The base hashes for each task in the recipe.
- </para></listitem>
- <listitem><para><filename>BB_BASEHASH_<filename:taskname></filename>:
- The base hashes for each dependent task.
- </para></listitem>
- <listitem><para><filename>BBHASHDEPS_<filename:taskname></filename>:
- The task dependencies for each task.
- </para></listitem>
- <listitem><para><filename>BB_TASKHASH</filename>:
- The hash of the currently running task.
- </para></listitem>
- </itemizedlist>
- </para>
- </section>
</section>
</chapter>