"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<chapter>
-<title>File download support</title>
+<title>File Download Support</title>
+
+ <para>
+ BitBake's <filename>fetch</filename> and
+ <filename>fetch2</filename> modules support downloading
+ files.
+ This chapter provides an overview of the fetching process
+ and also presents sections on each of the fetchers BitBake
+ supports.
+ <note>
+ The original <filename>fetch</filename> code, for all
+ practical purposes, has been replaced by
+ <filename>fetch2</filename> code.
+ Consequently, the information in this chapter does not
+ apply to <filename>fetch</filename>.
+ </note>
+ </para>
<section id='file-download-overview'>
<title>Overview</title>
<para>
- BitBake provides support to download files.
- This procedure is called fetching and is handled by the
- fetch and fetch2 modules.
- At this point, the original fetch code is considered to
- be replaced by fetch2 and this manual is only related
- to the fetch2 codebase.
+ When BitBake starts to execute, the very first thing
+ it does is to fetch the source files needed.
+ This section overviews the process.
+ For some additional information on fetching source files, see the
+ "<link linkend='source-fetching-dev-environment'>Source Fetching</link>"
+ section.
</para>
<para>
- The <filename>SRC_URI</filename> is normally used to
- tell BitBake which files to fetch.
- The next sections describe the available fetchers and
- their options.
- Each fetcher honors a set of variables and per
- URI parameters separated by a “;” consisting of a key and
- a value.
- The semantics of the variables and parameters are
- defined by the fetcher.
- BitBake tries to have consistent semantics between the
- different fetchers.
+ When BitBake goes looking for source files, it follows a search
+ order:
+ <orderedlist>
+ <listitem><para><emphasis>Pre-mirror Sites:</emphasis>
+ BitBake first uses pre-mirrors to try and find source
+ files.
+ These locations are defined using the
+ <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
+ variable.
+ </para></listitem>
+ <listitem><para><emphasis>Source URI:</emphasis>
+ If pre-mirrors fail, BitBake uses
+ <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>.
+ </para></listitem>
+ <listitem><para><emphasis>Mirror Sites:</emphasis>
+ If fetch failures occur using <filename>SRC_URI</filename>,
+ BitBake next uses mirror location as defined by the
+ <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
+ variable.
+ </para></listitem>
+ </orderedlist>
</para>
<para>
- The overall fetch process first attempts to fetch from
- <filename>PREMIRRORS</filename>.
- If these fail, the original <filename>SRC_URI</filename>
- is attempted.
- If that fails, BitBake falls back to
- <filename>MIRRORS</filename>.
Because cross-URLs are supported, it is possible to mirror
a Git repository on an HTTP server as a tarball.
Here are some examples that show commonly used mirror
definitions:
<literallayout class='monospaced'>
-PREMIRRORS ?= "\
- bzr://.*/.* http://somemirror.org/sources/ \n \
- cvs://.*/.* http://somemirror.org/sources/ \n \
- git://.*/.* http://somemirror.org/sources/ \n \
- hg://.*/.* http://somemirror.org/sources/ \n \
- osc://.*/.* http://somemirror.org/sources/ \n \
- p4://.*/.* http://somemirror.org/sources/ \n \
- svk://.*/.* http://somemirror.org/sources/ \n \
- svn://.*/.* http://somemirror.org/sources/ \n"
-
-MIRRORS =+ "\
- ftp://.*/.* http://somemirror.org/sources/ \n \
- http://.*/.* http://somemirror.org/sources/ \n \
- https://.*/.* http://somemirror.org/sources/ \n"
+ PREMIRRORS ?= "\
+ bzr://.*/.* http://somemirror.org/sources/ \n \
+ cvs://.*/.* http://somemirror.org/sources/ \n \
+ git://.*/.* http://somemirror.org/sources/ \n \
+ hg://.*/.* http://somemirror.org/sources/ \n \
+ osc://.*/.* http://somemirror.org/sources/ \n \
+ p4://.*/.* http://somemirror.org/sources/ \n \
+ svk://.*/.* http://somemirror.org/sources/ \n \
+ svn://.*/.* http://somemirror.org/sources/ \n"
+
+ MIRRORS =+ "\
+ ftp://.*/.* http://somemirror.org/sources/ \n \
+ http://.*/.* http://somemirror.org/sources/ \n \
+ https://.*/.* http://somemirror.org/sources/ \n"
</literallayout>
</para>
<para>
- Non-local downloaded output is placed
- into the directory specified by the
- <filename>DL_DIR</filename> variable.
- For non local archive downloads, the code can verify
- sha256 and md5 checksums for the download to ensure
- the file has been downloaded correctly.
- These can be specified in the following forms
- for md5 and sha256 checksums, respectively:
+ Any source files that are not local (i.e. downloaded from
+ the Internet) are placed into the download directory,
+ which is specified by
+ <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>.
+ </para>
+
+ <para>
+ For non-local archive downloads, the fetcher code can verify
+ sha256 and md5 checksums to ensure
+ the archives have been downloaded correctly.
+ You can specify these checksums by using the
+ <filename>SRC_URI</filename> variable with the appropriate
+ varflags as follows:
<literallayout class='monospaced'>
SRC_URI[md5sum]
SRC_URI[sha256sum]
</literallayout>
- You can also specify them as parameters on the
- <filename>SRC_URI</filename>:
+ You can also specify the checksums as parameters on the
+ <filename>SRC_URI</filename> as shown below:
<literallayout class='monospaced'>
SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
</literallayout>
- If <filename>BB_STRICT_CHECKSUM</filename> is set, any download
- without a checksum will trigger an error message.
- In cases where multiple files are listed in
+ If
+ <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link>
+ is set, any download without a checksum triggers an error message.
+ In cases where multiple files are listed using
<filename>SRC_URI</filename>, the name parameter is used
assign names to the URLs and these are then specified
in the checksums using the following form:
</para>
</section>
- <section id='local-file-fetcher'>
- <title>Local file fetcher</title>
+ <section id='bb-fetchers'>
+ <title>Fetchers</title>
<para>
- The URN for the local file fetcher is file.
- The filename can be either absolute or relative.
- If the filename is relative,
- <filename>FILESPATH</filename> and failing that
- <filename>FILESDIR</filename> will be used to find the
- appropriate relative file.
- The metadata usually extend these variables to include
- variations of the values in <filename>OVERRIDES</filename>.
- Single files and complete directories can be specified.
- <literallayout class='monospaced'>
+ As mentioned in the previous section, the
+ <filename>SRC_URI</filename> is normally used to
+ tell BitBake which files to fetch.
+ And, the fetcher BitBake uses depends on the how
+ <filename>SRC_URI</filename> is set.
+ </para>
+
+ <para>
+ These next few sections describe the available fetchers and
+ their options.
+ Each fetcher honors a set of variables URI parameters,
+ which are separated by semi-colon characters and consist
+ of a key and a value.
+ The semantics of the variables and parameters are
+ defined by the fetcher.
+ BitBake tries to have consistent semantics between the
+ different fetchers.
+ </para>
+
+ <section id='local-file-fetcher'>
+ <title>Local file fetcher</title>
+
+ <para>
+ The URN for the local file fetcher is file.
+ </para>
+
+ <para>
+ The filename can be either absolute or relative.
+ If the filename is relative,
+ <link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>
+ is used.
+ Failing that,
+ <link linkend='var-FILESDIR'><filename>FILESDIR</filename></link>
+ is used to find the appropriate relative file.
+ </para>
+
+ <para>
+ The metadata usually extend these variables to include
+ variations of the values in
+ <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>.
+ Single files and complete directories can be specified.
+ <literallayout class='monospaced'>
SRC_URI = "file://relativefile.patch"
SRC_URI = "file://relativefile.patch;this=ignored"
SRC_URI = "file:///Users/ich/very_important_software"
- </literallayout>
- </para>
- </section>
+ </literallayout>
+ </para>
+ </section>
- <section id='cvs-fetcher'>
- <title>CVS fetcher</title>
+ <section id='cvs-fetcher'>
+ <title>CVS fetcher</title>
- <para>
- The URN for the CVS fetcher is cvs.
- This fetcher honors the variables <filename>CVSDIR</filename>,
- <filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>,
- <filename>UPDATECOMMAND_cvs</filename>.
- <filename>DL_DIR</filename> specifies where a
- temporary checkout is saved.
- <filename>SRCDATE</filename> specifies which date to
- use when doing the fetching (the special value of "now"
- will cause the checkout to be updated on every build).
- <filename>FETCHCOMMAND</filename> and
- <filename>UPDATECOMMAND</filename> specify which executables
- to use for the CVS checkout or update.
- </para>
+ <para>
+ The URN for the CVS fetcher is cvs.
+ </para>
- <para>
- The supported parameters are module, tag, date,
- method, localdir, rshand scmdata.
- The module specifies which module to check out,
- the tag describes which CVS TAG should be used for
- the checkout.
- By default, the TAG is empty.
- A date can be specified to override the
- <filename>SRCDATE</filename> of the
- configuration to checkout a specific date.
- The special value of "now" will cause the checkout to be
- updated on every build.
- method is by default pserver.
- If ext is used the rsh parameter will be evaluated
- and <filename>CVS_RSH</filename> will be set.
- Finally, localdir is used to checkout into a special
- directory relative to <filename>CVSDIR</filename>.
- <literallayout class='monospaced'>
+ <para>
+ This fetcher honors the variables <filename>CVSDIR</filename>,
+ <filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>,
+ <filename>UPDATECOMMAND_cvs</filename>.
+ The
+ <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
+ variable specifies where a
+ temporary checkout is saved.
+ The
+ <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link>
+ variable specifies which date to
+ use when doing the fetching.
+ The special value of "now" causes the checkout to be
+ updated on every build.
+ The <filename>FETCHCOMMAND</filename> and
+ <filename>UPDATECOMMAND</filename> variables specify the executables
+ to use for the CVS checkout or update.
+ </para>
+
+ <para>
+ The supported parameters are as follows:
+ <itemizedlist>
+ <listitem><para><emphasis>"module":</emphasis>
+ Specifies the module to check out.
+ </para></listitem>
+ <listitem><para><emphasis>"tag":</emphasis>
+ Describes which CVS TAG should be used for
+ the checkout.
+ By default, the TAG is empty.
+ </para></listitem>
+ <listitem><para><emphasis>"date":</emphasis>
+ Specifies a date.
+ If no "date" is specified, the
+ <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link>
+ of the configuration is used to checkout a specific date.
+ The special value of "now" causes the checkout to be
+ updated on every build.
+ </para></listitem>
+ <listitem><para><emphasis>"method":</emphasis>
+ By default <filename>pserver</filename>.
+ If "method" is set to "ext", BitBake examines the "rsh"
+ parameter and sets <filename>CVS_RSH</filename>.
+ </para></listitem>
+ <listitem><para><emphasis>"localdir":</emphasis>
+ Used to checkout force into a special
+ directory relative to <filename>CVSDIR</filename>.
+ </para></listitem>
+ <listitem><para><emphasis>"rsh"</emphasis>
+ Used in conjunction with the "method" parameter.
+ </para></listitem>
+ <listitem><para><emphasis>"scmdata":</emphasis>
+ I need a description for this.
+ </para></listitem>
+ </itemizedlist>
+ Following are two examples using cvs:
+ <literallayout class='monospaced'>
SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
- </literallayout>
- </para>
- </section>
+ </literallayout>
+ </para>
+ </section>
- <section id='http-ftp-fetcher'>
- <title>HTTP/FTP fetcher</title>
+ <section id='http-ftp-fetcher'>
+ <title>HTTP/FTP fetcher</title>
- <para>
- The URNs for the HTTP/FTP fetcher are http, https, and ftp.
- This fetcher honors the variables
- <filename>FETCHCOMMAND_wget</filename>.
- <filename>FETCHCOMMAND</filename> contains the command used
- for fetching.
- “${URI}” and “${FILES}” will be replaced by the URI and
- basename of the file to be fetched.
- <literallayout class='monospaced'>
+ <para>
+ The URNs for the HTTP/FTP fetcher are http, https, and ftp.
+ </para>
+
+ <para>
+ This fetcher honors the variables
+ <filename>FETCHCOMMAND_wget</filename>.
+ The <filename>FETCHCOMMAND</filename> variable
+ contains the command used for fetching.
+ “${URI}” and “${FILES}” are replaced by the URI and
+ the base name of the file to be fetched.
+ <literallayout class='monospaced'>
SRC_URI = "http://oe.handhelds.org/not_there.aac"
SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac"
SRC_URI = "ftp://you@oe.handheld.sorg/home/you/secret.plan"
- </literallayout>
- </para>
- </section>
+ </literallayout>
+ </para>
+ </section>
- <section id='svn-fetcher'>
- <title>SVN Fetcher</title>
+ <section id='svn-fetcher'>
+ <title>SVN Fetcher</title>
- <para>
- The URN for the SVN fetcher is svn.
- </para>
+ The URN for the SVN fetcher is svn.
+ </para>
- <para>
- This fetcher honors the variables
- <filename>FETCHCOMMAND_svn</filename>,
- <filename>SVNDIR</filename>,
- and <filename>SRCREV</filename>.
- <filename>FETCHCOMMAND</filename> contains the
- subversion command.
- <filename>SRCREV</filename> specifies which revision
- to use when doing the fetching.
- </para>
+ <para>
+ This fetcher honors the variables
+ <filename>FETCHCOMMAND_svn</filename>,
+ <filename>SVNDIR</filename>,
+ and
+ <link linkend='var-SRCREV'><filename>SRCREV</filename></link>.
+ The <filename>FETCHCOMMAND</filename> variable contains the
+ <filename>subversion</filename> command.
+ The <filename>SRCREV</filename> variable specifies which revision
+ to use when doing the fetching.
+ </para>
- <para>
- The supported parameters are proto, rev and scmdata.
- proto is the Subversion protocol, rev is the
- Subversion revision.
- If scmdata is set to “keep”, the “.svn” directories will
- be available during compile-time.
- <literallayout class='monospaced'>
+ <para>
+ The supported parameters are as follows:
+ <itemizedlist>
+ <listitem><para><emphasis>"proto":</emphasis>
+ The Subversion protocol.
+ </para></listitem>
+ <listitem><para><emphasis>"rev":</emphasis>
+ The Subversion revision.
+ </para></listitem>
+ <listitem><para><emphasis>"scmdata":</emphasis>
+ Set to "keep" causes the “.svn” directories
+ to be available during compile-time.
+ </para></listitem>
+ </itemizedlist>
+ Following are two examples using svn:
+ <literallayout class='monospaced'>
SRC_URI = "svn://svn.oe.handhelds.org/svn;module=vip;proto=http;rev=667"
SRC_URI = "svn://svn.oe.handhelds.org/svn/;module=opie;proto=svn+ssh;date=20060126"
- </literallayout>
- </para>
- </section>
+ </literallayout>
+ </para>
+ </section>
- <section id='git-fetcher'>
- <title>GIT Fetcher</title>
+ <section id='git-fetcher'>
+ <title>GIT Fetcher</title>
- <para>
- The URN for the GIT Fetcher is git.
- </para>
+ The URN for the Git Fetcher is git.
+ </para>
- <para>
- The variable <filename>GITDIR</filename> will be used as the
- base directory where the Git tree is cloned to.
- </para>
+ The variable <filename>GITDIR</filename> is used as the
+ base directory in which the Git tree is cloned.
+ </para>
- <para>
- The parameters are tag, protocol, and scmdata.
- The tag parameter is a Git tag, the default is “master”.
- The protocol tag is the Git protocol to use and defaults to “git”
- if a hostname is set, otherwise it is “file”.
- If scmdata is set to “keep”, the “.git” directory will be available
- during compile-time.
- <literallayout class='monospaced'>
+ <para>
+ The supported parameters are as follows:
+ <itemizedlist>
+ <listitem><para><emphasis>"tag":</emphasis>
+ The Git tag.
+ The default is "master".
+ </para></listitem>
+ <listitem><para><emphasis>"protocol":</emphasis>
+ The Git protocol.
+ The default is "git" when a hostname is set.
+ If a hostname is not set, the Git protocol is "file".
+ </para></listitem>
+ <listitem><para><emphasis>"scmdata":</emphasis>
+ When set to “keep”, the “.git” directory is available
+ during compile-time.
+ </para></listitem>
+ </itemizedlist>
+ Following are two examples using git:
+ <literallayout class='monospaced'>
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1"
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"
- </literallayout>
- </para>
+ </literallayout>
+ </para>
+ </section>
</section>
</chapter>
projects / scm / bb / tizen-distro.git / commitdiff