1 Nest Labs Build - GNU Autotools
2 ===============================
4 [![Build Status][nlbuild-autotools-travis-svg]][nlbuild-autotools-travis]
6 [nlbuild-autotools-travis]: https://travis-ci.org/nestlabs/nlbuild-autotools
7 [nlbuild-autotools-travis-svg]: https://travis-ci.org/nestlabs/nlbuild-autotools.svg?branch=master
11 The Nest Labs Build - GNU Autotools (nlbuild-autotools) provides a
12 customized, turnkey build system framework, based on GNU autotools, for
13 standalone Nest Labs (or other) software packages that need to support
14 not only building on and targeting against standalone build host
15 systems but also embedded target systems using GCC-based or
16 -compatible toolchains.
18 In addition, nlbuild-autotools endeavors to make it easy to support:
20 * Unit and Functional Tests (via 'make check')
21 * Code Coverage (via 'make coverage')
22 * Code Formatting (via 'make pretty' or 'make pretty-check')
24 * Distribution Generation (via 'make dist' or 'make docdist')
26 # Users and System Integrators
30 This project is typically subtreed (or git submoduled) into a target
31 project repository and serves as the seed for that project's build
34 Assuming that you already have a project repository established in
35 git, perform the following in your project repository:
38 1. % git remote add nlbuild-autotools ssh://<PATH_TO_REPOSITORY>/nlbuild-autotools.git
39 2. % git fetch nlbuild-autotools
42 You can place the nlbuild-autotools package anywhere in your project;
43 however, by convention, "third_party/nlbuild-autotools/repo" is recommended:
46 3. % mkdir third_party
47 4. % git subtree add --prefix=third_party/nlbuild-autotools/repo --squash --message="Add subtree mirror of repository 'ssh://<PATH_TO_REPOSITORY>/nlbuild-autotools.git' branch 'master' at commit 'HEAD'." nlbuild-autotools HEAD
50 At this point, you now have the nlbuild-autotools package integrated
51 into your project. The next step is using the
52 nlbuild-autotools-provided examples as templates. To do this, a
53 convenience script has been provided that will get you started. You
54 can tune and customize the results, as needed, for your project. From
55 the top level of your project tree:
58 5. % third_party/nlbuild-autotools/repo/scripts/mkskeleton -I third_party/nlbuild-autotools/repo --package-description "My Fantastic Package" --package-name "mfp"
61 ## Supported Build Host Systems
63 For Linux users, you likely already have GNU autotools installed through your system's distribution (e.g. Ubuntu). However, if your GNU autotools packages are incomplete or downrevision relative to what's required from nlbuild-autotools, nlbuild-autotools can be built and installed or can be downloaded and expanded in your local nlbuild-autotools tree.
65 The nlbuild-autotools system supports and has been tested against the
66 following POSIX-based build host systems:
71 * x86_64-unknown-linux-gnu
73 ### Build and Install {#Build_and_Install}
75 Simply invoke `make tools` at the top-level of your nlbuild-autotools
76 tree or, for example, from your package or project in which you have
77 integrated nlbuild-autotools:
80 make -C third_party/nlbuild-autotools/repo tools
83 ### Download and Expand
85 Alongside nlbuild-autotools distributions are pre-built
86 architecture-independent and -dependent distributions of the form:
88 * nlbuild-autotools-common-_version_.tar.{gz,xz}
89 * nlbuild-autotools-_host_-_version_.tar.{gz,xz}
91 Find and download the appropriate pair of nlbuild-autotools-common and
92 nlbuild-autotools-_host_ for your system and then expand them from the
93 top-level of your nlbuild-autotools tree with a command similar to the
97 % tar --directory tools/host -zxvf nlbuild-autotools-common-1.1.tar.gz
98 % tar --directory tools/host -zxvf nlbuild-autotools-x86_64-unknown-linux-gnu-1.1.tar.gz
101 Please see the [FAQ](#FAQ) section for more background on why this package
102 provides options for these pre-built tools.
104 ## Package Organization
106 The nlbuild-autotools package is laid out as follows:
108 | Directory | Description |
109 |--------------------------------------|------------------------------------------------------------------------------------------|
110 | autoconf/ | GNU autoconf infrastructure provided by nlbuild-autotools. |
111 | autoconf/m4/ | GNU m4 macros for configure.ac provided by nlbuild-autotools. |
112 | automake/ | GNU automake Makefile.am header and footer infrastructure provided by nlbuild-autotools. |
113 | automake/post/ | GNU automake Makefile.am footers. |
114 | automake/post.am | GNU automake Makefile.am footer included by every makefile. |
115 | automake/pre/ | GNU automake Makefile.am headers. |
116 | automake/pre.am | GNU automake Makefile.am header included by every makefile. |
117 | examples/ | Example template files for starting your own nlbuild-autotools-based project. |
118 | scripts/ | Automation scripts for regenerating the build system and for managing package versions. |
119 | tools/ | Qualified packages of and pre-built instances of GNU autotools. |
120 | tools/host/ | Pre-built instances of GNU autotools (if installed). |
121 | tools/host/i686-pc-cygwin/ | Pre-built instances of GNU autotools for 32-bit Cygwin (if installed). |
122 | tools/host/i686-pc-linux-gnu/ | Pre-built instances of GNU autotools for 32-bit Linux (if installed). |
123 | tools/host/x86_64-apple-darwin/ | Pre-built instances of GNU autotools for 64-bit Mac OS X (if installed). |
124 | tools/host/x86_64-unknown-linux-gnu/ | Pre-built instances of GNU autotools for 64-bit Linux (if installed). |
125 | tools/packages/ | Qualified packages for GNU autotools. |
126 ## Internal Package Dependencies and Repositories
128 Your package may have dependencies on other packages that can either
129 be inlined into your package or can be specified externally. If your
130 package has such dependencies, nlbuild-autotools contains support to
131 facilitate easy standalone tests and a successful 'make distcheck'
132 target (which effectively reqires 'configure' with no arguments to
133 produce a successful build) without incurring the costs of inlining
134 these dependencies into your own package.
136 nlbuild-autotools supports this by providing a means to pull down
137 external git package repositories that your package depends on using
138 either git clone (default) or git submodules when you use and support
139 --with-<package>=internal as a location for your dependent packages.
141 The example 'Makefile-bootstrap' has been provided as infrastructure to
142 make this easy for you as a package maintainer and for your package
143 users. The bootstrap makefile supports both the generic 'repos' target
144 to pull down all repositories on which your project depends as well as
145 relative path targets to where those repositories might live in your
146 project when they are internal (e.g,
147 third_party/package/repo).
149 Consequently, you can, for example, invoke:
152 % make -f Makefile-bootstrap repos
158 % make -f Makefile-bootstrap third_party/package/repo
161 to pull down all repositories or just the repository, for example,
162 placed at 'third_party/package/repos'.
164 The bootstrap makefile generated for your package is yours to edit and
165 extend. In fact, hooks have been added so that you can do
166 package-specific work, including recursively pulling down repositories
169 However, an even better and easier approach for your package users is
170 to integrate the bootstrap makefile repository process into your
171 configure script such that when an "internal" package location is
172 detected, it invokes the bootstrap makefile to perform this work on
173 behalf of the user. For example:
176 # Depending on whether my-package has been configured for an internal
177 # location, its directory stem within this package needs to be set
178 # accordingly. In addition, if the location is internal, then we need
179 # to attempt to pull it down using the bootstrap makefile.
181 if test "${nl_with_my_package}" = "internal"; then
182 maybe_my_package_dirstem="my-package/repo"
183 my_package_dirstem="third_party/${maybe_my_package_dirstem}"
185 AC_MSG_NOTICE([attempting to create internal ${my_package_dirstem}])
186 ${MAKE-make} --no-print-directory -C ${ac_confdir} -f Makefile-bootstrap ${my_package_dirstem}
188 if test $? -ne 0; then
189 AC_MSG_ERROR([failed to create ${my_package_dirstem}. Please check your network connection or the correctness of 'repos.conf'])
192 maybe_my_package_dirstem=""
195 AC_SUBST(MY_PACKAGE_SUBDIRS, [${maybe_my_package_dirstem}])
196 AM_CONDITIONAL([PACKAGE_WITH_MY_PACKAGE_INTERNAL], [test "${nl_with_my_package}" = "internal"])
199 Note, the use of AC_SUBST on MY_PACKAGE_SUBDIRS to provide a mechanism
200 to conditionally populate SUBDIRS in the appropriate automake file
201 locations such that GNU autoconf does not generate syntax errors about
202 the potential absence of the subdirectory at bootstrap time.
204 Of course, in either case, network connectivity is required to reach
205 the external git server hosting the packages on which your project
208 In addition to the 'repos' target, the bootstrap makefile also
209 supports the 'clean-repos' target that undoes the work of the 'repos'
210 target. When using either the 'clone' or 'submodule' pull methods, it
211 will clean-up all of the synchronized repositories. When using the
212 'submodule' pull method, it also is careful to ensure it does not
213 disturb existing git or git submodule state your project might be
216 The infrastructure all works, of course, whether you are working in or
217 out of git and whether you have colocated or non-colocated source and
222 This dependent repository feature of nlbuild-autotools uses a file,
223 'repos.conf', at the top level of your project source directory to
224 determine what external project repositories your package might want
225 to pull down, the location of their git server, the branch you want to
226 pull, and the location in your project in which you want to place
229 The format of 'repos.conf' _almost_ precisely follows that used by git
230 submodules with two notable additions, the 'pull' section and the
231 'commit' key. The 'pull' section allows you to optionally specify the
232 'method' key as 'clone' or 'submodule' (defaulting to 'clone' when the
233 key is not present). This selects whether 'git clone' or 'git
234 submodule' is used to pull down repositories. The 'commit' key allows
235 you to specify not only what branch to checkout but, more precisely,
236 what commit or tag to checkout rather than just _HEAD_ of a
237 branch. More information is available in 'Makefile-bootstrap' or with
238 `man gitmodules` or `git help gitmodules`.
242 Q: Why does nlbuild-autotools have an option for its own built versions
243 of GNU autotools rather than leveraging whatever versions exist on
244 the build host system?
246 A: Some build host systems such as Mac OS X may not have GNU autotools
247 at all. Other build host systems, such as Linux, may have different
248 distributions or different versions of those distributions in which
249 the versions of GNU autotools are apt to be different.
251 This differences lead to different primary and secondary autotools
252 output and, as a consequence, a divergent user and support
253 experience. To avoid this, this package provides a pre-built,
254 qualified set of GNU autotools along with a comprehensive,
255 standalone set of scripts to drive them without reliance on those
256 versions of the tools on the build host system.
258 Q: When I rebootstrap my package, I see that a number of files related to
259 nlbuild-autotools have unexpectedly changed. Why is this happening?
261 A: From time to time, the packages that comprise GNU autotools change
262 upstream. Frequently, common host operating systems (e.g., Ubuntu) take
263 a stable snapshot of the current autotools for a major release (e.g.,
264 Ubuntu 14). On the next major release (e.g., Ubuntu 16), another snapshot
267 If your package was first bootstrapped with one version of
268 autotools and those bootstrap-generated files checked-in but you
269 later bootstrap with another version of autotools, then you will
270 likely observe this behavior.
272 There are two solutions to this problem. First, to ensure a
273 consistent set of bootstrap-generated files, you can build the
274 autotools included with nlbuild-autotools. The bootstrap process
275 will always prefer to use those versions rather than those
276 available on the build host when they are available. See the
277 section [Build and Install](#Build_and_Install) for more
280 The second way to ensure a consistent set of bootstrap-generated
281 files is to not check them in. This does, however, require that
282 package users, rather than package maintainers, perform the
283 bootstrap process and does require that package users, rather than
284 package maintainers, have autotools available on the build host.
288 If you are maintaining nlbuild-autotools, you have several key things to know:
290 1. Generating nlbuild-autotools distributions
291 2. Generating optional nlbuild-autotools prebuilt binary distributions.
292 3. Upgrading GNU autotools packages.
294 ### Generating Distributions
296 To generate a nlbuild-autotools distribution, simply invoke:
302 The package version will come from tags in the source code control
303 system, if available; otherwise, from '.default-version'. The version
304 can be overridden from the PACKAGE_VERSION or VERSION environment
307 The resulting archive will be at the top of the package build
310 ### Generating Optional Prebuilt Binary Distributions
312 To generate an optional nlbuild-autotools prebuilt binary
313 distribution, simply invoke:
319 The package version will come from the source code control system, if
320 available; otherwise, from `.default-version`. The version can be
321 overridden from the _PACKAGE_VERSION_ or _VERSION_ environment variables.
323 The resulting archives will be at the top of the package build
326 ### Upgrading GNU Autotools Packages
328 To change or upgrade GNU autotools packages used by nlbuild-autotools,
329 edit the metadata for each package in tools/packages/_package_/
330 _package_.{url,version}.
334 nlbuild-autools follows the [Semantic Versioning guidelines](http://semver.org/)
335 for release cycle transparency and to maintain backwards compatibility.
339 nlbuild-autools is released under the [Apache License, Version 2.0 license](https://opensource.org/licenses/Apache-2.0).
340 See the `LICENSE` file for more information.